Rawtherapee: Test macOS build using clang 3.9.1

[Benitoite]

https://filebin.net/rzowftk5rz26n3b7/RawTherapee_OSX_10.9_64_5.0-r1-gtk3-72-g29a72d15.zip

The main things I had to do in addition to the instructions in Rawpedia were to change the bash environment variables CC and CXX to point to the clang 3.9.1 installed by macports instead of apple's, and substituted 1.15 for 1.14 in the libiconv patching instructions.

[Beep6581]

@Benitoite I have some questions:

• Which versions of macOS should this run on?
• Why does the version in the zip and dmg filenames not match the version in AboutThisBuild.txt? The filenames show this is based on a commit from March 2, while AboutThisBuild shows a commit which does not exist.

If you can make a build using the latest dev branch, and someone verifies that it works fine, then I would like to advertise it on G+ to get more testing.

[Benitoite]

I think theoretically it should run on 10.9 and up...
The zip and dmg and everything was generated by the make macosx_bundle command. I'm not sure why there would be a discrepancy.
This was build from a brand new clone of the project without selecting a specific branch. git checkout dev says I'm Already on 'dev'

[Beep6581]

@ion12 @partha1b can you guys confirm that running the script does that, before we invest time in digging into this again?

[Partha1b]

@Benitoite What are your precise issues?

The minimum version is 10.9 as stated in the CMakeList.txt file: -mmacosx-version-min=10.9
What discrepancy are you referring to?

CC: @Beep6581, @ion12

[Benitoite]

@partha1b looks like the file names have a different commit suffix than what is I believe correctly reported in the aboutthisbuild, and the file names having been auto generated by the make macosx_bundle script.

[Beep6581]

@Partha1b

What discrepancy are you referring to?

The zip file is called
RawTherapee_OSX_10.9_64_5.0-r1-gtk3-72-g29a72d15.zip
and it contains
RawTherapee_OSX_10.9_64_5.0-r1-gtk3-72-g29a72d15.dmg
while AboutThisBuild.txt shows

Version: 5.0-r1-gtk3-474-gd8e0207b

[Partha1b]

@Beep6581 & @Benitoite

The zip file and the dmg are named with the created variable PROJECT_FULL_VERSION which is obtained from git describe --tags --always. The AboutThisBuild.txt is created by cmake.

This is not a Mac issue. 🙂

[heckflosse]

@Partha1b hmmm

1. PROJECT_FULL_VERSION is only in maxosx_bundle.sh
2. git describe --tags --always reports 5.0-r1-gtk3-475-g7553d86 fot the latest commit

So, the -474- (or -475- ) in AboutThisBuild.txt seems to be correct while the -72- seems to be wrong. Or do I miss something?

Ingo

[Beep6581]

@Partha1b and rtdata/CMakeLists.txt calls the script for Apple:
https://github.com/Beep6581/RawTherapee/blob/dev/rtdata/CMakeLists.txt#L66

1. @Benitoite was the macosx_bundle.sh script called automatically or did you run it manually?
2. Were you on dev HEAD when running CMake and did you then check out commit 29a72d15 when running the script? If so, why?
3. Could you delete your "build" folder and make a new build using the latest dev?

[Partha1b]

@heckflosse

Let me reiterate so that we're on the same page.
In macosx_bundle.sh, we have
PROJECT_FULL_VERSION="$gitDescribe" where gitDescribe="$(git describe --tags --always)"

The zip and dmg files are named as
dmg_name="${PROJECT_NAME// /_}_OSX_${MINIMUM_SYSTEM_VERSION}_${PROC_BIT_DEPTH}_${PROJECT_FULL_VERSION}"

[heckflosse]

@Partha1b

git describe --tags --always reports 5.0-r1-gtk3-475-g7553d86 for the lates commit.

What's the name for your zip and dmg in this case?

[Partha1b]

@heckflosse
It's an issue reported by @Benitoite. I think @Beep6581 is already asking @Benitoite about it.

[ion12]

Testing yesterday, I get 5.0-r1-gtk3-474-gd8e0207b in the AboutThisBuild.txt and RawTherapee_OSX_10.9_64_5.0-r1-gtk3-474-gd8e0207b for the zip/dmg.
@Benitoite this could be a local issue on your end, can you try as @Beep6581 suggested to delete and clone RT from scratch and see if you still end up with the discrepency?

[ion12]

Also @Benitoite could you check if in your build there any any missing icon issues? Specifically are you missing the folder expander triangles in the tree view?

I think I've discovered a regression that breaks the link to the Adawaita icons with the latest changes to the build script and executable loader script.
re-enabling
export XDG_DATA_DIRS="${resources}/share
in executable_loader.in seems to resolves it.

ping @Beep6581 @Partha1b there are 4 path exports from the old script that I commented out, can we confirm exactly what is needed and what is not in addition to the currently enabled path exports that @Partha1b said were essential?

[ion12]

@Benitoite commented: The main things I had to do in addition to the instructions in Rawpedia were to change the bash environment variables CC and CXX to point to the clang 3.9.1 installed by macports instead of apple's, and substituted 1.15 for 1.14 in the libiconv patching instructions.

@Beep6581 it seems you made some changes to rawpedia recently -

1. As long as the instructions target macports, we need
   -DCMAKE_C_COMPILER="clang-mp-X.X"
-DCMAKE_CXX_COMPILER="clang++-mp-X.X"
   to remain in the instructions. Otherwise it's confusing. If you don't want to list these in the main cmake commands, they at least need to be mentioned as a requirement for macports in the paragraph before.

2. if
   -DPROC_LABEL="generic processor"
   is no longer necessary, then it should also be removed from the preceding paragraph and not just the cmake commands.

3. Could you explain why
   make -j8 install -> make -j4 install
   and what is the functions of -DWITH_LTO="OFF" ?

4. We already include an updated libiconv 1.15 in the sources, we should update rawpedia too :)

[Beep6581]

Morning

can we confirm exactly what is needed and what is not

As a macOS non-user, I cannot.

As long as the instructions target macports, we need

I'm quite sure whatever changes there arose from the long discussion on GitHub.
I added those things back.

Could you explain why -j4

Quad-core CPUs are far less popular than dual-core ones, that's all.

Why -DWITH_LTO="OFF"

To let the person reading the guide know that such an option exists and they could set it to "on".

We already include an updated libiconv 1.15 in the sources, we should update rawpedia too :)

Done. I also changed -mmacosx-version-min= to 10.9 for libiconv, hope that's ok.

[Benitoite]

Beep6581:

@Benitoite was the macosx_bundle.sh script called automatically or did you run it manually?
Were you on dev HEAD when running CMake and did you then check out commit 29a72d1 when running the script? If so, why?
Could you delete your "build" folder and make a new build using the latest dev?

I used make macosx_bundle as instructed by Rawpedia. I think my problem lied in the fact that I had used the parent directory as the code repository back in time, and I had been just recently cloning from inside of that folder forgetting I had tried to compile it in the past. So deleting the parent folder resolved that. I am using this folder structure for whatever reason,..

repo-rt {
               Rawtherapee {
               build {

And maybe that is what was confusing the macosx_bundle process. I have to build from an empty folder inside the clone, is that correct?

[Benitoite]

Was able to get RawTherapee_OSX_10.9_64_5.0-r1-gtk3-475-g7553d86b.dmg by building in the correct folder.

@ion12 Yes those triangle icons are missing. Clicking where they should be does expand the folder, but the folders don't load when clicked or double clicked upon. Perhaps I need to delete some old prefs file? The file browser preferences is set to show all the extensions.
Also, the program only runs once. Trying to launch after the first launch always crashes. Both the version from the first post and RawTherapee_OSX_10.9_64_5.0-r1-gtk3-475-g7553d86b.dmg I generated this morning.

[Partha1b]

@ion12 Yes, you need the XDG_DATA_HOME variable. Are you sure you need to set XDG_DATA_DIRS?

@Beep6581 Yes, setting macosx minimum version is necessary for all dylibs that you are using in building RT.

@Benitoite When RT crashes at launch, what is the error message? Also, how are you launching RT?

[Beep6581]

@Benitoite it should be:

repo-rt/build
repo-rt/cmake
repo-rt/doc
repo-rt/licenses
repo-rt/rtdata
repo-rt/rtengine
repo-rt/rtexif
repo-rt/rtgui
repo-rt/tools

You should cd into the build folder when running cmake and compiling.

[Benitoite]

@partha1b All I'm able to get is the system crash log... here's the beginning of it:

Process:               rawtherapee-bin [1833]
Path:                  /Applications/RawTherapee.app/Contents/MacOS/rawtherapee-bin
Identifier:            com.rawtherapee.rawtherapee
Version:               5.0.475 (5.0.475)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           rawtherapee-bin [1833]
User ID:               501

Date/Time:             2017-05-01 07:48:53.117 -0700
OS Version:            Mac OS X 10.12.4 (16E195)
Report Version:        12
Anonymous UUID:        567E9ED4-5465-6075-F9A6-523A0FD4EEEA

Sleep/Wake UUID:       06B18D1D-CC80-40A7-93D4-F1C096DA60B7

Time Awake Since Boot: 920000 seconds

System Integrity Protection: enabled

Crashed Thread:        14  pool

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000001
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0x1:
--> 
    __TEXT                 00000001032ce000-00000001039eb000 [ 7284K] r-x/rwx SM=COW  /Applications/RawTherapee.app/Contents/MacOS/rawtherapee-bin

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fffcd09034a mach_msg_trap + 10
1   libsystem_kernel.dylib          0x00007fffcd08f797 mach_msg + 55
2   com.apple.CoreFoundation        0x00007fffb7253854 __CFRunLoopServiceMachPort + 212
3   com.apple.CoreFoundation        0x00007fffb7252cd1 __CFRunLoopRun + 1361
4   com.apple.CoreFoundation        0x00007fffb7252524 CFRunLoopRunSpecific + 420
5   com.apple.HIToolbox             0x00007fffb67b2ebc RunCurrentEventLoopInMode + 240
6   com.apple.HIToolbox             0x00007fffb67b2cf1 ReceiveNextEventCommon + 432
7   com.apple.HIToolbox             0x00007fffb67b2b26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
8   com.apple.AppKit                0x00007fffb4d4de24 _DPSNextEvent + 1120
9   com.apple.AppKit                0x00007fffb54c985e -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
10  libgdk-3.0.dylib                0x0000000104e21141 poll_func + 172
11  libglib-2.0.0.dylib             0x00000001041ed726 g_main_context_iterate + 326
12  libglib-2.0.0.dylib             0x00000001041ed9d3 g_main_loop_run + 207
13  libgtk-3.0.dylib                0x000000010490a97a gtk_main + 74
14  libgtkmm-3.0.1.dylib            0x000000010433349a Gtk::Main::run(Gtk::Window&) + 156
15  rawtherapee-bin                 0x000000010341c44c main + 8492
16  libdyld.dylib                   0x00007fffccf69235 start + 1

Copied app in dmg to Applications folder, then launching by double-clicking on that icon.

[Partha1b]

@Benitoite I am assuming you patched libiconv and rebuilt as per instructions?

[Beep6581]

@Benitoite run RT through GDB to get a meaningful backtrace. Instructions here http://rawpedia.rawtherapee.com/How_to_write_useful_bug_reports

[Benitoite]

@Partha1b

I did the libiconv patch... btw Does libiconv need to be downloaded and patched each time, or does doing it once install patched files permanently into the build system ?

[Benitoite]

@beep6581 will do when I get home from the beach.

[Partha1b]

@Benitoite I don't think so. However, I have no idea about macport stuff and you'll have to ask @ion12 about it. 🙁

[Benitoite]

Libiconv comes in manually during the following of the patching instructions as opposed to macports.

[ion12]

@Benitoite if you followed the rawpedia instructions you should have backed up and replaced macports' libiconv.2.dylib with the patched one. You should only need to do this once.
I don't know why the crash at startup but it doesn't read as any wrong libiconv version errors that I've seen.

The missing Adwaita icons issue is not on your end, it is a problem in the src.
@Partha1b yes, re-enabling XDG_DATA_DIRS in the executable loader resolves this issue.
For what it's worth, if I temporarily rename /opt/local (that's where macports installs all the dependencies - libraries, themes, icons etc.) That ALSO resolves the missing icons issues, even without XDG_DATA_DIRS re-enabled.
Unforunately I don't have a second machine without macports to test on at the moment. Hope to have one soon though and set it up with a fresh 10.12 install.
@Benitoite if you get a build running, you can try opening RawTherapee.app/Contents/MacOS/rawtherapee in a text editor and uncommenting the line
# export XDG_DATA_DIRS="${resources}/share"
and confirm whether this resolves the missing icons for you.

[Benitoite]

@Beep6581 I ran a debug build thru gdb (I executed sudo gdb ˜/repo-rt/build/debug/MacOS/rawtherapee) , but the crash happens before any information can be triggered:

Reading symbols from ./rawtherapee...done.
(gdb) r
Starting program: /Users/tune/repo-rt/build/debug/MacOS/rawtherapee 
During startup program terminated with signal SIGTRAP, Trace/breakpoint trap.
(gdb) set pagination off
(gdb) set logging file log.txt
(gdb) set logging on
Copying output to log.txt.
(gdb) thread apply all bt full
(gdb) q

resulting in an empty log.txt

[Partha1b]

@Benitoite When you see the SIGTRAP signal and the gdb prompt returns, please type bt to get a backtrace.

[Benitoite]

@Partha1b I tried deleting all the .cache and .config folders for RawTherapee*, now getting this...

Reading symbols from ./rawtherapee...done.
(gdb) r
Starting program: /Users/tune/repo-rt/build/debug/MacOS/rawtherapee 
During startup program terminated with signal ?, Unknown signal.
(gdb) bt
No stack.
(gdb) 

Attempting to launch from command line looks like this...

./rawtherapee

(rawtherapee:41817): Gdk-WARNING **: GdkQuartzDisplay does not implement the monitor vfuncs

(rawtherapee:41817): GLib-GObject-WARNING **: invalid cast from 'GtkMenuBar' to 'GtkWindow'

(rawtherapee:41817): Gtk-CRITICAL **: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(rawtherapee:41817): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
Wrong JPEG library version: library is 80, caller expects 90
Wrong JPEG library version: library is 80, caller expects 90
Wrong JPEG library version: library is 80, caller expects 90
Segmentation fault: 11

[Benitoite]

Maybe I have too many jpeg codes. btw port info jpeg reports:

jpeg @9b (graphics)
Variants:             universal

Description:          This package contains C software to implement JPEG image encoding, decoding, and transcoding. This software implements JPEG baseline,
                      extended-sequential, and progressive compression processes.
Homepage:             http://www.ijg.org/

Conflicts with:       libjpeg-turbo, mozjpeg
Platforms:            darwin, freebsd, sunos
License:              IJG
Maintainers:          Email: [email protected]
                      Policy: openmaintainer

Built again after attempting to uninstall all of them via port and re imported just the jpeg package.
At this point I'm able to run the debug version directly ˜/repo-rt/build/debug/MacOS/rawtherapee and it starts up but running thru gdb always crashes. The terminal window does collect some information, I don't know if it's as much as a proper backtrace.

[Benitoite]

@ion12 happy to report uncommenting that line did make the triangles show up in the release build I just made...
With the debug build I was playing with earlier, there was no problem with the triangles.

So, I was able to make macosx_bundle a working zip (with triangles) if I edited the same line in executable_loader.in before invoking that command. https://filebin.net/q9oh76kznl8rq5an/RawTherapee_OSX_10.9_64_5.0-r1-gtk3-475-g7553d86b.zip

[Benitoite]

At this point I'm just wondering if the line could be uncommented or could it be made into an Apple conditional statement?

[ion12]

Sorry for not responding sooner, a bit busy this week.
@Benitoite Didn't have time to do an exhaustive check, but your build seems to run fine on my machine.
You are probably missing @Partha1b's cairo patch, which we never got around to including in the source code or the rawpedia instructions. Still not a 100% sure if it's needed when compiling against macports' dependencies and didn't have time to confirm.

@Benitoite Yes, XDG_DATA_DIRS can be uncommented in the source code, in rt-repo/tools/osx/executable_loader.in - it is an Apple-only script anyway, so no need for conditionals.

Before you do that in your local repo - could you confirm something :
Comment out again XDG_DATA_DIRS in the bundled loader, temporarily rename the /opt folder on your drive to anything but 'opt' and run RT. (be sure to rename back after the test).
The build should run fine, and the icons should be detected correctly even without XDG_DATA_DIRS enabled.
It seems that the icons are not detected correctly when running on a machine that has macports' depencies in place and no XDG_DATA_DIRS setup in the bundled loader.

[ion12]

@Beep6581 @Partha1b
There are still some library/resource linking issues I want to be absolutely sure are set right.
I also think the pixbuff and im-module loaders cache files can/should be re-written to make them portable.
I'll try to get to it over the weekend or next week, as soon as I have some spare time.

I finally have access to a spare machine, and I will test everything from scratch on a virgin Sierra.

@Beep6581 In the meantime, if you test @Benitoite's build with other users, could we refer any relevant debugging feedback in the forum back to github? Maybe concentrate everything into one new github issue? It's getting harder to follow when the same issues are starting to get duplicated over several github threads + the forum as well ;)

[Beep6581]

@ion12 good to hear.

I advertised @Benitoite 's build in Google+ https://plus.google.com/b/106783532637761598368/+rawtherapee/posts/ZyNwHYpA564

[Benitoite]

@ion12 @partha Yes I am using macports and do still have the infamous cairo bug in my latest builds(4 months ago I found a jpg generated by google earth that crashes RT) https://filebin.net/mquyuao4lbmn8soq/cuChi2016school-Sat.jpg

[Benitoite]

@ion12

Before you do that in your local repo - could you confirm something :
Comment out again XDG_DATA_DIRS in the bundled loader, temporarily rename the /opt folder on your drive to anything but 'opt' and run RT. (be sure to rename back after the test).
The build should run fine, and the icons should be detected correctly even without XDG_DATA_DIRS enabled.
It seems that the icons are not detected correctly when running on a machine that has macports' depencies in place and no XDG_DATA_DIRS setup in the bundled loader.

Did that and commented-out build ran fine as you suggested.

[ion12]

Thanks @Benitoite. I hope to get back to this soon.

[Beep6581]

Hi @Benitoite
We just releases 5.1-rc1 http://rawtherapee.com/blog/rawtherapee-5.1-rc1-released
Any chance you could make a new build for macOS? OpenMP-enabled, as before.

[Benitoite]

Sure @Beep6581
https://filebin.net/619ep8s9ynht6l7q/RawTherapee_OSX_10.9_64_5.1-rc1.zip

Version: 5.1-rc1
Branch: 5.1-rc1
Commit: 22d00440
Commit date: 2017-05-10
Compiler: clang-mp-3 3.9.1
Processor: generic x86
System: Apple
Bit depth: 64 bits
Gtkmm: V3.16.0
Build type: release
Build flags: -arch x86_64 -std=c++11 -mtune=generic -Werror=unused-label -mmacosx-version-min=10.9 -flto -fopenmp=libomp -Werror=unknown-pragmas -Wall -Wno-unused-result -Wno-deprecated-declarations -O3 -DNDEBUG
Link flags:  -mtune=generic -headerpad_max_install_names -flto
OpenMP support: ON
MMAP support: ON

[Beep6581]

Thank you, build made public here: https://discuss.pixls.us/t/rawtherapee-5-1-rc1-released/4146?u=morgan_hardwood

[ion12]

@Beep6581 I feel really bad for not getting back to this as soon as I had hoped to, but my work deadlines are proving too stressful at the moment and I don't think I'll have time before the date you announced as the next stable planned date :(

Hopefully the builds @Benitoite is providing are working for other users, and the current state of the build scripts is good enough for the release.

I will have a go at all the checks and improvements I believe are still worth doing as soon as I can find the time to get a fresh build environment up and running.

[Beep6581]

I believe that to be the case. It should reduce the stress and let you and other macOS contributors agree to a better and easier to follow build environment for 5.2.