Stockfish: New master could not compiled in MacOS

[ArminHHJ]

I download the newest version.
Start compile with: make build ARCH=x86-64-bmi2 comp=clang

Error and stopping comp.
misc.cpp:325:10: error: use of undeclared identifier 'aligned_alloc'; did you
mean 'std_aligned_alloc'?
return aligned_alloc(alignment, size);
^~~~~
std_aligned_alloc
misc.cpp:323:7: note: 'std_aligned_alloc' declared here
void* std_aligned_alloc(size_t alignment, size_t size) {
^
1 error generated.
make[1]: * [misc.o] Error 1
make: * [build] Error 2
Armins-MacBook-Pro:src armin$

[vondele]

What is the clang / macos / xcode version ? compilation on MacOS is part of the CI, so it presumably is a compiler version aspect

Our CI uses xcode12

[vondele]

Also a useful test would be if you would test the following change to the code:

diff --git a/src/misc.cpp b/src/misc.cpp
index 3d7c75e55..f4a25a80a 100644
--- a/src/misc.cpp
+++ b/src/misc.cpp
@@ -322,7 +322,7 @@ void prefetch(void* addr) {

 void* std_aligned_alloc(size_t alignment, size_t size) {
 #if defined(__APPLE__)
-  return aligned_alloc(alignment, size);
+  return std::aligned_alloc(alignment, size);
 #elif defined(_WIN32)
   return _mm_malloc(size, alignment);
 #else

[ddugovic]

Interesting, I wonder how to update my CI to xcode12 (or otherwise resolve version differences).
https://travis-ci.org/github/ddugovic/Stockfish/jobs/715767573
https://github.com/ddugovic/Stockfish/blob/master/.travis.yml

[vondele]

so, yes, strange our CI passes on osx https://travis-ci.org/github/official-stockfish/Stockfish/jobs/715765891
but I see no obvious differences between the travis.yml files.
Do you have Makefile differences, in particular:

ifeq ($(KERNEL),Darwin)
        CXXFLAGS += -arch $(arch) -mmacosx-version-min=10.15
        LDFLAGS += -arch $(arch) -mmacosx-version-min=10.15
endif

[TonHaver]

I had the same problem.
Errors did not appear after upgrading to XCode Commandline Tools 11.2 from https://developer.apple.com/download/more/

I kept having a warning, but a executable is produced, which can be run on commandline and produces a bench.
(Still having problems to run it with NNUE in Hiarcs Chess Explorer but that's another issue, I think)

Warning left is:
ld: warning: linking module flags 'SDK Version': IDs have conflicting values ('[2 x i32] [i32 10, i32 15]' from misc.o with '[2 x i32] [i32 10, i32 14]' from ld-temp.o)

[vondele]

@TonHaver that warning should maybe go away with a make clean followed by a clean make ... build

[TonHaver]

@vondele Yes this helped, thanks

[TonHaver]

While using XCode 11.2 did make the errors and warnings go away, and the executable seems running OK in classical mode, trying to use NNUE reveals a problem and a crash. But that might only because I'm still on Mojave MacOS 10.14.6

Makefile mentions minimum version Darwin 10.15 , so NNUE functionality just isn't available on older OS versions

setoption name Use NNUE value true
dyld: lazy symbol binding failed: Symbol not found: _aligned_alloc
Referenced from: /Applications/ChessEngines/Stockfish20200807/src/./stockfish (which was built for Mac OS X 10.15)
Expected in: /usr/lib/libSystem.B.dylib

[ddugovic]

@vondele Unfortunately I do not see any relevant Makefile differences when comparing:
https://github.com/ddugovic/Stockfish/blob/master/src/Makefile
https://github.com/official-stockfish/Stockfish/blob/master/src/Makefile

Looking at the travis CI comparison, my fork contains (although I don't know if something else is the root cause):

```
+

• #
• # Build snapshot
• #
• 
  
  • if [[ "${TRAVIS_PULL_REQUEST}" = "false" && "$DEPLOY" == "stockfish" ]]; then make clean && git apply ../deploy/old-gcc.patch && docker build --tag oldstable-slim-for-sf -f ../deploy/oldstable-slim.dockerfile ../deploy && docker run --volume "$PWD/..:/home/builder/Stockfish" --user $(id -u) oldstable-slim-for-sf; fi
  
  
• 
  
  • if [[ "${TRAVIS_PULL_REQUEST}" = "false" && "$DEPLOY" == "stockfish-osx" ]]; then make clean && make build ARCH=x86-64 EXE=stockfish-osx-x86_64; fi
    
    +```
  
  

[vondele]

so does ../deploy/old-gcc.patch play a role?

[ddugovic]

Well... the compile error is 'aligned_alloc' is not a member of 'std':

g++ -Wall -Wcast-qual -fno-exceptions -std=c++17 -fprofile-generate -pedantic -Wextra -Wshadow -m64 -DANTI -DANTIHELPMATE -DATOMIC -DBUGHOUSE -DCRAZYHOUSE -DDISPLACEDGRID -DEXTINCTION -DGIVEAWAY -DGRID -DHELPMATE -DHORDE -DKNIGHTRELAY -DKOTH -DLOOP -DLOSERS -DPLACEMENT -DRACE -DRELAY -DSLIPPEDGRID -DSUICIDE -DTHREECHECK -DTWOKINGS -DTWOKINGSSYMMETRIC -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2 -flto   -c -o misc.o misc.cpp
misc.cpp: In function 'void* std_aligned_alloc(size_t, size_t)':
misc.cpp:332:10: error: 'aligned_alloc' is not a member of 'std'
   return std::aligned_alloc(alignment, size);
          ^~~

so I don't see how https://github.com/ddugovic/Stockfish/blob/f4ee250044efece3513325f5f0fb86ee54ff1548/deploy/old-gcc.patch#L1-L13

diff --git a/src/thread.h b/src/thread.h
index 3aa5bb51..ac72d15d 100644
--- a/src/thread.h
+++ b/src/thread.h
@@ -84,7 +84,7 @@ public:

 struct MainThread : public Thread {

-  using Thread::Thread;
+  explicit MainThread(size_t n) : Thread(n) { }

   void search() override;
   void check_time();

could cause that. Maybe I should report 'aligned_alloc' is not a member of 'std' to GCC maintainers?

[edward-512]

I can check as well tomorrow when i get back to my Mac. I am running Xcode 12 on the latest public beta of 10.16.

[vondele]

Well... the compile error is 'aligned_alloc' is not a member of 'std':

Actually, that's confusing....

void* std_aligned_alloc(size_t alignment, size_t size) {
#if defined(__APPLE__)
  return aligned_alloc(alignment, size);
#elif defined(_WIN32)
  return _mm_malloc(size, alignment);
#else
  return std::aligned_alloc(alignment, size);
#endif
}

So, that should only happen if __APPLE__ is not defined, while you're on OSX?

[ddugovic]

Ah, https://github.com/ddugovic/Stockfish/blob/f4ee250044efece3513325f5f0fb86ee54ff1548/deploy/oldstable-slim.dockerfile#L1 must be outdated:

FROM debian:oldstable-slim

Also see https://github.com/coelckers/gzdoom/pull/904 for ideas... maybe https://github.com/official-stockfish/Stockfish/pull/2927 works?

[vondele]

Does #2927 fix your CI?

[ddugovic]

Yes, https://travis-ci.org/github/ddugovic/Stockfish/jobs/715833496 just completed successfully.

[TonHaver]

https://github.com/official-stockfish/Stockfish/pull/2927 didn't work on MacOs 10.14

This change gives me NNUE Evaluation in bench

void* std_aligned_alloc(size_t alignment, size_t size) {
#if (defined(__APPLE__) && defined(_LIBCPP_HAS_C11_FEATURES)) || defined(__ANDROID__) || defined(__OpenBSD__) || (defined(__GLIBCXX__) && !defined(_GLIBCXX_HAVE_ALIGNED_ALLOC) )
  return aligned_alloc(alignment, size);
#elif (defined(_WIN32) || (defined(__APPLE__) && !defined(_LIBCPP_HAS_C11_FEATURES)))
  return _mm_malloc(size, alignment);
#else
  return std::aligned_alloc(alignment, size);
#endif
}

I'm not really sure what I'm doing here (going back to 32bit memory?), but it seems that _GLIBCXX_HAVE_ALIGNED_ALLOC is not allways correct. Can't test if first part of the if still works on Catalina MacOS 10.15

[vondele]

OK, interesting, did you need to adjust the Makefile as well (-mmacosx-version-min=10.15)

[TonHaver]

No, it just ignores that, it seems.

I started reading here: https://github.com/Homebrew/homebrew-core/issues/46393 , found some interesting links there, and then tried _mm_malloc

[vondele]

OK, to be correct you need to use similar defines for the std_aligned_free.

[TonHaver]

OK, will try that tomorrow.
And I might be able to test it on MacOs 10.12 or 10.13 too. But not 10.15 .....

[daylen]

aligned_alloc is just not available on macOS versions earlier than 10.15. (The header comment for aligned_alloc in macOS 10.15 says that 10.15 is the _first version_ where that function is available.) Potentially you could try with _mm_malloc if you are using 10.14 or earlier… or you could just update to 10.15, if that's possible in your configuration.

If _mm_malloc works, we could try to adopt that in Stockfish, but the #ifs will get complicated with checking of macOS version numbers, etc. and I think we'd rather avoid that. macOS 10.15 market share as a percentage of macOS users is growing!

[daylen]

Oh, I see you're trying to toggle between whether to use aligned_alloc or _mm_malloc based on defined(_LIBCPP_HAS_C11_FEATURES). I can test and see if we're still picking aligned_alloc in 10.15.

[edward-512]

Works perfectly on my mac on xcode 12...

[edward-512]

However, I have had a long-term issue with using profile-build on Mac. I always seem to run into this issue:
Using "make profile-build ARCH=x86-64-bmi2"
ld: library not found for -lgcov clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [stockfish] Error 1 make[1]: *** [gcc-profile-make] Error 2 make: *** [profile-build] Error 2 edward@Edwards-MacBook-Pro
However, I have found a workaround by using homebrew's gcc-10, and by replacing all gcc to gcc-10 in the makefile, I can make profile builds work. Could this be incorporated into the code in some way?

[vondele]

profile build is a different issue, we should not tackle it here. You can specify a custom compiler without modification via
CXX=gcc-10 make ...
or
make .... COMPILER=gcc-10

[edward-512]

Ok, cool 😎. Just mentioning that it compiles fine for me on macOS 10.16.

[TonHaver]

After changes made to both std_alligned_alloc and std_aligned_free I could succesfully compile on MacOS 10.14 and 10.13 and run a NNUE bench
Tried some older systems too (10.6 and 10.9) but C++17 wasn't available there. Can't find a machine with 10.12

[edward-512]

Interesting... There is a flag in the makefile with “ -mmacosx-version-min=10.15”, if that helps at all.

[ArminHHJ]

@TonHaver, where can I get the changes to test

[TonHaver]

@ArminHHJ https://github.com/TonHaver/Stockfish/blob/master/src/misc.cpp

[TonHaver]

Interesting... There is a flag in the makefile with “ -mmacosx-version-min=10.15”, if that helps at all.

It seems that flag is ignored. I don't see a warning on 10.13 or 10.14

[ArminHHJ]

great, it works. Check in HCE, NN is loaded. Now I do a turnier in CuteChess 😁

Von meinem iPhone gesendet

Am 08.08.2020 um 16:59 schrieb TonHaver [e-mail]:


@ArminHHJ https://github.com/TonHaver/Stockfish/blob/master/src/misc.cpp

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[vondele]

@TonHaver I don't think the flag is ignored, could you try to see what the minimum version of the flag could be, i.e. can we compile on 10.13, 10.14, 10.15 without warning with this modification to the source, if we set macosx-version-min=10.13

[TonHaver]

@vondele I can only test on 10.14 and 10.13 After modification of misc.cpp both compiled without warnings. It didn't matter if I set -mmacosx-version-min to 10.13 , 10.14 , 10.15 or 10.16
All binaries were able to show a NNUE bench

The flag does seem to do something on the 10.14 system at least. The binary file size was different depending on the setting of that flag. (10.15 or 10.16 made no difference)
On the 10.13 system file sizes remained the same

[vondele]

So, it looks like I would go for the patch in https://github.com/official-stockfish/Stockfish/issues/2921#issuecomment-670644058 with (the same changes to free), and change the Makefile flag to macosx-version-min=10.13

@ddugovic if you agree, could you update your PR?

[ddugovic]

@vondele Thanks, I'm trying a build now:
https://travis-ci.org/github/ddugovic/Stockfish/builds/716181411
https://github.com/ddugovic/Stockfish/commit/4692b3f4070bfd6633530a18a1fd3b1c2cf5440f

[ddugovic]

Looks good to me, thanks.

[ArminHHJ]

The new version (master 08.08.) and change from TonHager (misc..ccp) works great with Sergio-NN (01.08.) on MacOS (10.14.6). Much stronger than Stockfish 11 (26.07.), on par with BrainFish (last version 24.07.), but not as strong as SF-Honey-Build (19.07.) with BainBooks and Sergio-NN (01.08.).

Checkup tournament 09.08.20,
CuteChess on MacOS, 0.1 sec/move, HERT500-8ply, TB 6men

Name.................................................Elo......+/-.....Games.....Score......Draws
SF+NN-XI-1907-Brain-SV0801.........71.......17.......1000..........60.0%.....39.3%
SF+NNUE-0808+TH-SV0801........19......16......1000........52.7%....40.9%
Brainfish-2407-bmi2-AJ...................3.........16.......1000.........50.4%.....42.3%
Stockfish-11-2607-bmi2-AJ............-94......17........1000.........36.8%.....35.1%
2000 of 2000 games finished.

Thanks for fixing this error for MacOS, fantastic work :)