Pytorch_geometric: can't upgrade/install torch-spline-conv and torch-scatter on Mac

[cysmnl]

UserWarning:

                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 4.9 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 4.9 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                              !! WARNING !!

fatal error: 'forward_list' file not found

include

         ^~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1

These are the errors I get trying to pip or manually install torch-spline-conv and torch-scatter.

I'm on a Mac OSX 10.12.6
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0

[rusty1s]

Hi,

I‘m not sure if I can help you, as this seems to be a problem with Mac and the new Extension API. Please verify that the official extension-cpp examples work on your machine and report back.

[cysmnl]

Hi,
I could not compile the extension-cpp/cpp folder via python jit.py
There's an issue with my system defaulting to the old clang complier which uses C++ 4.2.1.. I tried installing gcc8 with homebrew and linked it, but checking c++ -v still points to clang.

[rusty1s]

You need to use clang on macOS, and personally, this works fine for me :(

[cysmnl]

After reinstalling anaconda and installing a new environment, I was able to pip install torch-scatter just fine. But now I have trouble importing it.

(torch) cyml:~ cyrus$ python
Python 3.7.0 (default, Jun 28 2018, 07:39:16)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.

import torch
import torch_scatter
Traceback (most recent call last):
File "", line 1, in
File "/Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/torch_scatter/__init__.py", line 3, in
from .mul import scatter_mul
File "/Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/torch_scatter/mul.py", line 3, in
from torch_scatter.utils.ext import get_func
File "/Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/torch_scatter/utils/ext.py", line 2, in
import scatter_cpu
ImportError: dlopen(/Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/scatter_cpu.cpython-37m-darwin.so, 2): Symbol not found: __ZN2at5ErrorC1ENS_14SourceLocationESs
Referenced from: /Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/scatter_cpu.cpython-37m-darwin.so
Expected in: flat namespace
in /Users/cyrus/anaconda3/envs/torch/lib/python3.7/site-packages/scatter_cpu.cpython-37m-darwin.so

[rusty1s]

Please try installing from source:
python setup.py install

[PhillipMarshall]

I get the same error message on Linux "Symbol not found: __ZN2at5ErrorC1ENS_14SourceLocationESs".
After installing from source the error is still the same.

[cysmnl]

I am using Homebrew GCC 8.2.0 with an anaconda environment (python 3.7, GCC 4.0.1)
Tried installing pytorch_scatter-1.0.4 from source, no luck. I get the same error.
I can compile pytorch/extension-cpp/cpp, but still get error "Symbol not found: __ZN2at5ErrorC1ENS_14SourceLocationESs" when I import lltm_cpp.

Everything installs fine in redhat Linux in an anaconda environment (python 3.7, GCC 7.2.0)

[rusty1s]

I could reproduce the error on a mac. I will look into it.

[fighting41love]

Same error.
Mac OS mojave, GCC=5.2.0, python=3.6
ImportError: dlopen(~/lib/python3.6/site-packages/scatter_cpu.cpython-36m-darwin.so, 2): Symbol not found: __ZN2at5ErrorC1ENS_14SourceLocationESs Referenced from:~/lib/python3.6/site-packages/catter_cpu.cpython-36m-darwin.so Expected in: flat namespace in~/lib/python3.6/site-packages/scatter_cpu.cpython-36m-darwin.so

[chengaf]

After installing from source, the error still exists
undefined symbol: _ZN2at6detail20DynamicCUDAInterface10set_deviceE

[rusty1s]

I could resolve this issue by setting up a clean miniconda environment. It seems to be fixable by clearing the pip cache (https://github.com/rusty1s/pytorch_scatter/issues/7).

[ludwigwinkler]

pip install -vvv torch-scatter (phd)
Created temporary directory: /private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-ephem-wheel-cache-plrbxau5
Created temporary directory: /private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-req-tracker-mc_3i5pl
Created requirements tracker '/private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-req-tracker-mc_3i5pl'
Created temporary directory: /private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-install-2c7znfo_
Collecting torch-scatter
1 location(s) to search for versions of torch-scatter:

• https://pypi.org/simple/torch-scatter/
  Getting page https://pypi.org/simple/torch-scatter/
  Looking up "https://pypi.org/simple/torch-scatter/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/torch-scatter/ HTTP/1.1" 304 0
  Analyzing links from page https://pypi.org/simple/torch-scatter/
  Found link https://files.pythonhosted.org/packages/29/96/566ac314e796d4b07209a3b88cc7a8d2e8582d55819e33f72e6c0e8d8216/torch_scatter-0.3.0.tar.gz#sha256=9e5e5a6efa4ef45f584e8611f83690d799370dd122b862646751ae112b685b50 (from https://pypi.org/simple/torch-scatter/), version: 0.3.0
  Found link https://files.pythonhosted.org/packages/6a/b0/ecffacddf573c147c70c6e43ce05d24f007155ce3fb436959d3d2a24da46/torch_scatter-1.0.2.tar.gz#sha256=ccda794c25265b3450206b7fb0bf74f16a0b45f3f72d9547a42e44648a32faee (from https://pypi.org/simple/torch-scatter/), version: 1.0.2
  Found link https://files.pythonhosted.org/packages/08/09/07b106f3e74246f4ecf6517013a053b6dd7486c4f889d81f39adc662431f/torch_scatter-1.0.3.tar.gz#sha256=e626993194819ba65cdf89a52fbbb7780569d9e157bc63dbef13ead6b7a33930 (from https://pypi.org/simple/torch-scatter/), version: 1.0.3
  Found link https://files.pythonhosted.org/packages/2d/70/df2bc259d9606f00854ca43b6839f9047ec44900563435e0067584c93864/torch_scatter-1.0.4.tar.gz#sha256=ec004d687e47da9d5477407849d815629fc8b571ee87aeeebf6af8ed6f16defc (from https://pypi.org/simple/torch-scatter/), version: 1.0.4
  Found link https://files.pythonhosted.org/packages/2f/97/c50a6aeaedc6924180c6f5810d2a7405ce11aa9b82ba4284badad549d665/torch_scatter-1.1.0.tar.gz#sha256=e534cc2ecb2f9d9b559b1513cd411737d26ea5585d1d65ff571fec55f42a49de (from https://pypi.org/simple/torch-scatter/), version: 1.1.0
  Found link https://files.pythonhosted.org/packages/91/5f/eb1d3ef3810cb1165859d40db4d9ee6d7f1dfef97d7e5c34010055f43d95/torch_scatter-1.1.1.tar.gz#sha256=9db7f2c0a5cddf6cfde633e33db7c2c94eaab163e9f8edb46460d6414cc97917 (from https://pypi.org/simple/torch-scatter/), version: 1.1.1
  Using version 1.1.1 (newest of versions: 0.3.0, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.1.1)
  Using cached wheel link: file:///Users/ludwigwinkler/Library/Caches/pip/wheels/e6/c3/70/18a170af616d37cccfa39b1f55c8fd3db6efb1bdaa4f9eb340/torch_scatter-1.1.1-cp37-cp37m-macosx_10_14_x86_64.whl
  Added torch-scatter from file:///Users/ludwigwinkler/Library/Caches/pip/wheels/e6/c3/70/18a170af616d37cccfa39b1f55c8fd3db6efb1bdaa4f9eb340/torch_scatter-1.1.1-cp37-cp37m-macosx_10_14_x86_64.whl to build tracker '/private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-req-tracker-mc_3i5pl'
  Removed torch-scatter from file:///Users/ludwigwinkler/Library/Caches/pip/wheels/e6/c3/70/18a170af616d37cccfa39b1f55c8fd3db6efb1bdaa4f9eb340/torch_scatter-1.1.1-cp37-cp37m-macosx_10_14_x86_64.whl from build tracker '/private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-req-tracker-mc_3i5pl'
  Installing collected packages: torch-scatter

Successfully installed torch-scatter-1.1.1
Cleaning up...
Removed build tracker '/private/var/folders/dk/g2vbp9_j2h73w35cywjv1sw40000gn/T/pip-req-tracker-mc_3i5pl'

FYI,
weirdly enough I could only make torch-scatter install in my conda environment after having it installed in my root python3 via pip3.
So clearing the wheels in the root pip cache was actually the wrong thing to do in my case (then again, it's in my specific case).
I'm not familiar with the inner workings of pip/conda to give recommendations, just wanted to add it to the thread. =)

PS: Thank you very much for the awesome library!

[koustuvsinha]

Installing clang compiler tools within my conda environment worked for me:

conda install -y clang_osx-64 clangxx_osx-64 gfortran_osx-64