Grpc: Compilation error on macOS (issue with boringssl)

Created on 25 Oct 2018 · 50Comments · Source: grpc/grpc

Compile error both using CPython 3.6 and PyPy 6.0.0 (3.5.3) from MacPorts

```pypy3 --version
Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Jul 30 2018, 08:59:51)
[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]

```pypy3 --version
Python 3.6.6

Log:
```b'[C] Compiling src/boringssl/err_data.c'
b"In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:115:
/opt/local/include/openssl/e_os2.h:276:11: error: 'OPENSSL_EXPORT' macro redefined [-Werror,-Wmacro-redefined]

define OPENSSL_EXPORT extern

third_party/boringssl/include/openssl/base.h:182:9: note: previous definition is here

define OPENSSL_EXPORT

In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:83:31: error: redefinition of typedef 'ASN1_INTEGER' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_INTEGER;
^
third_party/boringssl/include/openssl/base.h:244:31: note: previous definition is here
typedef struct asn1_string_st ASN1_INTEGER;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:84:31: error: redefinition of typedef 'ASN1_ENUMERATED' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_ENUMERATED;
^
third_party/boringssl/include/openssl/base.h:240:31: note: previous definition is here
typedef struct asn1_string_st ASN1_ENUMERATED;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:85:31: error: redefinition of typedef 'ASN1_BIT_STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_BIT_STRING;
^
third_party/boringssl/include/openssl/base.h:238:31: note: previous definition is here
typedef struct asn1_string_st ASN1_BIT_STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:86:31: error: redefinition of typedef 'ASN1_OCTET_STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_OCTET_STRING;
^
third_party/boringssl/include/openssl/base.h:245:31: note: previous definition is here
typedef struct asn1_string_st ASN1_OCTET_STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:87:31: error: redefinition of typedef 'ASN1_PRINTABLESTRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_PRINTABLESTRING;
^
third_party/boringssl/include/openssl/base.h:246:31: note: previous definition is here
typedef struct asn1_string_st ASN1_PRINTABLESTRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:88:31: error: redefinition of typedef 'ASN1_T61STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_T61STRING;
^
third_party/boringssl/include/openssl/base.h:248:31: note: previous definition is here
typedef struct asn1_string_st ASN1_T61STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:89:31: error: redefinition of typedef 'ASN1_IA5STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_IA5STRING;
^
third_party/boringssl/include/openssl/base.h:243:31: note: previous definition is here
typedef struct asn1_string_st ASN1_IA5STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:90:31: error: redefinition of typedef 'ASN1_GENERALSTRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_GENERALSTRING;
^
third_party/boringssl/include/openssl/base.h:242:31: note: previous definition is here
typedef struct asn1_string_st ASN1_GENERALSTRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:91:31: error: redefinition of typedef 'ASN1_UNIVERSALSTRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_UNIVERSALSTRING;
^
third_party/boringssl/include/openssl/base.h:250:31: note: previous definition is here
typedef struct asn1_string_st ASN1_UNIVERSALSTRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:92:31: error: redefinition of typedef 'ASN1_BMPSTRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_BMPSTRING;
^
third_party/boringssl/include/openssl/base.h:239:31: note: previous definition is here
typedef struct asn1_string_st ASN1_BMPSTRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:93:31: error: redefinition of typedef 'ASN1_UTCTIME' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_UTCTIME;
^
third_party/boringssl/include/openssl/base.h:251:31: note: previous definition is here
typedef struct asn1_string_st ASN1_UTCTIME;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:94:31: error: redefinition of typedef 'ASN1_TIME' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_TIME;
^
third_party/boringssl/include/openssl/base.h:249:31: note: previous definition is here
typedef struct asn1_string_st ASN1_TIME;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:95:31: error: redefinition of typedef 'ASN1_GENERALIZEDTIME' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_GENERALIZEDTIME;
^
third_party/boringssl/include/openssl/base.h:241:31: note: previous definition is here
typedef struct asn1_string_st ASN1_GENERALIZEDTIME;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:96:31: error: redefinition of typedef 'ASN1_VISIBLESTRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_VISIBLESTRING;
^
third_party/boringssl/include/openssl/base.h:253:31: note: previous definition is here
typedef struct asn1_string_st ASN1_VISIBLESTRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:97:31: error: redefinition of typedef 'ASN1_UTF8STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_UTF8STRING;
^
third_party/boringssl/include/openssl/base.h:252:31: note: previous definition is here
typedef struct asn1_string_st ASN1_UTF8STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:98:31: error: redefinition of typedef 'ASN1_STRING' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef struct asn1_string_st ASN1_STRING;
^
third_party/boringssl/include/openssl/base.h:247:31: note: previous definition is here
typedef struct asn1_string_st ASN1_STRING;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:99:13: error: redefinition of typedef 'ASN1_BOOLEAN' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef int ASN1_BOOLEAN;
^
third_party/boringssl/include/openssl/base.h:233:13: note: previous definition is here
typedef int ASN1_BOOLEAN;
^
In file included from src/boringssl/err_data.c:18:
In file included from /opt/local/include/openssl/err.h:122:
/opt/local/include/openssl/ossl_typ.h:100:13: error: redefinition of typedef 'ASN1_NULL' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef int ASN1_NULL;
^
third_party/boringssl/include/openssl/base.h:234:13: note: previous definition is here
typedef int ASN1_NULL;
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: * [../pip-install-so7ktzhu/grpcio/objs/opt/src/boringssl/err_data.o] Error 1
"

<!--

This form is for bug reports and feature requests ONLY!
For general questions and troubleshooting, please ask/look for answers here:
- grpc.io mailing list: https://groups.google.com/forum/#!forum/grpc-io
- StackOverflow, with "grpc" tag: http://stackoverflow.com/questions/tagged/grpc

Issues specific to *grpc-java*, *grpc-go*, *grpc-node*, *grpc-dart*, *grpc-web* should be created in the repository they belong to (e.g. https://github.com/grpc/grpc-LANGUAGE/issues/new)
-->

### What version of gRPC and what language are you using?
the last commit on the *master* branch

### What operating system (Linux, Windows, …) and version?
macOS, 17.7.0 Darwin Kernel

### What runtime / compiler are you using (e.g. python version or version of gcc)
Apple LLVM version 9.1.0 (clang-902.0.39.2)

### What did you do?
`python3 setup.py build` or `pypy3 setup.py build`

Standalone **boringssl** from the *master* branch can be compiled as expected with no errors:

mkdir build
cd build
cmake ..
make
```

lanPython platformacOS toolchaiPythoPython 2 toolchaiPythoPython 3

Source

Mottl

Most helpful comment

same issue. OSX El Captain 10.11.6, Python 2.7.11 pyenv

log of pip install -vvv grpcio:

gist.github.com/erickertz/6b04284778bc267b2c1bc38b1755b5d4

erickertz on 25 Oct 2018

👍3

All 50 comments

same issue. OSX El Captain 10.11.6, Python 2.7.11 pyenv

log of pip install -vvv grpcio:

gist.github.com/erickertz/6b04284778bc267b2c1bc38b1755b5d4

erickertz on 25 Oct 2018

👍3

Same issue

kendfinger on 25 Oct 2018

So, let's go a bit over this. I'll start with the last mention first.

same issue. OSX El Captain 10.11.6, Python 2.7.11 pyenv

log of pip install -vvv grpcio:

gist.github.com/erickertz/6b04284778bc267b2c1bc38b1755b5d4

I'm guessing you're saying you've installed python through pyenv. I'm also going to assume you have pyenv installed through brew.

So on a clean mac, (unfortunately on High Sierra, I can't rollback to El Captain), I am doing the following:

Install brew following the instructions at https://brew.sh/
Install pyenv using brew install pyenv and modifying my ~/.bash_profile file according to the installation instructions.
Install python 2.7.11 using pyenv install 2.7.11 and activate it using pyenv global 2.7.11. Confirming I am using the pyenv 2.7.11 version by doing pip --version which shows it's running from my local pyenv folder.
Install grpcio using pip install -vvv grpcio.

--> This works properly.

So either this is an El Captain problem since that'd be the only difference, or there are other things about your environment you haven't specified. I had to do a few assumptions in how you got things installed to get there, but following the reproduction steps I just mentioned, grpcio installs fine.

nicolasnoble on 25 Oct 2018

Could you try the same using MacPorts?

sudo port install openssl pypy3
sudo /opt/local/lib/pypy3/bin/pypy3 -m ensurepip
/opt/local/lib/pypy3/bin/pip3 install grpcio --user

Mottl on 25 Oct 2018

And the same for CPython:

sudo port install openssl python36
git clone [email protected]:grpc/grpc.git
cd grpc
python setup.py build

Mottl on 25 Oct 2018

Yes, I'm currently re-installing XCode to get a fresh Mac installation, this takes some time :-)

nicolasnoble on 25 Oct 2018

I appreciate the assistance.

Mottl on 25 Oct 2018

Tried in High Sierra 10.13.6, Python 2.7.11 (by pyenv) Python 3.4 (compiled) Python 3.7 (package), failed to reproduce this error...

@Mottl
Have you compiled or installed OpenSSL or BoringSSL manually before? Did you update the submodules by git submodule update --init --resursive?

lidizheng on 25 Oct 2018

@Mottl Also, can you use command printenv to print the value of environment variables?

lidizheng on 25 Oct 2018

I can confirm the problem installing python with macport with an out of the box installation (without pyenv). I wasn't able to reproduce using brew and/or pyenv.

Digging very quickly, this seems to be a macports issue where it will inject OpenSSL's headers into the compilation line, which interferes with our own headers. I'm not sure what we could do right now about it, as it's Bad Practice (tm) from ports... I'll talk with @lidizheng about digging this further.

nicolasnoble on 25 Oct 2018

I think _openssl_ was installed as binary via macports. I haven't installed _BoringSSL_ beforehand.
This is the info from _python36_ macports package:
```python36 @3.6.7 (lang)
Variants: optimizations, universal

Description: Python is an interpreted, interactive, object-oriented programming language.
Homepage: https://www.python.org/

Extract Dependencies: xz
Build Dependencies: pkgconfig
Library Dependencies: bzip2, expat, gettext, libedit, libffi, ncurses, openssl, sqlite3, xz, zlib
Runtime Dependencies: python_select, python3_select
Platforms: darwin
License: PSF
Maintainers: Email: [email protected], GitHub: jmroot
Policy: openmaintainer
```
_openssl_ is listed as a requirement for python36

Did you update the submodules by git submodule update --init --resursive?

No, just make git clone on _grpc_ and all third parties libs were cloned automatically.

I can confirm the problem installing python with macport with an out of the box installation (without pyenv).

Great, so you don't need the printenv anymore.

Mottl on 25 Oct 2018

Have you managed to run _grpc_ with _PyPy_ v6.0.0 (3.5.3)? If so, could you just upload pypy3 setup.py bdist_wheel?

Mottl on 25 Oct 2018

/opt/local/include is the default installation path for headers for MacPorts and
-I /opt/local/include appended to all cc commands for sure.

Conflicting OpenSSL headers live in /opt/local/include/openssl/*

Mottl on 25 Oct 2018

Here is the line that causes the problem:
cc -Ithird_party/protobuf/src -Ithird_party/googletest/googletest/include -Ithird_party/googletest/googlemock/include -I/opt/local/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/address_sorting/include -g -Wall -Wextra -Werror -Wno-long-long -Wno-unused-parameter -DOSATOMIC_USE_INLINED=1 -Wno-deprecated-declarations -Ithird_party/nanopb -DPB_FIELD_32BIT -O2 -Wframe-larger-than=16384 -fPIC -I. -Iinclude -I/Users/user/grpc/gens -I/opt/local/include -I/usr/local/include -DNDEBUG -DINSTALL_PREFIX=\"/usr/local\" -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -I/opt/local/include -Ithird_party/boringssl/include -fvisibility=hidden -DOPENSSL_NO_ASM -D_GNU_SOURCE -DWIN32_LEAN_AND_MEAN -D_HAS_EXCEPTIONS=0 -DNOMINMAX -std=c99 -Wsign-conversion -Wconversion -Wshadow -Wextra-semi -Wno-sign-conversion -Wno-conversion -Wno-unused-value -Wno-unknown-pragmas -Wno-implicit-function-declaration -Wno-unused-variable -Wno-sign-compare -Wno-implicit-fallthrough -Wno-extra-semi -MMD -MF /Users/user/grpc/objs/opt/src/boringssl/err_data.dep -c -o /Users/user/grpc/objs/opt/src/boringssl/err_data.o src/boringssl/err_data.c

if I strip -I/opt/local/include it compiles. Commenting lines around 375 in Makefile doesn't make an effect...

Mottl on 25 Oct 2018

Rawr. I was against putting these in the Makefile in the first place for reasons like these.

nicolasnoble on 26 Oct 2018

Still can't figure out from where else it takes -I/opt/local/include. Any suggestions?

Mottl on 26 Oct 2018

@Mottl It maybe in https://github.com/grpc/grpc/blob/master/setup.py#L183

lidizheng on 26 Oct 2018

@lidizheng, I doubt it's there. Can't find something that generates /opt/local/include.

Mottl on 26 Oct 2018

I still can't reproduce the error locally... I'm using the MacPorts to install openssl and Python3.6.

Can you try to run this setup.py and give me the print out variables?
https://gist.github.com/lidizheng/13c0cc9b09ac1f52a59c85a52ea4c3fd#file-setup-py-L187
It just add couple lines of printing to master's setup.py.

lidizheng on 26 Oct 2018

Please keep in mind I'm using python 3...

python setup.py setup
('src/python/grpcio', 'include', '.', 'third_party/boringssl/include', 'third_party/zlib', 'third_party/nanopb', 'third_party/cares', 'third_party/cares/cares', 'third_party/cares/config_darwin', 'third_party/address_sorting/include')
CORE_INCLUDE : ('include', '.')
SSL_INCLUDE : ('third_party/boringssl/include',)
ZLIB_INCLUDE : ('third_party/zlib',)
NANOPB_INCLUDE : ('third_party/nanopb',)
CARES_INCLUDE : ('third_party/cares', 'third_party/cares/cares', 'third_party/cares/config_darwin')
ADDRESS_SORTING_INCLUDE : ('third_party/address_sorting/include',)
EXTENSION_INCLUDE_DIRECTORIES : ('src/python/grpcio', 'include', '.', 'third_party/boringssl/include', 'third_party/zlib', 'third_party/nanopb', 'third_party/cares', 'third_party/cares/cares', 'third_party/cares/config_darwin', 'third_party/address_sorting/include')

Mottl on 26 Oct 2018

Just a few updates: I was able to reproduce the problem with pypy3, and I didn't attempt first using your python36 instructions. @lidizheng is going to look at pypy3 now after seeing it repro on my machine.

I just looked at the python36 stuff, and several things to note:

1) you can't git clone grpc like this. You need to close recursively (add --recursive to the clone command line).
2) your instructions don't get to a point where you can even run python setup.py build:

# python setup.py build
Traceback (most recent call last):
  File "setup.py", line 21, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

nicolasnoble on 26 Oct 2018

Just a few updates: I was able to reproduce the problem with pypy3, and I didn't attempt first using your python36 instructions. @lidizheng is going to look at pypy3 now after seeing it repro on my machine.

I just looked at the python36 stuff, and several things to note:

you can't git clone grpc like this. You need to close recursively (add --recursive to the clone command line).

Same result

your instructions don't get to a point where you can even run python setup.py build:
# python setup.py build
Traceback (most recent call last):
  File "setup.py", line 21, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

pip install setuptools

Mottl on 26 Oct 2018

I ended up doing python -m ensurepip.

The problem reproduces there too in all cases. @lidizheng is going to try reinstalling his work environment to get to the same state as mine :-)

nicolasnoble on 26 Oct 2018

@nicolasnoble, @lidizheng
Multiple -I/opt/local/include are generated in Makefile by pkg_config. For example:

CPPFLAGS += $(shell $(PKG_CONFIG) --cflags zlib)

Output:

pkg-config --cflags zlib
-I/opt/local/include

Seems like we need the separate CPPFLAGS for _boringssl_. Any thoughts?

Mottl on 26 Oct 2018

So, let's go a bit over this. I'll start with the last mention first.

same issue. OSX El Captain 10.11.6, Python 2.7.11 pyenv
log of pip install -vvv grpcio:
gist.github.com/erickertz/6b04284778bc267b2c1bc38b1755b5d4

I'm guessing you're saying you've installed python through pyenv. I'm also going to assume you have pyenv installed through brew.

So on a clean mac, (unfortunately on High Sierra, I can't rollback to El Captain), I am doing the following:

Install brew following the instructions at https://brew.sh/

Install pyenv using brew install pyenv and modifying my ~/.bash_profile file according to the installation instructions.

Install python 2.7.11 using pyenv install 2.7.11 and activate it using pyenv global 2.7.11. Confirming I am using the pyenv 2.7.11 version by doing pip --version which shows it's running from my local pyenv folder.

Install grpcio using pip install -vvv grpcio.
--> This works properly.

So either this is an El Captain problem since that'd be the only difference, or there are other things about your environment you haven't specified. I had to do a few assumptions in how you got things installed to get there, but following the reproduction steps I just mentioned, grpcio installs fine.

OK, thanks for testing and trying to reproduce. All of your assumptions seem about right. If there is anything else about my environment you would like to know, just ask. I'm only trying to help.

erickertz on 26 Oct 2018

Hi, @nicolasnoble
are there any news regarding this problem?

Mottl on 29 Oct 2018

@Mottl I am trying to obtain another MacBook to try to reproduce the error. I followed your exact commands and reinstall all dev-tools in my Mac, the error still doesn't show up. If I got any luck, I will let you know.

lidizheng on 29 Oct 2018

@lidizheng, thanks a lot!
Have you tried installing both zlib and openssl using MacPorts? Is that not enough to reproduce the error? Do you need some help from me? The root of the problem is pkg-config --cflags zlib. If you provide me with the updated Makefile with a separate CPPFLAGS for boringssl I will definitely can test it (and improve it to a ready-state if needed).

Mottl on 29 Oct 2018

@Mottl I tried install both zlib and openssl using MacPorts. And then pypy3 and ensurepip, it will indeed trigger building grpcio from source distribution. But in my Mac, the output of pkg-config --cflags zlib is empty... So it compiled successfully.

lidizheng on 29 Oct 2018

@lidizheng, here is the contents of zlib package:

port contents zlib
Port zlib contains:
  /opt/local/include/zconf.h
  /opt/local/include/zlib.h
  /opt/local/lib/libz.1.2.11.dylib
  /opt/local/lib/libz.1.dylib
  /opt/local/lib/libz.a
  /opt/local/lib/libz.dylib
  /opt/local/lib/pkgconfig/zlib.pc
  /opt/local/share/doc/zlib/ChangeLog
  /opt/local/share/doc/zlib/FAQ
  /opt/local/share/doc/zlib/README
  /opt/local/share/doc/zlib/zlib.3.pdf
  /opt/local/share/examples/zlib/README.examples
  /opt/local/share/examples/zlib/enough.c
  /opt/local/share/examples/zlib/fitblk.c
  /opt/local/share/examples/zlib/gun.c
  /opt/local/share/examples/zlib/gzappend.c
  /opt/local/share/examples/zlib/gzjoin.c
  /opt/local/share/examples/zlib/gzlog.c
  /opt/local/share/examples/zlib/gzlog.h
  /opt/local/share/examples/zlib/zlib_how.html
  /opt/local/share/examples/zlib/zpipe.c
  /opt/local/share/examples/zlib/zran.c
  /opt/local/share/man/man3/zlib.3.gz

please notice /opt/local/lib/pkgconfig/zlib.pc

The contents of /opt/local/lib/pkgconfig/zlib.pc:

prefix=/opt/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
sharedlibdir=${libdir}
includedir=${prefix}/include

Name: zlib
Description: zlib compression library
Version: 1.2.11

Requires:
Libs: -L${libdir} -L${sharedlibdir} -lz
Cflags: -I${includedir}

May be you just put this/opt/local/lib/pkgconfig/zlib.pc to your system if for some reason it doesn't exist?

Mottl on 29 Oct 2018

The same for /opt/local/lib/pkgconfig/openssl.pc:

prefix=/opt/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: OpenSSL
Description: Secure Sockets Layer and cryptography libraries and tools
Version: 1.0.2p
Requires: libssl libcrypto

and for /opt/local/lib/pkgconfig/protobuf.pc:
```prefix=/opt/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: Protocol Buffers
Description: Google's Data Interchange Format
Version: 2.6.1
Libs: -L${libdir} -lprotobuf -D_THREAD_SAFE
Libs.private: -lz
Cflags: -I${includedir} -D_THREAD_SAFE

Commented out because it crashes pkg-config sigh:

http://bugs.freedesktop.org/show_bug.cgi?id=13265

Conflicts: protobuf-lite

```

Mottl on 29 Oct 2018

Makefile uses $(PKG_CONFIG) for these libraries:

CPPFLAGS += $(shell $(PKG_CONFIG) --cflags zlib)
CPPFLAGS += $(shell $(PKG_CONFIG) --cflags libcares)
CPPFLAGS := $(shell $(PKG_CONFIG) --cflags openssl) $(CPPFLAGS)
CPPFLAGS := $(shell $(PKG_CONFIG) --cflags protobuf) $(CPPFLAGS)

Mottl on 29 Oct 2018

@Mottl Thanks for listing the pkg-config files, I have them installed using MacPorts. I guess I installed all of them once before while compiling Python manually.

lidizheng on 29 Oct 2018

I don't think it does matters... Makefile is static (not generated by python) and it just invokes pkg-config --cflags zlib. Do you havezlib.pc, protobuf.pc and openssl.pc in /opt/local/lib/pkgconfig/?

Mottl on 29 Oct 2018

Yes, I have them. I am reading this MacPorts ticket
python extension module builds missing -I/opt/local/include. It might be something injected by this change...

lidizheng on 29 Oct 2018

Yes, that could be related. Please, keep in mind that Makefile has also these lines:

ifeq ($(SYSTEM),Darwin)
ifneq ($(wildcard /usr/local/ssl/include),)
INCLUDES += /usr/local/ssl/include
endif
ifneq ($(wildcard /opt/local/include),)
INCLUDES += /opt/local/include
endif
ifneq ($(wildcard /usr/local/include),)
INCLUDES += /usr/local/include
endif
LIBS = m z
ifneq ($(wildcard /usr/local/ssl/lib),)
LDFLAGS += -L/usr/local/ssl/lib
endif
ifneq ($(wildcard /opt/local/lib),)
LDFLAGS += -L/opt/local/lib
endif
ifneq ($(wildcard /usr/local/lib),)
LDFLAGS += -L/usr/local/lib
endif
endif

So /opt/local/include will be added toINCLUDES anyway.

Mottl on 29 Oct 2018

But the Makefile isn't used by setup.py now, is it?

nicolasnoble on 29 Oct 2018

Indeed, src/python/grpcio/commands.py line 277:

make_process = subprocess.Popen(
    ['make'] + extra_defines + targets,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE)
make_out, make_err = make_process.communicate()

commands.py is used by setup.py (check BuildExt class)

Mottl on 29 Oct 2018

@Mottl I managed to reproduce the error (finally) by specify the PKG_CONFIG_PATH=/opt/local/lib/pkgconfig/. I guess if pkg-config is installed by MacPorts, this problem will occur.

lidizheng on 29 Oct 2018

@Mottl There is a conflict between custom libraries and system libraries... If you force the Makefile to compile with the custom libraries (boringSSL etc.), it will work. Can you try this?

REQUIRE_CUSTOM_LIBRARIES_opt=1 /opt/local/bin/pypy3 setup.py build

lidizheng on 29 Oct 2018

Great, man! You did it! Compiled with both CPython and PyPy.
What is REQUIRE_CUSTOM_LIBRARIES_opt? Where it comes from?

Mottl on 29 Oct 2018

The root cause here is a mismatch between Makefile and commands.py. In currently logic, the Makefile will use system libraries as default (https://github.com/grpc/grpc/issues/11655), but in commands.py is forcing to compile libboringssl.a at here.

So, my solution will be:

Making Python build process aware of REQUIRE_CUSTOM_LIBRARIES_$(CONFIG) variables, and adjust build target accordingly.
While building libboringssl.a detect whether HAS_SYSTEM_OPENSSL_ALPN is true or not, if it is true, log and exit to prevent future problems.

lidizheng on 29 Oct 2018

Great, thanks a lot! Are you planning to make a commit/PR?

Mottl on 29 Oct 2018

@Mottl Yep, but it could take some time to ensure it is stable and doesn't break any thing.

lidizheng on 29 Oct 2018

Seems like there is a fix in the works here, but just wanted to add that I just updated to Mojave and am still having the same issue.

erickertz on 30 Oct 2018

@erickertz Can you confirm it is the same issue with same error? Can you show the error log by adding -vvv flag?

lidizheng on 31 Oct 2018

@lidizheng sure thing:

https://gist.github.com/erickertz/0be9a44a42d2334174156303b72c6ef1

erickertz on 31 Oct 2018

@erickertz The PR just got merged, can you try to install from the master branch, and see if the problem still exists or not? This fix will be included in next release, which will be weeks from now.

lidizheng on 31 Oct 2018

@lidizheng
Installed with PyPy 6.0.0 (3.5.3) just fine. Thanks a lot!

Mottl on 1 Nov 2018

@Mottl Thank you for following the issue and providing suggestions ;)

lidizheng on 1 Nov 2018

Was this page helpful?

0 / 5 - 0 ratings