Azure-sdk-for-python: Compile error when attempting to install azure 2.0.0rc5 on a CentOS based AWS machine

[christonog]

I tried to install the azure python SDK 2.0.0rc5 on an AWS linux machine. The machine OS is the Amazon linux flavor, based off of CentOs. AMI Id is "ami-f5f41398." We will be using AWS running Ansible to manage servers on other cloud providers, such as Azure.

I tried installing azure 2.0.0rc5: pip install "azure==2.0.0rc5" as recommended here:

http://docs.ansible.com/ansible/guide_azure.html

Here is the output I've received when attempting the above command:

$ sudo pip install "azure==2.0.0rc5"
Collecting azure==2.0.0rc5
  Using cached azure-2.0.0rc5-py2.py3-none-any.whl
Collecting azure-graphrbac==0.30.0rc5 (from azure==2.0.0rc5)
  Using cached azure_graphrbac-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-servicemanagement-legacy==0.20.3 (from azure==2.0.0rc5)
  Using cached azure_servicemanagement_legacy-0.20.3-py2.py3-none-any.whl
Collecting azure-batch==0.30.0rc5 (from azure==2.0.0rc5)
  Using cached azure_batch-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-storage==0.32.0 (from azure==2.0.0rc5)
  Using cached azure_storage-0.32.0-py2-none-any.whl
Collecting azure-servicebus==0.20.2 (from azure==2.0.0rc5)
  Using cached azure_servicebus-0.20.2-py2.py3-none-any.whl
Collecting azure-mgmt==0.30.0rc5 (from azure==2.0.0rc5)
  Using cached azure_mgmt-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-common[autorest]==1.1.4 (from azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_common-1.1.4-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/site-packages (from azure-servicemanagement-legacy==0.20.3->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): futures in /usr/lib/python2.7/dist-packages (from azure-storage==0.32.0->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): azure-nspkg in /usr/local/lib/python2.7/site-packages (from azure-storage==0.32.0->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /usr/lib/python2.7/dist-packages (from azure-storage==0.32.0->azure==2.0.0rc5)
Collecting azure-mgmt-logic==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_logic-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-batch==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_batch-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-authorization==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_authorization-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-resource==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_resource-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-commerce==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_commerce-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-redis==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_redis-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-cognitiveservices==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_cognitiveservices-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-notificationhubs==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_notificationhubs-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-powerbiembedded==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_powerbiembedded-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-cdn==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_cdn-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-network==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_network-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-keyvault==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_keyvault-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-storage==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_storage-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-compute==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_compute-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-scheduler==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_scheduler-0.30.0rc5-py2.py3-none-any.whl
Collecting azure-mgmt-web==0.30.0rc5 (from azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_web-0.30.0rc5-py2.py3-none-any.whl
Collecting msrest<0.5.0,>=0.4.0; extra == "autorest" (from azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached msrest-0.4.3-py2-none-any.whl
Collecting msrestazure<0.5.0,>=0.4.0; extra == "autorest" (from azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached msrestazure-0.4.2-py2-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from python-dateutil->azure-storage==0.32.0->azure==2.0.0rc5)
Collecting azure-mgmt-nspkg (from azure-mgmt-logic==0.30.0rc5->azure-mgmt==0.30.0rc5->azure==2.0.0rc5)
  Using cached azure_mgmt_nspkg-1.0.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): requests-oauthlib>=0.5.0 in /usr/local/lib/python2.7/site-packages (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): isodate>=0.5.4 in /usr/local/lib/python2.7/site-packages (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /usr/local/lib/python2.7/site-packages (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Collecting keyring>=5.6 (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached keyring-9.3.1-py2.py3-none-any.whl
Collecting chardet>=2.3.0 (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached chardet-2.3.0.tar.gz
Collecting certifi>=2015.9.6.2 (from msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached certifi-2016.8.31-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): oauthlib>=0.6.2 in /usr/local/lib/python2.7/site-packages (from requests-oauthlib>=0.5.0->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Collecting secretstorage; sys_platform == "linux2" or sys_platform == "linux" (from keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached SecretStorage-2.3.1.tar.gz
Collecting cryptography (from secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached cryptography-1.5.tar.gz
Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /usr/local/lib/python2.7/site-packages (from cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.8 in /usr/local/lib/python2.7/site-packages (from cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=11.3 in /usr/lib/python2.7/dist-packages (from cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Requirement already satisfied (use --upgrade to upgrade): ipaddress in /usr/local/lib/python2.7/site-packages (from cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Collecting cffi>=1.4.1 (from cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
  Using cached cffi-1.8.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/local/lib/python2.7/site-packages (from cffi>=1.4.1->cryptography->secretstorage; sys_platform == "linux2" or sys_platform == "linux"->keyring>=5.6->msrest<0.5.0,>=0.4.0; extra == "autorest"->azure-common[autorest]==1.1.4->azure-graphrbac==0.30.0rc5->azure==2.0.0rc5)
Installing collected packages: cffi, cryptography, secretstorage, keyring, chardet, certifi, msrest, msrestazure, azure-common, azure-graphrbac, azure-servicemanagement-legacy, azure-batch, azure-storage, azure-servicebus, azure-mgmt-nspkg, azure-mgmt-logic, azure-mgmt-batch, azure-mgmt-authorization, azure-mgmt-resource, azure-mgmt-commerce, azure-mgmt-redis, azure-mgmt-cognitiveservices, azure-mgmt-notificationhubs, azure-mgmt-powerbiembedded, azure-mgmt-cdn, azure-mgmt-network, azure-mgmt-keyvault, azure-mgmt-storage, azure-mgmt-compute, azure-mgmt-scheduler, azure-mgmt-web, azure-mgmt, azure
  Running setup.py install for cffi ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-J17b3Z/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Rk5LMI-record/install-record.txt --single-version-externally-managed --compile:
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/cffi
    copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/ffiplatform.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/verifier.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/commontypes.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/vengine_gen.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/__init__.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/model.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/api.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/cparser.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/recompiler.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/lock.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/_cffi_include.h -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/parse_c_type.h -> build/lib.linux-x86_64-2.7/cffi
    copying cffi/_embedding.h -> build/lib.linux-x86_64-2.7/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/c
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
     #include <ffi.h>
                     ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-J17b3Z/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Rk5LMI-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-J17b3Z/cffi/

Is there something I am missing? Installing the current stable release on the box completes successfully.

[lmazuel]

It seems pip has some difficulties to identify the platform tag of your machine, since it downloads the source code of cryptography and tries to compile it, instead of downloading one of the wheel files that does not require compilation. Updating pip might help. You can also download the right wheel on PyPI and install it prior to the SDK installation.

Otherwise, you will be able to compile cryptography using the directives given on the cryptography website:
https://cryptography.io/en/latest/installation/#building-cryptography-on-linux

EDIT: Actually, I (surprisingly) just see that cryptography has no wheel for linux, so you need to follow the cryptography tutorial (basically install the needed packages).

EDIT2: The cryptography team indirectly confirm that there is no wheel for linux right now:
https://github.com/pyca/cryptography/issues/2872#issuecomment-241439083

[lmazuel]

Hi @christonog could you confirm this unlocked you?
Thanks!

[lmazuel]

Closing due to inactivity. Please re-comment here or open a new issue if you still have problems with that.
Thanks!

[christonog]

@imazuel Oddly didn't get the prior notifications for this issue. Will check out the link and report back.