Conan: Add support for system ssl store usage

[Aalmann]

Hi @memsharded and @lasote,

since Python 3.4 and 2.7.9 the usage of system SSL store (like Windows certificate store) is available.
This can also be implemented with requests used in conan for up-/download and would be very helpful especially for enterprise/corporate networks with "private" CAs which are deposite on each machine.

In our case we have some internal root CAs and only internal available servers which ssl certs are signed by the internal CAs. The CAs are available at system level in the corresponding ssl stores which are also used by browsers and scm tools like git (by using schannel config on Windows). Only conan does not trust these certs and we either have to ssl_verify=false or add these certs in cacert.pem. The usage of "self-managed" cacert.pem is often discussed in the web (for every tool which comes with such a file or "private" storage) and is a high risk if a CA is compromised.

Another positive effect is, that this would also work with smart cards / PKI cards if certs are stored there and verified using a card reader and pin.

A good example of how to implement it with just a few lines of code can be found at https://stackoverflow.com/questions/42981429/ssl-failure-on-windows-using-python-requests second answer

Probably the best place for implementation within conan could be conan_api.py at get_basic_requester(cache) and should only take an hour for implementation and test. Adding a switch/option in conan config would be ok to get it work with cacert.pem or system ssl store (like CONAN_USE_SYSTEM_DEFAULT_CERTS=True)

It would be very very very very nice and helpful for me, my team and company if you can implement this feature as soon as possible.

Thanks in advance.
Best Aalmann

@verenchen and @ahauan4: fyi

[Aalmann]

This could probably fix #3668 and #2976 and #2687

[lasote]

Sounds good. I agree with an opt-in mechanism in the conan-conf/env_var

[lasote]

@Aalmann about #2687. How could it help with the python 3.4 deprecation? wrong link?

[Aalmann]

@lasote yep, wrong link. I just read about ssl. Sorry, was a long day at work. :smiley:

[lasote]

To investigate for 1.14 (if the schedule allows it) how good works: https://docs.python.org/2/library/ssl.html#ssl.SSLContext.load_default_certs in different operating systems. Only if it is robust we are going to adopt it.

[tonka3000]

I would love to have that feature in conan. I have the same problems as @Aalmann , because at work we also have internal root CA and we always have to turn off ssl verify on CI and client maschine.

[Aalmann]

@tonka3000
You don't have to turn it off. Just add your CAs to cacert.pem in conan home. A much better way would be to distribute the patched file with conan config install.
But the best way is, if conan uses the system cert store.

[tonka3000]

Hey @Aalmann ,

Yeah, conan config install would be better. Maybe I will switch to that until the issue is hopefully implemented. Thanks for the hint.

The most painful point are the CI servers. conan is installed on demand and the only real useable option for us without breaking existing CI code from other projects was the ssl verfiy off.

I hope conan will get the system ssl store feature, because it would be much easier in enterprises to handle ssl/tls.

[Aalmann]

You should give it a try. At least for cacert.pem, profiles, standard settings and remotes. 😉
Just one command and everything is set up (except user logon) if you use shell or batch scripts.

Only the artifactory Jenkins plugin for pipeline jobs does not really support it out of the box and one has to do some additional steps.

[szegel]

I would like the Conan client to use system cert's store, AND/OR have an environmental variable like $CONAN_CERT so I do not have to copy my cert into my $CONAN_USER_HOME directory. It would make things easier, and more convenient.

[memsharded]

Hi @szegel

Unfortunately, it seems that system ssl support is a bit more challenging than expected, and https://github.com/conan-io/conan/pull/5659/ is blocked, apparently because different behavior of the requests library, that makes difficult to provide a solid solution. Lets try to revisit this in next iteration 1.29, and at least check the status, and try to unblock if possible.

In the meantime, providing custom paths is a very easy feature. I have provided a PR #7398 , which should be relatively low risk and maybe can be added to 1.28. At the moment I have proposed with conan.conf vars, not env-vars, because env-vars pollute a bit more the environment, let me know if good enough.

[szegel]

Yea! That looks like a good temporary solution. Thanks @memsharded

[memsharded]

7398 was merged, Conan 1.28 will have support for custom paths (outside the Conan cache, for client cert and key files)