Repo2docker: Support for different pypi server

[rnestler]

It would be nice if repo2docker would support specifying a different pypi server to use. I'd like to use it to create docker images which use python packages which are hosted on our own devpi index.
It is also useful to speed up creation of the docker images, because the packages are downloaded much faster.

[rnestler]

One way to solve it would be to allow injecting a custom pip.conf which then could just contain something like:

[global]
index-url=https://pypi.example.com/
trusted-host=pypi.example.com

[rnestler]

One can work around it by installing the packages that are only available on the local pypi server in a postBuild step:

#!/bin/bash
mkdir -p ~/.config/pip/
cp pip.conf ~/.config/pip/pip.conf
pip2 install -r local_requirements.txt
pip install -r local_requirements3.txt

[betatim]

Have you tried http://pip.readthedocs.io/en/latest/reference/pip_install/#requirements-file-format to specify the extra command-line arguments?

If the pip.conf is in the same directory as pip is being executed does it pick it up? And if not can we specify a path to a configuration file as command line argument to pip in a way that doesn't upset it when that file doesn't exist?

Using postBuild seems like a good workaround until we have something else. It is designed as a low effort escape hatch for the case where you want to do something a bit out of the ordinary/until there is better support in repo2docker.

[rnestler]

Have you tried http://pip.readthedocs.io/en/latest/reference/pip_install/#requirements-file-format to specify the extra command-line arguments?

Yes I tried. It woks as in it picks up the the server, but since it doesn't trust the certificate It would also need the --trusted-host option which isn't supported by the requirements file (There is only a subset of arguments supported).
It would also work to add the certificate to the system certificate store:

sudo cp CACertificate.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

But this would need to happen before pip is executed.

If the pip.conf is in the same directory as pip is being executed does it pick it up? And if not can we specify a path to a configuration file as command line argument to pip in a way that doesn't upset it when that file doesn't exist?

The repo content is copied to $HOME right? So I could just create .config/pip/pip.conf in the repo and it should probably just work.

[rnestler]

The repo content is copied to $HOME right? So I could just create .config/pip/pip.conf in the repo and it should probably just work.

It actually worked! I'd still prefer to be able to add the certificate so that I wouldn't need the trusted-host=pypi.example.com option, which degrades security.

[betatim]

The repo content is copied to $HOME right? So I could just create .config/pip/pip.conf in the repo and it should probably just work.

Noice! Can we find a good place in the repo2docker documentation to add this?

Why do we need to add the certificate? Is it because your certificate is self-signed or would this be a problem with all certificates?

I can't think of a good way to add the certificate so I am giving preference to "let's not add any new config files/standard for things that only one person wants to do". Is there a directory in the user's home directory where we can place the certificate for it to form part of the chain?

[rnestler]

Noice! Can we find a good place in the repo2docker documentation to add this?

Maybe in the FAQ?

[rnestler]

Why do we need to add the certificate? Is it because your certificate is self-signed or would this be a problem with all certificates?

The servers certificate uses a certificate which is signed by our internal CA. So we just need to deploy the certificate of our Root-CA to every system.
I guess it's quite common for companies to have their own certificate infrastructure.

[betatim]

Adding to the FAQ sounds good to me.

The servers certificate uses a certificate which is signed by our internal CA.

That is reasonable. Wanted to confirm this was the source of the issue and not something else. I don't have a good suggestion for what to do. What do you think of maybe being able to provide the certificate as an argument to repo2docker --trusted-cert=... which then gets injected from the outside? (Feels like a hack) Maybe someone else has a good idea