Readthedocs.org: Build Fails to Create Conda Environment
Details
- Project URL: https://readthedocs.org/projects/tethys-platform/
- Build URL (if applicable): https://readthedocs.org/projects/tethys-platform/builds/5043136/
- Read the Docs username (if applicable): swainn
Expected Result
I expect the docs for the dev branch of my project to build.
Actual Result
I encounter a failure during the creation of the conda environment (see full output below).
Things I have tried:
(1) Found this suggestion on the Conda issues page, so I attempted wiping the environment several times.
(2) Reverted the back to commit with last successful docs build, but still failed with same error.
(3) Created this Conda environment my dev machine successfully (using miniconda).
Here is the error:
conda env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.yml
Fetching package metadata .........
Solving package specifications: ..........
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda/dev
envs directories : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda
/home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/conda/.pkgs
/home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
$ /usr/local/miniconda/bin/conda-env create --name dev --file /home/docs/checkouts/readthedocs.org/user_builds/tethys-platform/checkouts/dev/docs-conda_env.yml
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
return_value = func(*args, **kwargs)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 111, in execute
installer.install(prefix, pkg_specs, args, env)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/installers/conda.py", line 38, in install
raise CondaRuntimeError('RuntimeError: %s' % e)
CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part' for writing ([Errno 13] Permission denied: u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part').
Fetching packages ...
yaml-0.1.6-0.t 0% | | ETA: --:--:-- 0.00 B/s
swainn
All 54 comments
Hi, we have the same problem since about 1 week:
xarray: https://readthedocs.org/projects/xray/builds/5051404/
fmaussion
on 21 Feb 2017
It seems this issue can be remedied by upgrading to conda 4.2.13. See: https://github.com/conda/conda/issues/3448#issuecomment-281493109
swainn
on 23 Feb 2017
Thanks. Do we have any control on the conda version installed on RTD?
fmaussion
on 23 Feb 2017
What is striking me is that we are the only ones reporting this. I'd expect such an issue to affect much more people?
fmaussion
on 23 Feb 2017
I have the same error:
https://readthedocs.org/projects/fermipy/builds/5067364/
woodmd
on 24 Feb 2017
Our docs aren't built since two weeks now... :-(
Did anyone come up with a solution?
fmaussion
on 28 Feb 2017
Same here. We've had no docs builds succeed for nearly two weeks and no response from RTD...
swainn
on 28 Feb 2017
Same problem for us too: https://readthedocs.org/projects/aospy/builds/5097720/
spencerahill
on 3 Mar 2017
Similar issue here: http://readthedocs.org/projects/gsshapy/builds/5091350. Builds having issues for over 2 weeks.
snowman2
on 3 Mar 2017
Same issue on our builds, for the same period of time as others are reporting:
https://readthedocs.org/projects/ga4gh-reference-implementation/builds/5097461/
https://readthedocs.org/projects/ga4gh-client/builds/5097407/
https://readthedocs.org/projects/ga4gh-schemas/builds/5051360/
dcolligan
on 3 Mar 2017
We're considering dropping RTD because of this. Is there something we can do to help the update of conda happen on RTD?
david4096
on 7 Mar 2017
@agjohnson now that we are talking about upgrading the packages from conda-forge in #2378, is it possible to upgrade the conda version also? Since it's a bugfix version I think there is no problem to upgrade from 4.2.12 to 4.2.13 (people says this problem is fixed there). What do you think?
Considering how RTD internally works with conda (https://github.com/rtfd/readthedocs-docker-images/blob/master/Dockerfile#L40), it seems that this upgrade won't be available until the next docker image is built and deployed to production.
Other (some of them, _quick-and-dirty_) solutions I have on the top of my head:
- run
conda update condabefore each of the builds (kind of dangerous since RTD can't guarantee a specific version) - add a config in the
ymlto pin the conda version before running the build - upgrade the conda version _just once_ inside the docker container used to build the documentation (hotfix)
The latest version of conda seems to be 4.3.14-py36_0 in my pc when running conda update conda
humitos
on 7 Mar 2017
I don't fully understand conda, and we need people on our core team who can actually track and update the usage. What is the correct fix for this? Is it something that we broke, or did conda break?
ericholscher
on 7 Mar 2017
I'm happy to upgrade conda if that is the correct fix -- but I'd like to know the root cause of what broke, so that we can prevent it from happening again.
ericholscher
on 7 Mar 2017
Here is a discussion on the issue on conda: https://github.com/conda/conda/issues/3448#issuecomment-280816176
snowman2
on 7 Mar 2017
So this was an issue with the version of conda from when we created the last set of docker images? Is there a way to pin an install of conda to a specific verison? The official install docs only mention a way to install the "latest".
ericholscher
on 7 Mar 2017
The current plan is to rebuild the 2.0 docker image, which I believe will cause it to install the latest version of conda. I'd like to figure out a way to pin the version to prevent this issue going forward. We've had a lot of breakage in the conda tooling specifically, and I'd like to knowingly upgrade it, instead of it happening randomly on image rebuilds.
ericholscher
on 7 Mar 2017
Image rebuild is going here: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/ -- it will probably take ~30 mins
ericholscher
on 7 Mar 2017
I'd like to figure out a way to pin the version to prevent this issue going forward.
There is a blog post about that: http://www.damian.oquanta.info/posts/how-to-pin-conda.html
Anyway, it says that conda will upgrade automatically on each install (I just tried that at my pc and it's true) and if you want to not do so, you have to pin conda itself by using the pinned file (https://conda.io/docs/faq.html#pinning-packages)
[12:44] humitos: ~> conda install scipy
Fetching package metadata .........
Solving package specifications: .
Package plan for installation in environment /home/humitos/.miniconda3:
The following NEW packages will be INSTALLED:
libgfortran: 3.0.0-1
mkl: 2017.0.1-0
numpy: 1.12.0-py36_0
scipy: 0.18.1-np112py36_1
The following packages will be UPDATED:
conda: 4.3.11-py36_0 --> 4.3.14-py36_0
Proceed ([y]/n)?
On the other hand, I didn't find a way to install for the first time an specific version of conda. It seems that you have to install the latest one and then you can do conda update conda=x.x.x if you want to pin it.
humitos
on 7 Mar 2017
Sounds like a way forward at least. So we can do the network install, and then do the conda update conda=x.x.x after that in the Dockerfile.
ericholscher
on 7 Mar 2017
I'm not convinced that won't break at some point in the future, but at least it makes the builds more deterministic :)
ericholscher
on 7 Mar 2017
Please don't deprecate conda support. Is makes using/distributing Python packages soo much easier.
snowman2
on 7 Mar 2017
Moving conversation around the Docker image to rtfd/readthedocs-docker-images#22
A new build image likely won't hit our servers for a few hours, the build process takes a while.
Also, I'll say that for anyone wondering why their project is still hitting this issue, consider contributing your time to resolve this. We are community run service and need both your input and your contributions.
agjohnson
on 7 Mar 2017
New image builds have wrapped up, and I've installed them across the servers. Can folks give a test to confirm the fix here?
agjohnson
on 7 Mar 2017
snowman2
on 7 Mar 2017
Sorry, still failing here too: https://readthedocs.org/projects/aospy/builds/5117255/
spencerahill
on 7 Mar 2017
Looks like it's still using conda/4.3.11?
ericholscher
on 7 Mar 2017
Any conda users understand the output here, and know why it would be continuing to use the old version? https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/
ericholscher
on 7 Mar 2017
Looks like we want to point it at https://repo.continuum.io/miniconda/Miniconda2-4.3.11-Linux-x86_64.sh
ericholscher
on 7 Mar 2017
Another option is to run an older tagged and proven version of conda from: https://repo.continuum.io/miniconda/
Otherwise, I think you need to run: conda update --yes conda
The reason is that Miniconda is not updated as frequently as Anaconda is.
snowman2
on 7 Mar 2017
https://github.com/rtfd/readthedocs-docker-images/pull/23/files is a proposed fix.
ericholscher
on 7 Mar 2017
Actually, I think 4.2.12 (2016-11-03) would be a better version to use. The date of release of 4.3.11 (2017-02-14) is the date of the start of the build fails with RTD.
https://repo.continuum.io/miniconda/Miniconda2-4.2.12-Linux-x86_64.sh
snowman2
on 7 Mar 2017
It seems that 4.3.11 is the last released version of miniconda? So the latest released version is broken?
ericholscher
on 7 Mar 2017
It seems that 4.2.12 is the last released verison, and the above bug says that 4.2.13 is the one that needs to be used, but isn't released here: https://repo.continuum.io/miniconda/ ?
ericholscher
on 7 Mar 2017
4.3.11 is the version failing on my current build and for @humitos. 4.2.12 failed for the original poster @swainn. So, maybe try 4.2.11 (2016-10-24).
snowman2
on 7 Mar 2017
wait, is the latest version of conda failing to create an environment and install some packages? Instead of us keep upgrading the RTD server to a random version of conda, can you (people with failing builds) test this locally by creating the proper conda environment with the different versions and confirm which version of conda is not having troubles in your pc?
If conda is broken there is nothing that we can do at RTD but just wait for a fix on them :)
humitos
on 7 Mar 2017
Agreed -- I don't really know how to fix this, other than shooting in the dark at random released or unreleased versions of miniconda. I'd love to know a known good version, and be able to go with it moving forward.
ericholscher
on 7 Mar 2017
This build (https://readthedocs.org/projects/aospy/builds/5117255/) has a different error and I'm guessing the Permission denied problem could be related because the miniconda is installed as root and then it's executed as docs:
Installed as root: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L37
User changed to
docs: https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile#L44
(the docker hub image build related: https://hub.docker.com/r/readthedocs/build/builds/bgpgd7axkbaihm6wkexya6h/)
humitos
on 7 Mar 2017
If this is a permission issue: https://github.com/rtfd/readthedocs-docker-images/issues/24
Testing this shouldn't be difficult.
agjohnson
on 7 Mar 2017
That doesn't explain why this recently started failing though?
ericholscher
on 7 Mar 2017
There were two major changes in this timeframe: we rerolled the build images (upgrading conda), and the new image is based on ubuntu 16.04 now. I have no clue what the internals of the shell script do, but if it was doing something different in 14.04 vs 16.04, this could be the cause.
agjohnson
on 7 Mar 2017
Just to add more context here. I tried to build one of the failing project outside RTD (env, docker img, etc) and it worked:
[14:59] humitos: repositories> git clone https://github.com/spencerahill/aospy.git
Clonar en 芦aospy禄...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 484.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
[14:59] humitos: repositories> cd aospy/
/home/humitos/readthedocs/repositories/aospy
[14:59] humitos: aospy> conda env create --name aospy_latest --file docs/environment.yml
Fetching package metadata ...........
Solving package specifications: .
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:03 53.90 kB/s
[... snipped ...]
matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:14 543.61 kB/s
#
# To activate this environment, use:
# > source activate aospy_latest
#
# To deactivate this environment, use:
# > source deactivate aospy_latest
#
[15:13] humitos: aospy> conda info
Current conda install:
platform : linux-64
conda version : 4.3.11
conda is private : False
conda-env version : 4.3.11
conda-build version : not installed
python version : 3.6.0.final.0
requests version : 2.12.4
root environment : /home/humitos/.miniconda3 (writable)
default environment : /home/humitos/.miniconda3
envs directories : /home/humitos/.miniconda3/envs
/home/humitos/.conda/envs
package cache : /home/humitos/.miniconda3/pkgs
/home/humitos/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
user-agent : conda/4.3.11 requests/2.12.4 CPython/3.6.0 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
UID:GID : 1000:1000
[15:48] humitos: aospy>
After this, I booted the readthedocs/build:2.0 and ran into this permission problem again:
docs@40764e8adc4f:/$ conda info
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
docs@40764e8adc4f:/$ git clone https://github.com/spencerahill/aospy.git
fatal: could not create work tree dir 'aospy': Permission denied
docs@40764e8adc4f:/$ cd
docs@40764e8adc4f:~$ git clone https://github.com/spencerahill/aospy.git
Cloning into 'aospy'...
remote: Counting objects: 2156, done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2156 (delta 7), reused 1 (delta 1), pack-reused 2133
Receiving objects: 100% (2156/2156), 3.62 MiB | 502.00 KiB/s, done.
Resolving deltas: 100% (1468/1468), done.
Checking connectivity... done.
docs@40764e8adc4f:~$ cd aospy/
docs@40764e8adc4f:~/aospy$ conda env create --name latest --file docs/environment.yml
Fetching package metadata .........
Solving package specifications: ..........
Fetching packages ...
ca-certificate 100% |##############################################################################################################################################| Time: 0:00:00 240.00 kB/s
[ ... snipped ... ]
xz-5.2.2-0.tar 100% |##############################################################################################################################################| Time: 0:00:02 356.06 kB/s
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
`$ /usr/local/miniconda/bin/conda-env create --name latest --file docs/environment.yml`
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python2.7/site-packages/conda/exceptions.py", line 479, in conda_exception_handler
return_value = func(*args, **kwargs)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 111, in execute
installer.install(prefix, pkg_specs, args, env)
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/installers/conda.py", line 38, in install
raise CondaRuntimeError('RuntimeError: %s' % e)
CondaRuntimeError: Runtime error: RuntimeError: Runtime error: Could not open u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part' for writing ([Errno 13] Permission denied: u'/usr/local/miniconda/pkgs/yaml-0.1.6-0.tar.bz2.part').
docs@40764e8adc4f:~/aospy$ ls -la /usr/local/miniconda/pkgs/
total 112
drwxr-xr-x 20 root root 4096 Feb 5 07:43 .
drwxr-xr-x 11 root root 4096 Feb 5 07:43 ..
-rw-rw-r-- 1 root root 2097 Nov 3 19:04 .cio-config.py
-rw-rw-r-- 1 root root 17834 Nov 3 19:04 .install.py
drwxr-xr-x 2 root root 4096 Feb 5 07:43 cache
drwxr-xr-x 6 root root 4096 Feb 5 07:43 conda-4.2.12-py27_0
drwxr-xr-x 3 root root 4096 Feb 5 07:43 conda-env-2.6.0-0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 enum34-1.1.6-py27_0
drwxr-xr-x 7 root root 4096 Feb 5 07:43 openssl-1.0.2j-0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 pip-8.1.2-py27_0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 pycosat-0.6.1-py27_1
drwxr-xr-x 4 root root 4096 Feb 5 07:43 pycrypto-2.6.1-py27_4
drwxr-xr-x 7 root root 4096 Feb 5 07:43 python-2.7.12-1
drwxr-xr-x 6 root root 4096 Feb 5 07:43 readline-6.2-2
drwxr-xr-x 4 root root 4096 Feb 5 07:43 requests-2.11.1-py27_0
drwxr-xr-x 4 root root 4096 Feb 5 07:43 ruamel_yaml-0.11.14-py27_0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 setuptools-27.2.0-py27_0
drwxr-xr-x 6 root root 4096 Feb 5 07:43 sqlite-3.13.0-0
drwxr-xr-x 6 root root 4096 Feb 5 07:43 tk-8.5.18-0
-rw-rw-r-- 1 root root 1777 Nov 3 19:04 urls
-rw-rw-r-- 1 root root 1216 Nov 3 19:04 urls.txt
drwxr-xr-x 5 root root 4096 Feb 5 07:43 wheel-0.29.0-py27_0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 yaml-0.1.6-0
drwxr-xr-x 5 root root 4096 Feb 5 07:43 zlib-1.2.8-3
docs@40764e8adc4f:~/aospy$ ls -la /home/docs/.conda/envs/.pkgs
total 46044
drwxr-xr-x 3 docs docs 4096 Mar 7 20:55 .
drwxr-xr-x 4 docs docs 4096 Mar 7 20:53 ..
-rw-r--r-- 1 docs docs 169592 Mar 7 20:53 ca-certificates-2017.1.23-0.tar.bz2
drwxr-xr-x 2 docs docs 4096 Mar 7 20:53 cache
-rw-r--r-- 1 docs docs 5892278 Mar 7 20:54 gettext-0.19.7-1.tar.bz2
-rw-r--r-- 1 docs docs 120285 Mar 7 20:54 graphite2-1.3.9-0.tar.bz2
-rw-r--r-- 1 docs docs 23833004 Mar 7 20:54 icu-58.1-1.tar.bz2
-rw-r--r-- 1 docs docs 939817 Mar 7 20:55 jpeg-9b-0.tar.bz2
-rw-r--r-- 1 docs docs 47736 Mar 7 20:55 libffi-3.2.1-3.tar.bz2
-rw-r--r-- 1 docs docs 287481 Mar 7 20:55 libgfortran-3.0.0-1.tar.bz2
-rw-r--r-- 1 docs docs 2077443 Mar 7 20:55 libiconv-1.14-4.tar.bz2
-rw-r--r-- 1 docs docs 1121442 Mar 7 20:55 ncurses-5.9-10.tar.bz2
-rw-r--r-- 1 docs docs 742096 Mar 7 20:55 pcre-8.39-0.tar.bz2
-rw-r--r-- 1 docs docs 3799933 Mar 7 20:55 pixman-0.34.0-0.tar.bz2
-rw-r--r-- 1 docs docs 5160488 Mar 7 20:55 sqlite-3.13.0-1.tar.bz2
-rw-r--r-- 1 docs docs 2038478 Mar 7 20:55 tk-8.5.19-1.tar.bz2
-rw-r--r-- 1 docs docs 998 Mar 7 20:55 urls.txt
-rw-r--r-- 1 docs docs 873657 Mar 7 20:55 xz-5.2.2-0.tar.bz2
docs@40764e8adc4f:~/aospy$
So, the issue is that we have two directories set as package cache, one that is accessible by the docs user and another one that only root has permission. For some reason, some packages goes to the /home/docs/.conda/envs/.pkgs (where the docs user has permission) and for some reason there are packages that goes to /usr/local/miniconda/pkgs and it fails because we are executing conda as docs user.
Another test was trying to update the conda itself inside the docker image and it also says we don't have permissions to do it:
docs@40764e8adc4f:~$ conda info
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 2.7.12.final.0
requests version : 2.11.1
root environment : /usr/local/miniconda (read only)
default environment : /usr/local/miniconda
envs directories : /home/docs/.conda/envs
/usr/local/miniconda/envs
package cache : /home/docs/.conda/envs/.pkgs
/usr/local/miniconda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
docs@40764e8adc4f:~$ conda update conda
Fetching package metadata .......
Solving package specifications: ..........
Package plan for installation in environment /usr/local/miniconda:
The following packages will be downloaded:
package | build
---------------------------|-----------------
libffi-3.2.1 | 1 38 KB
idna-2.2 | py27_0 122 KB
ipaddress-1.0.18 | py27_0 31 KB
pyasn1-0.1.9 | py27_0 54 KB
pycparser-2.17 | py27_0 153 KB
requests-2.13.0 | py27_0 776 KB
six-1.10.0 | py27_0 16 KB
cffi-1.9.1 | py27_0 325 KB
cryptography-1.7.1 | py27_0 848 KB
pyopenssl-16.2.0 | py27_0 68 KB
conda-4.3.14 | py27_0 481 KB
------------------------------------------------------------
Total: 2.8 MB
The following NEW packages will be INSTALLED:
cffi: 1.9.1-py27_0 (soft-link)
cryptography: 1.7.1-py27_0 (soft-link)
idna: 2.2-py27_0 (soft-link)
ipaddress: 1.0.18-py27_0 (soft-link)
libffi: 3.2.1-1 (soft-link)
pyasn1: 0.1.9-py27_0 (soft-link)
pycparser: 2.17-py27_0 (soft-link)
pyopenssl: 16.2.0-py27_0 (soft-link)
six: 1.10.0-py27_0 (soft-link)
The following packages will be UPDATED:
conda: 4.2.12-py27_0 --> 4.3.14-py27_0 (soft-link)
requests: 2.11.1-py27_0 --> 2.13.0-py27_0 (soft-link)
CondaIOError: IO error: Missing write permissions in: /usr/local/miniconda
#
# You don't appear to have the necessary permissions to update packages
# into the install area '/usr/local/miniconda'.
# However you can clone this environment into your home directory and
# then make changes to it.
# This may be done using the command:
#
# $ conda create -n my_root --clone=/usr/local/miniconda
docs@40764e8adc4f:~$
Following the suggestion from the previous command is not a good idea :grin:
docs@40764e8adc4f:~$ conda create -n my_root --clone=/usr/local/miniconda
Source: /usr/local/miniconda
Destination: /home/docs/.conda/envs/my_root
The following packages cannot be cloned out of the root environment:
- conda-4.2.12-py27_0
Packages: 16
Files: 0
Linking packages ...
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[tk ]|######################################## WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13] | 18%
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pycosat ]|########################################################################################## WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]
[pip WARNING conda.lock:touch(53): Failed to create lock, do not run conda in parallel processes [errno 13]############################################# | 87%
[ COMPLETE ]|#################################################################################################################################################################| 100%
#
# To activate this environment, use:
# > source activate my_root
#
# To deactivate this environment, use:
# > source deactivate my_root
#
docs@40764e8adc4f:~$
I installed miniconda with the docs user and it was successful:
docs@40764e8adc4f:~$ curl -O https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27.4M 100 27.4M 0 0 162k 0 0:02:53 0:02:53 --:--:-- 141k
docs@40764e8adc4f:~$ bash Miniconda2-latest-Linux-x86_64.sh -b -p /home/docs/miniconda2
PREFIX=/home/docs/miniconda2
installing: python-2.7.13-0 ...
installing: cffi-1.9.1-py27_0 ...
[ ... snipped ... ]
docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda info
Current conda install:
platform : linux-64
conda version : 4.3.11
conda is private : False
conda-env version : 4.3.11
conda-build version : not installed
python version : 2.7.13.final.0
requests version : 2.12.4
root environment : /home/docs/miniconda2 (writable)
default environment : /home/docs/miniconda2
envs directories : /home/docs/miniconda2/envs
/home/docs/.conda/envs
package cache : /home/docs/miniconda2/pkgs
/home/docs/.conda/pkgs
channel URLs : https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : None
offline mode : False
user-agent : conda/4.3.11 requests/2.12.4 CPython/2.7.13 Linux/4.4.0-64-generic debian/stretch/sid glibc/2.23
UID:GID : 1005:205
docs@40764e8adc4f:~$ /home/docs/miniconda2/bin/conda env create --name latest --file aospy/docs/environment.yml
Fetching package metadata ...........
Solving package specifications: .
[ ... snipped ... ]
python-3.5.3-1 100% |##############################################################################################################################################| Time: 0:01:38 196.49 kB/s
[ ... snipped ... ]
matplotlib-1.5 100% |##############################################################################################################################################| Time: 0:00:16 477.15 kB/s
#
# To activate this environment, use:
# > source activate latest
#
# To deactivate this environment, use:
# > source deactivate latest
#
docs@40764e8adc4f:~$
So, my suggestion is to install miniconda after the docs user is created (and switched to) in the docker image as in this PR: https://github.com/rtfd/readthedocs-docker-images/pull/25
Plus, if we really want to pin a version as in https://github.com/rtfd/readthedocs-docker-images/pull/23 we need to use a setting to avoid the automatic upgrade (conda config --set auto_update_conda False -- https://conda.io/docs/config.html#update-conda-automatically-auto-update-conda) but I'm not sure if that is something good to do.
humitos
on 7 Mar 2017
I've tried downgrading conda, and now we get a YAML error from the conda env command. Hopefully the user permission fix will work, and we can hopefully get this to a working state.
ericholscher
on 8 Mar 2017
It appears to be this issue: https://github.com/conda/conda/issues/3738
How do we install a version of conda that isn't listed over at https://repo.continuum.io/miniconda/? Do we have to install a lower version of conda, and then use conda upgrade conda==X?
ericholscher
on 8 Mar 2017
I've got this failing locally, and will patch it and deploy a fix in the next couple hours, hopefully.
ericholscher
on 8 Mar 2017
My latest https://readthedocs.org/projects/schemas/builds/5118842/
Traceback (most recent call last):
File "/usr/local/miniconda/bin/conda-env", line 4, in <module>
import conda_env.cli.main
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main.py", line 35, in <module>
from . import main_attach
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/cli/main_attach.py", line 4, in <module>
from ..env import from_environment
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/env.py", line 17, in <module>
from . import yaml
File "/usr/local/miniconda/lib/python2.7/site-packages/conda_env/yaml.py", line 8, in <module>
import yaml
ImportError: No module named yaml
I believe this should _finally_ be fixed. Knock on wood.
ericholscher
on 8 Mar 2017
Thanks, looking good so far! https://readthedocs.org/projects/schemas/builds/5118911/
david4096
on 8 Mar 2017
Great success, thanks!
david4096
on 8 Mar 2017
Yes, success for us: https://readthedocs.org/projects/aospy/builds/5118901/
Thank you very much!
spencerahill
on 8 Mar 2017
Glad to hear this is working for folks. I'll close this for now, feel free to reopen here if it isn't resolved.
agjohnson
on 8 Mar 2017
I think it starts failing recently because a new version of yaml which was installed at root level, so when it tries to upgrade that package with de docs users it fails (check my log and when I said that I don't know why some packages goes to the root cache and some others to the docs cache). It's because of that...
humitos
on 8 Mar 2017
Congratulations to the RTD devs, this wasn't an easy one :)
astrojuanlu
on 8 Mar 2017
This is fixed for me:
https://readthedocs.org/projects/tethys-platform/builds/5122358/
I completely understand the challenges with community projects. Thank you for your time.
swainn
on 8 Mar 2017
Related issues
krzychb
路
4Comments
SylvainCorlay
路
3Comments
boscorelly
路
4Comments
et304383
路
4Comments
humitos
路
4Comments
Most helpful comment
Congratulations to the RTD devs, this wasn't an easy one :)