Readthedocs.org: Installing conda package ncurses fails with 'placeholder too short'

[takluyver]

I specified some packages to install in a conda environment, and it appeared to download them OK, but when trying to link them, I got:

[ncurses-5.9-5 /home/docs/checkouts/readthedocs.org/user_builds/takluyver]||  13%
Error: ERROR: placeholder '/root/miniconda3/envs/_build_placehold_placehold_placehold_placehold_placehold_p' too short in: ncurses-5.9-5

Full log here

This looks similar to conda/conda#1087, but it's not immediately clear what's going on. When I do the same steps on my local machine, it succeeds. I suspect it's a problem with the environment prefix being relatively long (/home/docs/checkouts/readthedocs.org/user_builds/takluyver-nbconvert-test/conda/docs-build-w-conda/). @ilanschnell may be able to shed more light.

[agjohnson]

This might be a problem of an old version of conda then? We're on 3.17.0 on the servers.

[takluyver]

That's certainly possible.

[ericholscher]

Yea, not sure how to diasnose this one. It is an error coming from conda, so likely their bug.

[takluyver]

Having a guess at what's going on:

Part of the point of conda is that packages are relocatable - they can be installed in an environment anywhere, and find other things relative to that environment. In some cases, packages use hardcoded absolute paths. When these occur inside a binary file, conda builds the package using a placeholder path (that's the bizarre _build_placehold_placehold... path), and on installation, rewrites the file to edit the environment path in there. That can't work if the environment path is longer than the placeholder, as it is in this case. That's partly because I've picked fairly long names for both the project and the branch, but other people will probably hit this too. By my count, the path structure RTD is using (/home/docs/checkouts/readthedocs.org/user_builds/{}/conda/{}/ leaves 23 characters to share between the two names before it exceeds that placeholder.

If I'm right, conda developers can alleviate the problem by rebuilding that package with a longer placeholder, but they can't easily remove the limitation entirely.

[ericholscher]

Ugh, yea. We ran into this with virtualenv as well. We ended up calling python directly, and then passing the bin path as an argument as a workaround for this. Not sure the best way to handle this with conda, as it appears to be happening inside conda, and not in our commands that we're executing.

[takluyver]

I tried renaming the project to something shorter (it's a test fork to experiment with conda builds, so the name isn't important), but the build directory is still using the old name (takluyver-nbconvert-test). Is there a way to force it to use the new name?

[pelson]

Same problem here. Any chance a symlink could be put on the filesystem such as:

ln -s /home/docs/checkouts/readthedocs.org/user_builds/ /rtd/user/builds/

That would give us an awful lot more room to play with.

A saving grace, this problem will only occur with packages with binary files which have the $PREFIX baked in. Unfortunately, one of my dependencies is one such package :disappointed:

[wholmgren]

I'm running into the same problem on a project that built fine ~1 month ago. I haven't changed anything that impacts the build environment. Should I pin my dependencies to older versions or is this problem tied to rtd's version of conda? Sad face, indeed.

[pelson]

@msarahan will be interested in this (linking it to the comment at https://github.com/conda/conda-build/pull/877#issuecomment-218936229)

[bgruening]

I still see this with:

[root@046e76217ae3 bioconda-recipes]# conda-build --version
conda-build 1.21.11+0.g5b44ab3.dirty

[msarahan]

Sorry, nothing in conda or conda-build has changed regarding prefix length in the 1.x release series. This will be one of the big features of the 2.x series. I am the maintainer of conda-build, and I am working to have this out as soon as I can - timeline is presently 1-2 weeks.

[bgruening]

@msarahan thanks for the good news!

[willingc]

@msarahan Thanks for the work on conda-build. I came across an error similar to @takluyver this morning. Let us know if you need someone to test the fix when its ready.

ERROR: placeholder '/opt/conda/envs/_build_placehold_placehold_placehold_placehold_placehold_placeho' too short in: ncurses-5.9-8

[msarahan]

I have tagged the 2.0.0beta2 release. Notes are up at https://github.com/conda/conda-build/releases/tag/2.0.0beta2

We're working on packaging it. When it is ready, it will be available with

conda install -c conda-canary conda-build

Note that this probably will also update your conda installation. Use care if you have important things in your root environment (perhaps install another miniconda install for testing).

In the meantime, if you're up for installing it from source, please go ahead and clone. I will be excited to hear how it works for you.

The final release is planned for Monday, August 29, depending on how many bugs come up.

[msarahan]

The 2.0.0beta3 release is available. This fixes issues with the entry points, fixes a deadlock encountered with falling back to a shorter prefix, and adds compatiblity with numpy for the setup.py jinja2 templating. This release is available on the conda-canary channel, as described above.

Release still on schedule for Monday.

[ghost]

We use Jenkins and our workspace path was already /var/jenkins/workspace/project-name which ran over 80 chars pretty easily once other stuff was cloned into the workspace. I finally tracked it down to the output path of conda build being the one that's too long. So I run conda build in a place like "/tmp/proj${BUILD_NUMBER}" and then mv it back into the workspace after.

More or less, just:

export tmpDir=/tmp/myproj${BUILD_NUMBER}
export CONDA_BLD_PATH=$tmpDir
conda build ...
mv $tmpDir/* {where you wanted it}

I had tried using conda-canary and latest conda, but it would fail on some other package we have not being built with it, or some other mess. Part of that is we have internal packages that may need to be rebuilt with a newer conda build to be compatible with path size changes. This worked for now, but I'd like to keep a watch on how this path issue evolves.

[jakirkham]

Would think this is long since resolved. Probably safe to close.

[willingc]

@jakirkham I would go ahead and close. cc/ @agjohnson @humitos

[humitos]

@jakirkham @willingc do we have an example of a project that wasn't builing before because of this issue and now is building properly?

If I undertsand correctly, this wasn't an error because of RTD but an error in conda, right? If that's the case, we should check what version we are using on RTD to build and see if the issue is fixed in that version already.

[msarahan]

It's not so much an error in conda as simply a property of packages. When you build a package, you do so in a folder with a particular length. Packages that embed paths in binaries then cannot be installed into paths longer than that creation-time length. About 2 years ago, the standard was 80 characters. We changed it to 255. Essentially all packages in the ecosystem have been rebuilt, so this problem should be extremely rare (only happening to users with very long install paths).

[jakirkham]

IOW this was a conda package issue (not a RTD issue) that has been resolved by improved tooling since used to rebuild many packages in the intervening time.

[humitos]

Thanks you all for the update. I'm closing this issue. Feel free to re-open if you consider :)