Readthedocs.org: Mention more the usage of the rtd configuration file

[dhermes]

This is proposed as an alternative to the "Install your project inside a virtualenv using setup.py install" option:

But really it could be a replacement? Due to the way Sphinx works, installing using setuptools (i.e. setup.py install) can be problematic. See a gist I have showcasing how small differences can occur. For example, with setuptools (on the left), my enum has no members but with pip (on the right) the members are visible:

The core issue is that setuptools will install an egg (which is zipped and remains zipped once installed) and pip will build a wheel and install it (which will end up unpacked in site-packages).

IIUC, Sphinx will try to locate the actual source (e.g. to determine class members). ISTM that when installed as an egg, Sphinx fails to find the source files since the entire package is zipped into a single file. (This is speculation, I haven't dug into the Sphinx source to check this.)

/cc @jonparrott

---

PS I'm aware one can just specify . in docs/requirements.txt, but this can still be problematic if, for example, running pip install for a given package relies on other dependencies already being installed.

[agjohnson]

We already have an option for this which should work. More info here:
http://docs.readthedocs.io/en/latest/yaml-config.html#python-pip-install

[dhermes]

That's great to know! I didn't realize the beta YAML config had features that were only available there. Thanks. Shall we close?

I feel like this issue should be re-purposed somehow, just because setup.py install installs eggs, which breaks many tools (esp. important is that it breaks Sphinx in some cases).

[agjohnson]

@dhermes any examples of how this breaks some packages? It would probably be worth exploring the failure case more if it is anything we can reproduce.

[dhermes]

@agjohnson I didn't mean it breaks the installation of packages, I meant it breaks tools. Sorry if that was confusing.

My example above showcases such a "breakage", caused by the fact that Sphinx can't find the source code when rendering the enum.

[agjohnson]

Oh sorry! I'm just over here bouncing between issues with wild abandon -- I didn't reread the original report.

That makes a lot of sense. More recently, we've discussed using pip here, as setup.py will not be required in the future. We could evaluate moving to pip install globally, or at least defaulting to using pip install and making python setup.py install the user-configurable backup. We'd want to be sure that fallout is limited though, and I'm not certain of the edge cases between the two methods for us.

@ericholscher might have more background here on the historical significance of python setup.py for our installation steps. Also, paging @agronholm, as you've had good insight into our package installation methods. I'm not seeing many downsides to using pip install by default though.

[ericholscher]

I know we tried this in the past, but made it optional for some reason. I think pip is likely standard enough now that we should probably just use it always. We can probably do a similar roll out behind a feature flag for it.

[dhermes]

Thanks guys, that's great!

After our (very painful) ongoing experience with namespace packages over in google-cloud-python, I would legitimately advise people to only use pip to install packages (or, to flip on it's head, never use setup.py install to install directly).

[agronholm]

After our (very painful) ongoing experience with namespace packages over in google-cloud-python, I would legitimately advise people to only use pip to install packages (or, to flip on it's head, never use setup.py install to install directly).

Seconded. I've seen many cases where running setup.py install lead to bizarre problems. Not only that, but this will cause easy_install to be used for installing dependencies, which, among other problems, does not support wheels. I, too, see no downsides to using pip globally.

[agjohnson]

We can probably do a similar roll out behind a feature flag for it.

👍 We could just invert the logic for the readthedocs.yml pip config option. It would be defaulted to True If users really need to override, the option can be set to False to use setup.py install instead.

[dhermes]

@agjohnson Just my two cents: I am a fairly heavy RTD user and it never occurred to me to use the .readthedocs.yml config until this thread. The checkbox in the UI could also use a refresh.

[agjohnson]

@dhermes We should be talking more about the config file and its use. Our plan is to eventually get rid of a lot of the dashboard page options and move them to the config -- making a lot of these options per-version/branch, but also simplifying our admin dashboard pages greatly. We'd plan on having some better UX around the dashboard when that happens.

[ezdac]

You should definitely mention this more in the build guides! Had I known about this it would have saved me a lot of trouble. (To be fair, I should probably have stumbled upon the yaml configuration if I would have done better research)

[ericholscher]

@ezdac updating our tutorial to use the YAML file would be a lovely contribution!

[Zac-HD]

It would be fantastic if this also exposed an option for the package to install!

As part of our recent move towards a monorepo (Python + Ruby + Rust implementations, website, etc), Hypothesis no longer has the setup.py in the repo root. Our readthedocs build therefore failed (with the install option checked) or was missing included docstrings similarly to the above.

We fixed that with some yml and requirements hacks - HypothesisWorks/hypothesis#1228 - but thought I'd comment FYI and for anyone else searching with a related problem.

[stsewd]

So, the default option is to not install the project, as some heavy projects don't need that, and would take some resources to build. I think the remaining issue is to document more the usage of the yaml config on the getting started guide. I'm documenting this on #4451

[stsewd]

Or maybe I can put that on another PR after #4451

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stsewd]

This is blocked by https://github.com/rtfd/readthedocs.org/issues/4279

[humitos]

The issue (#4279) that was blocking this one is already closed and #4451 merged. What is the actionable part of this one now?

I think the remaining issue is to document more the usage of the yaml config on the getting started guide.

Reading the getting started guide, seems to be some _fast_ to go live into RTD. I wouldn't go through our YAML file there. Maybe, I'd mention it on a .. tip:: or "What's next?" section at the bottom.

[stsewd]

We already have some docs that point to the config file https://docs.readthedocs.io/en/stable/intro/import-guide.html#building-your-documentation (that guide is linked from the first steps)

Also, we suggest the config file in the import menu, advanced settings and build page.