Readthedocs.org: Command killed due to excessive memory consumption

[muammar]

Details

• Read the Docs project URL: https://mlchem.readthedocs.io/en/latest/
• Build URL (if applicable): https://readthedocs.org/projects/mlchem/builds/8623080/
• Read the Docs username (if applicable):
• GitHub: https://github.com/muammar/mlchem/tree/master/docs/

Expected Result

The documentation created with moduleindex.

Actual Result

No module index and build failing with:

Collecting matplotlib==3.0.2 (from -r docs/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/e7/f9/5377596cb1c035c102396f5934237a046f80da69974026f90bee5db8b7ba/matplotlib-3.0.2-cp37-cp37m-manylinux1_x86_64.whl
Collecting numpy==1.16.1 (from -r docs/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/61/57/07c49e1a6d2706fb7336b3fb11dd285c1e96535c80833d7524f002f57086/numpy-1.16.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting setuptools==40.8.0 (from -r docs/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/d1/6a/4b2fcefd2ea0868810e92d519dacac1ddc64a2e53ba9e3422c3b62b378a6/setuptools-40.8.0-py2.py3-none-any.whl
Collecting torch==1.0.1 (from -r docs/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/4e/ba/d78f35357995297206790fe1af8688bc0403828c90f8e99190035196d56e/torch-1.0.1-cp37-cp37m-manylinux1_x86_64.whl (560.1MB)
Killed


Command killed due to excessive memory consumption

The issue might come from pytorch installation I suppose. Locally, the module index works as expected. I had implemented a Mockclass as shown here but that breaks local creation of the documentation in the sense that I don't get the module index page.

What would you suggest me to try and fix this issue?

Thanks!

[stsewd]

Those are really heavy dependencies https://github.com/muammar/mlchem/blob/master/docs/requirements.txt. If you don't rely on specific versions, our images have some of these libraries already installed https://docs.readthedocs.io/en/stable/config-file/v2.html#python-system-packages

[muammar]

@stsewd thanks for your fast reply. Until now I am not using anything very special so I assume this should work with any pytorch version. I will read the link you posted above and will write back.

[muammar]

This https://github.com/muammar/mlchem/blob/master/docs/environment.yml and this https://github.com/muammar/mlchem/blob/master/readthedocs.yml fixed it :)

https://mlchem.readthedocs.io/en/latest/py-modindex.html

Thanks for your guidance. I will proceed to close this report.

[schimpffabian]

This https://github.com/muammar/mlchem/blob/master/docs/environment.yml and this https://github.com/muammar/mlchem/blob/master/readthedocs.yml fixed it :)

@muammar it seems like the links are broken. It would be awesome if you could share your solution since I face the same error at the moment.

[achaiah]

@muammar @stsewd I too am facing the same issues. torch may be a large dependency but it's something people are including more and more often in their builds. Anyone have a good workaround?

[schimpffabian]

I am looking into http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports but didn't get it to work yet.

[schimpffabian]

@muammar @stsewd I too am facing the same issues. torch may be a large dependency but it's something people are including more and more often in their builds. Anyone have a good workaround?

I solved it in my project https://github.com/2fasc/Distributed_Malaria_Detection . I created a new requirements file where I got rid of all requirements that take a heavy toll on the memory. Save the deleted requirements like in http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports and you should be good.

[achaiah]

@2fasc Thanks that got me unstuck! I guess we fake it till sphinx makes it :)

[mrava87]

Hi,
trying to revive this discussion. In one of my projects (https://readthedocs.org/projects/pylops-gpu/) I use pytorch and the documentation used to build successfully last time I did it, now - likely because some of the dependencies have been updated - I get the usual Command killed due to excessive memory consumption error.

I understand that if torch is just needed in the package but not in the doc there are workarounds, but I want to use torch actively in the documentation building process to create sphinx-gallery tutorials. Anyone has managed to crack this and found a combination pip/conda and torch version that does not break memory consumption? Or any other suggestion?