Jupyter-book: nested subfolders are not found by 'jupyter-book toc' if no files are in each level

[amueller]

my folder structure was something like

- mybook
   - notebooks
      - chapter1
         - notebook1.ipynb
         - notebook2.ipynb

running jupyter-book toc mybook didn't find the notebooks.
Once I added an ipynb file into the notebooks folder, it worked.

Not sure if that's intentional.

[choldgraf]

interesting - so you had to do something like:

- mybook
   - notebooks
      - mynotebook.ipynb
      - chapter1
         - notebook1.ipynb
         - notebook2.ipynb

?

I think that is def a bug :-)

[amueller]

yeah exactly.

[choldgraf]

cool - will try and fix quickly for ya so you don't have to change your folder structure (if you're willing to work off of master)

[amueller]

I'm working off master now. and I dropped an empty index in every section which is probably not too bad. Probably each chapter should have some intro. But I thought it was worth reporting in case you didn't want that behavior.

[choldgraf]

Yeah - actually after working on this for 10 minutes or so I realized it will be a bit more complex to support this than I thought. It may not be an afternoon fix but let's leave this open to support it one day :-)

Note I added this into the docs in the meantime: https://github.com/executablebooks/jupyter-book/pull/570

[amueller]

I'm not sure if this is related to my folder structure, but I'm still struggling a bit.
I must be missing something.
My toc looks like

file: index
title: Applied Ml Python
sections:
- file: 01-introduction\index
  title: 01-introduction
  sections:
  - file: 01-introduction\01 - Data Loading
    title: 01 - data loading
  - file: 01-introduction\09 - Pipelines
    title: 09 - pipelines
- file: 02-supervised_learning\index
  title: 02-supervised Learning
  sections:
  - file: 02-supervised_learning\02 - Supervised Learning
    title: 02 - supervised learning
  - file: 02-supervised_learning\03 - Preprocessing
    title: 03 - preprocessing
  - file: 02-supervised_learning\04 - Review of Supervised Learning
    title: 04 - review of supervised learning

And I get errors like:

C:\Users\t3kci\checkout\Machine_Learning_in_Python\applied_ml_python\index.md:9: WARNING: toctree contains reference to nonexisting document '01-introduction\\index'
C:\Users\t3kci\checkout\Machine_Learning_in_Python\applied_ml_python\index.md:9: WARNING: toctree contains reference to nonexisting document '02-supervised_learning\\index'

looking for now-outdated files... none found
pickling environment... done
checking consistency... C:\Users\t3kci\checkout\Machine_Learning_in_Python\applied_ml_python\01-introduction\index.md: WARNING: document isn't included in any toctree
C:\Users\t3kci\checkout\Machine_Learning_in_Python\applied_ml_python\02-supervised_learning\index.md: WARNING: document isn't included in any toctree


Any idea? Don't want to bug you too much though. The generated page is empty apart from the first index page, i.e. has no toc.

[choldgraf]

Hmmm - I wonder if this is because you have empty files in those index.md files. Can you try adding a dummy title like "Index" for each one to see if that fixes it?

Sorry about the annoyance here! I don't think the jupyter-book toc command has been used much yet so these are valuable bugs to uncover

[amueller]

each of them already has a dummy title.

[amueller]

In case you want to have a look: https://github.com/amueller/Machine_Learning_in_Python

I removed the index files and I had the same problem btw. I wonder if there's a path issue, I'm on the OS that uses backslashs.

[choldgraf]

OK - a couple quick thoughts:

when I clone and try to build your book, I also get error/warnings but different ones. Mine are:

WARNING: toctree contains reference to document '03-unsupervised_learning/aml-11-calibration' that doesn't have a title: no link will be generated

and I think that's because many of your content files don't have titles inside them. Right now the title: field of the toc.yml file will over-ride the title in the navigation bar, but it doesn't set the title...you're still expected to have the title in the page itself. This feels confusing to me now that I see it...

For the errors you're reporting, I do suspect it might be an issue with (I'm assuming) windows. I actually run Windows myself so I can give it a shot probably tomorrow (I use WSL, so even though I have windows, I don't have a development environment set up in it)

One thing that comes to mind - I feel like we should ask people to put / in their _toc.yml files regardless of the OS. If we make the behavior of / change between OS's (at least when it comes to file paths) then we might be opening up a can of worms with CI/CD systems on repositories that were built with windows

[amueller]

I just set up my WSL, needed to reboot for that. downloading anaconda right now ;)

And I have no issue with adding titles, that one I would have probably figured out ;) You might want to change it but the error message seems informative.

None of these notebooks have been touched yet for it being an actual, think of me as the worst chaos money :)

[choldgraf]

those are the best kinds of chaos monkeys for beta releases :-)

[amueller]

Summary:
WSL works with WSL generated toc (i.e. what you tried I think)
WSL doesn't work with windows generated toc
windows doesn't work with windows generated toc
windows doesn't work with WSL generated toc.

I agree the yaml file probably should use / in all operating systems.

[choldgraf]

ok cool - that's very helpful, thanks. Will try and look into it soonishfully

[amueller]

thanks! should I open a separate issue? I'm happy to work in WSL for now.

[amueller]

semi-related: do you run an X server in your WSL?

[choldgraf]

Yeah actually that'd be helpful so we can track windows support.

I do, I think I just pipe it to windows though