Notebook: 404 error when starting notebook from a hidden folder

[choldgraf]

I'm getting a 404 error when I start a jupyter notebook and point it to a hidden folder.

Note: this is resolved, see update below

For me this was enough to recreate the problem:

mkdir test
mkdir .test

# works
jupyter notebook --notebook-dir=./test

# 404 error
jupyter notebook --notebook-dir=./.test

Update

This is not an error, but is expected behavior. Jupyter Notebooks don't serve hidden folders for safety reasons. Newer versions of the notebook server make it possible to launch the server from a hidden folder. Try running:

pip install --upgrade jupyter

[choldgraf]

(this started after I upgraded to 5.0, and it's also happening with jupyter lab FWIW

[tjnd89]

I'm also seeing the same thing after updating a Windows install. But in my case the folder is not really hidden. I have a bunch of notebooks stored in my .ipython folder. When running

#404 error
cd c:\users\tj\.ipython
jupyter notebook

#works
cd c:\users\tj\.ipython\test
jupyter notebook

If you right-click properties on the .ipython folder it will look like the read-only box is checked. But this is true for all folders in windows and is misleading (it's not really readonly). This can be verified using:

(C:\Anaconda3) C:\Users\tj>attrib .ipython
             C:\Users\tj\.ipython

Here's a copy of the server error message:

(C:\Anaconda3) C:\Users\tj\.ipython>jupyter notebook
[I 05:54:30.053 NotebookApp] Serving notebooks from local directory: C:\Users\tj\.ipython
[I 05:54:30.054 NotebookApp] 0 active kernels
[I 05:54:30.054 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=91fd95901828a7e9dbfccc166eaa977bb508bf8813a2f064
[I 05:54:30.054 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 05:54:30.082 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=91fd95901828a7e9dbfccc166eaa977bb508bf8813a2f064
[I 05:54:30.565 NotebookApp] Accepting one-time-token-authenticated connection from ::1
[I 05:54:30.567 NotebookApp] Refusing to serve hidden directory, via 404 Error
[W 05:54:30.658 NotebookApp] 404 GET /tree?token=bc1848cfa7a983646a888ae7ded3bfb6ec54cdc7ac9653f1 (::1) 94.25ms referer=None

[takluyver]

I think our 'is it hidden' check always respects the Unix convention (files starting with a . are hidden), as well as checking the hidden attribute on Windows.

We probably shouldn't be checking the directory where you start the notebook server, though.

[tjnd89]

On the windows side version the jupyter client 4.4.0 works fine with when starting in a directory with a leading . in the name). Something new was introduced in 5.0.0 that breaks this behavior.

[SamuelMarks]

Error still present

[xingshou]

I have the same issue in my Mac book.

jupyter            1.0.0      
jupyter-client     5.0.1      
jupyter-console    5.1.0      
jupyter-core       4.3.0

[vickorian]

This error occurs if you set c.NotebookApp.notebook_dir in jupyter_notebook_config to a drive letter rather than a folder. See issue# 3133 https://github.com/ContinuumIO/anaconda-issues/issues/3133

[kumarvem]

Thanks, after unhide folder it is working.
How to unhide folder in windows => http://www.wikihow.com/Unhide-Folders-in-Windows-7

[bharagavafirst]

even after checking hide folder in windows for anaconda folder im getting this message pls help

[bharagavafirst]

i also mark anaconda folder as un hidden and read only but this does not resolve my problem

[wel51x]

Are you guys EVER going to fix this shit?

[takluyver]

Please remain civil when asking people to fix bugs. Being rude about it does not make us eager to help you.

[wel51x]

Seems to be the only way to get you people to respond to things. The has been a problem for some six months now, yet people - myself included - continue to have a problem with it.

People would be "civil" if you guys would actually address the issue, or at least provide a status update when people indicate they're having problems. That's civil. Silence is as bad as insult.

[choldgraf]

Hi @wel51x, the beauty (and the challenge) of the open source world is that it relies on the contributions and constructive input of its community in order to get things done. If you've got constructive thoughts on how to fix it, please give them here or feel free to open a PR.

[wel51x]

Point taken. Forgive the rant, I was frustrated. My apologies. Pretty much a newbie.

In the meantime I expect I'll have to revert to 4.3 where I didn't have the problem.

[choldgraf]

Agree 1000% that code that's not working is super frustrating :-\ hopefully this bug will be squashed soon. If I have some time I can try digging in a bit more in the coming weeks.

[vickorian]

Do we know where the base code is referencing the intro point?

[choldgraf]

OK - so, I looked into this a little bit, and I don't believe that it's a bug, it's intended behavior. See here:

https://github.com/jupyter/notebook/blob/master/notebook/tree/handlers.py#L44

So clearly notebook explicitly is not allowing hidden folders. I played around with getting this to work and it seems like it'd be non-trivial to add support for hidden folders (somebody correct me if I'm wrong). So, I think the real problem here is that the error messages you get when starting from hidden folders are not helpful. From the user's perspective it feels like a bug and isn't clarified as intended, which is why folks are getting frustrated here.

A few ideas that could help this:

1. In the error message that is logged, rather than on top of saying "Refusing to serve hidden directory" I'd add "(hidden directories not yet supported)" or something along these lines.
2. In the 404 page the notebook throws, what if we added some more useful information there, so instead of just

{% extends "error.html" %}
{% block error_detail %}
<p>You are requesting a page that does not exist!</p>
{% endblock %}

we added a short list of possible fixes below that are common to jupyter, so the page looks something like:

---

You are requesting a page that does not exist!

_The following steps might help:_

1. Did you type in or click on a URL? Make sure there are no type-os and the path is correct.
2. Are you attempting to access a hidden folder? Hidden folders are not supported with the Notebook UI.
   

   3. ....anything else?

Thoughts? @takluyver and maybe @minrk has thoughts too? Let me know and then I will clarify this issue title/top-level comment (or I'll open a PR and turn this issue into a feature request)

[tjnd89]

The issue from my perspective is that under Windows, a directory beginning with "." and a directory that is marked as hidden are two different things. I would prefer it to still working with the former and would understand if it didn't with the latter.

[takluyver]

It is certainly by design - the original motivating example was the .ssh folder storing your private ssh keys. Refusing to serve files from hidden folders makes it a bit harder for potential attackers to get that information or to e.g. add themself to the authorized keys for ssh-ing in. It's still possible through running a kernel, but there are some classes of attack where HTTP requests would be possible but controlling a kernel would not. This is defence in depth: if an outer layer of security is imperfect, it limits what someone can do.

However, I think we could relax it a bit. In particular, if you've launched the notebook from a hidden folder, you probably want to serve that folder (possibly you don't think of it as hidden). So we could refuse to serve hidden subfolders within the starting directory, but not check the starting directory itself.

For the record, part of the reason we haven't been paying attention to this is that while Anaconda has been getting a lot of issues filed about this and linking to this issue, this issue itself has only been getting a couple of comments a month. I don't get notifications from Anaconda issues, so I didn't see all those related issues until I came to this page. That's not anyone's fault, it's just one of those things that happens.

[minrk]

I think stopping the is_hidden check just below the notebook root makes sense.

[choldgraf]

Cool - I think that if it's possible to start the notebook from a hidden folder, that'd nail the majority of the use-cases and we don't need to change documentation at all.

[choldgraf]

so is this as simple as just adding something like and not cm.is_root(path) to:

https://github.com/jupyter/notebook/blob/master/notebook/tree/handlers.py#L43

?

[minrk]

@choldgraf yes, I think so (though I would check root first, since it ought to be cheaper).

[takluyver]

And please keep it civil even when responding to other people who may not have been completely civil themselves. Sweary comments make everyone angry and unproductive.

The comment I'm referring to has been deleted, but everyone who's commented on this issue got it by email already. So I wanted to post this before anyone feels the need to make their own angry response. Take a deep breath and move on.

(I know this can be difficult - I bit back the urge to write something snarkier in my own response, and I could probably have bitten it further back than I did. Thanks @choldgraf for getting the discussion back on track after that.)

[takluyver]

I've opened #2907 to address this as we discussed, by allowing starting the notebook server in a hidden directory.

Looking at the code, I wonder if this was actually an oversight all along; I think the is_hidden function is only meant to check files and directories below the root, and was doing the wrong thing when passed the root directory itself.

[choldgraf]

🎉 🎉 🎉 🎉 woooo @takluyver 🎉 🎉 🎉 🎉

[khadekirti]

I faced the same issue. Here is what I have done :
1) Download Python & Install from: https://www.python.org/downloads/
2) Set variable path: "C:\Users\Python27" and "C:\Python27\Scripts"
3) In the command prompt type 'pip install jupyter'
4) Next, in the command prompt type 'jupyter notebook'

[kskand]

I am getting below error while opening 'jupyter notebook', i tried with other port also but it didn't work. Can anyone help me on this ? I am using windows 10.
(C:\Users\ks00349483\AppData\Local\Continuum\Anaconda3) C:\Users\ks00349483>jupyter notebook
[I 16:21:49.067 NotebookApp] The port 8888 is already in use, trying another port.
[I 16:21:49.237 NotebookApp] Serving notebooks from local directory: C:\Users\ks00349483
[I 16:21:49.237 NotebookApp] 0 active kernels
[I 16:21:49.237 NotebookApp] The Jupyter Notebook is running at: http://localhost:8889/?token=e69ede421e2df0425d8ac9d8c36f1ffaa32a18d1297d338c
[I 16:21:49.237 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 16:21:49.252 NotebookApp]

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8889/?token=e69ede421e2df0425d8ac9d8c36f1ffaa32a18d1297d338c

[I 16:21:50.440 NotebookApp] Accepting one-time-token-authenticated connection from ::1
[I 16:21:50.456 NotebookApp] Refusing to serve hidden directory, via 404 Error
[W 16:21:50.640 NotebookApp] 404 GET /tree?token=0cb117662c05f2133d736476ab685bf019cd19a17cc0becd (::1) 199.62ms referer=None

[kskand]

@goanpeca : Thanks for you quick response but it didn't help. When i installed Anaconda last week at that it worked but this week, suddenly i started getting this error.

[takluyver]

What does jupyter notebook --version give you? Notebook version 5.2 should have it fixed, so if you're using an older version, update.

[rezwanh001]

When I run anaconda-navigator , I am getting following issues. How can I solve this ?

[I 10:45:03.733 NotebookApp] Writing notebook server cookie secret to /run/user/1000/jupyter/notebook_cookie_secret
[I 10:45:05.584 NotebookApp] Serving notebooks from local directory: /home/rezwan
[I 10:45:05.584 NotebookApp] 0 active kernels 
[I 10:45:05.584 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=fbaf89288e9faf90768125f2f2bbb8bbe40e74f2a816c0ab
[I 10:45:05.584 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 10:45:05.585 NotebookApp] 

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=fbaf89288e9faf90768125f2f2bbb8bbe40e74f2a816c0ab
[6098:6136:1121/104506.194884:ERROR:browser_gpu_channel_host_factory.cc(108)] Failed to launch GPU process.
[I 10:45:06.198 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1

[takluyver]

I think the only bit there that's actually an error is the line about browser_gpu_channel_host_factory - people are discussing that on #2836.

[hichoe95]

maybe you should try 'pip3 install --upgrade jupyter '

i solved it with this way

[zhouzhou77]

@hichoe95 Thank you so much , I solved this proplem too ..

[rkranjancs1014]

just check if your directory is hidden

[ecampana]

I wanted to add that this error message also appears when you try to configure jupyter to use a directory where you don't have file access. For example,

c.NotebookApp.notebook_dir = u'/home/myaccount/notebooks'

where "notebooks" is a directory not accessible able to you. This unlikely scenario happened to me when IT copied a folder to my home directory but did not properly set the file permissions on it.