Azure-docs: Jupyter Hub returns 500 error after successful login

Created on 12 Feb 2019 · 13Comments · Source: MicrosoftDocs/azure-docs

The DLVM/DSVM templates often break, so you can't get easy access to Jupyter Lab. It's hit or miss, with a 500 error being returned after a successful login most of the time I create a new DLVM/DSVM. Please provide a workaround in the docs. Something like the following.

1) Go to https://{your_ip}:8000. If you got in, GREAT! It worked for you! If not, follow the steps below
2) Go to the Azure Portal, and add a Network rule to allow incoming TPC traffic to your DLVM on Port 8888 (for example)
3) Open Terminal (if using a Mac) or something else if using Windows/Linux
4) SSH to the DLVM with "ssh {username}:{DLVM_IP_address}
5) Start Jupyter with "jupyter notebook --no-browser --port=8888 --ip=* --notebook-dir=~/notebooks"
using --ip=* lets you login from ip addresses other than 'localhost'
6) Copy resulting URL with the access token in it.
7) Paste in a LOCAL browser, but don't hit Enter
8) Replace the private name of the DLVM/DSVM with the ip address or public DNS name
for instance http://fastai:8888/?token=... becomes http://12.34.56.78:8888/?token=...
9) Hit Enter and you're in! Congratulations! You've rocked the workaround and can now get to work!

Unfortunately, it appears getting Jupyter access on DSVM and DLVM instance creation falls into the "Too Hard" box. By having a workaround Azure can support users who have limited Linux proficiency, but still want to use the Ubuntu DSVM (since it has production support for things like PyTorch and fastai).

Alternatively, for readers of this post, you can use deep learning SaaS services like crestle.ai, paperspace.com (or it's super easy tool, Gradient), Kaggle kernels, FloydHub, or a million other really good Jupyter hosting services. Heck, you can even use notebooks.azure.com, however, to use a GPU, you'll still have to configure a DLVM or DSVM yourself anyway... :-(

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 27c1a5f0-b0df-88f4-feaf-90bc17eeb69e
Version Independent ID: 3d5b36f7-e892-8d1d-fa14-a2e5d407a25c
Content: Create an Ubuntu Linux Data Science Virtual Machine - Azure
Content Source: articles/machine-learning/data-science-virtual-machine/dsvm-ubuntu-intro.md
Service: machine-learning
GitHub Login: @gopitk
Microsoft Alias: gokuma

assigned-to-author machine-learninsvc product-issue triaged

Source

StevenBorg

👍2

Most helpful comment

Use LOWER CASE ONLY for user names and all is well!

StevenBorg on 13 Feb 2019

👍2 🎉1 😄1

All 13 comments

Thanks for the feedback and sorry about the issue you are facing @StevenBorg . How often have you seen this issue and is it consistent on your DSVM instance? The jupyterhub log file (/var/log/jupyterhub.log) will give some indication what may be going on. Are you seeing any errors in the Jupyterhub logs? You can also try to stop and restart the jupyterhub service with sudo systemctl stop jupyterhub and sudo systemctl start jupyterhub. (Should not be needed usually. Just to see whether this "fixes" the condition).

Also what version of the Ubuntu DSVM image are you using? You can run this command: curl -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2017-08-01" | python -m json.tool on an SSH session in bash. It will output a whole lot of metadata on the VM. The version field is one of them. More info is here. There was a bug one of the older version where occasionally the jupyterhub service will be unable to spawn jupyter session which was fixed back in December.

gopitk on 12 Feb 2019

Gopi,

I created 4 different instances from the Azure Portal yesterday and today. Each day was consistent – successful login to Jupyter but Error 500 when I get there. (I can tell it’s successful since when I mess up the password it won’t let me through to the 500 error.)

I’ve also tried several times over the past 2 years, with varying success. Usually the Jupyter notebook requires some love to get it working. After a series of frustrations, I just started using an AWS deep learning instance, since they just work OOB. However, I just took a job at Microsoft, and I start Feb 25th, so I figured I’d better get used to the Azure DSVM/DLVMs. :-) I’ve always been a fan of the Azure ML Service, so I play there for my operationalization, etc. (I’ve always been a huge fan of Azure over AWS.). But it was just easier to use AWS when I needed to get a quick deep learning environment up and running.

I’ll go spin up another one and get some of these details for you, so you have more to go on. But I’m not much of a Linux guru, so I might have to take a while. :-). (Plus, my kids are at home for a snow day, so I get interrupted a lot.)

Alternatively, if you spin up a new one, I’m sure you’ll see the same behavior. (Unless it’s due to me using a Mac, but I’ve used Chrome and Safari, both with the same result.)

:-)

Steve

From: Gopi Kumar notifications@github.com
Reply-To: MicrosoftDocs/azure-docs reply@reply.github.com
Date: Tuesday, February 12, 2019 at 12:23 PM
To: MicrosoftDocs/azure-docs azure-docs@noreply.github.com
Cc: Steven Borg steve@theborg.family, Mention mention@noreply.github.com
Subject: Re: [MicrosoftDocs/azure-docs] Jupyter Hub returns 500 error after successful login (#24813)

Thanks for the feedback and sorry about the issue you are facing @StevenBorg . How often have you seen this issue and is it consistent on your DSVM instance? The jupyterhub log file (/var/log/jupyterhub.log) will give some indication what may be going on. Are you seeing any errors in the Jupyterhub logs? You can also try to stop and restart the jupyterhub service with sudo systemctl stop jupyterhub and sudo systemctl start jupyterhub. (Should not be needed usually. Just to see whether this "fixes" the condition).

Also what version of the Ubuntu DSVM image are you using? You can run this command: curl -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2017-08-01" | python -m json.tool on an SSH session in bash. It will output a whole lot of metadata on the VM. The version field is one of them. More info is here. There was a bug one of the older version where occasionally the jupyterhub service will be unable to spawn jupyter session which was fixed back in December where there was occasional error.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

StevenBorg on 12 Feb 2019

Also, I should mention I spun up 3 different Ubuntu images…

A special one with a configuration for Fast.ai – from a github repo
That one may have been version bound to an old version, so I went to the portal to build out latest versions
DLVM on Ubuntu
DSVM on Ubuntu

Then another DLVM on Ubuntu where I went in a fixed everything to make it work, so I could post the ticket.

:-)

Steve

From: Steven Borg steve@theborg.family
Date: Tuesday, February 12, 2019 at 1:51 PM
To: MicrosoftDocs/azure-docs reply@reply.github.com, MicrosoftDocs/azure-docs azure-docs@noreply.github.com
Cc: Mention mention@noreply.github.com
Subject: Re: [MicrosoftDocs/azure-docs] Jupyter Hub returns 500 error after successful login (#24813)

Gopi,

Alternatively, if you spin up a new one, I’m sure you’ll see the same behavior. (Unless it’s due to me using a Mac, but I’ve used Chrome and Safari, both with the same result.)

:-)

Steve

Thanks for the feedback and sorry about the issue you are facing @StevenBorg . How often have you seen this issue and is it consistent on your DSVM instance? The jupyterhub log file (/var/log/jupyterhub.log) will give some indication what may be going on. Are you seeing any errors in the Jupyterhub logs? You can also try to stop and restart the jupyterhub service with sudo systemctl stop jupyterhub and sudo systemctl start jupyterhub. (Should not be needed usually. Just to see whether this "fixes" the condition).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

StevenBorg on 12 Feb 2019

I just tried to create from the FastAI template (which did have the previous issue from Sept 2018 resolved) and I am unable to simulate the issue. When you get a chance please share error messages from jupyterhub.log when you tried to login and tghe output from the image version command I pasted above ?

gopitk on 12 Feb 2019

Will do! Heading in now. Just recreated a DLVM straight from the portal and running into the same issue.

I’ll get you the results of the commands in a few!

From: Gopi Kumar notifications@github.com
Reply-To: MicrosoftDocs/azure-docs reply@reply.github.com
Date: Tuesday, February 12, 2019 at 2:14 PM
To: MicrosoftDocs/azure-docs azure-docs@noreply.github.com
Cc: Steven Borg steve@theborg.family, Mention mention@noreply.github.com
Subject: Re: [MicrosoftDocs/azure-docs] Jupyter Hub returns 500 error after successful login (#24813)

t tried to create from the FastAI template (which did have the previous issue from Sept 2018 resolved) and I am unable to simulate the issue. When you get a chance please share error messages from jupyterhub.log when you tried to login and tghe output from the image version command I pasted above ?

StevenBorg on 12 Feb 2019

See below for errors and logs. Looks like MAYBE the problem is that I’m using an upper case character in my user name??? Testing now.

Here’s a quick demo of creating the DLVM (while I get to the scripts below)

After building, I go to https://myDNS.westus2.cloudapp.azure.com:8000 and go through the warnings. Finally clicking the Proceed link…

Which brings me to the login page:

Entering a bad password simply lets me know it’s not right:

But entering the correct password gets me to here:

Now, let me jump into ssh and some commands!

Here’s the version: 19.01.03

% Total % Received % Xferd Average Speed Time Time Time Current

                             Dload  Upload   Total   Spent    Left  Speed

100 665 100 665 0 0 216k 0 --:--:-- --:--:-- --:--:-- 216k

{

"compute": {

    "location": "westus2",

    "name": "fastai",

    "offer": "linux-data-science-vm-ubuntu",

    "osType": "Linux",

    "placementGroupId": "",

    "platformFaultDomain": "0",

    "platformUpdateDomain": "0",

    "publisher": "microsoft-ads",

    "resourceGroupName": "fastai",

    "sku": "linuxdsvmubuntu",

    "subscriptionId": "da807d2a-38f8-407b-b918-fe4de3364f67",

    "tags": "Application:DataScience",

    "version": "19.01.03",

    "vmId": "b8fa1d46-963c-4271-b619-0d73c2a30acf",

    "vmSize": "Standard_NC6"

},

"network": {

    "interface": [

        {

            "ipv4": {

                "ipAddress": [

                    {

                        "privateIpAddress": "192.168.1.4",

                        "publicIpAddress": "52.183.45.185"

                    }

                ],

                "subnet": [

                    {

                        "address": "192.168.1.0",

                        "prefix": "28"

                    }

                ]

            },

            "ipv6": {

                "ipAddress": []

            },

            "macAddress": "000D3AFDB14C"

        }

    ]

}

}

Here’s the key part of the log, which includes this in the python error stack trace: KeyError: 'User azureadmin does not exist.'

Is it possible that upper case characters aren’t allowed as User Names? It lets me use them elsewhere, but it looks like my AzureAdmin user has somehow been converted to azureadmin, which is a different string since it’s case sensitive.

[I 2019-02-12 22:18:32.508 JupyterHub log:158] 302 GET / -> /hub (@::ffff:50.46.121.75) 0.97ms

[I 2019-02-12 22:18:32.561 JupyterHub log:158] 302 GET /hub -> /hub/ (@::ffff:50.46.121.75) 0.42ms

[I 2019-02-12 22:18:32.602 JupyterHub log:158] 302 GET /hub/ -> /hub/login (@::ffff:50.46.121.75) 0.44ms

[I 2019-02-12 22:18:32.878 JupyterHub log:158] 200 GET /hub/login (@::ffff:50.46.121.75) 262.13ms

[W 2019-02-12 22:19:22.364 JupyterHub auth:618] PAM Authentication failed (AzureAdmin@::ffff:50.46.121.75): [PAM Error 7] Authentication failure

[W 2019-02-12 22:19:22.365 JupyterHub base:504] Failed login for AzureAdmin

[I 2019-02-12 22:19:22.366 JupyterHub log:158] 200 POST /hub/login?next= (@::ffff:50.46.121.75) 2393.51ms

[E 2019-02-12 22:20:23.545 JupyterHub web:1621] Uncaught exception POST /hub/login?next= (::ffff:50.46.121.75)

HTTPServerRequest(protocol='https', host='dsvmvuykhdzkfl7va.westus2.cloudapp.azure.com:8000', method='POST', uri='/hub/login?next=', version='HTTP/1.1', remote_ip='::ffff:50.46.121.75')

Traceback (most recent call last):

  File "/data/anaconda/envs/py35/lib/python3.5/site-packages/tornado/web.py", line 1543, in _execute

    result = yield result

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/futures.py", line 294, in result

    raise self._exception

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/tasks.py", line 242, in _step

    result = coro.throw(exc)

  File "/data/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/handlers/login.py", line 81, in post

    user = await self.login_user(data)

  File "/data/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 483, in login_user

    await maybe_future(self.authenticator.add_user(user))

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/futures.py", line 381, in __iter__

    yield self  # This tells Task to wait for completion.

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/tasks.py", line 310, in _wakeup

    future.result()

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/futures.py", line 294, in result

    raise self._exception

  File "/data/anaconda/envs/py35/lib/python3.5/asyncio/tasks.py", line 240, in _step

    result = coro.send(None)

  File "/data/anaconda/envs/py35/lib/python3.5/site-packages/jupyterhub/auth.py", line 525, in add_user

    raise KeyError("User %s does not exist." % user.name)

KeyError: 'User azureadmin does not exist.'

[E 2019-02-12 22:20:23.835 JupyterHub log:150] {

  "X-Forwarded-Proto": "https",

  "X-Forwarded-Host": "dsvmvuykhdzkfl7va.westus2.cloudapp.azure.com:8000",

  "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36",

  "Accept-Language": "en-US,en;q=0.9",

  "Cache-Control": "max-age=0",

  "Upgrade-Insecure-Requests": "1",

  "X-Forwarded-For": "::ffff:50.46.121.75",

  "Referer": "https://dsvmvuykhdzkfl7va.westus2.cloudapp.azure.com:8000/hub/login?next=",

  "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",

  "Connection": "close",

  "X-Forwarded-Port": "8000",

  "Content-Type": "application/x-www-form-urlencoded",

  "Cookie": "MSCC=1549919420; _ga=GA1.2.947189185.1549919421; _gid=GA1.2.720362469.1549919421; AMCVS_EA76ADE95776D2EC7F000101%40AdobeOrg=1; _mkto_trk=id:157-GQE-382&token:_mch-azure.com-1549922098587-42526; AMCV_EA76ADE95776D2EC7F000101%40AdobeOrg=-179204249%7CMCMID%7C51833578486996441414172429931544220796%7CMCAAMLH-1550526898%7C9%7CMCAAMB-1550526914%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCOPTOUT-1549929298s%7CNONE%7CMCAID%7CNONE%7CMCCIDH%7C2058593835",

  "Accept-Encoding": "gzip, deflate, br",

  "Content-Length": "45",

  "Origin": "https://dsvmvuykhdzkfl7va.westus2.cloudapp.azure.com:8000",

  "Host": "dsvmvuykhdzkfl7va.westus2.cloudapp.azure.com:8000"

}

[E 2019-02-12 22:20:23.836 JupyterHub log:158] 500 POST /hub/login?next= (@::ffff:50.46.121.75) 359.34ms

And the Jupyter full Jupyter log:

[I 2018-12-05 03:58:25.938 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-05 03:58:25.938 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-05 03:58:25.941 JupyterHub app:1055] Writing cookie_secret to /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-05 03:58:25.959 alembic.runtime.migration migration:117] Context impl SQLiteImpl.

[I 2018-12-05 03:58:25.959 alembic.runtime.migration migration:122] Will assume non-transactional DDL.

[I 2018-12-05 03:58:26.028 alembic.runtime.migration migration:327] Running stamp_revision -> 896818069c98

[I 2018-12-05 03:58:26.471 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-05 03:58:26.472 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-05 03:58:26.472 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-05 03:58:26.472 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-05 03:58:26.509 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-05 03:58:26.510 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-05 03:58:26.857 JupyterHub proxy:301] Checking routes

[I 2018-12-05 03:58:26.857 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-05 03:58:26.860 JupyterHub app:1912] JupyterHub is now running at http://:8000

[I 2018-12-05 04:03:26.864 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:08:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:13:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:18:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:23:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:28:26.865 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:33:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:38:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:43:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:48:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:53:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 04:58:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:03:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:08:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:13:26.864 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:18:26.863 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:23:26.862 JupyterHub proxy:301] Checking routes

[C 2018-12-05 05:27:34.227 JupyterHub app:1929] Received SIGTERM, shutting down

[I 2018-12-05 05:27:34.229 JupyterHub app:1706] Cleaning up single-user servers...

[I 2018-12-05 05:27:34.230 JupyterHub proxy:627] Cleaning up proxy[56210]...

[I 2018-12-05 05:27:34.232 JupyterHub app:1739] ...done

[I 2018-12-05 05:29:40.758 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-05 05:29:40.758 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-05 05:29:40.853 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-05 05:29:41.126 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-05 05:29:41.127 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-05 05:29:41.128 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-05 05:29:41.128 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-05 05:29:41.165 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-05 05:29:41.167 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-05 05:29:42.026 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:29:42.027 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-05 05:29:42.030 JupyterHub app:1912] JupyterHub is now running at http://:8000

[C 2018-12-05 05:30:41.645 JupyterHub app:1929] Received SIGTERM, shutting down

[I 2018-12-05 05:30:41.646 JupyterHub app:1706] Cleaning up single-user servers...

[I 2018-12-05 05:30:41.646 JupyterHub proxy:627] Cleaning up proxy[3385]...

[I 2018-12-05 05:30:41.647 JupyterHub app:1739] ...done

[I 2018-12-05 05:35:13.226 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-05 05:35:13.231 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-05 05:35:13.316 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-05 05:35:13.576 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-05 05:35:13.576 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-05 05:35:13.577 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-05 05:35:13.577 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-05 05:35:13.608 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-05 05:35:13.609 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-05 05:35:15.711 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:35:15.711 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-05 05:35:15.714 JupyterHub app:1912] JupyterHub is now running at http://:8000

[I 2018-12-05 05:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 05:55:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:45:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 06:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 07:55:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 08:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 09:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:45:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 10:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:35:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 11:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:45:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 12:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 13:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 14:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:05:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 15:55:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:30:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 16:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 17:55:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:10:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:40:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 18:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:05:15.717 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:15:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:35:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:45:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:50:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 19:55:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:00:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:20:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:25:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:30:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:35:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:40:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:45:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:50:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 20:55:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:00:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:05:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:10:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:15:15.715 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:20:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:25:15.716 JupyterHub proxy:301] Checking routes

[I 2018-12-05 21:30:15.715 JupyterHub proxy:301] Checking routes

[C 2018-12-05 21:32:43.837 JupyterHub app:1929] Received SIGTERM, shutting down

[I 2018-12-05 21:32:43.837 JupyterHub app:1706] Cleaning up single-user servers...

[I 2018-12-05 21:32:43.837 JupyterHub proxy:627] Cleaning up proxy[2698]...

[I 2018-12-05 21:32:43.838 JupyterHub app:1739] ...done

[I 2018-12-08 00:58:52.138 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-08 00:58:52.305 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-08 00:58:52.745 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-08 00:58:53.915 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-08 00:58:53.916 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-08 00:58:53.916 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-08 00:58:53.917 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-08 00:58:53.959 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-08 00:58:53.961 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-08 00:59:05.279 JupyterHub proxy:301] Checking routes

[I 2018-12-08 00:59:05.280 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-08 00:59:05.283 JupyterHub app:1912] JupyterHub is now running at http://:8000

[I 2018-12-08 01:04:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:09:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:14:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:19:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:24:05.285 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:29:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:34:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:39:05.285 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:44:05.286 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:49:05.286 JupyterHub proxy:301] Checking routes

[C 2018-12-08 01:50:05.826 JupyterHub app:1929] Received SIGTERM, shutting down

[I 2018-12-08 01:50:05.839 JupyterHub app:1706] Cleaning up single-user servers...

[I 2018-12-08 01:50:05.839 JupyterHub proxy:627] Cleaning up proxy[3151]...

[I 2018-12-08 01:50:05.840 JupyterHub app:1739] ...done

[I 2018-12-08 01:50:29.199 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-08 01:50:29.200 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-08 01:50:29.209 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-08 01:50:29.250 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-08 01:50:29.251 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-08 01:50:29.252 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-08 01:50:29.252 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-08 01:50:29.318 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-08 01:50:29.321 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-08 01:50:30.116 JupyterHub proxy:301] Checking routes

[I 2018-12-08 01:50:30.117 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-08 01:50:30.120 JupyterHub app:1912] JupyterHub is now running at http://:8000

[I 2018-12-08 01:55:30.123 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:00:30.123 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:05:30.123 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:10:30.123 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:15:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:20:30.123 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:25:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:30:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:35:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:40:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:45:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:50:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 02:55:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:00:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:05:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:10:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:15:30.125 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:20:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:25:30.125 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:30:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:35:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:40:30.125 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:45:30.125 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:50:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 03:55:30.122 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:00:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:05:30.124 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:10:30.123 JupyterHub proxy:301] Checking routes

[C 2018-12-08 04:12:33.904 JupyterHub app:1929] Received SIGTERM, shutting down

[I 2018-12-08 04:12:33.906 JupyterHub app:1706] Cleaning up single-user servers...

[I 2018-12-08 04:12:33.906 JupyterHub proxy:627] Cleaning up proxy[2352]...

[I 2018-12-08 04:12:33.907 JupyterHub app:1739] ...done

[I 2018-12-08 04:13:01.289 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4

[I 2018-12-08 04:13:01.290 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4

[I 2018-12-08 04:13:01.296 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret

[I 2018-12-08 04:13:01.340 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN

[W 2018-12-08 04:13:01.341 JupyterHub app:1173] No admin users, admin interface will be unavailable.

[W 2018-12-08 04:13:01.341 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config.

[I 2018-12-08 04:13:01.342 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.

[I 2018-12-08 04:13:01.387 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/

[I 2018-12-08 04:13:01.396 JupyterHub proxy:567] Starting proxy @ http://:8000

[I 2018-12-08 04:13:02.215 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:13:02.215 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:8081

[I 2018-12-08 04:13:02.220 JupyterHub app:1912] JupyterHub is now running at http://:8000

[I 2018-12-08 04:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:28:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:38:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:48:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:53:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 04:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:23:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:28:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:33:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:43:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:48:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:53:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 05:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:13:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:18:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:23:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:28:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:48:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 06:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:33:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:48:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 07:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:23:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 08:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:33:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 09:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:53:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 10:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 11:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 12:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 13:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 14:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 15:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 16:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:18:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:43:02.225 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 17:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:48:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 18:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 19:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 20:58:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:28:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 21:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:28:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 22:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:03:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:28:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-08 23:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:03:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 00:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:03:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 01:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:43:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 02:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:38:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 03:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 04:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 05:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:23:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:28:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 06:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:43:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 07:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 08:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:33:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 09:58:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:13:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:33:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 10:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:18:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 11:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:38:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:48:02.224 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 12:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:33:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 13:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:13:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:43:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:48:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 14:58:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:13:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:48:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 15:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:53:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 16:58:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:23:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:43:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:48:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 17:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:08:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:38:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 18:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:33:02.221 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 19:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:03:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:08:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:23:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:28:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:43:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:53:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 20:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:18:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:33:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:38:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:43:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:48:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:53:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 21:58:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:03:02.223 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:08:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:13:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:18:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:23:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:28:02.222 JupyterHub proxy:301] Checking routes

[I 2018-12-09 22:33:02.222 Jupyter

StevenBorg on 12 Feb 2019

Thanks @StevenBorg . Check with our dev and it seems to be an issue in Jupyterhub. Somehow it expects local accounts to be lower cased. Sorry for the trouble. We will look at updating the doc. Can you create another account on the box with lower cased userid? Yes. Kinda weird :(

gopitk on 12 Feb 2019

BOOM! Lower case user ID works like a charm!

No wonder it was sporadic for me! Sometimes I’d use lower case and others I’d use upper case. Grrr…

:-)

Steve

From: Gopi Kumar notifications@github.com
Reply-To: MicrosoftDocs/azure-docs reply@reply.github.com
Date: Tuesday, February 12, 2019 at 2:58 PM
To: MicrosoftDocs/azure-docs azure-docs@noreply.github.com
Cc: Steven Borg steve@theborg.family, Mention mention@noreply.github.com
Subject: Re: [MicrosoftDocs/azure-docs] Jupyter Hub returns 500 error after successful login (#24813)

he trouble. We will look at updating the doc. Can you create another account on the box with lower cased userid? Yes. Kinda weird :(

StevenBorg on 12 Feb 2019

Use LOWER CASE ONLY for user names and all is well!

StevenBorg on 13 Feb 2019

👍2 🎉1 😄1

Gopi!

No way! You’re also the approver for the pull request I just sent in to update Keras to include a reference to Microsoft Cognitive Toolkit!

Weird…. Very small world.

😃

Steve

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

StevenBorg on 13 Feb 2019

please-close

gopitk on 14 Feb 2019

I had this exact same error in June. So it’s definitely not fixed. Sure enough, I created a user with a lowercase name and the problem went away. This is a serious usability issue for anyone using an iPad to create and manage accounts online. The reason the login/user was up-cased was as a result of the Azure Portal form that automatically up-cased the account name in the form field. I could have changed it, but never anticipated a problem such as this. Thanks for posting the thread @StevenBorg @gopitk!

lisah2u on 27 Jun 2019

Use LOWER CASE ONLY for user names and all is well!

You definitely rock!! this works for me

Just in case

Create new DSVM users
If multiple users share a DSVM, you can avoid blocking each other by creating and using a DSVM user for each notebook user:

On the Azure Portal, navigate to your virtual machine.
Under Support + troubleshooting in the left margin, select Reset password.
Enter a new username and password and select Update. (Existing usernames are not affected.)
Repeat the previous step for any additional users.
https://docs.microsoft.com/en-us/azure/notebooks/use-data-science-virtual-machine#accessing-azure-notebooks-files-from-the-dsvm