Black: Error writing grammar tables

[EWouters]

I get an error upon launching Jupyter Lab with the jupyterlab_code_formatter extension enabled:

[I 10:37:25.686 LabApp] Node v10.8.0

[I 10:37:26.075 LabApp] Build is up to date
Generating grammar tables from C:\Users\%USERNAME%\Anaconda3\lib\site-packages\blib2to3\Grammar.txt
Writing grammar tables to C:\Users\%USERNAME%\AppData\Local\black\black\Cache\18.9b0\Grammar3.7.1.final.0.pickle
Writing failed: [Errno 2] No such file or directory: 'C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0\\Grammar3.7.1.final.0.pickle'
Generating grammar tables from C:\Users\%USERNAME%\Anaconda3\lib\site-packages\blib2to3\PatternGrammar.txt
Writing grammar tables to C:\Users\%USERNAME%\AppData\Local\black\black\Cache\18.9b0\PatternGrammar3.7.1.final.0.pickle
Writing failed: [Errno 2] No such file or directory: 'C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0\\PatternGrammar3.7.1.final.0.pickle'

After manually creating the folder C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0 and relaunching Jupyter lab the error disappears (even though the folder seems to stay empty).

The error appears to originate from this line. It might be related to #192. I think this line fails because the cache directory does not exist, but I don't understand why it doesn't exist.

Other users confirmed this happens on linux and with version 19.10b as well in this issue.

I tried to find the cause of this issue in the extension, but it doesn't seem to do anything out of the ordinary with the cache folder, so I was hoping someone here can tell me why the cache folder would not exist?

[EWouters]

Looks like I found the cause, thanks for the fix @AlJohri!

[AlJohri]

Thanks @EWouters! If you the bandwidth you can make a PR to updated the vendored blib2to3 as described in the conversation here: https://github.com/psf/black/pull/1224

I haven't gotten the chance.

[copperwiring]

Has there been a PR on this yet?

[ichard26]

I'm reopening this issue since 1. the issue still exists and 2. the fix #1223 proposes isn't ideal.

I see two possible solutions to this issue:

1. Upon importing Black, the cache directory where the grammar table cache is stored is created. (#1223 suggests going this route)
2. When Black is called using the CLI and actually finishes doing something, the cache directory is created if needed and the grammar table cache¹ is then written to disk.

While option 1 is simpler, the cache directory would be created when simply importing Black. Applications that use Black as a library might never use the cache, yet the cache dir would still be created, not an ideal side-effect. Option 2 is a bit more involved but it avoids the unnecessary side-effect mentioned above. Anyway the other caches like the cache which records which files are well-formatted already behave this way. It's only the grammar table cache that tries to write to disk when Black is imported.

---

1. which is handled completely differently compared to the other caches Black maintains but still lives in the same directory

[VasuBhog]

I am still seeing this error with
papermill 2.2.2
black 20.8b1
Mac OS: 10.15.7

Is there any update or workaround for this error?
Upon papermill execution:
Generating grammar tables from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/blib2to3/Grammar.txt Writing grammar tables to /Users/xxx/Library/Caches/black/20.8b1/Grammar3.6.8.final.0.pickle Writing failed: [Errno 2] No such file or directory: '/Users/xxx/Library/Caches/black/20.8b1/tmpp31361aj' Generating grammar tables from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/blib2to3/PatternGrammar.txt Writing grammar tables to /Users/xxx/Library/Caches/black/20.8b1/PatternGrammar3.6.8.final.0.pickle Writing failed: [Errno 2] No such file or directory: '/Users/xxx/Library/Caches/black/20.8b1/tmpcq8qn401'

[AlJohri]

@VasuBhog As a work around, use the black CLI at least once locally or run: python -c "import logging; logging.basicConfig(level='INFO'); import black" once in your python environment (see https://github.com/psf/black/issues/1223 for details).