Black: Black fails due to incompatible cache format

[serhiy-storchaka]

Currently (19.3b0) Black uses the highest protocol for pickling the cache. In Python 3.8 it is 5. Protocol 5 is not supported on versions < 3.8. As a consequence, after using Black on 3.8 it becomes unusable on older Pythons.

Possible solutions:

• Use the default protocol for pickling.
• Include the protocol number or the Python version in the name of the cache file.
• Ignore unpickling errors and continue as there is no cache.

These options can be applied together.

[cs01]

I just ran into this. The workaround for now is to remove the cache. Cache locations are described in black's readme and vary by os. See https://github.com/psf/black#ignoring-unmodified-files.

[zsol]

This was also bugging the hell out of me while working on #943 and #711 so I might get around to fixing it soon

[vaneseltine]

https://github.com/psf/black/pull/1034 adds a test and fixes the issue.

I'd be tempted to make a bare except here, so that we simply return {} if there's any problem at all trying to read the cache. That would be a somewhat lazier but more future proof approach. Instead, I stuck with the current issue and added ValueError to the exceptions that read_cache will properly handle.

[wimglenn]

Exact error message to help google:

ValueError: unsupported pickle protocol: 5

Invalidating caches:

• linux rm -rf ~/.cache/black
• macOS rm -rf ~/Library/Caches/black
• winblows rmdir /Q/S "%HOMEPATH%/AppData/Local/black"

[vaneseltine]

Good tip.

Also possibly worth noting, in case of any odd problems, that the exact cache directory being used can be pulled from black's constants:

import black
print(black.CACHE_DIR)

[zsol]

This shouldn't be necessary anymore

[TomFaulkner]

This shouldn't be necessary anymore

Ran into this today on a just installed black 19.3b0 on Python 3.8.0.

[vaneseltine]

This shouldn't be necessary anymore

Ran into this today on a just installed black 19.3b0 on Python 3.8.0.

This is fixed in the latest stable release, 19.10b0. (19.3b0 was released in March.)

For many different reasons people can't or don't use the latest versions, so having the removal information seems handy to me. :smile:

[TomFaulkner]

I had just pip installed it without pinning a version. Weird that I got an out of date version. I'll reinstall when I'm on that machine and see if I get the up-to-date version.

[cooperlees]

I believe this is all fixed.