Black: INTERNAL ERROR: Black produced code that is not equivalent to the source.

[GriceTurrble]

Working on the tests in one of my projects, converting some from using unittest to pytest. While adding some fixtures and moving things around, Black reported it could no longer format one of the files.

To Reproduce This commit contains the file tests/apis/inboundshipments/test_requests.py. Run Black with all default arguments, and see an error pop up.

(direct link to the file in that commit)

Expected behavior No error, reformat or leave alone.

Environment (please complete the following information):

• Version: 19.10b0 installed from PyPI
• OS and Python version:
  
  
  
  • locally: Windows 10, Python 3.8.5
  
  
  • externally: we run GitHub Actions with a formatting check, using ubuntu-latest and latest Py3 they have available. This workflow run shows the same error (the workflow code currently runs pip install black).
  
  

Does this bug also happen on master? Checked with online formatter, and no, the same bug does not occur. Good to know!

Additional context Seems like a new release would fix it, or I could change our test suite and my local tools to target latest from your master branch. Any insight on exact cause of the problem would be good to know, so I can watch out for anything I may be introducing.

Thank you.

[JelleZijlstra]

Thanks for the report! Since this is fixed on master and we're planning to have a new release soon, I'm not sure there's anything else for us to do.

It would be helpful to get a smaller repro case for issues like this—the file you link is more than a thousand lines long. That would make it easier for us to see if it's similar to a previously reported crash.

[ichard26]

Simple repro case:

class somerandomclass:
    # fmt: off
    @test()
    # fmt: on
    def foo(): 
        pass

def thisIsAFunction():
    pass

And this was the bad code in the 1000+ lines file:

    # fmt: off
    @pytest.mark.parametrize("address", [
        {},  # empty dict
        "this is not a dict",  # not a dict at all.
    ])
    # fmt: on
    def test_address_raises_exceptions(self, address, inboundshipments_api):
        """Empty address dict should raise MWSError."""
        with pytest.raises(MWSError):
            inboundshipments_api.set_ship_from_address(address)

    # fmt: off
    @pytest.mark.parametrize("address", [
        # name missing
        {"address_1": "500 Summat Cully Lane", "city": "Gilead"},
        # address_1 missing
        {"name": "Roland Deschain", "city": "Gilead"},
        # city missing
        {"name": "Roland Deschain", "address_1": "500 Summat Cully Lane"},
    ])
    # fmt: on
    def test_required_address_keys_missing(self, address, inboundshipments_api):
        """Any missing required key should raise MWSError"""
        with pytest.raises(MWSError):
            inboundshipments_api.set_ship_from_address(address)

If you remove this chunk in the file, the file formats correctly and without error. The problem seems to be that the fmt: off and fmt: on are unlinked. This causes the fmt: off comments to apply everywhere and this in itself causes misbehaviour, look at https://github.com/psf/black/issues/569.

Doing a git bisect, the commit that fixed this was 892eddacd215d685e136686b7f629ade70adca83 from PR https://github.com/psf/black/pull/1325 ... how aptly named! Also this bug was already filed as https://github.com/psf/black/issues/560.

[GriceTurrble]

I was just about to mention that. Removing my # fmt flags near the decorators allows it to format properly.

Thanks @ichard26 !