semgrep fails when Python file has trailing indentation

Created on 17 Mar 2020 · 7Comments · Source: returntocorp/semgrep

When there is trailing tab in a Python file semgrep fails.

Failing: https://sgrep.live/5J3
Trailing tab removed: https://sgrep.live/JjQ

Expected behavior
Expected semgrep to succeed, or error and clearly told me the Python syntax was invalid.

bug external-user good first issue python

Source

dlukeomalley

Most helpful comment

@jajajasalu2 I think your PR in returntocorp/pfff#102 likely fixes this. 🎉

DrewDennison on 3 May 2020

🎉2

All 7 comments

@jajajasalu2 I think your PR in returntocorp/pfff#102 likely fixes this. 🎉

DrewDennison on 3 May 2020

🎉2

@DrewDennison unfortunately no, that PR is in but https://semgrep.live/5J3?version=develop still fails to parse

ievans on 6 May 2020

FWIW, when I run semgrep manually on this file from the command line, everything is happy. @DrewDennison is it possible this is an issue with semgrep.live?

nbrahms on 28 May 2020

I used semgrep-core as of 5df168ab0550c

nbrahms on 28 May 2020

Are you sure? Running on https://github.com/returntocorp/semgrep/pull/843/commits/119d3ce4d22979bdbf823affea87d14bd592691c which should be the same as 5df168ab0550c still gets the parse error

brendongo on 28 May 2020

https://semgrep.live/5J3?version=119d3ce4d22979bdbf823affea87d14bd592691c for convenience

brendongo on 28 May 2020

Hi,

Just my two cents on this: https://github.com/returntocorp/pfff/pull/102 does fix an underlying lexing issue but not this bug. The last few tokens for the erroneous code are

INDENT
NEWLINE
DEDENT
NEWLINE
EOF

For which I don't see a corresponding production in the python parser. Python 3.7 doesn't give me a syntax error either, so I guess this should parse.