semgrep 0.14.0 python API issues

[ajinabraham]

Describe the bug
I am trying to upgrade to semgrep 0.14.0.

Settingquiet=True is not respected by the progress bar and initial message, runningxrules.
Ref: https://github.com/returntocorp/semgrep/blob/develop/semgrep/semgrep/semgrep_main.py#L250
Also, this will trigger an exception.

Example:

    io_capture = StringIO()
    output_handler = OutputHandler(
        OutputSettings(
            output_format=OutputFormat.JSON,
            output_destination=None,
            quiet=True,
            error_on_findings=False,
            strict=True,
        ),
        stdout=io_capture,
    )
    semgrep_main.main(
        output_handler=output_handler,
        target=[str(t) for t in targets],
        pattern="",
        lang="",
        config=str(config),
        **kwargs,
    )
    output_handler.close()
    return json.loads(io_capture.getvalue())

When quiet=Truesemgrep shows running x rules message and progress bar and triggers and exception,

running 73 rules...
  8%|█████▊                                                                |6/73
 in _invoke_semgrep
    return json.loads(io_capture.getvalue())
  File "/xxx/.pyenv/versions/3.7.7/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/xxx/.pyenv/versions/3.7.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/xxx/.pyenv/versions/3.7.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It looks like when quiet=True, io_capture.getvalue() returns and empty string. Everything works as expected when quite is set to True

Environment
0.14.0 Pypi

[dlukeomalley]

Thank you for filing Ajin. On the surface this looks related to #1228. I'll mark it as a duplicate for now and make sure we look into both.

[brendongo]

Root cause: running semgrep as cli sets some global verbosity flags that this API code path does not set. These types of issues will be fixed when we get https://github.com/returntocorp/semgrep/pull/1188 over the line but for now I will add the flag setting code to this function to fix.

[brendongo]

Actually adding

import semgrep.util
semgrep.util.set_flags(False, True, False) # Verbose, Quiet, Force_color

before calling invoke_semgrep should get you desired functionality.

I'll work on getting #1188 overthe line so there's no weird global like this.

[dlukeomalley]

@brendongo Are we good to close this now that #1188 is closed?

[brendongo]

Let me double check that this is also fixed. A little different since the entry point here is the API not the cli

[brendongo]

invoke_semgrep should now no longer print anything to stderr or stdout and added a test that enforces this