Super-linter: How can I pass command line arguments to Pylint in Azure DevOps?

[Connell1]

I am running Super-Linter in an Azure DevOps pipeline and passing the VALIDATE_PYTHON_PYLINT environment variable to perform linting using Pylint. When I run the pipeline, I get a load of errors of the form E0401: Unable to import 'pyspark.sql' (import-error). According to the Pylint documentation, module names should be passed as command line arguments. Is there some way I can pass command line args from Super-Linter to Pylint?

More details is this Stack Overflow question.

[admiralAwkbar]

@Connell1 Hey! thanks for the question...
It seems the option would be to ignore the import errors in the following way:

import someModule # pylint: disable=import-error

Or you could ignore them in the .pylintrc

ignored-modules=module1,module2,...

We are currently looking at ways to help users pass more command lines to the linters to help with corner cases and look to have something in the near future.

thanks!

[GaboFDC]

Also, you can see related #157 to see how it was solved in GitHub actions, maybe you can extrapolate from there to solve in Azure.

[Connell1]

Thanks for the responses. I tried the work around suggested in #157 but haven't been able to get it working. I have the following code in my pipeline. It still fails to find the python modules so I think my value for the PYTHONPATH variable in the last line is wrong. Can anyone let me know what it should be please?

jobs:
  - job: PythonLint
    displayName: Python Lint
    pool:
      vmImage: ubuntu-latest
    steps:
    - script: sudo apt-get install -y --no-install-recommends python3-venv python3-setuptools
      displayName: Install missing python packages

    - script:  |
        python3 -m pip install pipenv
        python3 -m pipenv sync --python 3.6
        python3 -m pipenv install --dev
        python3 -m pipenv lock

      env:
        PIPENV_VENV_IN_PROJECT: 1
      displayName: Install dependencies with pipenv

    - script:  mv .venv /home/vsts/work/_temp/_github_workflow
      displayName: Move the dependencies

    - script: |
        docker pull github/super-linter:latest
        docker run -e RUN_LOCAL=true -e VALIDATE_PYTHON_PYLINT=true -v $(System.DefaultWorkingDirectory)/Azure/Integration:/tmp/lint github/super-linter  -v PYTHONPATH="/github/workflow/.venv/lib/python3.6/site-packages"
      displayName: 'Code Scan using GitHub Super-Linter'

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[alerque]

@github-actions You are being too aggressive, this question hasn't gotten a satisfactory answer and is really an outstanding problem in this project that needs fixing. Closing the issue won't make the problem go away.

[GaboFDC]

I think you can fix it by the lint rules file, also try setting PYTHONPATH as specified on the workarround in #157

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[alerque]

No @github-actions this issue is not stale, the project is stale. The maintainers' last documented stance on this is:

We are currently looking at ways to help users pass more command lines to the linters [...]

There still isn't a provision for the need here other than really ugly work arounds.

[GaboFDC]

You can use the linter file to specify the rules you want to avoid, yo don't necesarely have to doit by the CLi args.
Also as an open sopurce project any PR adding more funtionalities is higly welcomed.

[Connell1]

It is not enough to just turn off some individual rules. If I can't import a module and I just disable the import-error rule, I will still get errors on every line that calls a function from the imported module.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[alerque]

Wasn't stale. Until the stale-bot is smart enough to notice when a project collaborator is not the last person to comment and takes that into consideration, it shouldn't be allowed to go around closing things.