Super-linter: 'VALIDATE_ALL_CODEBASE: false' does not work

[dshevtsov]

Describe the bug

In my workflow, I want to scan only new or edited files, hence I have had set VALIDATE_ALL_CODEBASE: false (the setting).

When the workflow runs on a pull request (PR) that contains changes to .md files only, it fails on .js files that have not even been expected to be scanned, because the PR does not contain any changes to .js files.

Expected behavior

When the workflow with the VALIDATE_ALL_CODEBASE: false settings runs on a pull request, it scans only new or edited files as stated in technical documentation.

Steps to Reproduce

Adding a link to real case at public repository in the next clause.

Logs

https://github.com/magento/devdocs/pull/8474/checks?check_run_id=1651122030

Additional context

[GaboFDC]

Maybe this is a case of documentation not been clear or missing.
The way we know what files to check when VALIDATE_ALL_CODEBASE=false is to compare the current branch with the DEFAULT_BRANCH. So in this case the target of the pr and the default branch are different, so the changed files, are not the same.

[dshevtsov]

Maybe this is a case of documentation not been clear or missing.

@GaboFDC yes, I think this is the case. My expectations were that the tool would've checked changes between the Base branch and the Head branch of the pull request.

So basically the tool allows testing only pull requests to a single branch, which is the default branch of the repository.
I cannot use the tool for changes between other branches. Correct?

[iainlane]

Is this a restriction that is part of the design of super linter, or would you accept a PR to change this to check out the target branch? I'm new to actions, but from reading the docs it looks like github.base_ref could be used as a source to diff against, for pull requests.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.