Renovate: pip_requirements: Renovate does not parse unbound requirements

[alecgerona]

I've toggled my project on the dashboard with a repository that has this settings file already: renovate.json

{
  "extends": [
    "config:base",
    ":githubComToken(XXXX)"
  ],
  "pip_requirements": {
    "fileMatch": [
      "^requirements/.*\\.txt$"
    ]
  }
}

The webhook is created, user is added with the proper permissions etc. My requirement files are all under the requirements folder (base.txt, etc.). I've let it sit for a few days since I know about the existing issues Renovate has with Gitlab. But there's still no MR to onboard or pin my requirements (I've left them blank to let Renovate deal with them) even though jobs are being executed on time (I've also seen some updates on my gitlab-ci file and dockerfiles but still no MRs. The most recent job id to reflect my latest settings change is #214805857.

I've also confirmed that this isn't a stale repository since the initial commit was just pushed last week on a private organization in Gitlab. Hope this is just a silly mistake on my end.

[rarkins]

There's nothing unexpected about how the app is behaving so this is not an app issue.

Look at job 214805857. You can see the fileMatch matched 3 files however no dependencies were extracted. It would seem that maybe Renovate doesn't know how to parse your requirements files. Please try to reproduce the problem in a public github repo with an example file so that we can take a look.

[alecgerona]

Hmm. This is probably the effect of not pinning the requirements myself. I'll try to reproduce this in a test repository.

[alecgerona]

Hi @rarkins, I've pinned the dependencies myself and now it works. Looks like Renovate can't indeed pin unversioned dependencies by itself. Should this be documented or be turned into a feature request?

[rarkins]

Please reproduce first, as I'd like to understand why the file was ignored. Then it can be a feature request potentially

[alecgerona]

I tried it on this repository: https://github.com/alecgerona/unversioned-requirements-test
Same setup as before, all unpinned. No PRs generated.

[rarkins]

I have renamed and transferred to the main repo. To resolve this issue, we should parse/detect the requirements, although still not pin them. At least then the files and dependencies will all show up in logs.

Currently we do not pin unbound requirements in any language (including ones that allow for a value like *), so that needs to be a separate feature to consider, keeping in mind backwards compatibility. We only pin them if there is some range already existing, e.g. 1.*

[alecgerona]

Awesome. Let me know if I can contribute anything for this.

[rarkins]

Sure, you're welcome to submit a PR. We'd need a tweaking of our regex, it's defined here: https://github.com/renovatebot/renovate/blob/6ddda0fa4b95b3090d08fba9ce26db4f9cf640b1/lib/manager/pip_requirements/extract.ts#L15

[alecgerona]

Will definitely check this out. Can I be invited to the Slack channel for more info?

[rarkins]

Sure, send an email so we can invite you

[alecgerona]

Cool. I sent you an email.

[alecgerona]

Might be related to this, Renovate also doesn't parse requirements that has a version that doesn't follow semantic versioning. Great example is black code formatter. Its version is XX.YbZ.

[rarkins]

@alecgerona you may be able to get it to work by using a package rule to configure loose versioning for that package.

[alecgerona]

With something like

"packageRules": [
    {
      "packageNames": ["black"],
      "versioning": "loose"
    }
  ]

?
Will give it a try.

[alecgerona]

Hi @rarkins, been getting this:

{
            "depName": "black",
            "updates": [],
            "depIndex": 0,
            "warnings": [],
            "datasource": "pypi",
            "skipReason": "unsupported-value",
            "fromVersion": "19.10b0",
            "currentValue": "==19.10b0"
},

[rarkins]

Sorry, that makes sense. Loose versioning doesn't understand the == part as it's python-specific. I'm not sure why we reject this version string with our default pep440 versioning. Anyway, it's unrelated to this issue so I'd prefer not to complicate it with further discussion. Please raise a new issue for us to look into supporting this syntax with out pep440 versioning scheme.