Renovate: Support for Crystal & Crystal Shards

[alexanderadam]

What would you like Renovate to be able to do?

Using Renovate with @crystal-lang and it's dependency manager Shards. You can find its documentation here.

Describe the solution you'd like

Renovate should work like it does with other dependency managers.

Describe alternatives you've considered

Dependabot, but it doesn't support Crystal neither.

[rarkins]

Datasource: seems like they are git, github, bitbucket or gitlab references, so we shouldn't need any new datasources at least in the initial drop.

Versioning: Looks like it supports a subset of Ruby versioning as per the description in https://github.com/crystal-lang/shards/blob/master/SPEC.md#version

Manager: Seems like a simple YAML parsing extract should be all it takes. No lock files.

[rarkins]

@alexanderadam could you create a public repo that contains an example shard file with the various use cases you'd expect to work? Ideally each of them should be out of date so that it results in some PRs.

[alexanderadam]

Wow, you answer fast! Awesome! :pray:

Seems like a simple YAML parsing extract should be all it takes.

Well, at least dependencies are just written in YAML. I'm not sure what it needs otherwise.

No lock files.

What do you mean? There are lock files.

They are also documented in the shards reference?

could you create a public repo that contains an example shard file with the various use cases you'd expect to work?

Would some of the examples at github be sufficient as well?

What exactly do you mean by "use cases"?
I guess @ysbaddaden, @waj, @straight-shoota, @bcardiff and the other contributors could help out in case you're thinking of particular shards features as "use cases".

[rarkins]

OK, I missed the lock files part. Makes it a little more complex but we've done it plenty of times before.

Examples are useful, but to proceed we'd want a hand-crafted repo that deliberately includes all the use cases we intend to support, has each deliberately out-of-date, etc.

[alexanderadam]

to proceed we'd want a hand-crafted repo that deliberately includes all the use cases we intend to support, has each deliberately out-of-date, etc.

Sorry but I'm still struggling what other use cases except out-of-date are relevant.
And for out-of-date dependencies, probably _any_ old repo would do, right?

So for instance:

1. I searched for old lock files
2. Found & checked out this old geoip repository: izikaj/geoip
3. Called shards install to install the outdated dependencies and then called shards update to see this:

Afterwards the lockfile should be updated accordingly:


It would really help if you could elaborate what use cases (other than the standard out-of-date one) you need.

Thank you in advance! :pray:

[rarkins]

What I mean is a shard file that deliberately has all the different types of dependencies defined (git, github, gitlab, using an exact version vs ranges, etc). And then a lock file to match of course. It's hard to develop initially against a messy "real" repository instead of one that's been deliberately created to test all use cases.

[rarkins]

1. Create a new repo
2. Create a shard file with no dependencies
3. Add dependencies until you've covered the different types defined in the spec
4. Try to make sure each dependency is out of date so that we have something to test

[alexanderadam]

Thank you for the explaination.

I created this repository, which has only outdated dependencies. But some the old dependencies are currently fixed but renovatebot would increase a dependency in the shard.yml automatically, right? Otherwise I would have to update the versions again.

Just tell me, in case I should change anything or in case I misunderstood something.

[rarkins]

At a quick glance, that looks perfect. Renovate will update the yml and then run the lock command to update it accordingly in the same commit.

[bcardiff]

For a start shards has an outdated command. I guess you should try to consume that data. Maybe there needs to be a data friendly format instead of human friendly output.

If there needs to be more integration that the one offered by outdated command I would have in mind that:

• Maybe shards could include a command to list versions of a package. That is mostly listing tags. Shards already has a cache to improve the time consumed on that task. Plus, there are some further optimization that might come in the future.

• A difference shards has from bundler is that you can have dependencies along the whole tree against branches and not only released versions via tags. This allows for example to depend on fresh shards that are not yet released.

[rarkins]

Thanks for the info. We always prefer doing our own version lookups and resolution anyway for flexibility, so we don't need the outdated command output for now.

• A difference shards has from bundler is that you can have dependencies along the whole tree against branches and not only released versions via tags. This allows for example to depend on fresh shards that are not yet released.

By this do you mean you can depend on any git ref such as branches? Are those unreleased fresh shards ever versioned? Because if they are not versioned, people probably wouldn't want us interfering and trying to upgrade them anyway.

[bcardiff]

By this do you mean you can depend on any git ref such as branches?

Yes

Are those unreleased fresh shards ever versioned?

Maybe. It depends on the author. If the requirement in shard.yml mentions a branch, maybe you want to know if the lock file is outdated regarding the branch. It's a corner case definitely.

There is also no central repository of shards. There is no ruby-gems equivalent.

---

Usually library shards do not check in the shard.lock file. There are different opinions why/if this should be done. That's probably the reason why you didn't notice a shard.lock in the first place.

[waghanza]

I'm also interested in crystal. I think, I'll try to contribute to this PR.

I used dependabot once, but I'm not sure it the lock files should be parsed @rarkins (I've no lock in the project I need renovatebot https://github.com/the-benchmarker/web-frameworks)

[rarkins]

It's not essential to parse lock files. It's even ok to not have lock file updating capabilities initially, however in that case we'd make crystal disabled by default so that only those without lock files can opt-in to using it. Adding lock file updating afterwards should not be too hard.