Shards: git command failed: git ls-tree

[kostya]

name: bla
version: 0.1.0

dependencies:
  http_parser:
    github: kostya/http_parser.cr

license: MIT

crystal deps
git command failed: git ls-tree -r --full-tree --name-only v0.8 -- shard.yml (). Maybe you didn't install git?

git --version
git version 2.8.3

if i add line branch: master it works.

[ysbaddaden]

The problem are the tags. Shards is looking for a v0.8 tag but it can't be found because it's 0.8 (missing a leading v). Specifying a refs (branch, commit or even tag) works because the refs will exist, and won't be the expected version tag.

I tried to support both namings (with or without a leading v), but it complicates the resolver.

[jhass]

Might still be worth to attempt to improve the error output here.

[ysbaddaden]

Definitely.

I don't think we should parse the Git message. Different versions may have different messages. Maybe buffer and display the command output in addition to the command?

[jhass]

Yeah, would break on localization anyhow.

Also I think a general "does git work" test before the first of any git operations with an appropriate error message would be good, and then changing the error message from "does your git work?" here to something else.

[kostya]

why not add branch: master by default?

[ysbaddaden]

Because shards target the last release by default.

It couldn't find version tags, but it found the shard.yml in master that specifies a version, and thus it tried to find this version. Maybe this is an odd behavior of an edge case (the version tags should exist).

[straight-shoota]

The error message was improved a little bit, but the actual issue still exists: shards breaks when a git tag doesn't start with v.

As the issues linking here show, this is an ongoing problem. And it's not just the shard's maintainer's fault (they could just use v tags), because shards accepts plain tags but can't treat them properly.

When reading the versions from git tags, in Shards::RELEASE_VERSION the leading v is optional and stripped off. However, #git_refs(version) prepends a v unless the version already starts with it (which should rarely be the case because it's stripped off).

This could be solved by enforcing git tags to start with v. This would at least fail clean and could provide a meaningful error message. It's not a good solution, though, because people are using plain git tags without v prefix. They're also already in the history of many shards.

I think this could be solved by storing the version as a git tag in it's original form, without stripping, and only convert to a non-prefixed string when it's used externally. Maybe a simple GitTag wrapper struct could be introduced to make this relatively straightforward.

[RX14]

Supporting both means that there can be two tags vx.x.x and x.x.x which point to different commits. Which one should be used? Only one should be supported, and this should break sooner rather than later.

[straight-shoota]

I don't think rivaling tags pointing to different commits is a real issue. If that is the case, it's surely the maintainers fault and shards can't be blamed for picking either of them. If there are both vx.x.x and x.x.x the only sane explanation is that they point to the same commit.

The better part of all repositories consistently use only one tag scheme. Some may switch from one to the other, but then each version tag is still unambiguously defined.

I happen to have collected data of around 400 most popular shards. Of these, 289 have a tag starting with v, 12 have a tag starting with a digit and the rest doesn't seem to have a tag (as reported by git describe --tags). So people tend to prefer v prefix, but it's not a singleton.

I'm not sure if shards should enforce a git tagging scheme.

Allowing an optional v prefix doesn't seem like a huge burden.

On the other hand, there needs to be some rules for what is considered a version tag (i.e. matching regex /^v?([\d\.]+)$/), so it's not a big difference if the requirements were a little bit stricter.

[RX14]

If that is the case, it's surely the maintainers fault and shards can't be blamed for picking either of them.

However, it would hugely help if shards told them sooner rather than later if they messed up tagging. It's probably easier to do than you think.

[straight-shoota]

Yeah, maybe it's better to just be strict. I realized that it's not just about keeping the output from git tag --list in it's original form internally, but also when interfacing with external version specification. That means everywhere a tag is expected would need to try again with the other variant if the first one fails.

Maintainers can simply add v-prefixed tags. We could even provide a simple script for doing that automatically.

In this context, perhaps a shards validate tool would be helpful which could verify that tags follow the expected format and match the version specified in spec.yml. Plus maybe additional checks. That's two steps ahead, though.

[straight-shoota]

Can this issue be re-opened?

[ysbaddaden]

No. A shard version MUST be represented with a Git tag starting with a v.

[ysbaddaden]

72% of the shards you reviewed have the leading v, and only 3% of them didn't have it. It's not worth supporting, and yes, Shards can enforce the leading v.

We can have a validate command. It could review the Git history, checking for mismatches between Git tags and commited shard.yml, even validate shard.yml as per the SPEC... but that's another topic.

[straight-shoota]

@ysbaddaden I'm fine with enforcing it. The issue is, shards currently does not enforce the v prefix.

Shards recognizes tags without v as valid version identifiers but then can't use them as valid ref for accessing it in the git repo. That leads to an error as shown in the OP.

To fix this, in RELEASE_VERSION regex 'v' needs to be present unconditionally.

[ysbaddaden]

Oh, that's broken, indeed, but not really the scope of this issue. Can you open a new issue, or just open a pull-request, if you have a solution already?