Mbed-cli: Random deploy problem

[JanneKiiskila]

Seems there is some weird things happening with mbed deploy, it's now failing. With mbed deploy --verbose it actually tells this:

jankii01@ubuntu:~/mbed/mbed-cloud-client-example-internal$ mbed deploy --verbose
[mbed] Working path "/home/jankii01/mbed/mbed-cloud-client-example-internal" (program)
[mbed] Updating library "pal-platform" to rev #bc8a1d84d7bf (tag: 2.2.0-RC1)
[mbed] Fetching revisions from remote repository to "pal-platform"
[mbed] Exec "git fetch --all --tags" in "/home/jankii01/mbed/mbed-cloud-client-example-internal/pal-platform"
Fetching origin
[mbed] Checkout "bc8a1d84d7bff7cb6a523fa03e353e23b4c2ae00" in pal-platform
[mbed] Revision "bc8a1d84d7bff7cb6a523fa03e353e23b4c2ae00" matches a branch "origin/HEAD" reference. Re-associating with branch
[mbed] Exec "git checkout origin/HEAD -q" in "/home/jankii01/mbed/mbed-cloud-client-example-internal/pal-platform"
[mbed] Updating library "mbed-os" to rev #a8f0c33eaa2c (tag: mbed-os-5.11.2)
[mbed] Fetching revisions from remote repository to "mbed-os"
[mbed] Exec "git fetch --all --tags" in "/home/jankii01/mbed/mbed-cloud-client-example-internal/mbed-os"
Fetching origin
From https://github.com/ARMmbed/mbed-os
 ! [rejected]              latest     -> latest  (would clobber existing tag)
error: Could not fetch origin

That actually explains why it goes wrong, but this info should be shown also w/o --verbose (rejected would clobber) as that explains what this is about.

I think the root cause is now the usage of a tag to indicate something that moves, which should not be done @adbridge @0xc0170 - tags are forever, you do not - ever - change them. Plus you do not make a tag and branch with the same name. That's a bug in git, if you ask me - it should decline to add either a tag or branch with the same name, if one already exists.

You can have a branch called "latest" and that can keep moving, but a tag cannot move. Ref: git documentation on tagging.

[JanneKiiskila]

Looking at the mbed-os via GitHub now shows we don't have a tag called latest anymore. But, all of the local copies/forks will have it and that seems to cause the issue.

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-822

[screamerbg]

@JanneKiiskila tag "latest" pointing to the latest Mbed OS 5 release has always been part of the release process of Mbed OS for the past almost 3 years. It's the first time we're receiving report that git refuses to fetch tags due to tags clobber. The fact that git provides the functionality to move tags says enough whether that's a "never" or a "up to your consideration" practice. Also, if you have comments about the release process of Mbed OS and what and how it's being released, I'd encourage you to raise this with the appropriate product owners and architects for this, not in the CLI tools repository where the tool serves the needs of the OS releases.

Regarding the issue, can you send me a copy of your codebase for further investigation?

[kjbracey-arm]

It's fixed behaviour in Git 2.20. Previously git fetch --tags would let tags move, even though plain git fetch didn't.

Release notes:

• "git push" into refs/tags/* hierarchy is rejected without getting
  forced, but "git fetch" (misguidedly) used the "fast forwarding"
  rule used for the refs/heads/* hierarchy; this has been corrected,
  which means some fetches of tags that did not fail with older
  version of Git will fail without "--force" with this version.

• The rules used by "git push" and "git fetch" to determine if a ref
  can or cannot be updated were inconsistent; specifically, fetching
  to update existing tags were allowed even though tags are supposed
  to be unmoving anchoring points. "git fetch" was taught to forbid
  updates to existing tags without the "--force" option.

So changing the command to git fetch --tags --force would make it go through and still work on older Gits.

Would be better to stop abusing the tags in the Mbed OS repo, but as a short-term tools fix.

[0xc0170]

Good catch with the release changes in Git.

Would be better to stop abusing the tags in the Mbed OS repo, but as a short-term tools fix.

We can work on this as a separate task how this can be avoided.

[ciarmcom]

Thank you for raising this issue. Please note we have updated our policies and
now only defects should be raised directly in GitHub. Going forward questions and
enhancements will be considered in our forums, https://forums.mbed.com/ . If this
issue is still relevant please re-raise it there.
This GitHub issue will now be closed.

[adbridge]

Note the 'latest' tag will no longer be used when we move to the new release model in January.