Collectd: The source packages of collectd-5.9.2 aren't generated by the formal procedure

[ashie]

• Version of collectd: collectd-5.9.2 on git

Expected behavior

version-gen.sh script should generate "5.9.2" on collectd-5.9.2 tag.

Actual behavior

version-gen.sh script generates "5.9.1.7.gdfb9dd0 on collectd-5.9.2 tag.

Steps to reproduce

$ git clone https://github.com/collectd/collectd.git
$ cd collectd
$ git checkout collectd-5.9.2
$ ./version-gen.sh

Cause of the issue

collectd-5.9.2 tag isn't annotated.

[faxm0dem]

I can add that the file hasn't been uploaded to collectd.org either (the spec file depends on that)

[ashie]

I've confirmed that it can be fixed locally by adding the annotation:

git tag -f collectd-5.9.2 -m "Tag for version 5.9.2"

[ashie]

Now I'm wondering how were collectd-5.9.2 packages generated?

In general, source packages for Autotoolized software should be generated by:

$ make dist

but they doesn't seem use it, because:

• They include garbage files such as autom4te.cache.
• PACKAGE_VERSION in the configure script in collectd-5.9.2.tar.bz2 is still '5.8.1.git'

[hnyman]

@mrunge @octo

collectd-5.9.2.tar.bz2 is not a bzip2 file, but instead it seems to be a .xz file which has an .tar file inside. (so instead of a tar bzip2 archive, we got a tar archive inside an xz archive.)

This leads to errors in real life usage:

bzcat: /Openwrt/r7800/dl/collectd-5.9.2.tar.bz2 is not a bzip2 file.
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

And like mentioned above, the file contains several megabytes of autom4te.cache etc.

[pseiderer]

And another shortcoming since the 5.9.x source release tar files, the (generated) file src/liboconfig/parser.c is missing, forcing a local bison installation (or host-bison in case of cross-compile, as detected by the buildroot autobuilds, see [1] for details)..., is this a 'bug' of the tarball or intentional?

[1] http://lists.busybox.net/pipermail/buildroot/2019-October/261555.html

[rpv-tomsk]

All these issues are bugs, because release process done by another person than before.
All that is done w/o explicit permission from project "owner", so results are expected.

[faxm0dem]

this is starting to get really weird @mrunge your email is bouncing back...

[mrunge]

Oh oh. sigh. Bear with me, I have been a few days cut off the net.

I'm sorry this happened, it has not been intentional.

Let's see what can be done to fix this here.

[mrunge]

So, I've updated the tarball on the release page here on github.

[ashie]

Thanks for the work :smiley:
BTW I think It's a good timing to add a document about the release procedure...

[hnyman]

There are currently different files .tar.bz2 to be downloaded at different locations leading to uncertainly on automated downloads:

https://collectd.org/files/collectd-5.9.2.tar.bz2
(hash dfcb2a2fa7de0ab02c9e6c1457bee2069957d4ffc9b428851661e9c5e5fc35b7)

https://github.com/collectd/collectd/releases/download/collectd-5.9.2/collectd-5.9.2.tar.bz2
(hash 917c483608b9b38438b121737b510c3d68f335c091bc286aa6ebcc0c8e372a09)

$ make package/collectd/download V=s
make[1]: Entering directory '/Openwrt/r7800'
make[2]: Entering directory '/Openwrt/r7800/feeds/packages/utils/collectd'
mkdir -p /Openwrt/r7800/dl
SHELL= flock /Openwrt/r7800/tmp/.collectd-5.9.2.tar.bz2.flock -c '      /Openwrt/r7800/scripts/download.pl "/Openwrt/r7800/dl" "collectd-5.9.2.tar.bz2" "7b220f8898a061f6e7f29a8c16697d1a198277f813da69474a67911097c0626b" "" "https://collectd.org/files/" "https://github.com/collectd/collectd/releases/download/collectd-5.9.2"    '
...
  1500K .......... .......... .......... .....                100% 29.1M=0.2s

2019-10-09 17:06:38 (6.44 MB/s) - written to stdout [1572628/1572628]

Hash of the downloaded file ... dfcb2a2fa7de0ab02c9e6c1457bee2069957d4ffc9b428851661e9c5e5fc35b7

+ wget --tries=5 --timeout=20 --no-check-certificate --output-document=- https://github.com/collectd/collectd/releases/download/collectd-5.9.2/collectd-5.9.2.tar.bz2
--2019-10-09 17:06:38--  https://github.com/collectd/collectd/releases/download/collectd-5.9.2/collectd-5.9.2.tar.bz2
Resolving github.com (github.com)... 140.82.118.4
...
  1750K .......... .......... .......... ........             100% 15.3M=0.9s

2019-10-09 17:06:40 (2.05 MB/s) - written to stdout [1831777/1831777]

Hash of the downloaded file ... 917c483608b9b38438b121737b510c3d68f335c091bc286aa6ebcc0c8e372a09

Please upload the correct file also to to collectd.org (if it is indeed the correct file. I haven't yet tested it as I stumbled at the download step.)

[mrunge]

I don't have access to collectd.org, and tbh, I am astonished, that collectd-5.9.2 shows up there.
I rebuilt the tar.bz2 file 2 days ago and replaced the file on github with the newer one.

sha256sum collectd-5.9.2.tar.bz2 
917c483608b9b38438b121737b510c3d68f335c091bc286aa6ebcc0c8e372a09  collectd-5.9.2.tar.bz2

[mrunge]

Just as a pointer, the release process is documented https://collectd.org/wiki/index.php/Release_process

[hnyman]

Maybe the process description is not quite complete, or it was not followed right:
The collectd 5.9.2 binary still claims to be 5.9.0 when built from collectd-5.9.2.tar.bz2 sources, as the file version-gen.sh in the 5.9 branch still says 5.9.0. It is not enough to bump the version in master, but it needs to be bumped also in the stable branch from which the release is made.

https://github.com/collectd/collectd/blob/collectd-5.9/version-gen.sh

Example:

root@router2:/tmp# collectd -?
collectd: unrecognized option: ?
Usage: collectd [OPTIONS]
...
  Data directory    /var/lib/collectd

collectd 5.9.0.git, http://collectd.org/
by Florian octo Forster <[email protected]>
for contributions see `AUTHORS'

It might be sensible to first fix things in the 5.9 branch and then release a new 5.9.3 that would contain the proper stuff and proper versioning. Then there wouldn't be competing source version floating around in the various foss source mirrors. E.g fossies.org has 5.9.2 now the first version of the 5.9.2 tar.xz (the first bz2 file properly identified?) that still claims to be 5.8.1 in the version-gen.sh
https://fossies.org/linux/privat/collectd-5.9.2.tar.xz/
https://fossies.org/linux/collectd/version-gen.sh

(Less than funny, but source jump from 5.9.0 to 5.9.2 decreases the version to 5.8.1...
https://fossies.org/diffs/collectd/5.9.0_vs_5.9.2/version-gen.sh-diff.html
They got the original faulty version of 5.9.2 ... )

Ps. not sure if the advice is still relevant/current, but there is advice about building the bzip2 distribution archive in https://github.com/collectd/collectd/blob/e1325d5ebebfcfc38d93ed453d01af24de4eb343/contrib/redhat/collectd.spec#L18
(that might also explain why the bz2 file still contains slightly different stuff in 5.9.2 than in 5.9.0).

[zebity]

@mrunge thanks Matthias missed this rather long trial...

BTW I came across the same issue with version-gen.sh file in the pkg-debian build as well.

As per @ashie the fix was to put in annotated tag:

git tag -a collectd-5.9.2-1 -m "Add 5.9.2 tag"

The version-gen.sh (DEFAULT_VERSION) is really a red herring as process always uses "git describe" result for version info.

I have just verified on my test build which is now generating: collectd-5.9.2.1.tar.bz2

Final point you have to explictly push tag...

git push origin collectd-5.9.2-1

(which you likely know, but I didn't and keep wondering where they where going ;-) )

[faxm0dem]

Signed tag works too