Zfs: Patch-release branch

[awehrfritz]

NOTE: This is more an feature/workflow request and not an issue with zfs _per se_, thus the issue template is not applicable and ignored here.

I am currently waiting for the next patch-release of the 0.8-series, i.e. 0.8.5, in the hope to get the fix from https://github.com/openzfs/zfs/pull/9967 (i.e. commit https://github.com/openzfs/zfs/commit/ab16c87e5560b5d8554386cfaded72e7f61465b8) into a released version and eventually into Debian/Fedora. However, it is unclear to me which changes/fixes/features will be considered for such a patch-release and I was wondering if there is a branch where all these things are queued-up for the next release. I found the zfs-0.8-release branch, but it seems that this branch is only updated once an actual release happens and it is unclear what the criteria are for changes to make it into a patch release.

I believe, this could be made more transparent, e.g. by introducing a branch for a particular version which only receives backwards compatible bug fixes as they are added to master branch (i.e. cherry-pick them). This allows one to keep track of what will make it into the next patch release which can then simply be done by tagging a particular version/commit.

I would appreciate if someone could clarify these issues and current development/release workflow.

This is somewhat related to https://github.com/openzfs/zfs/issues/10334, but I believe different enough to warrant a separate issue.

[awehrfritz]

I forgot to mention:
Currently, the zfs Debian packages for my 32-bit Armbian system (based on Debian Buster) are broken since the kernel is too new for the 0.7-series and the 0.8-series does not yet have a released version with the fix mentioned above.

I work around this issue by building my own packages based on the zfs-0.8-release with the additional commit that has the fix (https://github.com/openzfs/zfs/commit/ab16c87e5560b5d8554386c):

git clone https://github.com/zfsonlinux/zfs.git
git checkout --track origin/zfs-0.8-release 
git checkout -b zfs-0.8.4.git-ab16c87e
git cherry-pick ab16c87e5560b5d8554386cfaded72e7f61465b8

[gdevenyi]

This would indeed be an excellent workflow to add on top of #10334, as it would enable users to build "stable" bugfix-only snapshots. Its likely to also reduce the overall maintainer workload by merging in patches early on the branch instead of the all-in-one jobs the way releases are currently managed.

[eli-schwartz]

This is essentially identical to what I recommended here:
https://github.com/openzfs/zfs/pull/9161#issuecomment-534675049
https://github.com/openzfs/zfs/issues/9745#issuecomment-592638841
https://github.com/openzfs/zfs/pull/9961#issuecomment-613672623

I found the zfs-0.8-release branch, but it seems that this branch is only updated once an actual release happens and it is unclear what the criteria are for changes to make it into a patch release.

This has been a source of personal frustration to me. :(

[tonyhutter]

Yea, this has been an ongoing source of annoyance with users running the latest upstream kernels. I talked with @behlendorf about it, and we decided that the best solution would be to have a staging branch created for each upcoming release. That way, the community can open PRs against it, and everyone can get the back-ported build fixes much quicker than waiting for the point releases. It makes it easier for bleeding-edge kernel users to pull from that branch rather than having to manually cherry-pick the build fixes themselves. Eventually, we'll pull the staging branch patches and regular point release patches into the next point release.

I've just created a zfs-0.8.5-staging branch (https://github.com/openzfs/zfs/tree/zfs-0.8.5-staging). Feel free to open PRs against it :smile:

[eli-schwartz]

Nice. I'd like to backport 080102a1b686a36adaec3a11a9778783462429e8 but I ran into a very sticky problem... it modifies config/kernel-kmem.m4 but this file is not even distributed in the release tarball since the macros it defines in 0.8.4 are not even used so I presume autotools doesn't dist it.

There doesn't seem to be any way to create a patch that applies to git, but also applies to the tarball. It's the weirdest patch issue I think I've ever seen.

[tonyhutter]

@eli-schwartz feel free to modify the patch to make it work correctly in the branch. Maybe try modifying the kernel-kmem.m4 part to make the equivalent changes in whatever .m4 the staging branch uses, or just removing that section if it's not applicable. Or if it makes sense to include dependent patches to resolve the conflicts, you can include those in your PR as well.

[eli-schwartz]

Yeah, I just moved it to a new file. That way one patch can be downloaded and applied with patch -p1 regardless of whether the base is a release tarball or git checkout/snapshot.

[awehrfritz]

@eli-schwartz Thanks for pointing out your earlier comments on this, I have not seen them before - sorry for that.

Yea, this has been an ongoing source of annoyance with users running the latest upstream kernels.

Indeed, that is another area where I encounter issues every now and then. I guess my setup is a bit unfortunate in that I run Fedora Linux on my workstation, with /home on zfs, which typically runs the latest upstream kernels, and furthermore have a NAS system based on an ARMv7 32-bit SoC which is one of the less frequently used architectures for ZFS.

I talked with @behlendorf about it, and we decided that the best solution would be to have a staging branch created for each upcoming release. ... Eventually, we'll pull the staging branch patches and regular point release patches into the next point release.

That will do the trick. I would have thought that using the zfs-0.8-release branch directly to queue up the changes would be sufficient, but I might be missing something here. Having an additional staging branch works perfectly fine.

Thanks @tonyhutter for actioning this so quickly! Much appreciated!