Zfs: dkms modules not built automatically for Fedora update or upgrade

[array42]

Because of this Issue I did not upgrade to Fedora 27: https://github.com/zfsonlinux/zfs/issues/6814

The Upgrade from 25 to Version 26 did install the correct Version of zfs, but it did not build the dkms modules. I remember that I had to rebuild them once already manually after a normal upgrade.

Some commands if you face the same situation:
dkms status # nothing listed
dkms autoinstall # did nothing
(dkms remove ... if an old module exists in case of a normal update)
dkms add -m spl -v 0.7.3
dkms add -m zfs -v 0.7.3
dkms install -m spl -v 0.7.3
dkms install -m zfs -v 0.7.3

[stewartadam]

I've had the issue on upgrades within a release as well. Can't figure out what triggers it, but removing and re-adding as you do (& per the commands included in the zfs and spl %script sections) fixes it.

[alexmarkley]

I can confirm this is still an issue. I followed the recommended distribution upgrade path from Fedora 27 -> Fedora 28 and DKMS seemed to have "forgotten" about spl and zfs.

So when booting normally with the new FC28 kernel, dkms autoinstall does nothing and the pool cannot come up.

In order to bring things back to life, I had to:

dkms add -m spl -v 0.7.9
dkms add -m zfs -v 0.7.9
dkms autoinstall

[mskarbek]

It's DKMS fault. Basically, it can't handle the situation when you are updating both kernel and dkms module at the same time. During a normal update, you just blacklist the kernel (--exclude=kernel\*) and then update it separately. Unfortunately, this can't be done when jumping to the next OS release.
To be honest that's the reason I've switched from dkms to self-build kmod packages on Fedora.

[tonyhutter]

There's a similar issue to this for a F28 -> F29 upgrade described in: https://github.com/zfsonlinux There's a good reproducer in there too: https://github.com/zfsonlinux/zfs/issues/8089#issuecomment-436440183.

I'm changing the title of this issue to reflect that this isn't just a F25-> F26 issue:

- dkms modules not built automatically for Fedora update (or upgrade from 25 to 26)
+ dkms modules not built automatically for Fedora update or upgrade

[Lalufu]

Having squinted at the {zfs,spl}-dkms rpm package scripts for a bit I don't see a good way to solve this in its current form with the tools that RPM gives you.

Upgrading from X.fc28 to X.fc29 runs

%pretrans X.fc29
%pre X.fc29
%post X.fc29 (1)
%preun X.fc28 (2)
%postun X.fc28
%posttrans X.fc29

(trigger scripts omitted, since I believe there are no triggers involved in this)

(1) will build new modules from the code installed from X.fc29, while (2) will remove modules built in the past from the code installed from X.fc28. If X is the same for both we end up with no modules installed.

I don't see a way to solve this in the current script structure, since there's no real way to test for this in %preun X.fc28. Instead, could we move the building of new modules to %posttrans X.fc29? At this point in the transaction only a single version of {zfs,spl}-dkms should be installed, and it doesn't matter if %preun X.28 cleared out the old modules.

[tonyhutter]

@Lalufu excellent analysis. I've been looking into it further and wanted to add a few notes to what you wrote:

%pretrans X.fc29
%pre X.fc29
%post X.fc29 (1)
   - This is the step where fc29 gets built by dkms.  
     As part of the build, dkms automatically removes the previous
     modules before building the new ones.  It then builds the new
     modules.
%preun X.fc28 (2)
   - At this point, X.fc29 is be built and installed, but since it has the same X, 
     it's files get inadvertently nuked by fc28's uninstall.
%postun X.fc28
%posttrans X.fc29

The fix may be to update %preun X.fc28 (2) to see if we're upgrading or uninstalling. If we're uninstalling, then remove our files. If we're upgrading then do nothing, since will know dkms will have already removed our files in %post X.fc29 (1). I tested the following with spl-dkms, and it seems to fix the issue:

diff --git a/rpm/generic/spl-dkms.spec.in b/rpm/generic/spl-dkms.spec.in
index a8691d2..db7e2d4 100644
--- a/rpm/generic/spl-dkms.spec.in
+++ b/rpm/generic/spl-dkms.spec.in
@@ -78,6 +78,14 @@ echo -e "support or upgrade DKMS to a more current version."
 exit 1

 %preun
+if [ $1 -ne 0 ] ; then
+       # We're upgrading to a newer version of this package.  DKMS will
+       # automatically remove our package when it builds the new package
+       # so there's nothing to do at this step.
+       exit 0
+fi
+
+# If we're here then we're doing an uninstall (not upgrade).
 CONFIG_H="/var/lib/dkms/%{module}/%{version}/*/*/%{module}_config.h"
 SPEC_META_ALIAS="@PACKAGE@-@VERSION@-@RELEASE@"
 DKMS_META_ALIAS=`cat $CONFIG_H 2>/dev/null |

I need to do some more testing on this before I can put together a PR.

[Lalufu]

If DKMS actually removes old versions (how would it do that? Where does it keep track of them?), and that's what we want, then this should work, yes.

[mskarbek]

If DKMS actually removes old versions (how would it do that? Where does it keep track of them?), and that's what we want, then this should work, yes.

DKMS creates symlinks to all the modules managed by it in /var/lib/dkms and based on them decides what to remove.

[tonyhutter]

If DKMS actually removes old versions (how would it do that? Where does it keep track of them?) ...

Our dkms spec file actually does the build in %post via /usr/lib/dkms/common.postinst:

%post
for POSTINST in /usr/lib/dkms/common.postinst; do
    if [ -f $POSTINST ]; then
        $POSTINST %{module} %{version}
        exit $?
    fi
    echo "WARNING: $POSTINST does not exist."
done
echo -e "ERROR: DKMS version is too old and %{module} was not"
echo -e "built with legacy DKMS support."
echo -e "You must either rebuild %{module} with legacy postinst"
echo -e "support or upgrade DKMS to a more current version."
exit 1

common.postinst:

#We never want to keep an older version side by side to prevent conflicts
if [ -e "/var/lib/dkms/$NAME/$VERSION" ]; then
    echo "Removing old $NAME-$VERSION DKMS files..."
    dkms remove -m $NAME -v $VERSION --all
fi

https://github.com/dell/dkms/blob/master/dkms_common.postinst#L157

[greencopper]

The same problem exist on Debian Stretch.