ZFS on Linux is a poorly designed software solution to get ZFS up and running in Linux environments. Unlike FreeBSD, ZFS does not work with the Linux kernel natively. The developers of ZFS on Linux come up a creative hack. By injecting the ZFS into the kernel via DKMS, Linux kernel will understand what is ZFS. It works very well, and it really works with a single assumption: The system will never be updated or rebooted after installing ZFS on Linux. So what will happen after you update the system such as kernel or the system got rebooted? There is a good chance that your ZFS module will not be loaded. Otherwise you won’t be reading this article, right?

You just can’t assume that a MINI works like a semi truck. MINI was created for a totally different purposes. If you want to use a MINI like a semi-truck, then you have to accept the corresponding risks.

Long story short, below is how you get your data back:


You reboot your computer after a system upgrade (either updating to a new kernel, or updating the DKMS library, or both), and you notice that the ZFS is not loaded, i.e.,

#zpool import -a
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.


#/sbin/modprobe zfs
modprobe: FATAL: Module zfs not found.
or
modprobe: ERROR: could not insert 'zfs': Invalid argument



#dkms status
spl, 0.7.7, 3.10.0-693.17.1.el7.x86_64, x86_64: installed (original_module exists)
spl, 0.7.7, 3.10.0-693.21.1.el7.x86_64, x86_64: installed
zfs, 0.7.7, 3.10.0-693.17.1.el7.x86_64, x86_64: installed (original_module exists) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)
zfs, 0.7.7, 3.10.0-693.21.1.el7.x86_64, x86_64: installed (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)

What you need to do is to erase all the ZFS and related packages:

yum erase zfs zfs-dkms libzfs2 spl spl-dkms libzpool2 -y

Please reboot the system. This step is very important.

reboot

After that, try to install ZFS again.

yum install zfs -y

If the system complaints about mismatch dependent packages, try to remove the affected packages first and run the installation again.

After the installation, try to start the ZFS module:

/sbin/modprobe zfs

zpool import -a

If the ZFS is up and running, you may move to next section. If you see the following, please follow the steps below:


sudo /sbin/modprobe zfs

modprobe: ERROR: could not insert 'zfs': Invalid argument
  1. Reboot
  2. Clear the cache of the yum repository and try to update the system again. (sudo yum clean all)
  3. Reboot to the latest kernel
  4. Erase the ZFS and related packages and try it again.

If you already tried it for more than 5 times without any luck, don’t waste your time. You may want to bring the ZFS disks to another server. The new server should be able to recognize the ZFS disks. For the original server, you can connect to the ZFS disks on the new server via NFS using the original path. That will minimize the impact of changes.


Assuming that your ZFS is up and running, we will need to check the DKMS status:

#dkms status

If you see no error message, you are good to go.

spl, 0.7.8, 3.10.0-693.21.1.el7.x86_64, x86_64: installed (original_module exists)
zfs, 0.7.8, 3.10.0-693.21.1.el7.x86_64, x86_64: installed (original_module exists)

If you see an error message like the following, you may need to rebuild the modules again:

spl, 0.7.8, 3.10.0-693.21.1.el7.x86_64, x86_64: installed (original_module exists)
zfs, 0.7.8, 3.10.0-693.21.1.el7.x86_64, x86_64: installed (original_module exists) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)
sudo dkms remove zfs/0.7.8 --all
sudo dkms remove spl/0.7.8 --all
sudo dkms --force install spl/0.7.8
sudo dkms --force install zfs/0.7.8
sudo dkms status

Now you understand why no professional truckers use MINI for work. Good luck!

Our sponsors: