When I rebooted my computer and loaded my ZFS pool today, I got this error message:
#sudo zpool import -a
This pool uses the following feature(s) not supported by this system:
org.zfsonlinux:project_quota (space/object accounting based on project ID.)
com.delphix:spacemap_v2 (Space maps representing large segments are more efficient.)
All unsupported features are only required for writing to the pool.
The pool can be imported using '-o readonly=on'.
cannot import 'my_zpool': unsupported version or feature
On my another machine, I also saw something similar when I tried to create a new pool:
zpool create: invalid argument for this pool operation
This kind of error usually happens when you move your ZFS pool from one system to the other. For example, if your ZFS pool was created in ZFS v10, and you move it to a new system that can only handle ZFS v9, then this error message will show up. Obviously, this is simply not true in my case (and yours too). My system showed me this message after rebooting the server. It had nothing to do with moving the ZFS pool from one to the other. In short, this message is misleading, however it gave me some idea of what was going wrong.
Long story short. It is a known bug of the ZFS on Linux. This kind of problem happens when your Linux kernel is updated every time. If you want to get this resolved, you can only do two things. Never update your system kernel, or never reboot your server (so that the new kernel will not be loaded). If you can’t do any of these, then ZFS on Linux is not for you.
If you need to access your data now, you can mount it as read only, although this is not a long term solution:
sudo zpool import my_zpool -o readonly=on
Another way is to reboot your server to the older working kernel, assuming your old kernel is still available in your system.
So here is the reason why your system could not open your ZFS pool:
- You are running Linux kernel ver A and ZFS on Linux ver X, and your system is happy.
- A new kernel is release (e.g., ver B). Your system download it and the kernel is sitting under /boot
- Later, a new ZFS on Linux (e.g., ver Y) is available. In theory, when upgrading the ZFS on Linux, it supposes to compile the DKMS code with each kernel in the system. In the other words, your current kernel (ver A) and the new pending kernel (ver B) should know how to use the ZFS on Linux (both ver X and Y). Notice that I am using the word: “In theory”. And you probably know that things are not ideal in reality.
- So when your system is booted into the new kernel, for some reasons, your new kernel does not have the skill (ZFS on Linux ver Y) to open your ZFS pool, therefore you see that error message.
Here is how to solve the problem. Reinstalling the ZFS and DKMS packages is not going to solve the problem. You will need to rebuild the DKMS modules with your new kernel. First, reboot your computer to the latest kernel first. Here are my versions. Your versions may be different.
Old Kernel: 3.10.0-1062.9.1.el7.x86_64
New Kernel: 3.10.0-1062.12.1.el7.x86_64
Old DKMS ZFS Module: 0.8.2
New DKMS ZFS Module: 0.8.3
Remove your old kernels.
sudo package-cleanup --oldkernels --count=1 -y
Check your current DKMS status. It should contain some error:
sudo dkms status
Error! Could not locate dkms.conf file.
File: /var/lib/dkms/zfs/0.8.2/source/dkms.conf does not exist.
Clean up the DKMS folder:
#cd /var/lib/dkms/zfs/
#ls -al
# Move old libraries and old kernels to somewhere
0.8.2 <---- Move this to /tmp
0.8.3 <-- Keep
original_module
kernel-3.10.0-1062.9.1.el7.x86_64-x86_64 -> 0.8.3/3.10.0-1062.9.1.el7.x86_64/x86_64 <-- Move this to /tmp
Remove the old DKMS modules that are associated with old kernels:
sudo dkms remove zfs/0.8.2 --all;
Recompile the new DKMS module with the current kernel:
sudo dkms --force install zfs/0.8.3
Check your DKMS status again, it should be clean:
sudo dkms status
zfs, 0.8.3, 3.10.0-1062.12.1.el7.x86_64, x86_64: installed
If you see any old kernel that is associated with the new DKMS module, remove them, e.g.,
#sudo dkms status
zfs, 0.8.3, 3.10.0-1062.12.1.el7.x86_64, x86_64: installed (original_module exists)
zfs, 0.8.3, 3.10.0-1062.9.1.el7.x86_64, x86_64: built (original_module exists) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!) (WARNING! Missing some built modules!)
sudo dkms remove zfs/0.8.3 -k 3.10.0-1062.9.1.el7.x86_64
Now you may try to import your ZFS pool again. If it doesn't work, try to mount the ZFS pool in read only mode first, back up your data, rebuild the pool and restore it from backup.
Our sponsors:
