I received a Christmas gift from the RHEL 7 / CentOS 7 / Linux kernel team today. After my system got updated to the new kernel (3.10.0-957.1.3.el7.x86_64), my system gave me few surprises. As a sys-admin, I don’t want to see any surprise. What I really want is a working system. That’s one of the reasons why I always suggest people to use FreeBSD if possible. FreeBSD is a truly rock solid system.
Long story short. You have a working system. Your system receive a new kernel (e.g., 3.10.0-957.1.3.el7.x86_64). You decide to boot to that new kernel. Your system takes forever to boot. You go down to the server room, turn on the monitor and see the following messages:
[time stamp]dracut-initqueue : Warning: dracut-initqueue timeout - starting tmeout scripts [time stamp]dracut-initqueue : Warning: dracut-initqueue timeout - starting tmeout scripts ... [time stamp]dracut-initqueue : Warning: count not boot. [time stamp]dracut-initqueue : Warning: /dev/disk/by-uuid/XXX does not exists Starting Dracut Emergency Shell... Warning: /dev/disk/by-uuid/XXX does not exists Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report. dracut:/#
In my situation, my system has no problem to boot into the older kernel. It just does not like the new kernel. In my case, I check my /etc/fstab settings. I disable all of the non-standard devices.
#The following are standard. UUID=12347890-1234-9512-9518-963852710258 / xfs defaults 0 0 UUID=12347890-1234-4513-7532-963852710258 /boot xfs defaults 0 0 UUID=12347890-1234-9587-8526-963852710258 swap swap defaults 0 0 #The new kernel does not like it. I have to comment it out. #/storage/data/Dropbox.img /Dropbox/ ext4 defaults 0 0
That’s it. I simply mount the image after the system is booted and the problem is solved. This is done by mounting the image in a script and run it in /etc/crontab (@reboot)
The second surprise was even worse. In one of my Linux machines, the OS was installed in the USB drive (because all SATA ports have been used for raid storage). For some odd reasons, the new kernel cannot be booted because the system lives in a USB flash drive. So I tried to install a new CentOS 7 on the same hardware (the installation disk contains the latest kernel) and it is giving me the same results. I ended up installing the OS on a SATA hard drive, which is not what I want because my computer case does not have any extra space of another hard drive.
Sometimes, I think the Linux kernel / system engineers have way too much spare time.