Safely Update the Linux Kernel on Rocky Linux 9

Sat, 11 Jul 2020 22:14:00 -0400

Scope

This guide updates the kernel on Rocky Linux 9 and other compatible Enterprise Linux 9 rebuilds. The normal and preferred method is to install the distribution kernel from the standard repositories. An optional section covers ELRepo’s kernel-lt and kernel-ml packages for testing hardware that the distribution kernel does not support.

The original article targeted CentOS Linux 8, which reached end of life in December 2021. Migrate unsupported systems before changing their kernels.

Enterprise Linux kernels retain a stable version family while backporting security fixes and drivers. A higher upstream version number does not by itself mean that the system is more secure, stable, or supported.

Prepare for the Update

Identify the distribution, running kernel, architecture, and default boot kernel:

cat /etc/os-release
uname -r
uname -m
sudo grubby --default-kernel

List installed kernel packages and boot entries:

rpm -q kernel kernel-core --last
sudo grubby --info=ALL

Check available space in the boot filesystems:

df -h /boot /boot/efi

The /boot/efi path might not be a separate filesystem on a BIOS-based installation. Resolve a nearly full /boot before installing another kernel, but keep at least one known-good older kernel.

Before updating a remote server:

Confirm that a hypervisor console, physical console, or other out-of-band recovery method is available.
Back up important data and boot configuration.
Record the current kernel path reported by grubby --default-kernel.
Check whether storage, networking, GPU, security, or virtualization depends on an out-of-tree module.
Take a tested virtual-machine snapshot when the platform supports it and the storage design makes snapshots safe.

Inspect DKMS and installed third-party kernel modules when those tools are present:

command -v dkms >/dev/null && dkms status
rpm -qa | grep -E '^(kmod-|akmod-)'

Confirm that required module packages have a build compatible with the new kernel before rebooting.

Install the Supported Distribution Kernel

Refresh metadata and review all available updates:

sudo dnf check-update --refresh

The safest routine update keeps the kernel and related userspace packages at a consistent supported level:

sudo dnf upgrade --refresh

For a maintenance window intentionally limited to kernel packages, use the documented kernel update command:

sudo dnf update kernel

DNF treats kernels as install-only packages. It installs the new kernel alongside older versions instead of replacing the running kernel immediately.

List the installed kernels again and confirm that the new boot files exist:

rpm -q kernel-core --last
ls -lh /boot/vmlinuz-* /boot/initramfs-*.img
sudo grubby --info=ALL

The new distribution kernel normally becomes the default automatically. Confirm the selection:

sudo grubby --default-kernel

If the default is incorrect, copy the exact desired /boot/vmlinuz-... path from grubby --info=ALL and set it explicitly:

sudo grubby --set-default /boot/vmlinuz-VERSION.ARCH
sudo grubby --default-kernel

Replace VERSION.ARCH with an installed kernel filename. Do not use grub2-set-default 0; numeric positions can change when another kernel is installed.

Reboot and Verify

Schedule the interruption, then reboot:

sudo systemctl reboot

After the system returns, verify the running kernel and boot state:

uname -r
sudo grubby --default-kernel
systemctl --failed
sudo journalctl -b -p err..alert --no-pager

Test storage, networking, firewall rules, virtualization, containers, monitoring, and application services that depend on kernel features. Confirm that required modules loaded:

lsmod
sudo dmesg --level=err,warn

Warnings are not automatically failures. Compare them with the previous boot and investigate messages related to required hardware or services.

Roll Back to an Older Distribution Kernel

If the new kernel cannot boot, display the GRUB menu during startup by pressing Esc repeatedly and select a known-good older kernel. Console access is essential when the machine cannot reach the network.

If the system boots but the new kernel causes a problem, list the installed entries:

sudo grubby --info=ALL

Set the exact older kernel path as the default and reboot:

sudo grubby --set-default /boot/vmlinuz-OLD_VERSION.ARCH
sudo grubby --default-kernel
sudo systemctl reboot

After rebooting, verify that uname -r reports the selected version. Keep the failed kernel installed until logs and package state have been reviewed; removing it immediately can discard evidence needed for diagnosis.

Manage Retained Kernels

DNF’s installonly_limit controls how many versions of install-only packages are retained. Inspect the configured value:

grep -R '^installonly_limit' /etc/dnf/dnf.conf /etc/dnf/dnf.conf.d 2>/dev/null

DNF normally keeps three versions. A value of at least 2 preserves a rollback kernel; 1 is not permitted. Do not manually remove the running kernel or every older kernel.

After the new kernel has passed testing and a known-good rollback kernel remains, preview obsolete install-only packages:

sudo dnf remove --oldinstallonly --setopt installonly_limit=3 --assumeno

Review the proposed transaction. If it preserves the running kernel, the default kernel, and a tested fallback, run it interactively without --assumeno:

sudo dnf remove --oldinstallonly --setopt installonly_limit=3

Do not remove kernels by copying a version prefix into a broad wildcard command. That can remove the active kernel, dependencies, or the only bootable fallback.

Optional: Test an ELRepo Kernel

Use the distribution kernel whenever it supports the hardware. ELRepo describes its alternative kernels as a last resort for hardware testing and provides them without distribution support. They can expose the system to compatibility, security, performance, or data-integrity problems.

kernel-lt follows an upstream long-term-support branch.
kernel-ml follows the upstream mainline stable branch.

Both install alongside the Rocky Linux kernel. Keep the Rocky Linux kernel and its boot entry intact.

Check Secure Boot

Check the firmware state before installing an ELRepo kernel:

sudo dnf install -y mokutil
sudo mokutil --sb-state

ELRepo states that kernel-lt and kernel-ml are not signed for Secure Boot. They will not boot under a standard Secure Boot policy. Do not disable Secure Boot merely to obtain a newer version number; use the supported distribution kernel unless there is a tested requirement and an approved signing or firmware policy.

Install the ELRepo Repository

Import both current ELRepo RPM signing keys and install the EL9 repository package from its stable URL:

sudo rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
sudo rpm --import https://www.elrepo.org/RPM-GPG-KEY-v2-elrepo.org
sudo dnf install https://www.elrepo.org/elrepo-release-9.el9.elrepo.noarch.rpm
sudo dnf update elrepo-release

Display the enabled repository definition and available alternative kernels:

sudo dnf repolist elrepo
sudo dnf --disablerepo='*' --enablerepo=elrepo-kernel list available 'kernel-lt*' 'kernel-ml*'

Install One Kernel Family

Install the long-term kernel for a conservative hardware test:

sudo dnf --enablerepo=elrepo-kernel install kernel-lt

Alternatively, install the mainline stable kernel when the required support is unavailable in kernel-lt:

sudo dnf --enablerepo=elrepo-kernel install kernel-ml

Do not install both without a specific test plan. If an out-of-tree module must be compiled, install the matching kernel-lt-devel or kernel-ml-devel package rather than the similarly named headers package:

sudo dnf --enablerepo=elrepo-kernel install kernel-lt-devel

Replace kernel-lt-devel with kernel-ml-devel when testing kernel-ml.

Select and Test the ELRepo Kernel

List the new boot entry and copy its exact kernel path:

sudo grubby --info=ALL

Set the selected ELRepo kernel as the default:

sudo grubby --set-default /boot/vmlinuz-ELREPO_VERSION.ARCH
sudo grubby --default-kernel

Replace ELREPO_VERSION.ARCH with the exact installed filename. Reboot and verify:

sudo systemctl reboot

After the host returns:

uname -r
systemctl --failed
sudo journalctl -b -p err..alert --no-pager

An ELRepo kernel version normally contains .el9.elrepo. Test every required service and device before considering it usable.

Return from ELRepo to the Distribution Kernel

List the boot entries, select the exact Rocky Linux kernel path, and reboot:

sudo grubby --info=ALL
sudo grubby --set-default /boot/vmlinuz-ROCKY_VERSION.ARCH
sudo grubby --default-kernel
sudo systemctl reboot

Confirm that the running kernel is the distribution build before removing ELRepo packages:

uname -r
rpm -qf /boot/vmlinuz-$(uname -r)

Remove only the ELRepo kernel family that is no longer required:

sudo dnf remove 'kernel-lt*'

For a mainline test, use:

sudo dnf remove 'kernel-ml*'

Review the removal transaction before accepting it. Never remove the ELRepo kernel while it is still running, and never remove the remaining distribution kernel packages.

Troubleshooting

The New Kernel Does Not Appear in GRUB

Confirm that the package and boot files exist:

rpm -q kernel-core --last
ls -l /boot/vmlinuz-* /boot/loader/entries/
sudo grubby --info=ALL

Inspect the package transaction and kernel installation logs before manually rebuilding GRUB:

sudo dnf history info last
sudo journalctl -b --no-pager | grep -Ei 'kernel-install|grub|dracut'

Enterprise Linux 9 normally manages Boot Loader Specification entries automatically. Do not overwrite EFI loader files with commands copied from older RHEL or CentOS releases.

A Required Module Is Missing

Identify the device and expected module:

lspci -nnk
modinfo MODULE_NAME

Replace MODULE_NAME with the required module. Boot the known-good kernel, then install or rebuild the matching signed module package. Do not force-load an incompatible module.

The System Boots but Networking or Storage Fails

Boot the previous kernel from GRUB, make it the default with grubby, and collect logs from the failed boot:

sudo journalctl --list-boots
sudo journalctl -b -1 -k --no-pager

The -1 boot is usually the immediately previous boot, but confirm the boot list before relying on that index.

Elrepo on Di