Hi there. In this post, I will give general info about compiling linux kernel and address the solution of a problem I encountered. This post was actually going to be a part of the next post, but this problem has costed me so much time, that it's worth writing about it in a separate blog post. The title may give a hint about the next post.
Compiling Kernel in Linux
Compiling the kernel is not something that should be done on a daily basis, unless you are running linux on a very special device e.g. an embedded system or a brand new hardware that the standard kernel does not support. Or you are using an experimental distro such as LFS or such as Gentoo where the kernel does not come pre-compiled. On the other hand, to be able to compile the kernel is a must to learn and master the system you are running at the lowest level. And it is easier than compiling another source code, once you have met few prerequisites and know the basics.
In this post, I will compile kernel v5.13.12. What I first need, is the source code of course. Btw, kernel version only matters for .config file, which I will mention it in next paragraphs. The rest is applicable to (almost) all kernel versions. Any kernel is available at www.kernel.org/pub under linux/kernel subdirectory. I have linked .xz file above due to its smaller size but .gz version of same data is also available.
I have extracted this file. Some necessary programs for compilation (such as make, gcc and gcc-c++) were already installed on my machine. The only missing package was ncurses-devel and it is only required for make menuconfig not the compilation itself however menuconfig makes choosing kernel properties quite easy. The prerequisites for the kernel is not limited to these. A detailed list for the v5.13 is available at kernel.org documentation. But it is not mandatory to have all the packages in that list installed. For example, if you are not going to compile anything related to PPP, pppd is not needed or OpenSSL is only required if kernel modules are to be signed.
Here is some preliminary info before starting with compilation: make clean deletes all compiled files and makes the code ready for recompilation (from scratch). make mrproper, which is special for linux kernel makefile, deletes .config file and all other generated files [1]. So, before I started, I also cleaned the directory with this command and set most default kernel properties with make defconfig. These properties, I am mentioning, are kept in .config file, so defconfig actually creates a default .config. These can be viewed or edited with a text editor or using make menuconfig command over ncurses TUI. Since there are 4845 lines in .config file, it is not logical to edit all properties manually.
.config file is different for each version, because each minor version adds or removes some features in kernel. Thus, I cannot compile using a .config file of another version due to compatibility. But I can adapt an older .config file to a newer kernel with make oldconfig command. This command only asks the user for the newly added properties which isn't in old .config. Having this said, it also doesn't make much sense if you are trying to adapt a v2.6 config file to v5.13.
 | |
| make menuconfig TUI |
I've run make menuconfig command and the menu above has appeared. Lines, ending with "--->" indicate a submenu. Each line in this menu (unless it has a submenu) correspond to a line in .config. In documents, features that must be enabled are given with [*] and features that must be disabled are given with [ ]. And [M] means that the feature is modular, which I explain later. Following image shows the kernel features, recommended to configure LFS with systemd:
 |
| Kernel features for LFS systemd |
For example, features such as "Auditing Support", "Control Group support" and "Configure standard kernel ..." are under "General setup"; "open by fhandle syscalls" is under "Configure standard kernel ...". The names in square brackets at the end of each line are the item names in .config file. Btw, I couldn't find "Enable seccomp ..." feature, which should be under "Processor type and features" (wrong at source). This feature is under "General architecture-dependent options" in v5.13 according to Kernelconfig.io. In older versions such as v5.9.9, it is under "Processor type and features". A specific item can be searched among all items by its name by pressing '/' in menuconfig. I can also change this feature in .config file (below). Usually, first letters of menu entries (but not always) in visible area, are their keyboard shortcuts. If I enter "Processor type..." submenu, I can navigate through the features that start with "E", by pressing "E" key on keyboard.
687:CONFIG_HAVE_ARCH_SECCOMP=y
688:CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
689:CONFIG_SECCOMP=y
690:CONFIG_SECCOMP_FILTER=y
691:# CONFIG_SECCOMP_CACHE_DEBUG is not set
CONFIG_SECCOMP is already enable on line 689. By substituting 'y' with an 'n', I can disable it if it's not needed. All these features go in the .config file. For [M], here is an example:
 |
| Kernel features for using LVM in BLFS |
M denotes that the feature will be compiled as a kernel module. Not all kernel functions have to be in compiled kernel, named vmlinuz-... under /boot directory. To keep kernel as small as possible, some functions can be kept as modules under /usr/lib/modules and loaded only if needed. In this way, LVM modules on a machine without an LVM disk don't need to be loaded into memory or sound card modules on a VM without a sound card.
Module files have .ko (kernel object) extension. They are loaded with insmod or modprobe and listed with lsmod. Modules allow new functionality to be added to the linux kernel without the need to recompile it [2]. Module configurations are kept in .conf files under /etc/modprobe.d/. With these files, arguments can be passed to the modules or modules can be disabled completely. Kernel modules are a much broader topic that I can cover here.
After setting the required options with menuconfig or manually, I've run make to compile the kernel. Of course, it does not get compiled instantly. With default options, it takes about 5 minutes on my computer. Since some features are compiled as modules by default, these modules need to be copied to the corresponding module directory with make modules_install after make.
After compilation, the file arch/x86/boot/bzImage (kernel) can be copied under /boot with name vmlinuz-... and loaded with grub. Every distribution copies its .config file, used for compilation, to /boot with name config-$(uname -r). This means that, if I want to recompile the same kernel I am using, I can copy that config file in /boot to the directory, where I compile the kernel, with name ".config" and run make again. Let's assume, I want to recompile Fedora kernel v5.13.12. I search kernel package from the build system koji and download kernel-core package for x86_64 architecture. Then I extract it using rpm2cpio kernel-core-5.13.12-200.fc34.x86_64.rpm | cpio -idmv command and copy lib/modules/5.13.12-200.fc34.x86_64/config file to linux-5.13.12 directory with name ".config" and compile it. I eventually get the same kernel as Fedora (additional reading: How to install a kernel from koji). However, because Fedora kernel is built to run on every machine, having that many features enabled (e.g. Fibre Channel even though it is modular), increases compile time drastically. It took 5-6 hours on my computer.
After this much preliminary information, I can discuss about the problem I had.
Compiling the Linux Kernel for Vmware
First, I've installed two VMs, one in VBox and one in vmware. They are same except minor differences. As I explained above, I have added some features to the defconfig, compiled the kernel, then copied it to /boot, created initramfs and grub config to load them. So far so good.
As it can be seen from the output below, I checked the devices and their kernel modules with lspci -k command:
In Vmware, besides the common modules like ata_piix, vmw_vmci, pcieport, there is another module called mptspi for SCSI controller. In VBox output (bottom part of above image), there are only standard drivers. ahci is used as SATA controller and there is no mptspi. I did some research on this module on internet. The configuration of mptbase, mptscsih, mptspi drivers is referred as "Fusion MPT ScsiHost drivers for SPI" [3], which is under "Device Drivers/Fusion MPT device support" menu with item name is CONFIG_FUSION according to kernelconfig.io. In default .config AHCI is turned on but Fusion MPT is off. That's why VBox is working fine but vmware isn't. And since this option is enabled in CentOS kernel, there is no problem with it on both platforms.
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_ACARD_AHCI is not set
$ grep FUSION .config
# CONFIG_FUSION is not set
References:
[1]: Why both make clean and make mrproper are used?
[2]: https://en.wikipedia.org/wiki/Loadable_kernel_module
[3]: https://cateee.net/lkddb/web-lkddb/FUSION_SPI.html
- : https://www.youtube.com/watch?v=WiZ05pnHZqM
No comments:
Post a Comment