Last updated: 11-05-2014 00:12

1. Introduction to the GNU/Linux Kernel
2. The Linux kernel evolution
   • 2.1. Kernel release types
3. Kernel Development
   • 3.1. Distribution kernels
4. Kernel Modules
   • 4.1. List the modules on a system
   • 4.2. Other Kernel module tools
5. Configuration and compilation of the Kernel
   • 5.1. Download and untar the kernel
   • 5.2. libncurses and libncurses-devel
   • 5.3. Configure the kernel
   • 5.4. Compile the main kernel, modules and install modules
   • 5.5. Install the kernel modules
   • 5.6. Install the new kernel
   • 5.7. Reboot
   • 5.8. Review the running kernel configuration options
   • 5.9. Debian Kernel
6. Patching the kernel

1. Introduction to the GNU/Linux Kernel

The Linux kernel dates back to 1991, when Linus Torvalds made it available to the community. It is one of the few operating systems (OS) that while extensively used, also makes its source code available. The kernel of the GNU/Linux system is the heart of the operating system. Its structure can be divided into a series of management components designed to:

• Manage processes: what tasks will be run, in what order and with what priority. An important aspect is the scheduling of the CPU: how is the CPU's time to run tasks optimised with the best possible performance or interactivity with users?
• Intercommunication of processes and synchronisation: how do tasks communicate with each other, with what different mechanisms and how can groups of tasks be synchronised?
• Input/output management (I/O): control of peripherals and management of associated resources.
• Memory management: optimising use of the memory, paginating service, and virtual memory.
• File management: how the system controls and organises the files present in the system and access to them.

In proprietary systems like Microsoft Windows, the kernel is hidden below the layers of the OS software and access is limited to registers via a Registry Utility. The Linux kernel offers an open source solution with the ensuing permissions for modifying, correcting, generating new versions and updates very quickly by anyone anywhere with the required knowledge for doing so.

2. The Linux kernel evolution

The Linux kernel evolution can be considered in terms of Source Lines of Code (SLOC). Starting with the initial versions having a couple of thousand lines to the current versions exceeding 15 billion SLOC.

The following diagram gives a graphical representation of the growth of the kernel size based on lines of code.

The management of the Linux Kernel is vested in the Linux Foundation which counts Linus Torvalds the founder of Linux as one of its employees.

2.1. Kernel release types

In order to control Kernel releases they are classified under the following headings:

• Prepatch
  • Prepatch or RC kernels are mainline kernel pre-releases that are mostly aimed at other kernel developers and Linux enthusiasts.
  • They must be compiled from source and usually contain new features that must be tested before they can be put into a stable release.
  • Prepatch kernels are maintained and released by Linus Torvalds.
• Mainline
  • Mainline tree is maintained by Linus Torvalds.
  • It's the tree where all new features are introduced and where all the exciting new development happens.
  • New mainline kernels are released every 2-3 months.
• Stable
  • After each mainline kernel is released, it is considered stable.
  • Any bug fixes for a stable kernel are backported from the mainline tree and applied by a designated stable kernel maintainer.
  • There are usually only a few bugfix kernel releases until next mainline kernel becomes available; unless it is designated a longterm maintenance kernel.
  • Stable kernel updates are released on as-needed basis, usually 2-3 a month.
• Longterm
  • There are usually several longterm maintenance kernel releases provided for the purposes of backporting bugfixes for older kernel trees.
  • Only important bugfixes are applied to such kernels and they don't usually see very frequent releases, especially for older trees.

3. Kernel Development

As kernels move from the mainline into the stable category, two things can happen:

• They can reach End of Life (EOL) after a few bugfix revisions, which means that kernel maintainers will release no more bugfixes for this kernel version, or
• They can be put into longterm maintenance, which means that maintainers will provide bugfixes for this kernel revision for a much longer period of time.

If the kernel version is marked EOL, consider upgrading to the next major version as there will be no more bugfixes provided for the kernel version.

3.1. Distribution kernels

Many Linux distributions provide their own longterm maintenance kernels that may or may not be based on those maintained by kernel developers. These kernel releases are not hosted at the kernel storage site and kernel developers can provide no support for them.

3.1.1. How to check the current Kernel

  $ uname -r
  
  3.11.0-18-generic
  

4. Kernel Modules

The Linux kernel incorporates a mechanism to load modules for certain functions on a need basis and they can be similarly unloaded when not required. These modules serve to extend the functionality of the kernel without the need to reboot the system. Without modules it would be necessary to build monolithic kernels and add new functionality directly into the kernel image. Besides having larger kernels, this has the disadvantage of requiring a rebuild and a reboot of the kernel each and every time new functionality is required. Here is a list of some of the more common modules on a typical Linux system.

Network

• Netlink
• Network scheduler
• Netfilter is a set of hooks inside the Linux kernel that allows kernel modules to register callback functions with the network stack. This is used for iptables firewall.
• Wireless drivers, iwlwifi, ipw2x00, brcm80211 etc...

Graphics

• Direct Rendering Manager (DRM) is a device-independent kernel-level device driver that provides support for the Direct Rendering Infrastructure (DRI). It allow user applications to access the video hardware. It provides hardware acceleration for Open Graphics Library (OpenGL). Without DRI, programs have to use the CPU while rendering (indirect rendering), which degrades overall performance.
• Graphics Execution Manager (GEM) memory management for device drivers for graphics chipsets. GEM is part of the DRM.
• Graphics drivers, vesa, geforce, nv, fglrx, radeon, r200, r300g, r600g,radeonsi, xorg-video-i810, xorg-video-intel, freedreno, etc ...

Sound

• Advanced Linux Sound Architecture (ALSA) Application Programming Interface (API) for sound card device drivers.

Input

• Event Device (evdev) handles input from keyboards, mice, joysticks, etc..

4.1. List the modules on a system

You can see what modules are already loaded into the kernel by running the lsmod utility, which:

• Lists the loaded modules in the kernel. (extracted from the pseudofile /proc/modules).
• It lists the names and dependencies with others, the size of the module in bytes.
• The module use counter.

  $ lsmod
  
  Module                  Size  Used by
  arc4                   12608  0 
  ppp_mppe               13002  0 
  ppp_async              17413  0 
  crc_ccitt              12707  1 ppp_async
  pci_stub               12622  1 
  vboxpci                23194  0 
  vboxnetadp             25670  0 
  vboxnetflt             27613  0 
  
  ~~~~~                  ~~~~~  ~
  
  wmi                    19070  0 
  video                  19318  1 i915
  

4.2. Other Kernel module tools

• modprobe: tries the loading of a module and its dependencies.
• insmod: loads a specific module.
• depmod: analyses dependencies between modules and creates a file of dependencies.
• rmmod: removes a module from the kernel.

5. Configuration and compilation of the Kernel

5.1. Download and untar the kernel

Download the latest stable kernel from the kernel storage site.

  $ cd /usr/src/
  
  $ sudo wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.14.tar.gz
  
  $ sudo tar -xzvf linux-3.14.tar.gz
  

5.2. libncurses and libncurses-devel

Install libncurses and libncurses-devel packages before going any further, these are essential to the menuconfig utility.

  $ sudo apt-get install libncurses5 libncurses5-dev
  

5.3. Configure the kernel

Use the commandline make menuconfig utility to make the required selections.

  $ cd linux-3.14
  
  $ sudo make menuconfig
  

5.4. Compile the main kernel, modules and install modules

  $ sudo make
  
  $ sudo make modules
  

5.5. Install the kernel modules

$ sudo make modules_install

This wll create a directory named /lib/modules/3.14/ on the system.

5.6. Install the new kernel

$ sudo make install

As a result of this step the following files are created in the /boot directory. The GRUB bootloader grub.cfg will also be updated at the same time so a manual update of GRUB is unnecessary.

• vmlinuz-3.14 - The actual kernel
• System.map-3.14 - The symbols exported by the kernel
• initrd.img-3.14 - initrd image is temporary root file system used during boot process
• config-3.14 - The kernel configuration file

5.7. Reboot

The system must be rebooted to implement the newly compiled kernel. If during the boot a problem occurs with the new kernel, it is possible to select the old kernel from the grub menu and boot into the old kernel to troubleshoot. Confirm the kernel with the uname -r command once booted.

  $ sudo init 6 
  
  $ uname -r
  
  3.14-generic
  

5.8. Review the running kernel configuration options

  $ sudo cat /boot/config-`uname -r` > /tmp/kernel.config.txt
  
  $ cat /tmp/kernel.config-3.11.0-18-generic.txt
  
  #
  # Automatically generated file; DO NOT EDIT.
  # Linux/x86_64 3.11.0-18-generic Kernel Configuration
  #
  CONFIG_64BIT=y
  CONFIG_X86_64=y
  CONFIG_X86=y
  CONFIG_INSTRUCTION_DECODER=y
  CONFIG_OUTPUT_FORMAT="elf64-x86-64"
  CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
  
  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~  ~~ 
  
  CONFIG_CORDIC=m
  CONFIG_DDR=y
  CONFIG_MPILIB=y
  CONFIG_SIGNATURE=y
  CONFIG_OID_REGISTRY=y
  CONFIG_UCS2_STRING=y
  CONFIG_FONT_SUPPORT=y
  # CONFIG_FONTS is not set
  CONFIG_FONT_8x8=y
  CONFIG_FONT_8x16=y
  

5.9. Debian Kernel

The Debian Kernel Handbook is a great resource when building or patching a kernel for the Debian distribution.

6. Patching the kernel

In some cases the application of patches to the kernel is common. The patch file in relation to the Linux kernel is an ASCII text file that contains the differences between the original source code and the new code, with additional information on file names and code lines. patch program is used to apply the patch to the tree of the kernel source code in /usr/src.

The patches are usually necessary when special hardware requires some modification of the kernel or some bugs have been detected subsequent to a wide distribution of a kernel version or else a new specific feature is to be added.

The patch is usually distributed in a compressed file of either bz2 (bunzip2), gzip .gz or LZMA .xz format, here is an example from v2.14:

• patch-3.14.gz

Once the patch is ready it must be applied. This process is usually documented in a readme file that accompanies the patch, but generally the process follows the steps of decompressing the patch in the source files directory and applying it over the sources of the kernel, for example:

  $ cd /usr/src/linux-3.14
  
  $ sudo tar -xzvf patch-3.14.gz
  
  $ sudo patch -p1 -i patch-3.14  < or >   $ sudo patch -p1 < patch-3.14
  

Once the patch is applied the kernel will have to be recompiled in order to generate it again.

The patches can be obtained from the kernel storage site or from the repositories of the various Linux distributions.