Sunday, November 25, 2012

Hi, fellas.....

I hoped that you have learned something from my previous articles. This article is going to give you a brief overview on how to develop one's own kernel. As we know, kernel is like the heart of any operating system. It defines user space and system space clearly and carries out the instructions given by the user. Shell is the inter-actor between kernel and the user. So this kernel plays a very important role while the system is being run. So some of you may have wondered that why isn't it possible to write a custom kernel? it is possible, but is highly risky. Only advanced users of gnu/linux operating systems are advised to try writing custom kernel.
I think that this article is the most advanced article when compared to my previous articles. So read every word of this article carefully.

Some keywords that should be learned before creating customized kernel:


fakeroot: Wondering that what is fakeroot? To gain the root facilities in any gnu/linux operating systems, you need to enter a command

sudo -i

Then shell will request you for the password. Enter it and you will get root control over your gnu/linux operating system.
Fakeroot works in the similar way. As the name suggests, it will provide a fake root environment. This command is used when we want to enable some thing like dpkg-buildpackage -rfakeroot, i.e. to remove the need to become root for a package build. This is done by setting LD_PRELOAD to libfakeroot. So, fakeroot provides wrappers around getuid, chown, chmod, mknod, stat etc, thereby creating a fake root environment. If you don't understand any of this, you do not need fakeroot!!!

kexec-tools: kexec-tools provides the /sbin/kexec binary that facilitates a new kernel to boot using the kernel's kexec feature either on a normal or a panic reboot. This package contains the /sbin/kexec binary and ancillary utilities that together form the userspace component of the kernel's kexec feature.

makedumpfile: As the name suggests, makedumpfile will create a dump file, i.e., it will stripe down the original dump file to a smaller dump file, thus saving the memory, smaller dump files would require less RAM. If the dump level is set to a level producing a large dump file, (0 produces the largest dump files, 31 the smallest), the dump file might become very large on systems with a lot of RAM. Under such situations the package 'makedumpfile' can help to us.

kernel-wedge: It is the udeb package builder for Debian-Installer, i.e., kernel-wedge splits udeb packages out of a kernel image deb. The udebs are used by the Debian-Installer.

Let us start creating our own kernel(s). Some of you may know that many kernels can co-exist in a GNU/LINUX operating system. So creating a custom kernel will affect all the kernels which exist in a GNU/LINUX operating system.

Step 1: There are few Prerequisites that has to be satisfied before creating custom kernel.
 The 'git'('git-core' in newer versions of ubuntu) provides git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which can be installed. You will need to install both of these packages. We'll need these 2 packages before creating a customized kernel.

sudo apt-get -y install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge git-core libncurses5 libncurses5-dev libelf-dev libdw-dev binutils-dev


Note: In older versions of Ubuntu, we should replace libncurses5-dev with ncurses-dev.

P.S: If you are a developer or have messed with similar stuff, then you most probably have some or even all of those packages already installed. But it doesn’t hurt to execute this command anyways. You also have to make sure that all the dependencies for the Linux build are satisfied. So, just type the following:

sudo apt-get build-dep kernel-package linux-meta


Step 2: Now you'll have everything that is required to create a customized kernel. So why don't we start creating customized kernel? Oh... Hold on!! It is better if you create a work-space and try creating the kernel. The default work-space would be you home directory. So let us create a path to your home directory through this command:


cd $home



Then clone the repository in the home directory.

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git


The above command would clone the default git repositories.
There is another command which will clone the natty git repositories. 

git clone git://kernel.ubuntu.com/ubuntu/ubuntu-natty.git source


It is left you to choose between default git repositories and natty git repositories. 

Note: Above commands would take some time to get executed. Patience is very important. 

Step 3: Take a note of the current kernel that you are using. Use this command to check the version of kernel that you are using. In my case, it is 3.5.0-19-generic.

uname -r


After knowing the exact version string, it will be easy to get the right patch for proper Reiser4 support. Make sure you also select the right patch for your own kernel version. Please note that Linux stable versions appear like flies.

Note: From here there will be two different types of creating customized kernel. 

1st type: Customizing the kernel by using patches. 
Now create a file by the name 'patches', and create a path to that new file through the terminal.

mkdir patches && cd patches/


Now download the patches to the file patches via this command:

wget http://www.kernel.org/pub/linux/kernel/people/edward/reiser4/reiser4-for-2.6/reiser4-for-2.6.38.patch.gz


Note: This command is only for 2.6.38.xx kernel versions. For higher versions of kernel, i.e., higher than 3.x.xx.xx versions of kernels) download the suitable patches.

Now expand the compressed patch file:


gunzip reiser4-for-2.6.38.patch.gz



Great!! Now the patches are ready to be incorporated into the Ubuntu kernel. Since we've put all the patches in a single directory, it is damn easy to incorporate all the patches into kernel. Let us do it:

cd ../source/

patch -p1 < ../patches/reiser4-for-2.6.38.patch


Now comes the configuration part. Do it very carefully. Suppose that you’re not a customization maniac, the default config file of the standard Ubuntu kernel, i.e. the one describing the features of the currently running kernel, will suit you well. You’re going to make a few changes here and there, but when you’re using the default config as a base the whole process is pretty easy and totally safe.


cp -vi /boot/config-$(uname -r) .config

make oldconfig




If everything went well you will be presented with the following question. Agree by pressing the [Y] key.

BFS cpu scheduler (SCHED_BFS) [Y/n/?] (NEW)



Immediately after that you’re going to be asked about the timer frequency. Go for a low one if you have a laptop or a not-so-powerful computer. 


Timer frequency 1. 100 HZ (HZ_100) 

 2. 250 HZ (HZ_250_NODEFAULT) (NEW) 

 3. 300 HZ (HZ_300)

4. 1000 HZ (HZ_1000) 

 5. 1500 HZ (HZ_1500) (NEW) 

 6. 2000 HZ (HZ_2000) (NEW) 

 7. 3000 HZ (HZ_3000) (NEW) 

 8. 4000 HZ (HZ_4000) (NEW) 

 9. 5000 HZ (HZ_5000) (NEW) 

 10. 7500 HZ (HZ_7500) (NEW) 

 11. 10000 HZ (HZ_10000) (NEW)

choice[1-11?]:


Next, just press [Enter] to all the other questions until you see this one:


Reiser4 (EXPERIMENTAL) (REISER4_FS) [N/m/y/?] (NEW)



Instead of incorporating Reiser4 support into our kernel we choose to build it as a dynamically loadable module instead, so we press the [M] key. We don’t want debugging support for Reiser4, so in the following question we press [N].


Enable reiser4 debug mode (REISER4_DEBUG) [N/y/?] (NEW)




For any other question just select the default answer by pressing [Enter]. In case you’d like to configure any special options, you can also type

make menuconfig




Build it like a real builder:

We’re almost done. We just have to build our new kernel, after all this hard work.

make-kpkg clean



For the next command you’ll most definitely have to make two adjustments. First, give to the –append-to-version parameter a name you’d like your new kernel to have. Second, change the value of –overlay-dir to the full path of the directory the kernel source resides.


nice fakeroot make-kpkg --initrd --append-to-version=-lockard --overlay dir=/home/gtklocker/dev/kernel/source kernel-image kernel-headers


Be patient and wait for the compilation to finish. Depending on the raw processing power or lack thereof- of your computer’s CPU, the whole process may take up to an hour or even more. In any case, a gazillion of messages will fly by your terminal window. Sit back, grab a bowl of pop corns or some kur kur dish and enjoy.

Installation:
If everything went well and I really can’t imagine a reason if they don’t, after the compilation finishes you’ll find a couple of DEB files in ~/dev/kernel. (P.S: Everything till here should go perfectly without any glitches or errors). Use below commands to find the DEB files in ~/dev/kernel directory.

cd ..ls -lh


Install those DEB files. 


sudo dpkg -i linux-*.deb



And now it’s one of those rare times where in order for the changes to take effect you need to reboot your Linux system.

2nd type: Using default kernel config file for customization.

Note: If you're using 2nd type to customize your kernel, then ignore step 2. But carry on with step 1. 


Change to the directory where you want to clone the git tree. In this example we will use $HOME

cd $HOME


Clone the mainline kernel git tree.


git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git


Change to linux directory.


cd linux


Copy the kernel config file from your existing system to the kernel tree.


cp /boot/config-`uname -r` .config


Bring the config file up to date. Answer any questions that get prompted. Unless you know you are interested in a particular feature, accepting the default option by pressing Enter should be a safe choice.


make oldconfig


In cases where your kernel source is significantly newer than the existing config file, you'll be presented with all of the new config options for which there is no existing config file setting. You can either sit there and keep hitting Enter to take the default (generally safe), or you can just run


yes '' | make oldconfig


which emulates exactly the same thing and saves you all that time.
(optional) If you need to make any kernel config changes, do the following and save your changes when prompted:


make menuconfig


Note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be (linux-image*.deb will be 200-300 MB instead of 20-30 MB). To turn this off, go into "Kernel hacking"; then, under "Kernel debugging", turn OFF "Compile the kernel with debug info".
Edit scripts/setlocalversion and comment out the line "echo "+"". If this is not done, make-kpkg will fail
to build the packages with an error "package linux-image-[kernel version]-custom+ not in control info".


sed -rie 's/echo "\+"/#echo "\+"/' scripts/setlocalversion


In addition, if you want to build from a git non-HEAD version, also comment out the lines extra plus at about line 168:

- if test "${LOCALVERSION+set}" != "set"; then - scm=$(scm_version --short) - res="$res${scm:++}" - fi + #if test "${LOCALVERSION+set}" != "set"; then + # scm=$(scm_version --short) + # res="$res${scm:++}" + #fi


Clean the kernel source directory.


make-kpkg clean


Build the linux-image and linux-header .deb files (CONCURRENCY_LEVEL can also be set manually to how many CPUs/cores to use to build the kernel). This process takes a lot of time.


CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom kernel_i





With this command the package names will be something like linux-image-2.6.24-rc5-custom and linux-headers-2.6.24-rc5-custom, and in that case the version will be 2.6.24-rc5-custom-10.00.Custom. You may change the string custom into something else by changing the --append-to-version option. You may also change the default 10.00.Custom into something else by using the --revision option.
Change to one directory level up (this is where the linux-image and linux-header .deb files were put)

cd ..


Now install the .deb files. In this example, the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb. You may receive warnings about '/lib/firmware/2.6.24-rc5-custom/' - this is expected and will only be problematic if the driver you are trying to test requires firmware.

sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb 

sudo dpkg -i linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb


You now need to make the new kernel bootable.
Get your module names.

cd /lib/modules ls

Update.sudo update-initramfs -ck module-name-for-new-kernel (in our case "update-initramfs -ck 2.6.24-rc5-custom_2.6.24-rc5-custom-10.00") sudo update-grub


You are now ready to boot into your new kernel. Just make sure you select the new kernel when you boot.

sudo reboot




Warning:Building and using a custom kernel will make it very difficult to get support for your system. While it is a learning experience to compile your own kernel, you will not be allowed to file bugs on the custom-built kernel (if you do, they will be Rejected without further explanation). So it is left to you to customize the kernel. 



15 comments:

  1. What you have said is just the update of the current kernel to the new one which linus provides you over the git for free.

    ReplyDelete
  2. Oh, well... May I know that who are you? It says that you're anonymous person. yeah, you are right. There is another method for this which will surely customize the kernel. I'll publish that in my next article.

    ReplyDelete
  3. No seriously, what you are doing is all above the linux kernel. You just update it and tell it where to take the dump. I want to customize my kernel in deep. Still deep, like getting into the programs what they've written and customize it myself, I mean in the smaller parts.

    ReplyDelete
    Replies
    1. yeah, well... this dump is one of the advanced features of kernel(you may know it). By using dump, you can recover the crashed kernel. If you want to customize it still very deep, then I'll publish another article about it to satisfy your curiosity...

      Delete
    2. Thank you. That would be more helpful.

      Delete
    3. Welcome... So may I know that who are you? I'm full of curiosity.....

      Delete
    4. Hope you enjoy my riddles.

      Delete
  4. Why is a raven like a writing desk?

    ReplyDelete
    Replies
    1. Oh.... That riddle was written by Lewis Carrol who claimed that there is no answer for it. But I guess that 'never' was written as 'nevar'. When written backwards, it becomes 'raven'. A pun by Lewis Carrol.

      Delete
    2. nice. Now that was easy to crack. Next:
      I have a little lilliputian in me as well as Blefuscian. I have never spoken to anyone. I am old, still won't speak to anyone. Who am i?

      Delete
    3. Oh, man.... This riddle is hard to crack.... I'll try to crack it....

      Delete
    4. And one more thing. I wonder why have you blocked copy from your blog. It is just a tedious process that one goes through.

      Delete
    5. Okay.... if you feel that it is a tedious task, then I'll re-enable copy....

      Delete
  5. You can change the background so that the text will be clearly visible and much more easier to read.

    ReplyDelete
  6. @sunil: yeah, I'll do it :) I want my blog to be comfortable to the readers :)

    ReplyDelete