Hi, fellas.....
I hoped that you have learned something from my previous articles. This article is going to give you a brief overview on how to develop one's own kernel. As we know, kernel is like the heart of any operating system. It defines user space and system space clearly and carries out the instructions given by the user. Shell is the inter-actor between kernel and the user. So this kernel plays a very important role while the system is being run. So some of you may have wondered that why isn't it possible to write a custom kernel? it is possible, but is highly risky. Only advanced users of gnu/linux operating systems are advised to try writing custom kernel.
I think that this article is the most advanced article when compared to my previous articles. So read every word of this article carefully.
Some keywords that should be learned before creating customized kernel:
fakeroot: Wondering that what is fakeroot? To gain the root facilities in any gnu/linux operating systems, you need to enter a command
sudo -i
kexec-tools: kexec-tools provides the /sbin/kexec binary that facilitates a new kernel to boot using the kernel's kexec feature either on a normal or a panic reboot. This package contains the /sbin/kexec binary and ancillary utilities that together form the userspace component of the kernel's kexec feature.
makedumpfile: As the name suggests, makedumpfile will create a dump file, i.e., it will stripe down the original dump file to a smaller dump file, thus saving the memory, smaller dump files would require less RAM. If the dump level is set to a level producing a large dump file, (0 produces the largest dump files, 31 the smallest), the dump file might become very large on systems with a lot of RAM. Under such situations the package 'makedumpfile' can help to us.
kernel-wedge: It is the udeb package builder for Debian-Installer, i.e., kernel-wedge splits udeb packages out of a kernel image deb. The udebs are used by the Debian-Installer.
Let us start creating our own kernel(s). Some of you may know that many kernels can co-exist in a GNU/LINUX operating system. So creating a custom kernel will affect all the kernels which exist in a GNU/LINUX operating system.
Step 1: There are few Prerequisites that has to be satisfied before creating custom kernel.
The 'git'('git-core' in newer versions of ubuntu) provides git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which can be installed. You will need to install both of these packages. We'll need these 2 packages before creating a customized kernel.
Note: In older versions of Ubuntu, we should replace libncurses5-dev with ncurses-dev.
P.S: If you are a developer or have messed with similar stuff, then you most probably have some or even all of those packages already installed. But it doesn’t hurt to execute this command anyways. You also have to make sure that all the dependencies for the Linux build are satisfied. So, just type the following:
Step 2: Now you'll have everything that is required to create a customized kernel. So why don't we start creating customized kernel? Oh... Hold on!! It is better if you create a work-space and try creating the kernel. The default work-space would be you home directory. So let us create a path to your home directory through this command:
Then clone the repository in the home directory.
After knowing the exact version string, it will be easy to get the right patch for proper Reiser4 support. Make sure you also select the right patch for your own kernel version. Please note that Linux stable versions appear like flies.
If everything went well you will be presented with the following question. Agree by pressing the [Y] key.
Immediately after that you’re going to be asked about the timer frequency. Go for a low one if you have a laptop or a not-so-powerful computer.
Next, just press [Enter] to all the other questions until you see this one:
Instead of incorporating Reiser4 support into our kernel we choose to build it as a dynamically loadable module instead, so we press the [M] key. We don’t want debugging support for Reiser4, so in the following question we press [N].
For any other question just select the default answer by pressing [Enter]. In case you’d like to configure any special options, you can also type
Build it like a real builder:
We’re almost done. We just have to build our new kernel, after all this hard work.
For the next command you’ll most definitely have to make two adjustments. First, give to the –append-to-version parameter a name you’d like your new kernel to have. Second, change the value of –overlay-dir to the full path of the directory the kernel source resides.
Be patient and wait for the compilation to finish. Depending on the raw processing power or lack thereof- of your computer’s CPU, the whole process may take up to an hour or even more. In any case, a gazillion of messages will fly by your terminal window. Sit back, grab a bowl of pop corns or some kur kur dish and enjoy.
Installation:
If everything went well and I really can’t imagine a reason if they don’t, after the compilation finishes you’ll find a couple of DEB files in ~/dev/kernel. (P.S: Everything till here should go perfectly without any glitches or errors). Use below commands to find the DEB files in ~/dev/kernel directory.
And now it’s one of those rare times where in order for the changes to take effect you need to reboot your Linux system.
Change to the directory where you want to clone the git tree. In this example we will use $HOME
Clone the mainline kernel git tree.
Change to linux directory.
Copy the kernel config file from your existing system to the kernel tree.
Bring the config file up to date. Answer any questions that get prompted. Unless you know you are interested in a particular feature, accepting the default option by pressing Enter should be a safe choice.
In cases where your kernel source is significantly newer than the existing config file, you'll be presented with all of the new config options for which there is no existing config file setting. You can either sit there and keep hitting Enter to take the default (generally safe), or you can just run
which emulates exactly the same thing and saves you all that time.
(optional) If you need to make any kernel config changes, do the following and save your changes when prompted:
Note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be (linux-image*.deb will be 200-300 MB instead of 20-30 MB). To turn this off, go into "Kernel hacking"; then, under "Kernel debugging", turn OFF "Compile the kernel with debug info".
In addition, if you want to build from a git non-HEAD version, also comment out the lines extra plus at about line 168:
Clean the kernel source directory.
Build the linux-image and linux-header .deb files (CONCURRENCY_LEVEL can also be set manually to how many CPUs/cores to use to build the kernel). This process takes a lot of time.
With this command the package names will be something like linux-image-2.6.24-rc5-custom and linux-headers-2.6.24-rc5-custom, and in that case the version will be 2.6.24-rc5-custom-10.00.Custom. You may change the string custom into something else by changing the --append-to-version option. You may also change the default 10.00.Custom into something else by using the --revision option.
Change to one directory level up (this is where the linux-image and linux-header .deb files were put)
Now install the .deb files. In this example, the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb. You may receive warnings about '/lib/firmware/2.6.24-rc5-custom/' - this is expected and will only be problematic if the driver you are trying to test requires firmware.
sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
You now need to make the new kernel bootable.
Get your module names.
cd /lib/modules
ls
You are now ready to boot into your new kernel. Just make sure you select the new kernel when you boot.
I hoped that you have learned something from my previous articles. This article is going to give you a brief overview on how to develop one's own kernel. As we know, kernel is like the heart of any operating system. It defines user space and system space clearly and carries out the instructions given by the user. Shell is the inter-actor between kernel and the user. So this kernel plays a very important role while the system is being run. So some of you may have wondered that why isn't it possible to write a custom kernel? it is possible, but is highly risky. Only advanced users of gnu/linux operating systems are advised to try writing custom kernel.
I think that this article is the most advanced article when compared to my previous articles. So read every word of this article carefully.
Some keywords that should be learned before creating customized kernel:
sudo -i
Then shell will request you for the password. Enter it and you will get root control over your gnu/linux operating system.
Fakeroot works in the similar way. As the name suggests, it will provide a fake root environment. This command is used when we want to enable some thing like dpkg-buildpackage -rfakeroot, i.e. to remove the need to become root for a package build. This is done by setting LD_PRELOAD to libfakeroot. So, fakeroot provides wrappers around getuid, chown, chmod, mknod, stat etc, thereby creating a fake root environment. If you don't understand any of this, you do not need fakeroot!!!
Fakeroot works in the similar way. As the name suggests, it will provide a fake root environment. This command is used when we want to enable some thing like dpkg-buildpackage -rfakeroot, i.e. to remove the need to become root for a package build. This is done by setting LD_PRELOAD to libfakeroot. So, fakeroot provides wrappers around getuid, chown, chmod, mknod, stat etc, thereby creating a fake root environment. If you don't understand any of this, you do not need fakeroot!!!
Let us start creating our own kernel(s). Some of you may know that many kernels can co-exist in a GNU/LINUX operating system. So creating a custom kernel will affect all the kernels which exist in a GNU/LINUX operating system.
Step 1: There are few Prerequisites that has to be satisfied before creating custom kernel.
The 'git'('git-core' in newer versions of ubuntu) provides git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which can be installed. You will need to install both of these packages. We'll need these 2 packages before creating a customized kernel.
sudo apt-get -y install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge git-core libncurses5 libncurses5-dev libelf-dev libdw-dev binutils-dev
Note: In older versions of Ubuntu, we should replace libncurses5-dev with ncurses-dev.
P.S: If you are a developer or have messed with similar stuff, then you most probably have some or even all of those packages already installed. But it doesn’t hurt to execute this command anyways. You also have to make sure that all the dependencies for the Linux build are satisfied. So, just type the following:
sudo apt-get build-dep kernel-package linux-meta
Step 2: Now you'll have everything that is required to create a customized kernel. So why don't we start creating customized kernel? Oh... Hold on!! It is better if you create a work-space and try creating the kernel. The default work-space would be you home directory. So let us create a path to your home directory through this command:
cd $home
Then clone the repository in the home directory.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
The above command would clone the default git repositories.
There is another command which will clone the natty git repositories.
git clone git://kernel.ubuntu.com/ubuntu/ubuntu-natty.git source
It is left you to choose between default git repositories and natty git repositories.
Note: Above commands would take some time to get executed. Patience is very important.
Step 3: Take a note of the current kernel that you are using. Use this command to check the version of kernel that you are using. In my case, it is 3.5.0-19-generic.
uname -r
Note: From here there will be two different types of creating customized kernel.
1st type: Customizing the kernel by using patches.
Now create a file by the name 'patches', and create a path to that new file through the terminal.
1st type: Customizing the kernel by using patches.
Now create a file by the name 'patches', and create a path to that new file through the terminal.
mkdir patches && cd patches/
Now download the patches to the file patches via this command:
wget http://www.kernel.org/pub/linux/kernel/people/edward/reiser4/reiser4-for-2.6/reiser4-for-2.6.38.patch.gz
Note: This command is only for 2.6.38.xx kernel versions. For higher versions of kernel, i.e., higher than 3.x.xx.xx versions of kernels) download the suitable patches.
Great!! Now the patches are ready to be incorporated into the Ubuntu kernel. Since we've put all the patches in a single directory, it is damn easy to incorporate all the patches into kernel. Let us do it:
cd ../source/
Now comes the configuration part. Do it very carefully. Suppose that you’re not a customization maniac, the default config file of the standard Ubuntu kernel, i.e. the one describing the features of the currently running kernel, will suit you well. You’re going to make a few changes here and there, but when you’re using the default config as a base the whole process is pretty easy and totally safe.
cp -vi /boot/config-$(uname -r) .config
Now expand the compressed patch file:
gunzip reiser4-for-2.6.38.patch.gz
Great!! Now the patches are ready to be incorporated into the Ubuntu kernel. Since we've put all the patches in a single directory, it is damn easy to incorporate all the patches into kernel. Let us do it:
cd ../source/
patch -p1 < ../patches/reiser4-for-2.6.38.patch
Now comes the configuration part. Do it very carefully. Suppose that you’re not a customization maniac, the default config file of the standard Ubuntu kernel, i.e. the one describing the features of the currently running kernel, will suit you well. You’re going to make a few changes here and there, but when you’re using the default config as a base the whole process is pretty easy and totally safe.
cp -vi /boot/config-$(uname -r) .config
make oldconfig
If everything went well you will be presented with the following question. Agree by pressing the [Y] key.
BFS cpu scheduler (SCHED_BFS) [Y/n/?] (NEW)
Immediately after that you’re going to be asked about the timer frequency. Go for a low one if you have a laptop or a not-so-powerful computer.
Timer frequency 1. 100 HZ (HZ_100)
2. 250 HZ (HZ_250_NODEFAULT) (NEW)
3. 300 HZ (HZ_300)
4. 1000 HZ (HZ_1000)
5. 1500 HZ (HZ_1500) (NEW)
6. 2000 HZ (HZ_2000) (NEW)
7. 3000 HZ (HZ_3000) (NEW)
8. 4000 HZ (HZ_4000) (NEW)
9. 5000 HZ (HZ_5000) (NEW)
10. 7500 HZ (HZ_7500) (NEW)
11. 10000 HZ (HZ_10000) (NEW)
choice[1-11?]:
Reiser4 (EXPERIMENTAL) (REISER4_FS) [N/m/y/?] (NEW)
Instead of incorporating Reiser4 support into our kernel we choose to build it as a dynamically loadable module instead, so we press the [M] key. We don’t want debugging support for Reiser4, so in the following question we press [N].
Enable reiser4 debug mode (REISER4_DEBUG) [N/y/?] (NEW)
For any other question just select the default answer by pressing [Enter]. In case you’d like to configure any special options, you can also type
make menuconfig
Build it like a real builder:
We’re almost done. We just have to build our new kernel, after all this hard work.
make-kpkg clean
For the next command you’ll most definitely have to make two adjustments. First, give to the –append-to-version parameter a name you’d like your new kernel to have. Second, change the value of –overlay-dir to the full path of the directory the kernel source resides.
nice fakeroot make-kpkg --initrd --append-to-version=-lockard --overlay dir=/home/gtklocker/dev/kernel/source kernel-image kernel-headers
If everything went well and I really can’t imagine a reason if they don’t, after the compilation finishes you’ll find a couple of DEB files in ~/dev/kernel. (P.S: Everything till here should go perfectly without any glitches or errors). Use below commands to find the DEB files in ~/dev/kernel directory.
cd ..ls -lh
Install those DEB files.
sudo dpkg -i linux-*.deb
2nd type: Using default kernel config file for customization.
Note: If you're using 2nd type to customize your kernel, then ignore step 2. But carry on with step 1.
Change to the directory where you want to clone the git tree. In this example we will use $HOME
cd $HOME
Clone the mainline kernel git tree.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Change to linux directory.
cd linux
Copy the kernel config file from your existing system to the kernel tree.
cp /boot/config-`uname -r` .config
Bring the config file up to date. Answer any questions that get prompted. Unless you know you are interested in a particular feature, accepting the default option by pressing Enter should be a safe choice.
make oldconfig
In cases where your kernel source is significantly newer than the existing config file, you'll be presented with all of the new config options for which there is no existing config file setting. You can either sit there and keep hitting Enter to take the default (generally safe), or you can just run
yes '' | make oldconfig
which emulates exactly the same thing and saves you all that time.
(optional) If you need to make any kernel config changes, do the following and save your changes when prompted:
make menuconfig
Note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be (linux-image*.deb will be 200-300 MB instead of 20-30 MB). To turn this off, go into "Kernel hacking"; then, under "Kernel debugging", turn OFF "Compile the kernel with debug info".
Edit scripts/setlocalversion and comment out the line "echo "+"". If this is not done, make-kpkg will fail
to build the packages with an error "package linux-image-[kernel version]-custom+ not in control info".
to build the packages with an error "package linux-image-[kernel version]-custom+ not in control info".
sed -rie 's/echo "\+"/#echo "\+"/' scripts/setlocalversion
In addition, if you want to build from a git non-HEAD version, also comment out the lines extra plus at about line 168:
- if test "${LOCALVERSION+set}" != "set"; then - scm=$(scm_version --short) - res="$res${scm:++}" - fi + #if test "${LOCALVERSION+set}" != "set"; then + # scm=$(scm_version --short) + # res="$res${scm:++}" + #fi
Clean the kernel source directory.
make-kpkg clean
Build the linux-image and linux-header .deb files (CONCURRENCY_LEVEL can also be set manually to how many CPUs/cores to use to build the kernel). This process takes a lot of time.
CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom kernel_i
With this command the package names will be something like linux-image-2.6.24-rc5-custom and linux-headers-2.6.24-rc5-custom, and in that case the version will be 2.6.24-rc5-custom-10.00.Custom. You may change the string custom into something else by changing the --append-to-version option. You may also change the default 10.00.Custom into something else by using the --revision option.
Change to one directory level up (this is where the linux-image and linux-header .deb files were put)
cd ..
Now install the .deb files. In this example, the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb. You may receive warnings about '/lib/firmware/2.6.24-rc5-custom/' - this is expected and will only be problematic if the driver you are trying to test requires firmware.
sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
sudo dpkg -i linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
You now need to make the new kernel bootable.
Get your module names.
cd /lib/modules
ls
Update.sudo update-initramfs -ck module-name-for-new-kernel (in our case "update-initramfs -ck 2.6.24-rc5-custom_2.6.24-rc5-custom-10.00") sudo update-grub
You are now ready to boot into your new kernel. Just make sure you select the new kernel when you boot.
What you have said is just the update of the current kernel to the new one which linus provides you over the git for free.
ReplyDeleteOh, well... May I know that who are you? It says that you're anonymous person. yeah, you are right. There is another method for this which will surely customize the kernel. I'll publish that in my next article.
ReplyDeleteNo seriously, what you are doing is all above the linux kernel. You just update it and tell it where to take the dump. I want to customize my kernel in deep. Still deep, like getting into the programs what they've written and customize it myself, I mean in the smaller parts.
ReplyDeleteyeah, well... this dump is one of the advanced features of kernel(you may know it). By using dump, you can recover the crashed kernel. If you want to customize it still very deep, then I'll publish another article about it to satisfy your curiosity...
DeleteThank you. That would be more helpful.
DeleteWelcome... So may I know that who are you? I'm full of curiosity.....
DeleteHope you enjoy my riddles.
DeleteWhy is a raven like a writing desk?
ReplyDeleteOh.... That riddle was written by Lewis Carrol who claimed that there is no answer for it. But I guess that 'never' was written as 'nevar'. When written backwards, it becomes 'raven'. A pun by Lewis Carrol.
Deletenice. Now that was easy to crack. Next:
DeleteI have a little lilliputian in me as well as Blefuscian. I have never spoken to anyone. I am old, still won't speak to anyone. Who am i?
Oh, man.... This riddle is hard to crack.... I'll try to crack it....
DeleteAnd one more thing. I wonder why have you blocked copy from your blog. It is just a tedious process that one goes through.
DeleteOkay.... if you feel that it is a tedious task, then I'll re-enable copy....
DeleteYou can change the background so that the text will be clearly visible and much more easier to read.
ReplyDelete@sunil: yeah, I'll do it :) I want my blog to be comfortable to the readers :)
ReplyDelete