Computing resources required for kernel development

Computing resources required for kernel development

[Harsh chopra]

[-- Attachment #1.1: Type: text/plain, Size: 565 bytes --]

Hi everyone,
I have been following the Kernel Hacking steps to submit my first patch but
I am facing problem in the kernel compilation as its taking more than a day
may be due to low computing resources..
I have Ubuntu 20.04 LTS with 4GB  DDR3 ram 1 TB HDD, Intel i3  2.GHZ and 4
cores but I have been using virt-manager for the kernel development with
CentOS 8.1 (minimal req.) and  resources 70 GB storage, 2 GB ram and 2
cores.
Thus if anyone could suggest how much computing power to have or any other
method so that I could make my development process smooth.

[-- Attachment #1.2: Type: text/html, Size: 631 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Computing resources required for kernel development

[Suraj Upadhyay]

[-- Attachment #1.1: Type: text/plain, Size: 1262 bytes --]

On Sat 30 May, 2020, 20:57 Harsh chopra, <serviceprovider.tester@gmail.com>
wrote:

> Hi everyone,
> I have been following the Kernel Hacking steps to submit my first patch
> but I am facing problem in the kernel compilation as its taking more than a
> day may be due to low computing resources..
> I have Ubuntu 20.04 LTS with 4GB  DDR3 ram 1 TB HDD, Intel i3  2.GHZ and 4
> cores but I have been using virt-manager for the kernel development with
> CentOS 8.1 (minimal req.) and  resources 70 GB storage, 2 GB ram and 2
> cores.
>
Thus if anyone could suggest how much computing power to have or any other
> method so that I could make my development process smooth.
>

The configurations of your pc are fine.

My personal computer has the same specs with a slight difference that I
have 12 GiBs of main memory.

I had the same problem in the beginning (tho I started only a few months
ago).

I suggest you use `make -j4` or a higher number for compilation.

And now my system does the compilation in less than 30 minutes.

What options are you using while building the kernel ??


_______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>

[-- Attachment #1.2: Type: text/html, Size: 3006 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Computing resources required for kernel development

[Harsh chopra]

[-- Attachment #1.1: Type: text/plain, Size: 1407 bytes --]

I used
make -j2 all

On Sun, 31 May 2020, 00:14 Suraj Upadhyay, <usuraj35@gmail.com> wrote:

>
> On Sat 30 May, 2020, 20:57 Harsh chopra, <serviceprovider.tester@gmail.com>
> wrote:
>
>> Hi everyone,
>> I have been following the Kernel Hacking steps to submit my first patch
>> but I am facing problem in the kernel compilation as its taking more than a
>> day may be due to low computing resources..
>> I have Ubuntu 20.04 LTS with 4GB  DDR3 ram 1 TB HDD, Intel i3  2.GHZ and
>> 4 cores but I have been using virt-manager for the kernel development with
>> CentOS 8.1 (minimal req.) and  resources 70 GB storage, 2 GB ram and 2
>> cores.
>>
> Thus if anyone could suggest how much computing power to have or any other
>> method so that I could make my development process smooth.
>>
>
> The configurations of your pc are fine.
>
> My personal computer has the same specs with a slight difference that I
> have 12 GiBs of main memory.
>
> I had the same problem in the beginning (tho I started only a few months
> ago).
>
> I suggest you use `make -j4` or a higher number for compilation.
>
> And now my system does the compilation in less than 30 minutes.
>
> What options are you using while building the kernel ??
>
>
> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies@kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>

[-- Attachment #1.2: Type: text/html, Size: 3526 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Computing resources required for kernel development

[Greg KH]

On Sun, May 31, 2020 at 11:42:16AM +0530, Harsh chopra wrote:
> I used
> make -j2 all

The number for -j should be at least the number of cores/cpus in your
system, and usually double works well.  I use -j10 for my tiny laptop
and that runs quite nicely.

I suggest running the kcbench tool to get a sense of how fast your
machine is, and what the best -j number is for your system:
	https://gitlab.com/knurd42/kcbench.git

Also, don't build the "full" kernel, only build what you need for your
hardware + whatever driver/code you are working on.  Read up on the
'make localmodconfig' kernel build option to have it create a
configuration file that is _MUCH_ smaller than all options, which will
make the build times very quick.

Good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Computing resources required for kernel development

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 2938 bytes --]

On Sat, 30 May 2020 20:56:21 +0530, Harsh chopra said:

> I have been following the Kernel Hacking steps to submit my first patch but
> I am facing problem in the kernel compilation as its taking more than a day
> may be due to low computing resources..

On a Raspberry Pi 4 with 4G of RAM, overclocked to 2Ghz, and using a 256G USB3
SSD as the system disk instead of an SD card, a 'make allmodconfig' has to
build 28,302 .o files and from a tree that had a 'make clean', a 'make' takes:

real	585m58.387s
user	537m15.917s
sys	65m59.296s

Note that the RPi4 has 4 cores, so a 'make -j4' will go even faster, but you
will need a case that provides cooling for the CPU/GPU, especially if
overclocked. (And just a few days ago, the RPi crew dropped a new firmware that
allows booting straight from a USB device, so no SD card is needed at all)

Meanwhile, a config for my laptop that's been somewhat minimized by
using 'make localmodconfig' only has to build 4,996 .o files (and I could probably
trim that back a whole bunch), which makes kernel builds about 8 times faster than
an 'allmodconfig' :)

How to:
Step 1 - boot a current kernel.
Step 2 - plug in any and all USB disks, cameras, keyboards/mice and anything
else that you might have, so their drivers get loaded. Make sure that you get
everything, because if you miss something, the device won't work because there
won't be a driver.
Step 3 - cd to wherever you're doing your kernel builds
Step 4 - 'make localmodconfig'
Step 5 - make and install your new kernel config.

Note that if you're doing patches for code in modules you don't have loaded, you
may need to go do 'make menuconfig' to enable building the the module. 

> I have Ubuntu 20.04 LTS with 4GB  DDR3 ram 1 TB HDD, Intel i3  2.GHZ and 4
> cores but I have been using virt-manager for the kernel development with
> CentOS 8.1 (minimal req.) and  resources 70 GB storage, 2 GB ram and 2
> cores.

That's not that much different from the RPi I mentioned above except for the
HDD rather than an SSD.  Note that even a smaller SSD for just your kernel
source will get you most of the bang for your buck, because gcc, the most
heavily used *.h files, and other stuff like that should remain in memory so
after the first few modules are compiled, pretty much all your disk I/O is for
*.c files from the source tree.   And although USB3 is preferable, you'll get
a speedup even if you only have USB2 ports available.

> Thus if anyone could suggest how much computing power to have or any other
> method so that I could make my development process smooth.

Work smarter, not harder.

The two biggest things you can do to speed things up are getting an SSD for
your system disk, and 'localmodconfig' to minimize the amount of extraneous
stuff you have to build.  The SSD may be more money than you're willing
to spend, but the other suggestion is essentially free.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies