Help to setup enviornment

Help to setup enviornment

[Sumitra Sharma]

Thank you Julia for your response. I followed your instructions and  get rid of the earlier mentioned error. But on running the command "make -j2" it shows the error:
##
Killed
make[1]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] Error 137
make[1]: *** Deleting file 'vmlinux.o'
make: *** [Makefile:1231: vmlinux_o] Error 2
##
As per the system logs, the process is killed due to insufficent memory.
I am running the linux on windows machine in VMPlayer and I have provided 4.6 GB of RAM to this machine. My systems RAM has a limit of 8 GB. I would appreciate some more help in it. Thank you.  

Re: Help to setup enviornment

[Julia Lawall]

On Wed, 8 Mar 2023, Sumitra Sharma wrote:

> Thank you Julia for your response. I followed your instructions and  get rid of the earlier mentioned error. But on running the command "make -j2" it shows the error:
> ##
> Killed
> make[1]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] Error 137
> make[1]: *** Deleting file 'vmlinux.o'
> make: *** [Makefile:1231: vmlinux_o] Error 2
> ##
> As per the system logs, the process is killed due to insufficent memory.
> I am running the linux on windows machine in VMPlayer and I have provided 4.6 GB of RAM to this machine. My systems RAM has a limit of 8 GB. I would appreciate some more help in it. Thank you.

Maybe just add more memory?  6 instead of 4?

julia

Re: Help to setup enviornment

[Deepak R Varma]

On Wed, Mar 08, 2023 at 06:27:39PM +0100, Julia Lawall wrote:
> 
> 
> On Wed, 8 Mar 2023, Sumitra Sharma wrote:
> 
> > Thank you Julia for your response. I followed your instructions and  get rid of the earlier mentioned error. But on running the command "make -j2" it shows the error:
> > ##
> > Killed
> > make[1]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] Error 137
> > make[1]: *** Deleting file 'vmlinux.o'
> > make: *** [Makefile:1231: vmlinux_o] Error 2
> > ##
> > As per the system logs, the process is killed due to insufficent memory.
> > I am running the linux on windows machine in VMPlayer and I have provided 4.6 GB of RAM to this machine. My systems RAM has a limit of 8 GB. I would appreciate some more help in it. Thank you.
> 
> Maybe just add more memory?  6 instead of 4?

Hi Sumitra,
Less likely, but do ensure you have sufficient Hard drive space available to
accommodate intermediate files and the Linux Image files created during the build
process. I hope you have done the partitioning well during your base Linux OS
setup on the VM.

Deepak.

> 
> julia
> 

Re: Help to setup enviornment

[Fabio M. De Francesco]

On mercoledì 8 marzo 2023 18:27:39 CET Julia Lawall wrote:
> On Wed, 8 Mar 2023, Sumitra Sharma wrote:
> > Thank you Julia for your response. I followed your instructions and  get 
rid
> > of the earlier mentioned error. But on running the command "make -j2" it
> > shows the error: ##
> > Killed
> > make[1]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] Error 137
> > make[1]: *** Deleting file 'vmlinux.o'
> > make: *** [Makefile:1231: vmlinux_o] Error 2
> > ##
> > As per the system logs, the process is killed due to insufficient memory.
> > I am running the linux on windows machine in VMPlayer and I have provided
> > 4.6 GB of RAM to this machine. My systems RAM has a limit of 8 GB. I would
> > appreciate some more help in it. Thank you.
> Maybe just add more memory?  6 instead of 4?
> 
> julia

I'm pretty sure that Windows 10/11 won't run with less than 4GB left in the 
host.

Anyway, it's not clear whether we are talking about the lack of RAM (and where 
Sumitra sees too few memory... in the Linux VM or in the Windows host?) or 
about Storage?

@Sumitra, I wasn't aware of you running Linux in a VM, so my previous message 
may have confused you. I was assuming that you build in a Linux host and run 
(at least a Linux VM for tests.

Now your set-up is clear: Linux on Windows.

Can you please run `df -h` on your Linux VM and post the output? In the 
meantime post your `cat /proc/meminfo`. As said I am not sure about what kind 
of memory you are talking about. Let us see the output.

Thanks,

Fabio

Re: Help to setup enviornment

[Sumitra Sharma]

Hi Fabio, this is the below output:
##
sumitra@ubuntu:~$ df -h
##
Filesystem      Size  Used Avail Use% Mounted on
udev            2.2G     0  2.2G   0% /dev
tmpfs           449M  1.8M  447M   1% /run
/dev/sda5        39G   26G   12G  70% /
tmpfs           2.2G     0  2.2G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.2G     0  2.2G   0% /sys/fs/cgroup
/dev/loop0      128K  128K     0 100% /snap/bare/5
/dev/loop1      249M  249M     0 100% /snap/gnome-3-38-2004/99
/dev/loop2       66M   66M     0 100% /snap/gtk-common-themes/1519
/dev/loop3       64M   64M     0 100% /snap/core20/1828
/dev/loop4       62M   62M     0 100% /snap/core20/1328
/dev/loop5      347M  347M     0 100% /snap/gnome-3-38-2004/119
/dev/loop6       92M   92M     0 100% /snap/gtk-common-themes/1535
/dev/loop7       55M   55M     0 100% /snap/snap-store/558
/dev/loop8       46M   46M     0 100% /snap/snap-store/638
/dev/loop9       50M   50M     0 100% /snap/snapd/18357
/dev/sda1       511M  4.0K  511M   1% /boot/efi
tmpfs           449M   28K  449M   1% /run/user/1000
##

##
sumitra@ubuntu:~$ cat /proc/meminfo
##
MemTotal:        4593512 kB
MemFree:          128644 kB
MemAvailable:    2020080 kB
Buffers:           75072 kB
Cached:          1914592 kB
SwapCached:        55292 kB
Active:           352904 kB
Inactive:        3504468 kB
Active(anon):      14532 kB
Inactive(anon):  1853924 kB
Active(file):     338372 kB
Inactive(file):  1650544 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       1918356 kB
SwapFree:        1594028 kB
Dirty:            198312 kB
Writeback:          2700 kB
AnonPages:       1845196 kB
Mapped:           112436 kB
Shmem:               604 kB
KReclaimable:     193728 kB
Slab:             276024 kB
SReclaimable:     193728 kB
SUnreclaim:        82296 kB
KernelStack:       10080 kB
PageTables:        15768 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4215112 kB
Committed_AS:    4901348 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       58952 kB
VmallocChunk:          0 kB
Percpu:            94720 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      229184 kB
DirectMap2M:     3543040 kB
DirectMap1G:     1048576 kB
##

Apart from this, I was talking about the RAM and I have provided the hard disk space of 40GB to my VM machine. And I see a lack of RAM in the VM machine.

Re: Help to setup enviornment

[Julia Lawall]

On Wed, 8 Mar 2023, Sumitra Sharma wrote:

> Hi Fabio, this is the below output:
> ##
> sumitra@ubuntu:~$ df -h
> ##
> Filesystem      Size  Used Avail Use% Mounted on
> udev            2.2G     0  2.2G   0% /dev
> tmpfs           449M  1.8M  447M   1% /run
> /dev/sda5        39G   26G   12G  70% /
> tmpfs           2.2G     0  2.2G   0% /dev/shm
> tmpfs           5.0M  4.0K  5.0M   1% /run/lock
> tmpfs           2.2G     0  2.2G   0% /sys/fs/cgroup
> /dev/loop0      128K  128K     0 100% /snap/bare/5
> /dev/loop1      249M  249M     0 100% /snap/gnome-3-38-2004/99
> /dev/loop2       66M   66M     0 100% /snap/gtk-common-themes/1519
> /dev/loop3       64M   64M     0 100% /snap/core20/1828
> /dev/loop4       62M   62M     0 100% /snap/core20/1328
> /dev/loop5      347M  347M     0 100% /snap/gnome-3-38-2004/119
> /dev/loop6       92M   92M     0 100% /snap/gtk-common-themes/1535
> /dev/loop7       55M   55M     0 100% /snap/snap-store/558
> /dev/loop8       46M   46M     0 100% /snap/snap-store/638
> /dev/loop9       50M   50M     0 100% /snap/snapd/18357
> /dev/sda1       511M  4.0K  511M   1% /boot/efi
> tmpfs           449M   28K  449M   1% /run/user/1000
> ##
>
> ##
> sumitra@ubuntu:~$ cat /proc/meminfo
> ##
> MemTotal:        4593512 kB
> MemFree:          128644 kB
> MemAvailable:    2020080 kB
> Buffers:           75072 kB
> Cached:          1914592 kB
> SwapCached:        55292 kB
> Active:           352904 kB
> Inactive:        3504468 kB
> Active(anon):      14532 kB
> Inactive(anon):  1853924 kB
> Active(file):     338372 kB
> Inactive(file):  1650544 kB
> Unevictable:           0 kB
> Mlocked:               0 kB
> SwapTotal:       1918356 kB
> SwapFree:        1594028 kB
> Dirty:            198312 kB
> Writeback:          2700 kB
> AnonPages:       1845196 kB
> Mapped:           112436 kB
> Shmem:               604 kB
> KReclaimable:     193728 kB
> Slab:             276024 kB
> SReclaimable:     193728 kB
> SUnreclaim:        82296 kB
> KernelStack:       10080 kB
> PageTables:        15768 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:     4215112 kB
> Committed_AS:    4901348 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:       58952 kB
> VmallocChunk:          0 kB
> Percpu:            94720 kB
> HardwareCorrupted:     0 kB
> AnonHugePages:         0 kB
> ShmemHugePages:        0 kB
> ShmemPmdMapped:        0 kB
> FileHugePages:         0 kB
> FilePmdMapped:         0 kB
> HugePages_Total:       0
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> Hugetlb:               0 kB
> DirectMap4k:      229184 kB
> DirectMap2M:     3543040 kB
> DirectMap1G:     1048576 kB
> ##
>
> Apart from this, I was talking about the RAM and I have provided the hard disk space of 40GB to my VM machine. And I see a lack of RAM in the VM machine.

Did you try with a smaller configuration?  One might guess that the
problem is at link time, and having less to link could be helpful.

julia

Re: Help to setup enviornment

[Fabio M. De Francesco]

On giovedì 9 marzo 2023 08:16:28 CET Sumitra Sharma wrote:
> Hi Fabio, this is the below output:
> ##
> sumitra@ubuntu:~$ df -h
> ##
> Filesystem      Size  Used Avail Use% Mounted on
> udev            2.2G     0  2.2G   0% /dev
> tmpfs           449M  1.8M  447M   1% /run
> /dev/sda5        39G   26G   12G  70% /
> tmpfs           2.2G     0  2.2G   0% /dev/shm
> tmpfs           5.0M  4.0K  5.0M   1% /run/lock
> tmpfs           2.2G     0  2.2G   0% /sys/fs/cgroup
> /dev/loop0      128K  128K     0 100% /snap/bare/5
> /dev/loop1      249M  249M     0 100% /snap/gnome-3-38-2004/99
> /dev/loop2       66M   66M     0 100% /snap/gtk-common-themes/1519
> /dev/loop3       64M   64M     0 100% /snap/core20/1828
> /dev/loop4       62M   62M     0 100% /snap/core20/1328
> /dev/loop5      347M  347M     0 100% /snap/gnome-3-38-2004/119
> /dev/loop6       92M   92M     0 100% /snap/gtk-common-themes/1535
> /dev/loop7       55M   55M     0 100% /snap/snap-store/558
> /dev/loop8       46M   46M     0 100% /snap/snap-store/638
> /dev/loop9       50M   50M     0 100% /snap/snapd/18357
> /dev/sda1       511M  4.0K  511M   1% /boot/efi
> tmpfs           449M   28K  449M   1% /run/user/1000
> ##
> 
> ##
> sumitra@ubuntu:~$ cat /proc/meminfo
> ##
> MemTotal:        4593512 kB
> MemFree:          128644 kB
> MemAvailable:    2020080 kB
> Buffers:           75072 kB
> Cached:          1914592 kB
> SwapCached:        55292 kB
> Active:           352904 kB
> Inactive:        3504468 kB
> Active(anon):      14532 kB
> Inactive(anon):  1853924 kB
> Active(file):     338372 kB
> Inactive(file):  1650544 kB
> Unevictable:           0 kB
> Mlocked:               0 kB
> SwapTotal:       1918356 kB
> SwapFree:        1594028 kB
> Dirty:            198312 kB
> Writeback:          2700 kB
> AnonPages:       1845196 kB
> Mapped:           112436 kB
> Shmem:               604 kB
> KReclaimable:     193728 kB
> Slab:             276024 kB
> SReclaimable:     193728 kB
> SUnreclaim:        82296 kB
> KernelStack:       10080 kB
> PageTables:        15768 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:     4215112 kB
> Committed_AS:    4901348 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:       58952 kB
> VmallocChunk:          0 kB
> Percpu:            94720 kB
> HardwareCorrupted:     0 kB
> AnonHugePages:         0 kB
> ShmemHugePages:        0 kB
> ShmemPmdMapped:        0 kB
> FileHugePages:         0 kB
> FilePmdMapped:         0 kB
> HugePages_Total:       0
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> Hugetlb:               0 kB
> DirectMap4k:      229184 kB
> DirectMap2M:     3543040 kB
> DirectMap1G:     1048576 kB
> ##
> 
> Apart from this, I was talking about the RAM and I have provided the hard 
disk
> space of 40GB to my VM machine. And I see a lack of RAM in the VM machine.

Sorry, I couldn't answer before...

Anyway in the meantime I've read that you finally managed to build your own 
custom kernel increasing the reserved RAM from about 4GB to 6GB.

FYI, it looks like you have _not_ any shortage of RAM in the VM, if we look at 
/proc/meminfo at rest (I mean when you're not building the kernel). I assume 
you think you have no memory available due to the ratio of the following two 
lines:

> MemTotal:        4593512 kB
> MemFree:          128644 kB

Look at the third line:

> MemAvailable:    2020080 kB

Linux keeps all it can keep in RAM for future fast re-use, so it can avoid 
many unnecessary page-faults when processes try to access again many of the 
same memory addresses which were in use in the past and may be re-used later 
(locality principle).

This is the output in my system:

suse:~ # cat /proc/meminfo
MemTotal:       32768064 kB
MemFree:          535508 kB
MemAvailable:   22111212 kB

I have about 22GB of immediately re-usable RAM while, at the moment, about the 
95% of my 32 GB are in use. I can count on a good amount of RAM, despite we 
see that "MemFree: 535508 kB"(I'm also running a QEMU/KVM VM on top of my host 
with 6 GB of reserved RAM).

From your output, the other lines provide very helpful insights about how 
exactly your memory is used (I won't go into these details).

I'm pretty sure that 4.5GB are enough to run a system for the kind of 
development you are going to do in this phase of the project. I think that 
perhaps you are running too many processes/threads at the same time of the 
builds.

Are you sure you configured your system not to run unnecessary services? What 
processes are eating the most RAM in your VM? Can they be disabled? Do you run 
any other interactive processes while building? If so, can you avoid to run 
them at the same time?

Use the "ps" command with the proper options to figure out how your system is 
using memory space for each process/thread. I think that the 2GB you left to 
the host (Windows 10? Windows 11?) will make you in trouble sooner or later.

Last thing... I think you should also listen to Julia and disable those kernel 
options which build unnecessary features and drivers. Take a look at your 
.config file with `make menuconfig` (or similar commands: type `make help` for 
a summary) and disable those  unnecessary options that your distro enables by 
default if you don't need them (as a bonus you will compile and link much 
faster). 

Use the options help pages (when you run "make menuconfig" and are at an 
option you want to know more just select "help" or press 'h') if you are not 
sure about what you are disabling and be very careful to leave the options you 
are not sure about as they currently are.

Fabio

Re: Help to setup enviornment

[Sumitra Sharma]

Hi Julia

Upgrading the VM RAM from 4 to 6 worked. Thank you for the help.

Re: Help to setup enviornment

[Julia Lawall]

On Thu, 9 Mar 2023, Sumitra Sharma wrote:

> Hi Julia
>
> Upgrading the VM RAM from 4 to 6 worked. Thank you for the help.

Great!

julia

Re: Help to setup enviornment

[Julia Lawall]

On Thu, 9 Mar 2023, Sumitra Sharma wrote:

> Hi Julia
>
> Upgrading the VM RAM from 4 to 6 worked. Thank you for the help.

Sumitra,

Your email setup generates very strange reply to lines, eg:

Reply-To: dd924cf4-ddd6-45aa-292d-5762b769ca22@inria.fr

julia

Re: Help to setup enviornment

[Sumitra Sharma]

Surely julia. I will look to it and trying setting it up again.