Kernel Memory

Kernel Memory

[Vijay Chauhan]

Hello,

I am newbie.
It has been said "kernel memory is not pageable"
What does it mean? There is no concept of kernel virtual address?

Any simple explanation will help me to udnerstand.

Thanks,
Vijay

Kernel Memory

[HowKernel StuffWorks]

Hi,
  I am a kernel newbie too, Kernel memory is not pagable, because kernel
itself is responsible for paging. See this discussion,
http://kerneltrap.org/node/6404

  Paging happens for regular processes, i.e each process memory is divided
into a page of certain size(4kb in Linux), so it can swap for another page
that might be needed for that particular moment, it usually replaces an
page that have not been used for long time(see LRU,internal fragmentation).
Kernel is a process too, just like anyother process, but it differs from
others because it directly talks to the hardware and also it is the one
which takes care of paging and LRU algo's. So it does not make much sense,
for a kernel memory to be pageable, because, if it removes the page that
contains that has the paging algorithm, then there is  no way to come back,
you want to retrieve a page from disk_swap_area, but you cannot, because
the page that contains code to retrieve the instructions are paged :)...
This is not just for paging, since kernel controls everything, it is not
advisable to put its own code in swap area(see, virtual memory, virtual
memory = physical memory + swap space).

BTW, i am a kernel newbie too, this is my basic understanding, please feel
free to correct it, if I am wrong....

On Thu, Jun 21, 2012 at 8:27 AM, Vijay Chauhan <kernel.vijay@gmail.com>wrote:

> Hello,
>
> I am newbie.
> It has been said "kernel memory is not pageable"
> What does it mean? There is no concept of kernel virtual address?
>
> Any simple explanation will help me to udnerstand.
>
> Thanks,
> Vijay
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120621/282abbb9/attachment-0001.html 

Kernel Memory

[kishore sheik ahamed]

Hey Vijay

I am a newbie too. Just sharing what I could go through.

It is said that Kernel or atleast a part of kernel needs to be non paged
for fast interrupt access etc as pinned memory
Wiki says
Pinned/Locked/Fixed pages

Operating systems have memory areas that are pinned (never swapped to
secondary storage). For example,
interrupt<http://en.wikipedia.org/wiki/Interrupt>mechanisms rely on an
array of pointers to their handlers, such as
I/O <http://en.wikipedia.org/wiki/I/O> completion and page
fault<http://en.wikipedia.org/wiki/Page_fault>.
If the pages containing these pointers or the code that they invoke were
pageable, interrupt-handling would become far more complex and
time-consuming, particularly in the case of page fault interrupts. Hence,
some part of the page table structures is not pageable.

Some pages may be pinned for short periods of time, others may be pinned
for long periods of time, and still others may need to be permanently
pinned. For example:

   - The paging supervisor code and drivers for secondary storage devices
   on which pages reside must be permanently pinned, as otherwise paging
   wouldn't even work because the necessary code wouldn't be available.
   - Timing-dependent components may be pinned to avoid variable paging
   delays.
   - Data buffers <http://en.wikipedia.org/wiki/Data_buffer> that are
   accessed directly by peripheral devices that use direct memory
access<http://en.wikipedia.org/wiki/Direct_memory_access>or I/O
   channels <http://en.wikipedia.org/wiki/I/O_channel> must reside in
   pinned pages while the I/O operation is in progress because such devices
   and the buses <http://en.wikipedia.org/wiki/Bus_%28computing%29> to
   which they are attached expect to find data buffers located at physical
   memory addresses; regardless of whether the bus has a memory management
   unit for I/O <http://en.wikipedia.org/wiki/IOMMU>, transfers cannot be
   stopped if a page fault occurs and then restarted when the page fault has
   been processed.

There are other two discussion thread which say kernel is non-pageable and
now due to growing kernel Data structures it is allowed

http://kerneltrap.org/node/6404

http://kerneltrap.org/node/8206


Regards

Kishore


On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay@gmail.com>wrote:

> Hello,
>
> I am newbie.
> It has been said "kernel memory is not pageable"
> What does it mean? There is no concept of kernel virtual address?
>
> Any simple explanation will help me to udnerstand.
>
> Thanks,
> Vijay
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120621/9cd13cf5/attachment.html 

Kernel Memory

[Arun KS]

Hello Vijay,

On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay@gmail.com> wrote:
> Hello,
>
> I am newbie.
> It has been said "kernel memory is not pageable"
> What does it mean? There is no concept of kernel virtual address?
You might have heard about 3G/1G split. This 1GB is the virtual
address of the kernel. And whenever kernel try to access any address
in this range(for ARM architecture it is  0xC0000000 - 0xFFFFFFFF),
there should not be any page fault. That means MMU should be able to
convert your virtual address to physical. This 1GB contains your IO
address, RAM address.

Paging is a mechanism OS uses to pull the data(in pagesizes) to and
fro between system RAM and secondary memory.
Kernel memory is not pageable. This means memory allocated for the
kernel will not be pagged out. If you try to access any memory in
kernel with out creating page tables(this can be done by ioremap) you
will end up in OOPS. The main reason of kernel not being swapable or
pageable is as follows.
Think this way. What will happen if we have paged out that portion of
the logic which decides what to do when a page fault occurs? Who will
take care of the page fault then?

But if a user program hit a page fault(ie accessed address is not in
main memory), kernel will load the page from secondary memory if it is
a valid address. And if the address accesses is illegal, kernel kill
the user application(Segmentation fault).

Thanks,
Arun

>
> Any simple explanation will help me to udnerstand.
>
> Thanks,
> Vijay
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Kernel Memory

[AFT]

Vijay Chauhan <kernel.vijay@gmail.com> writes:

> Hello,
>
> I am newbie.
> It has been said "kernel memory is not pageable"
> What does it mean? There is no concept of kernel virtual address?
>

Yes. Kernel works on static adress space.

> Any simple explanation will help me to udnerstand.
>

I'm not sure if you want to understand "how kernel manages memory for
its internal DS". If its the case you should read the following
documents.

1) Read the chapter 8 of Linux Device driver 3rd edition.

http://lwn.net/Kernel/LDD3/

2) To understand slab allocator read the following papers by bonwick

   a) 94 paper describing slab allocator:
   
   http://static.usenix.org/publications/library/proceedings/bos94/full_papers/bonwick.a

   b) Its followup in 2001
   
   http://static.usenix.org/event/usenix01/full_papers/bonwick/bonwick_html/index.html

3) These should be enough. But if you want to know detailed architecture
of how virtual memory manager work you should read Gorman's book on
Linux virtual memory manager. Its a free pdf. can be found here:

http://ptgmedia.pearsoncmg.com/images/0131453483/downloads/gorman_book.pdf

Happy hacking.

Cheers
aft


> Thanks,
> Vijay
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Kernel Memory

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 17:57 Thu 21 Jun     , Vijay Chauhan wrote:
> Hello,
> 
> I am newbie.
> It has been said "kernel memory is not pageable"
> What does it mean? There is no concept of kernel virtual address?
> 
> Any simple explanation will help me to udnerstand.

The right term is actually "kernel memory is not swappable". Swapping means
writing inactive memory to disk and then using it for something else. Kernel
memory not being swappable is a design decicion made in the early linux days.
Operating systems which swap kernel memory need to isolate everything which
should not be swappd out (e.g. things needed for swap-in, realtime stuff,
security sensitive data, ...). This is quite a bit of work. I also guess it is
pretty pointless nowadays. Installed memory and is getting so huge that virtual
memory developers have a hard time trying to keep cpu-usage overhead for
swapping user space memory low.

> There is no concept of kernel virtual address?

Kernel memory uses virtual addresses as well. However, these the entire system
memory is continuously mapped somewhere in the virtual address space. The
drawback is that fragmentation turns allocation of large continuous memory
regions into a game of luck.

There is also an virtual address area (vmalloc) which is used to dynamically
map multiple scattered pages to a continuous region. But this is rather slow
and rarely used.

You might want to take a look at: http://lwn.net/Kernel/LDD3/

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

Kernel Memory

[Vijay Chauhan]

Thanks everyone. I think i got enough information for further study.

Thanks,
Vijay

On Thu, Jun 21, 2012 at 7:10 PM, kishore sheik ahamed
<linuxinme@gmail.com> wrote:
> Hey Vijay
>
> I am a newbie too. Just sharing what I could go through.
>
> It is said that Kernel or atleast a part of kernel needs to be non paged for
> fast interrupt access etc as pinned memory
> Wiki says
>
> Pinned/Locked/Fixed pages
>
> Operating systems have memory areas that are pinned (never swapped to
> secondary storage). For example, interrupt mechanisms rely on an array of
> pointers to their handlers, such as I/O completion and page fault. If the
> pages containing these pointers or the code that they invoke were pageable,
> interrupt-handling would become far more complex and time-consuming,
> particularly in the case of page fault interrupts. Hence, some part of the
> page table structures is not pageable.
>
> Some pages may be pinned for short periods of time, others may be pinned for
> long periods of time, and still others may need to be permanently pinned.
> For example:
>
> The paging supervisor code and drivers for secondary storage devices on
> which pages reside must be permanently pinned, as otherwise paging wouldn't
> even work because the necessary code wouldn't be available.
> Timing-dependent components may be pinned to avoid variable paging delays.
> Data buffers that are accessed directly by peripheral devices that use
> direct memory access or I/O channels must reside in pinned pages while the
> I/O operation is in progress because such devices and the buses to which
> they are attached expect to find data buffers located at physical memory
> addresses; regardless of whether the bus has a memory management unit for
> I/O, transfers cannot be stopped if a page fault occurs and then restarted
> when the page fault has been processed.
>
> There are other two discussion thread which say kernel is non-pageable and
> now due to growing kernel Data structures it is allowed
>
> http://kerneltrap.org/node/6404
>
> http://kerneltrap.org/node/8206
>
>
> Regards
>
> Kishore
>
>
>
> On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay@gmail.com>
> wrote:
>>
>> Hello,
>>
>> I am newbie.
>> It has been said "kernel memory is not pageable"
>> What does it mean? There is no concept of kernel virtual address?
>>
>> Any simple explanation will help me to udnerstand.
>>
>> Thanks,
>> Vijay
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>

Kernel memory

[Ronit Halder]

Hi,

Where in the memory kernel is located in the boot time?

regards,
Ronit

Kernel memory

[Valdis.Kletnieks at vt.edu]

On Tue, 17 May 2016 12:41:44 +0530, Ronit Halder said:
> Hi,
>
> Where in the memory kernel is located in the boot time?

During which exact phase of the boot, and does it actually matter?  And
physical or virtual address?

(Hint:  If you're not the bootstrap that unpacks the compressed kernel
into memory, or the code that sets up the ASLR code, or are trying to work
around an issue with a hardware device that insists on taking certain
physical addresses for itself, you probably don't really care where the
kernel is....)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160517/a4da7bca/attachment.bin