Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[sandeep kumar]

Hi all,
The following link gives the memory map for the arm architecture.
http://www.arm.linux.org.uk/developer/memory.txt

I have the following doubts..
1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should follow
the same memory map.
Is it hardly constrained or can be changed?
Where are this constraints are implemented in the kernel source tree?

2) while i was student, i read in OS concepts that, "Virtual memory gives an
illusion to a process,
that it has always a larger continuous address space (even more than RAM)
available to it."
So i thought i could allocate howmuch ever memory i want.
But seeing the above link,i observed there is some limitation in the address
space created by the vmalloc().
So i m now thinking that vmalloc has some limit.

Please make me clear these things....


With regards,
Sandeep Kumar Anantapalli,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110531/fbf69350/attachment.html 

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[Mulyadi Santosa]

Hi...

I am not ARM guy, but I'll see what I can share here..... hold your breath :)

On Tue, May 31, 2011 at 12:54, sandeep kumar <coolsandyforyou@gmail.com> wrote:
> Hi all,
> The following link gives the memory map for the arm architecture.
> http://www.arm.linux.org.uk/developer/memory.txt
>
> I have the following doubts..
> 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should follow
> the same memory map.
> Is it hardly constrained or can be changed?
> Where are?this constraints are implemented in the kernel source tree?

you mean, device memory map? well AFAIK that is dictated by
BIOS....kernel simply just follow along...

>
> 2) while i was student,?i read in?OS concepts?that, "Virtual memory gives?an
> illusion?to a?process,
> that it?has always a larger?continuous address space (even more than RAM)
> available to it."

that's true... but you need to count another limitation: addressable
or not by the MMU or at least processor itself?

let's say you have 16 GiB of virtual memory, composed of 4 GiB of RAM
+ 12 GiB swap. Theoritically, a single process should be able to use
them all, but assuming we have no PAE enabled, an 32 bit system could
only address up to 4 GiB

> So i thought i could allocate howmuch ever memory i want.

Also think about fragmentation...

> But seeing the above link,i observed?there is some limitation in the address
> space created by the vmalloc().
> So i m now thinking that vmalloc has some?limit.

Yup.....

maybe my old article could shed a light further for you:
http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[sandeep kumar]

>> Hi all,
>> The following link gives the memory map for the arm architecture.
>> http://www.arm.linux.org.uk/developer/memory.txt
>>
>>I have the following doubts..
>>1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should follow
>>the same memory map.
>>Is it hardly constrained or can be changed?
>>Where are this constraints are implemented in the kernel source tree?

>you mean, device memory map? well AFAIK that is dictated by
>BIOS....kernel simply just follow along...

Not the device memory map..others also like vmalloc()/ioremap region,
dma_alloc() (refer that link)
these regions.

i refered to qualcomm chipset memory map(based on ARM9). They just mentioned
only where different devices were mapped.
They doesnt have this specifc adress space constraints for vmalloc,
ioremap,dma_alloc etc.,
So i thought it is taken care by kernel.

Please tell me where these constraints will be implemented?

Another clarification, while assigning memory through vmalloc(), dma_alloc()
kernel first
should check the available address space 'taking the constraints into
consideration' before returning the addresses, right?




On Tue, May 31, 2011 at 12:32 PM, Mulyadi Santosa <mulyadi.santosa@gmail.com
> wrote:

> Hi...
>
> I am not ARM guy, but I'll see what I can share here..... hold your breath
> :)
>
> On Tue, May 31, 2011 at 12:54, sandeep kumar <coolsandyforyou@gmail.com>
> wrote:
> > Hi all,
> > The following link gives the memory map for the arm architecture.
> > http://www.arm.linux.org.uk/developer/memory.txt
> >
> > I have the following doubts..
> > 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should
> follow
> > the same memory map.
> > Is it hardly constrained or can be changed?
> > Where are this constraints are implemented in the kernel source tree?
>
> you mean, device memory map? well AFAIK that is dictated by
> BIOS....kernel simply just follow along...
>
> >
> > 2) while i was student, i read in OS concepts that, "Virtual memory
> gives an
> > illusion to a process,
> > that it has always a larger continuous address space (even more than RAM)
> > available to it."
>
> that's true... but you need to count another limitation: addressable
> or not by the MMU or at least processor itself?
>
> let's say you have 16 GiB of virtual memory, composed of 4 GiB of RAM
> + 12 GiB swap. Theoritically, a single process should be able to use
> them all, but assuming we have no PAE enabled, an 32 bit system could
> only address up to 4 GiB
>
> > So i thought i could allocate howmuch ever memory i want.
>
> Also think about fragmentation...
>
> > But seeing the above link,i observed there is some limitation in the
> address
> > space created by the vmalloc().
> > So i m now thinking that vmalloc has some limit.
>
> Yup.....
>
> maybe my old article could shed a light further for you:
> http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
>
> --
> regards,
>
> Mulyadi Santosa
> Freelance Linux trainer and consultant
>
> blog: the-hydra.blogspot.com
> training: mulyaditraining.blogspot.com
>



-- 
With regards,
Sandeep Kumar Anantapalli,
Senior Software Engineer,
Samsung India Software Operations,
Bangalore.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110531/06a06561/attachment.html 

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[Dave Hylands]

Hi Sandeep,

On Mon, May 30, 2011 at 10:54 PM, sandeep kumar
<coolsandyforyou@gmail.com> wrote:
> Hi all,
> The following link gives the memory map for the arm architecture.
> http://www.arm.linux.org.uk/developer/memory.txt
>
> I have the following doubts..
> 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should follow
> the same memory map.
> Is it hardly constrained or can be changed?
> Where are?this constraints are implemented in the kernel source tree?

You can have 2 different configurations for the same chip which
present totally different memory maps. Where things are physically in
memory varies from chip to chip, regardless of the fact that they
happen to use the same processor.

> 2) while i was student,?i read in?OS concepts?that, "Virtual memory gives?an
> illusion?to a?process,
> that it?has always a larger?continuous address space (even more than RAM)
> available to it."
> So i thought i could allocate howmuch ever memory i want.
> But seeing the above link,i observed?there is some limitation in the address
> space created by the vmalloc().
> So i m now thinking that vmalloc has some?limit.

Absolutely. You can only vmalloc as much memory as there is virtual
space reserved for vmalloc. And your vmalloc memory space can become
fragmented, so you could have lots of space, but not have enough
contiguous virtual space left to allocate a large object.

Also keep in mind that when you vmalloc stuff you're allocating in
units of pages, and there is a guard page between each allocation. So
if you had 512 Mb of virtual space (which is 131,072 pages), you'll be
able to vmalloc a maximum of 65536 objects or 1 page each. Fewer
objects if they're bigger than a page.

The actual allocation of vmalloc memory is determined by the macros
VMALLOC_START and VMALLOC_END. VMALLOC_START is defined in
arch/arm/include/asm/pgtable.h:
http://lxr.linux.no/linux+v2.6.39/arch/arm/include/asm/pgtable.h#L41
although it can be overriden by a particular architecture.

VMALLOC_END is normally defined in the machine's memory.h file, and it
sits somewhere between VMALLOC_START and the beginning of I/O space.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[Peter Teoh]

To answer your subject:   I think the straight answer is "no".   Many
reason, among them:

ARM is still 32-bit, at least at the present moment:

http://www.google.com/search?q=does+arm+have+64bit&num=100

so with hardware 32-bit based, doing MMU at the 64-bit level is still
not possible (without the MMU 64-bit hardware architecture, I don't
think it is possible to do any >4GB memory translation stuff.   Am I
not wrong?

On Tue, May 31, 2011 at 1:54 PM, sandeep kumar
<coolsandyforyou@gmail.com> wrote:
> Hi all,
> The following link gives the memory map for the arm architecture.
> http://www.arm.linux.org.uk/developer/memory.txt
>
> I have the following doubts..
> 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should follow
> the same memory map.
> Is it hardly constrained or can be changed?
> Where are?this constraints are implemented in the kernel source tree?
>
> 2) while i was student,?i read in?OS concepts?that, "Virtual memory gives?an
> illusion?to a?process,
> that it?has always a larger?continuous address space (even more than RAM)
> available to it."
> So i thought i could allocate howmuch ever memory i want.
> But seeing the above link,i observed?there is some limitation in the address
> space created by the vmalloc().
> So i m now thinking that vmalloc has some?limit.
>
> Please make me clear these things....
>
>
> With regards,
> Sandeep Kumar Anantapalli,
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>



-- 
Regards,
Peter Teoh

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[sandeep kumar]

Yes peter you r right..
But my main concern(which i dint convey properly in subject) is whether
virtual memory allocation has a limit or not.
I got it answered.

Thank you ..

On Tue, May 31, 2011 at 8:57 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:

> To answer your subject:   I think the straight answer is "no".   Many
> reason, among them:
>
> ARM is still 32-bit, at least at the present moment:
>
> http://www.google.com/search?q=does+arm+have+64bit&num=100
>
> so with hardware 32-bit based, doing MMU at the 64-bit level is still
> not possible (without the MMU 64-bit hardware architecture, I don't
> think it is possible to do any >4GB memory translation stuff.   Am I
> not wrong?
>
> On Tue, May 31, 2011 at 1:54 PM, sandeep kumar
> <coolsandyforyou@gmail.com> wrote:
> > Hi all,
> > The following link gives the memory map for the arm architecture.
> > http://www.arm.linux.org.uk/developer/memory.txt
> >
> > I have the following doubts..
> > 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should
> follow
> > the same memory map.
> > Is it hardly constrained or can be changed?
> > Where are this constraints are implemented in the kernel source tree?
> >
> > 2) while i was student, i read in OS concepts that, "Virtual memory
> gives an
> > illusion to a process,
> > that it has always a larger continuous address space (even more than RAM)
> > available to it."
> > So i thought i could allocate howmuch ever memory i want.
> > But seeing the above link,i observed there is some limitation in the
> address
> > space created by the vmalloc().
> > So i m now thinking that vmalloc has some limit.
> >
> > Please make me clear these things....
> >
> >
> > With regards,
> > Sandeep Kumar Anantapalli,
> >
> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies at kernelnewbies.org
> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >
> >
>
>
>
> --
> Regards,
> Peter Teoh
>



-- 
With regards,
Sandeep Kumar Anantapalli,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110601/e04fafbf/attachment-0001.html 

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[subin gangadharan]

Hi Dave,

>Also keep in mind that when you vmalloc stuff you're allocating in
>units of pages, and there is a guard page between each allocation. So
>if you had 512 Mb of virtual space (which is 131,072 pages), you'll be
>able to vmalloc a maximum of 65536 objects or 1 page each. Fewer
>objects if they're bigger than a page.

A quick question, Suppose if I alloc 1MB of memory using vmalloc,Is there a
guard page between each page or is it after the 1MB.

With Regards,
Subin K G




On Tue, May 31, 2011 at 10:19 AM, Dave Hylands <dhylands@gmail.com> wrote:

> Hi Sandeep,
>
> On Mon, May 30, 2011 at 10:54 PM, sandeep kumar
> <coolsandyforyou@gmail.com> wrote:
> > Hi all,
> > The following link gives the memory map for the arm architecture.
> > http://www.arm.linux.org.uk/developer/memory.txt
> >
> > I have the following doubts..
> > 1) Any chipset(based on arm) manufacturer(qualcom,samsung..) should
> follow
> > the same memory map.
> > Is it hardly constrained or can be changed?
> > Where are this constraints are implemented in the kernel source tree?
>
> You can have 2 different configurations for the same chip which
> present totally different memory maps. Where things are physically in
> memory varies from chip to chip, regardless of the fact that they
> happen to use the same processor.
>
> > 2) while i was student, i read in OS concepts that, "Virtual memory
> gives an
> > illusion to a process,
> > that it has always a larger continuous address space (even more than RAM)
> > available to it."
> > So i thought i could allocate howmuch ever memory i want.
> > But seeing the above link,i observed there is some limitation in the
> address
> > space created by the vmalloc().
> > So i m now thinking that vmalloc has some limit.
>
> Absolutely. You can only vmalloc as much memory as there is virtual
> space reserved for vmalloc. And your vmalloc memory space can become
> fragmented, so you could have lots of space, but not have enough
> contiguous virtual space left to allocate a large object.
>
> Also keep in mind that when you vmalloc stuff you're allocating in
> units of pages, and there is a guard page between each allocation. So
> if you had 512 Mb of virtual space (which is 131,072 pages), you'll be
> able to vmalloc a maximum of 65536 objects or 1 page each. Fewer
> objects if they're bigger than a page.
>
> The actual allocation of vmalloc memory is determined by the macros
> VMALLOC_START and VMALLOC_END. VMALLOC_START is defined in
> arch/arm/include/asm/pgtable.h:
> http://lxr.linux.no/linux+v2.6.39/arch/arm/include/asm/pgtable.h#L41
> although it can be overriden by a particular architecture.
>
> VMALLOC_END is normally defined in the machine's memory.h file, and it
> sits somewhere between VMALLOC_START and the beginning of I/O space.
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
With Regards
Subin Gangadharan

Everything should be made as simple as possible,but not simpler.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110822/4fdd9132/attachment.html 

Can i allocate 4GB virtual addresses (more than a certain limit) using vmalloc?

[Jonathan Neuschäfer]

On Mon, Aug 22, 2011 at 04:43:55PM -0500, subin gangadharan wrote:
> Hi Dave,
> 
> >Also keep in mind that when you vmalloc stuff you're allocating in
> >units of pages, and there is a guard page between each allocation. So
> >if you had 512 Mb of virtual space (which is 131,072 pages), you'll be
> >able to vmalloc a maximum of 65536 objects or 1 page each. Fewer
> >objects if they're bigger than a page.
> 
> A quick question, Suppose if I alloc 1MB of memory using vmalloc,Is there a
> guard page between each page or is it after the 1MB.

If you do it one vmalloc call your 1MiB should be continuous, i.e. if
there's a guard page it will be after your megabyte.

HTH,
	Jonathan Neusch?fer