* Re: x86: vmalloc and THP [not found] <53E99F86.5020100@scalemp.com> @ 2014-08-12 6:07 ` Kirill A. Shutemov 2014-08-12 12:28 ` Eric Dumazet 2014-08-12 16:20 ` Oren Twaig 0 siblings, 2 replies; 6+ messages in thread From: Kirill A. Shutemov @ 2014-08-12 6:07 UTC (permalink / raw) To: Oren Twaig Cc: linux-kernel@vger.kernel.org, linux-mm, Shai Fultheim (Shai@ScaleMP.com) On Tue, Aug 12, 2014 at 08:00:54AM +0300, Oren Twaig wrote: > <html style="direction: ltr;"> plain/text, please. >Hello, > >Does memory allocated using vmalloc() will be mapped using huge >pages either directly or later by THP ? No. It's neither aligned properly, nor physically contiguous. >If not, is there any fast way to change this behavior ? Maybe by >changing the granularity/alignment of such allocations to allow such >mapping ? What's the point to use vmalloc() in this case? -- Kirill A. Shutemov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: x86: vmalloc and THP 2014-08-12 6:07 ` x86: vmalloc and THP Kirill A. Shutemov @ 2014-08-12 12:28 ` Eric Dumazet 2014-08-12 15:01 ` Kirill A. Shutemov 2014-08-12 16:20 ` Oren Twaig 1 sibling, 1 reply; 6+ messages in thread From: Eric Dumazet @ 2014-08-12 12:28 UTC (permalink / raw) To: Kirill A. Shutemov Cc: Oren Twaig, linux-kernel@vger.kernel.org, linux-mm, Shai Fultheim (Shai@ScaleMP.com) On Tue, 2014-08-12 at 09:07 +0300, Kirill A. Shutemov wrote: > On Tue, Aug 12, 2014 at 08:00:54AM +0300, Oren Twaig wrote: > >Does memory allocated using vmalloc() will be mapped using huge > >pages either directly or later by THP ? > > No. It's neither aligned properly, nor physically contiguous. > > >If not, is there any fast way to change this behavior ? Maybe by > >changing the granularity/alignment of such allocations to allow such > >mapping ? > > What's the point to use vmalloc() in this case? Look at various large hashes we have in the system, all using vmalloc() : [ 0.006856] Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes) [ 0.033130] Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes) [ 1.197621] TCP established hash table entries: 524288 (order: 11, 8388608 bytes) I would imagine a performance difference if we were using hugepages. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: x86: vmalloc and THP 2014-08-12 12:28 ` Eric Dumazet @ 2014-08-12 15:01 ` Kirill A. Shutemov 0 siblings, 0 replies; 6+ messages in thread From: Kirill A. Shutemov @ 2014-08-12 15:01 UTC (permalink / raw) To: Eric Dumazet Cc: Oren Twaig, linux-kernel@vger.kernel.org, linux-mm, Shai Fultheim (Shai@ScaleMP.com) On Tue, Aug 12, 2014 at 05:28:52AM -0700, Eric Dumazet wrote: > On Tue, 2014-08-12 at 09:07 +0300, Kirill A. Shutemov wrote: > > On Tue, Aug 12, 2014 at 08:00:54AM +0300, Oren Twaig wrote: > > >If not, is there any fast way to change this behavior ? Maybe by > > >changing the granularity/alignment of such allocations to allow such > > >mapping ? > > > > What's the point to use vmalloc() in this case? > > Look at various large hashes we have in the system, all using > vmalloc() : > > [ 0.006856] Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes) > [ 0.033130] Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes) > [ 1.197621] TCP established hash table entries: 524288 (order: 11, 8388608 bytes) I see lower-order allocation in upstream code. Is it some distribution tweak? > I would imagine a performance difference if we were using hugepages. Okay, it's *probably* a valid point. The hash tables are only allocated with vmalloc() on NUMA system, if hashdist=1 (default on NUMA). It does it to distribute memory between nodes. vmalloc() in NUMA_NO_NODE case will allocate all memory with 0-order page allocations: no physical contiguous memory for hugepage mappings. I guess we could teach vmalloc() to interleave between nodes on PMD_SIZE chunks rather then on PAGE_SIZE if caller asks for big memory allocations. Although, I'm not sure it it would fit all vmalloc() users. We also would need to allocate PMD_SIZE-aligned virtual address range to be able to mapped allocated memory with pmds. It's *potentially* interesting research project. Any volunteers? -- Kirill A. Shutemov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: x86: vmalloc and THP 2014-08-12 6:07 ` x86: vmalloc and THP Kirill A. Shutemov 2014-08-12 12:28 ` Eric Dumazet @ 2014-08-12 16:20 ` Oren Twaig 2014-08-12 21:40 ` Kirill A. Shutemov 1 sibling, 1 reply; 6+ messages in thread From: Oren Twaig @ 2014-08-12 16:20 UTC (permalink / raw) To: Kirill A. Shutemov Cc: linux-kernel@vger.kernel.org, linux-mm, Shai Fultheim (Shai@ScaleMP.com) Hi Kirill, I saw the thread has developed nicely :), still - wanted to answer your question below. On 8/12/2014 9:07 AM, Kirill A. Shutemov wrote: > On Tue, Aug 12, 2014 at 08:00:54AM +0300, Oren Twaig wrote: >> <html style="direction: ltr;"> > plain/text, please. Yes - noticed the html, sent again in plain text. >> If not, is there any fast way to change this behavior ? Maybe by >> changing the granularity/alignment of such allocations to allow such >> mapping ? > What's the point to use vmalloc() in this case? I've noticed that some lock/s are using linear addresses which are located at 0xffffc901922b4500 and from what I understand from mm.txt (kernel 3.0.101): *ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space *So I'm not sure who/how/why this lock got allocated there, but obviously it is using that linear set. No ? > --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: x86: vmalloc and THP 2014-08-12 16:20 ` Oren Twaig @ 2014-08-12 21:40 ` Kirill A. Shutemov 0 siblings, 0 replies; 6+ messages in thread From: Kirill A. Shutemov @ 2014-08-12 21:40 UTC (permalink / raw) To: Oren Twaig Cc: linux-kernel@vger.kernel.org, linux-mm, Shai Fultheim (Shai@ScaleMP.com) On Tue, Aug 12, 2014 at 07:20:52PM +0300, Oren Twaig wrote: > >What's the point to use vmalloc() in this case? > I've noticed that some lock/s are using linear addresses which are > located at 0xffffc901922b4500 and from what I understand > from mm.txt (kernel 3.0.101): > *ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space > > *So I'm not sure who/how/why this lock got allocated there, but obviously > it is using that linear set. No ? It would be nice to know what lock it was, but nothing is inherently wrong with lock in vmalloc space. -- Kirill A. Shutemov ^ permalink raw reply [flat|nested] 6+ messages in thread
* x86: vmalloc and THP @ 2014-08-12 5:01 Oren Twaig 0 siblings, 0 replies; 6+ messages in thread From: Oren Twaig @ 2014-08-12 5:01 UTC (permalink / raw) To: linux-kernel@vger.kernel.org, linux-mm; +Cc: Shai Fultheim (Shai@ScaleMP.com) Hello, Does memory allocated using vmalloc() will be mapped using huge pages either directly or later by THP ? If not, is there any fast way to change this behavior ? Maybe by changing the granularity/alignment of such allocations to allow such mapping ? Thanks, Oren Twaig. --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-08-12 21:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <53E99F86.5020100@scalemp.com>
2014-08-12 6:07 ` x86: vmalloc and THP Kirill A. Shutemov
2014-08-12 12:28 ` Eric Dumazet
2014-08-12 15:01 ` Kirill A. Shutemov
2014-08-12 16:20 ` Oren Twaig
2014-08-12 21:40 ` Kirill A. Shutemov
2014-08-12 5:01 Oren Twaig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox