[Linux-ia64] CONFIG_IA64_PAGE_SIZE

[Linux-ia64] CONFIG_IA64_PAGE_SIZE

[Broadfoot, KenX]

Anyone in the know,

I am using a 16KB page size for my Lion systems.

Under the help section of make menuconfig is says:

4KB   best for ia32

8KB   best for ia64

16KB  best for ia64

64KB  Not for Itanium


I just want to know what is the gain or loss if I used 8KB instead of 16KB
page size.

Also why is the 64KB page size bad for Itanium?

Thanks.

--ken

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[David Mosberger]

>>>>> On Thu, 18 Jan 2001 09:55:32 -0800, "Broadfoot, KenX" <kenx.broadfoot@intel.com> said:

  Ken> Anyone in the know,

  Ken> I am using a 16KB page size for my Lion systems.

  Ken> Under the help section of make menuconfig is says:

  Ken> 4KB best for ia32

  Ken> 8KB best for ia64

  Ken> 16KB best for ia64

  Ken> 64KB Not for Itanium


  Ken> I just want to know what is the gain or loss if I used 8KB
  Ken> instead of 16KB page size.

Based on recent measurements by Don and myself, I'd recommend 16KB.
For example, local kernel compiles go noticably faster that way.

	--david

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[Don Dugger]

Ken-

I'm actually in the middle of studying the effects of varying page
sizes on IA64.  Preliminary results are that you suffer approximately
a 10% performance hit when you go from 16K pages to 4K pages (I haven't
looked at 8K pages yet).

If you are running IA32 apps on your IA64 system there are some other
concerns.  All IA32 apps implicitly assume a 4K page size.  To handle
this we've had to do some tricks in the kernel that, for the most part,
have negated paging for IA32 processes.  For technical reasons we wind
up reading in the entire text and data sections of IA32 processes rather
than paging them in during process execution.  This means IA32 processes
are slower to start.  Note that on a kernel with 4K pages IA32 processes
will exhibit normal paging behavior.  Also, there could be applications
that explicitly require 4K pages to work.  I haven't run across any yet
but for absolute compatibility you might need to run with 4K pages.

64K pages don't work right now.  There is code in the SCSI system that
effectively limits the page size to no more that 32K (an unsupported
page size on current IA64 CPU's) so the largest page size we can currently
specify is 16K.  I'm also in the process of fixing that SCSI code as I
want to benchmark the system with 64K pages also.

On Thu, Jan 18, 2001 at 09:55:32AM -0800, Broadfoot, KenX wrote:
> 
> Anyone in the know,
> 
> I am using a 16KB page size for my Lion systems.
> 
> Under the help section of make menuconfig is says:
> 
> 4KB   best for ia32
> 
> 8KB   best for ia64
> 
> 16KB  best for ia64
> 
> 64KB  Not for Itanium
> 
> 
> I just want to know what is the gain or loss if I used 8KB instead of 16KB
> page size.
> 
> Also why is the 64KB page size bad for Itanium?
> 
> Thanks.
> 
> --ken
> 
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
n0ano@valinux.com
Ph: 303/938-9838

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[David Mosberger]

>>>>> On Thu, 18 Jan 2001 11:21:38 -0700, Don Dugger <n0ano@valinux.com> said:

  Don> This means IA32 processes are slower to start.

This needs to be qualified: in my experience, on a kernel with >4K
page size, IA32 processes start up slower _the first time_ they are
invoked.  Assuming sufficient memory, subsequent invocations of the
same program appear to be much faster than with 4KB pages.  Quite
unexpected.

	--david

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[Rich Altmaier]

I very much like larger page sizes for their performance boosts,
typically the VM efficiency helps both code+data allocation and
helps IO operation setup/teardown.

However be aware of one significant downside to larger page sizes,
for workloads involving a large number of small processes.
Suppose you fire up 1000 processes on a system, then the
additional physical memory consumption is significant.
I actually don't know the correct calculation but guesstimate
that per-process data structures which increase in size
include kernel stack, task struct, each shared lib data segment,
user data segment.  If the pagesize is increased from 4K to 64K,
then 4 such elements yield  60K*4*1000 processes = 240Mbyte.

Some customers do complain about this (and I tell them to buy
more memory and enjoy the performance).
Thanks, Rich


>
>
> On Thu, Jan 18, 2001 at 09:55:32AM -0800, Broadfoot, KenX wrote:
> >
> > Anyone in the know,
> >
> > I am using a 16KB page size for my Lion systems.
> >
> > Under the help section of make menuconfig is says:
> >
> > 4KB   best for ia32
> >
> > 8KB   best for ia64
> >
> > 16KB  best for ia64
> >
> > 64KB  Not for Itanium
> >
> >
> > I just want to know what is the gain or loss if I used 8KB instead of 16KB
> > page size.
> >
> > Also why is the 64KB page size bad for Itanium?
> >
> > Thanks.
> >
> > --ken
> >

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[Don Dugger]

Actually, the situation isn't quite this bad.  Turns out the kernel
stack and kernel register stack are both contained in the static
sized task structure.  The task structure has a minimum size of
32K so the per process memory for these structures is the same
for all page sizes from 4K to 32K.

What will change is the amount of unused space in the last page
of the user's data section, shared library data section, stack
section and register stack section.  Using the standard assumption
that half of the last page is wasted and also assuming that each
process only uses 1 shared library then going from 4K to 64K
increases this wasted space by 30K so your calculation becomes
30K * 4 * 1K = 120M.

On Thu, Jan 18, 2001 at 12:43:06PM -0800, Rich Altmaier wrote:
> I very much like larger page sizes for their performance boosts,
> typically the VM efficiency helps both code+data allocation and
> helps IO operation setup/teardown.
> 
> However be aware of one significant downside to larger page sizes,
> for workloads involving a large number of small processes.
> Suppose you fire up 1000 processes on a system, then the
> additional physical memory consumption is significant.
> I actually don't know the correct calculation but guesstimate
> that per-process data structures which increase in size
> include kernel stack, task struct, each shared lib data segment,
> user data segment.  If the pagesize is increased from 4K to 64K,
> then 4 such elements yield  60K*4*1000 processes = 240Mbyte.
> 
> Some customers do complain about this (and I tell them to buy
> more memory and enjoy the performance).
> Thanks, Rich
> 
> 
> >
> >
> > On Thu, Jan 18, 2001 at 09:55:32AM -0800, Broadfoot, KenX wrote:
> > >
> > > Anyone in the know,
> > >
> > > I am using a 16KB page size for my Lion systems.
> > >
> > > Under the help section of make menuconfig is says:
> > >
> > > 4KB   best for ia32
> > >
> > > 8KB   best for ia64
> > >
> > > 16KB  best for ia64
> > >
> > > 64KB  Not for Itanium
> > >
> > >
> > > I just want to know what is the gain or loss if I used 8KB instead of 16KB
> > > page size.
> > >
> > > Also why is the 64KB page size bad for Itanium?
> > >
> > > Thanks.
> > >
> > > --ken
> > >
> 
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
n0ano@valinux.com
Ph: 303/938-9838

Re: [Linux-ia64] CONFIG_IA64_PAGE_SIZE

[David Mosberger]

>>>>> On Thu, 18 Jan 2001 12:43:06 -0800, Rich Altmaier <richa@engr.sgi.com> said:

  Rich> However be aware of one significant downside to larger page
  Rich> sizes, for workloads involving a large number of small
  Rich> processes.  Suppose you fire up 1000 processes on a system,
  Rich> then the additional physical memory consumption is
  Rich> significant.  I actually don't know the correct calculation
  Rich> but guesstimate that per-process data structures which
  Rich> increase in size include kernel stack, task struct, each
  Rich> shared lib data segment, user data segment.  If the pagesize
  Rich> is increased from 4K to 64K, then 4 such elements yield
  Rich> 60K*4*1000 processes = 240Mbyte.

The minimum task size was an important consideration in the design of
IA-64 Linux.  As Don pointed out, the process data structure is at
least 32KB no matter what page size is in use (to avoid stack
overflow), so this really makes a difference only when the page size
is 64KB.

However, the real biggie are the page tables.  Even the smallest task
needs at the least three page table pages, so here the page size can
have a big impact.

  Rich> Some customers do complain about this (and I tell them to buy
  Rich> more memory and enjoy the performance).

Good answer! ;-)

	--david