[patch] x86: allow ZONE_DMA to be configurable

[patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

ZONE_DMA is unnecessary for a large number of machines that do not
require addressing in the lower 16MB of memory because they do not use
ISA devices with 16-bit address registers (plus one page byte register).

This patch allows users to disable ZONE_DMA for x86 if they know they
will not be using such devices with their kernel.

This prevents the VM from unnecessarily reserving a ratio of memory
(defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
for such allocations when it will never be used.

Signed-off-by: David Rientjes <rientjes@google.com>
---
 arch/x86/Kconfig      |    7 ++++++-
 arch/x86/mm/init_32.c |    2 ++
 arch/x86/mm/init_64.c |    2 ++
 3 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -99,7 +99,12 @@ config MMU
 	def_bool y
 
 config ZONE_DMA
-	def_bool y
+	bool "DMA memory allocation support"
+	default y
+	help
+	  DMA memory allocation support allows old ISA devices with 16-bit
+	  address registers to allocate memory within the first 16MB of
+	  address space.  Disable if no such devices will be used.
 
 config SBUS
 	bool
diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
--- a/arch/x86/mm/init_32.c
+++ b/arch/x86/mm/init_32.c
@@ -740,8 +740,10 @@ static void __init zone_sizes_init(void)
 {
 	unsigned long max_zone_pfns[MAX_NR_ZONES];
 	memset(max_zone_pfns, 0, sizeof(max_zone_pfns));
+#ifdef CONFIG_ZONE_DMA
 	max_zone_pfns[ZONE_DMA] =
 		virt_to_phys((char *)MAX_DMA_ADDRESS) >> PAGE_SHIFT;
+#endif
 	max_zone_pfns[ZONE_NORMAL] = max_low_pfn;
 #ifdef CONFIG_HIGHMEM
 	max_zone_pfns[ZONE_HIGHMEM] = highend_pfn;
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -598,7 +598,9 @@ void __init paging_init(void)
 	unsigned long max_zone_pfns[MAX_NR_ZONES];
 
 	memset(max_zone_pfns, 0, sizeof(max_zone_pfns));
+#ifdef CONFIG_ZONE_DMA
 	max_zone_pfns[ZONE_DMA] = MAX_DMA_PFN;
+#endif
 	max_zone_pfns[ZONE_DMA32] = MAX_DMA32_PFN;
 	max_zone_pfns[ZONE_NORMAL] = max_pfn;
 

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/13/2010 05:15 PM, David Rientjes wrote:
> ZONE_DMA is unnecessary for a large number of machines that do not
> require addressing in the lower 16MB of memory because they do not use
> ISA devices with 16-bit address registers (plus one page byte register).
> 
> This patch allows users to disable ZONE_DMA for x86 if they know they
> will not be using such devices with their kernel.
> 
> This prevents the VM from unnecessarily reserving a ratio of memory
> (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> for such allocations when it will never be used.

This isn't true.

There are many, many devices other than ISA devices which need access to
a restricted-memory pool because of hardware DMA limitations.  This
seems like a really bad idea.

A much better idea would be to have a runtime option of setting the
reservation ratio, if recovering no more than 1/66th of a gigabyte
matters so much to you.

	-hpa

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Wed, 13 Oct 2010, H. Peter Anvin wrote:

> > ZONE_DMA is unnecessary for a large number of machines that do not
> > require addressing in the lower 16MB of memory because they do not use
> > ISA devices with 16-bit address registers (plus one page byte register).
> > 
> > This patch allows users to disable ZONE_DMA for x86 if they know they
> > will not be using such devices with their kernel.
> > 
> > This prevents the VM from unnecessarily reserving a ratio of memory
> > (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> > for such allocations when it will never be used.
> 
> This isn't true.
> 
> There are many, many devices other than ISA devices which need access to
> a restricted-memory pool because of hardware DMA limitations.  This
> seems like a really bad idea.
> 
> A much better idea would be to have a runtime option of setting the
> reservation ratio, if recovering no more than 1/66th of a gigabyte
> matters so much to you.
> 

You can already set the lowmem_reserve_ratio via 
/proc/sys/vm/lowmem_reserve_ratio.

We've run without ZONE_DMA for a couple years because we don't have such 
hardware limitations and would appreciate the ability to disable it with a 
config option rather than hacking the kernel to get it to compile.  I'm 
sure Casey would as well as you can see in the thread "A question about 
ZONE_DMA".

I don't see the harm in being able to conveniently disable the zone if you 
know what you're doing.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/13/2010 05:42 PM, David Rientjes wrote:
> You can already set the lowmem_reserve_ratio via 
> /proc/sys/vm/lowmem_reserve_ratio.

And why is it then so hard to put in a single line in /etc/sysctl.conf
for you?

> We've run without ZONE_DMA for a couple years because we don't have such 
> hardware limitations and would appreciate the ability to disable it with a 
> config option rather than hacking the kernel to get it to compile.  I'm 
> sure Casey would as well as you can see in the thread "A question about 
> ZONE_DMA".
> 
> I don't see the harm in being able to conveniently disable the zone if you 
> know what you're doing.

There are going to be a lot of very strange bug reports as a result.

	-hpa

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Wed, 13 Oct 2010, H. Peter Anvin wrote:

> > You can already set the lowmem_reserve_ratio via 
> > /proc/sys/vm/lowmem_reserve_ratio.
> 
> And why is it then so hard to put in a single line in /etc/sysctl.conf
> for you?
> 

Because disabling CONFIG_ZONE_DMA allows for additional extensions: 
disabling CONFIG_GENERIC_ISA_DMA and CONFIG_ISA_DMA_API.  We've also run 
for a couple years without those.

> > We've run without ZONE_DMA for a couple years because we don't have such 
> > hardware limitations and would appreciate the ability to disable it with a 
> > config option rather than hacking the kernel to get it to compile.  I'm 
> > sure Casey would as well as you can see in the thread "A question about 
> > ZONE_DMA".
> > 
> > I don't see the harm in being able to conveniently disable the zone if you 
> > know what you're doing.
> 
> There are going to be a lot of very strange bug reports as a result.
> 

We have config options all over the kernel that change the behavior of 
syscalls without worrying about bug reports popping up from users who 
knowingly have disabled the option, I don't see this as being any 
different.  If you're concerned about, we could add a

#ifndef CONFIG_ZONE_DMA
	WARN_ON_ONCE(gfp_mask & __GFP_DMA);
#endif

in the page allocator.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

And the value of those additional options is what?  I'd consider adding this to the sewer pit called CONFIG_EMBEDDED (with a BUG_ON, not a warning... sheesh) but only if there is any demonstrable value other than a trivial amount of code (kilobytes?) in exchange for a bunch of crap #ifdef.

"David Rientjes" <rientjes@google.com> wrote:

>On Wed, 13 Oct 2010, H. Peter Anvin wrote:
>
>> > You can already set the lowmem_reserve_ratio via 
>> > /proc/sys/vm/lowmem_reserve_ratio.
>> 
>> And why is it then so hard to put in a single line in /etc/sysctl.conf
>> for you?
>> 
>
>Because disabling CONFIG_ZONE_DMA allows for additional extensions: 
>disabling CONFIG_GENERIC_ISA_DMA and CONFIG_ISA_DMA_API.  We've also run 
>for a couple years without those.
>
>> > We've run without ZONE_DMA for a couple years because we don't have such 
>> > hardware limitations and would appreciate the ability to disable it with a 
>> > config option rather than hacking the kernel to get it to compile.  I'm 
>> > sure Casey would as well as you can see in the thread "A question about 
>> > ZONE_DMA".
>> > 
>> > I don't see the harm in being able to conveniently disable the zone if you 
>> > know what you're doing.
>> 
>> There are going to be a lot of very strange bug reports as a result.
>> 
>
>We have config options all over the kernel that change the behavior of 
>syscalls without worrying about bug reports popping up from users who 
>knowingly have disabled the option, I don't see this as being any 
>different.  If you're concerned about, we could add a
>
>#ifndef CONFIG_ZONE_DMA
>	WARN_ON_ONCE(gfp_mask & __GFP_DMA);
>#endif
>
>in the page allocator.

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Wed, 13 Oct 2010, H. Peter Anvin wrote:

> And the value of those additional options is what?  I'd consider adding 
> this to the sewer pit called CONFIG_EMBEDDED (with a BUG_ON, not a 
> warning... sheesh)

BUG_ON() could panic the machine which would be rather unfortunate if we 
simply tried to load a driver that the kernel no longer supports because 
it doesn't have DMA.  A WARN_ON() seems much more appropriate to identify 
what the problem was.  It's not a fatal condition.

> but only if there is any demonstrable value other 
> than a trivial amount of code (kilobytes?) in exchange for a bunch of 
> crap #ifdef.
> 

The data savings is about 1% and the text savings is about 0.1% with all 
three options disabled:

7922297	1245500	 989600	10157397	 9afd55	vmlinux.before
7914674	1232700	 989472	10136846	 9aad0e	vmlinux.after

This is the only #ifdef necessary to make CONFIG_ZONE_DMA=n compile and 
CONFIG_GENERIC_ISA_DMA=n would require two additional #ifdefs 
(CONFIG_ISA_DMA_API=n would require none).  We carry this patch 
internally, so it would be trivial to send follow-up patches that do that 
if this patch is merged.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

How do you know it is not a fatal condition?

"David Rientjes" <rientjes@google.com> wrote:

>On Wed, 13 Oct 2010, H. Peter Anvin wrote:
>
>> And the value of those additional options is what?  I'd consider adding 
>> this to the sewer pit called CONFIG_EMBEDDED (with a BUG_ON, not a 
>> warning... sheesh)
>
>BUG_ON() could panic the machine which would be rather unfortunate if we 
>simply tried to load a driver that the kernel no longer supports because 
>it doesn't have DMA.  A WARN_ON() seems much more appropriate to identify 
>what the problem was.  It's not a fatal condition.
>
>> but only if there is any demonstrable value other 
>> than a trivial amount of code (kilobytes?) in exchange for a bunch of 
>> crap #ifdef.
>> 
>
>The data savings is about 1% and the text savings is about 0.1% with all 
>three options disabled:
>
>7922297	1245500	 989600	10157397	 9afd55	vmlinux.before
>7914674	1232700	 989472	10136846	 9aad0e	vmlinux.after
>
>This is the only #ifdef necessary to make CONFIG_ZONE_DMA=n compile and 
>CONFIG_GENERIC_ISA_DMA=n would require two additional #ifdefs 
>(CONFIG_ISA_DMA_API=n would require none).  We carry this patch 
>internally, so it would be trivial to send follow-up patches that do that 
>if this patch is merged.

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Wed, 13 Oct 2010, H. Peter Anvin wrote:

> How do you know it is not a fatal condition?
> 

We lack any GFP_DMA | __GFP_NOFAIL allocations in the kernel, the page 
allocator is safe to return NULL.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/13/2010 06:48 PM, David Rientjes wrote:
> On Wed, 13 Oct 2010, H. Peter Anvin wrote:
> 
>> And the value of those additional options is what?  I'd consider adding 
>> this to the sewer pit called CONFIG_EMBEDDED (with a BUG_ON, not a 
>> warning... sheesh)
> 
> BUG_ON() could panic the machine which would be rather unfortunate if we 
> simply tried to load a driver that the kernel no longer supports because 
> it doesn't have DMA.  A WARN_ON() seems much more appropriate to identify 
> what the problem was.  It's not a fatal condition.
> 
>> but only if there is any demonstrable value other 
>> than a trivial amount of code (kilobytes?) in exchange for a bunch of 
>> crap #ifdef.
>>
> 
> The data savings is about 1% and the text savings is about 0.1% with all 
> three options disabled:
> 
> 7922297	1245500	 989600	10157397	 9afd55	vmlinux.before
> 7914674	1232700	 989472	10136846	 9aad0e	vmlinux.after
> 
> This is the only #ifdef necessary to make CONFIG_ZONE_DMA=n compile and 
> CONFIG_GENERIC_ISA_DMA=n would require two additional #ifdefs 
> (CONFIG_ISA_DMA_API=n would require none).  We carry this patch 
> internally, so it would be trivial to send follow-up patches that do that 
> if this patch is merged.

I guess that qualifies (barely, arguably, but let's not go there) as
something that would be able to be carried under CONFIG_EMBEDDED
(certainly not without).

However, I'd like to have the whole thing as a complete patch series.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Wed, 13 Oct 2010, H. Peter Anvin wrote:

> I guess that qualifies (barely, arguably, but let's not go there) as
> something that would be able to be carried under CONFIG_EMBEDDED
> (certainly not without).
> 

I'm not running a CONFIG_EMBEDDED kernel and I'm pretty sure Casey isn't 
either, so we'd still need to carry our own patches to change the Kconfig 
bool line anyway.

I'm suprised that we can't allow users who definitively know that they 
have no hardware limitations the ability to disable ZONE_DMA.  Would a 
strong caution in the Kconfig "help" text suffice or will you not be 
convinced?

Re: [patch] x86: allow ZONE_DMA to be configurable

[Andrew Morton]

On Wed, 13 Oct 2010 17:15:04 -0700 (PDT) David Rientjes <rientjes@google.com> wrote:

> ZONE_DMA is unnecessary for a large number of machines that do not
> require addressing in the lower 16MB of memory because they do not use
> ISA devices with 16-bit address registers (plus one page byte register).
> 
> This patch allows users to disable ZONE_DMA for x86 if they know they
> will not be using such devices with their kernel.
> 
> This prevents the VM from unnecessarily reserving a ratio of memory
> (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> for such allocations when it will never be used.
> 

I wonder how hard it would be to do this at runtime, probably with a
boot parameter.

I'd be a little concerned at the effects of this on page reclaim and
the page allocator - it might expose weird pre-existing bugs or
inefficiencies.  But we can cross that bridge when we fall off it, I
guess.

Re: [patch] x86: allow ZONE_DMA to be configurable

[KOSAKI Motohiro]

> On Wed, 13 Oct 2010 17:15:04 -0700 (PDT) David Rientjes <rientjes@google.com> wrote:
> 
> > ZONE_DMA is unnecessary for a large number of machines that do not
> > require addressing in the lower 16MB of memory because they do not use
> > ISA devices with 16-bit address registers (plus one page byte register).
> > 
> > This patch allows users to disable ZONE_DMA for x86 if they know they
> > will not be using such devices with their kernel.
> > 
> > This prevents the VM from unnecessarily reserving a ratio of memory
> > (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> > for such allocations when it will never be used.
> > 
> 
> I wonder how hard it would be to do this at runtime, probably with a
> boot parameter.
> 
> I'd be a little concerned at the effects of this on page reclaim and
> the page allocator - it might expose weird pre-existing bugs or
> inefficiencies.  But we can cross that bridge when we fall off it, I
> guess.

In this case, completely ZONE disablling is not required. 16MB saving
is not big matter. To prevent userland process use lower zone is necessary.

I have such patch, because 32bit highmem system has similar requirement.
(user process want to only use ZONE_HIGHMEM, otherwize ZONE_NORMAL easily
exshost)

I'll prepare to post them if necessary.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/13/2010 09:46 PM, David Rientjes wrote:
> 
> I'm not running a CONFIG_EMBEDDED kernel and I'm pretty sure Casey isn't 
> either, so we'd still need to carry our own patches to change the Kconfig 
> bool line anyway.
> 
> I'm suprised that we can't allow users who definitively know that they 
> have no hardware limitations the ability to disable ZONE_DMA.  Would a 
> strong caution in the Kconfig "help" text suffice or will you not be 
> convinced?

CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm doing".
That is *what it is* and *all it is*.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [patch] x86: allow ZONE_DMA to be configurable

[Pekka Enberg]

On 10/13/2010 09:46 PM, David Rientjes wrote:
>> I'm not running a CONFIG_EMBEDDED kernel and I'm pretty sure Casey isn't
>> either, so we'd still need to carry our own patches to change the Kconfig
>> bool line anyway.
>>
>> I'm suprised that we can't allow users who definitively know that they
>> have no hardware limitations the ability to disable ZONE_DMA.  Would a
>> strong caution in the Kconfig "help" text suffice or will you not be
>> convinced?

On Thu, Oct 14, 2010 at 6:38 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm doing".
> That is *what it is* and *all it is*.

Yup. The naming is quite unfortunate, though. Can we change it to
something else?

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/14/2010 09:00 AM, Pekka Enberg wrote:
> On 10/13/2010 09:46 PM, David Rientjes wrote:
>>> I'm not running a CONFIG_EMBEDDED kernel and I'm pretty sure Casey isn't
>>> either, so we'd still need to carry our own patches to change the Kconfig
>>> bool line anyway.
>>>
>>> I'm suprised that we can't allow users who definitively know that they
>>> have no hardware limitations the ability to disable ZONE_DMA.  Would a
>>> strong caution in the Kconfig "help" text suffice or will you not be
>>> convinced?
> 
> On Thu, Oct 14, 2010 at 6:38 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm doing".
>> That is *what it is* and *all it is*.
> 
> Yup. The naming is quite unfortunate, though. Can we change it to
> something else?
> 

Funny, this seems to come up repeatedly ;)

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [patch] x86: allow ZONE_DMA to be configurable

[Ingo Molnar]

* H. Peter Anvin <hpa@zytor.com> wrote:

> On 10/14/2010 09:00 AM, Pekka Enberg wrote:
>
> >> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm 
> >> doing". That is *what it is* and *all it is*.
> > 
> > Yup. The naming is quite unfortunate, though. Can we change it to 
> > something else?
> 
> Funny, this seems to come up repeatedly ;)

I'd suggest to rename CONFIG_EMBEDDED=y to CONFIG_EXPERT=y, but i'd also 
suggest to rename all sub-config-options that depend on CONFIG_EXPERT to 
have a CONFIG_EXPERT_ prefix.

So we'd have:

  CONFIG_EXPERT_CPU_SUP_INTEL=y
  CONFIG_EXPERT_MMAP_ALLOW_UNINITIALIZED=y

etc.

That way all the enabled 'expert options' become easily visible in the 
.config file and they advertise themselves properly - it's also easily 
greppable. In the source code it also becomes self-documenting, it's 
obvious when a dependency is 'rare' or 'special' - it has a 
CONFIG_EXPERT_ prefix.

Something for the KS i guess.

Thanks,

	Ingo

Re: [patch] x86: allow ZONE_DMA to be configurable

[Andrew Morton]

On Thu, 14 Oct 2010 20:34:48 +0200
Ingo Molnar <mingo@elte.hu> wrote:

> 
> * H. Peter Anvin <hpa@zytor.com> wrote:
> 
> > On 10/14/2010 09:00 AM, Pekka Enberg wrote:
> >
> > >> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm 
> > >> doing". That is *what it is* and *all it is*.
> > > 
> > > Yup. The naming is quite unfortunate, though. Can we change it to 
> > > something else?
> > 
> > Funny, this seems to come up repeatedly ;)
> 
> I'd suggest to rename CONFIG_EMBEDDED=y to CONFIG_EXPERT=y, but i'd also 
> suggest to rename all sub-config-options that depend on CONFIG_EXPERT to 
> have a CONFIG_EXPERT_ prefix.
> 
> So we'd have:
> 
>   CONFIG_EXPERT_CPU_SUP_INTEL=y
>   CONFIG_EXPERT_MMAP_ALLOW_UNINITIALIZED=y
> 
> etc.

yup.  CONFIG_EMBEDDED didn't make a lot of sense even when we first did it. 
I suppose CONFIG_EXPERT is OK, although everyone will select it because
they think they're experts ;)  CONFIG_PROPELLORHEAD!

> That way all the enabled 'expert options' become easily visible in the 
> .config file and they advertise themselves properly - it's also easily 
> greppable. In the source code it also becomes self-documenting, it's 
> obvious when a dependency is 'rare' or 'special' - it has a 
> CONFIG_EXPERT_ prefix.
> 
> Something for the KS i guess.

nah, just do it.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

I don't like the CONFIG_EXPERT prefix, just because it makes it hard to move things into and out of this rubric, and that's bad... (consider: "oh, that would mean this *huge* patch) ... not to mention that what is CONFIG_EXPERT on one platform may not be for another!  As far as the main switch is concerned, it's a better name of course...

"Andrew Morton" <akpm@linux-foundation.org> wrote:

>On Thu, 14 Oct 2010 20:34:48 +0200
>Ingo Molnar <mingo@elte.hu> wrote:
>
>> 
>> * H. Peter Anvin <hpa@zytor.com> wrote:
>> 
>> > On 10/14/2010 09:00 AM, Pekka Enberg wrote:
>> >
>> > >> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm 
>> > >> doing". That is *what it is* and *all it is*.
>> > > 
>> > > Yup. The naming is quite unfortunate, though. Can we change it to 
>> > > something else?
>> > 
>> > Funny, this seems to come up repeatedly ;)
>> 
>> I'd suggest to rename CONFIG_EMBEDDED=y to CONFIG_EXPERT=y, but i'd also 
>> suggest to rename all sub-config-options that depend on CONFIG_EXPERT to 
>> have a CONFIG_EXPERT_ prefix.
>> 
>> So we'd have:
>> 
>>   CONFIG_EXPERT_CPU_SUP_INTEL=y
>>   CONFIG_EXPERT_MMAP_ALLOW_UNINITIALIZED=y
>> 
>> etc.
>
>yup.  CONFIG_EMBEDDED didn't make a lot of sense even when we first did it. 
>I suppose CONFIG_EXPERT is OK, although everyone will select it because
>they think they're experts ;)  CONFIG_PROPELLORHEAD!
>
>> That way all the enabled 'expert options' become easily visible in the 
>> .config file and they advertise themselves properly - it's also easily 
>> greppable. In the source code it also becomes self-documenting, it's 
>> obvious when a dependency is 'rare' or 'special' - it has a 
>> CONFIG_EXPERT_ prefix.
>> 
>> Something for the KS i guess.
>
>nah, just do it.

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Thu, 14 Oct 2010, H. Peter Anvin wrote:

> > I'm not running a CONFIG_EMBEDDED kernel and I'm pretty sure Casey isn't 
> > either, so we'd still need to carry our own patches to change the Kconfig 
> > bool line anyway.
> > 
> > I'm suprised that we can't allow users who definitively know that they 
> > have no hardware limitations the ability to disable ZONE_DMA.  Would a 
> > strong caution in the Kconfig "help" text suffice or will you not be 
> > convinced?
> 
> CONFIG_EMBEDDED *is* "I am a user who (think I) know what I'm doing".
> That is *what it is* and *all it is*.
> 

Ok, I'll follow Ingo's suggestion of renaming this to CONFIG_EXPERT and 
then prepare a patchset that allows users to disable CONFIG_ZONE_DMA, 
CONFIG_GENERIC_ISA_DMA, and CONFIG_ISA_DMA_API on x86.

Thanks!

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Thu, 14 Oct 2010, Andrew Morton wrote:

> > ZONE_DMA is unnecessary for a large number of machines that do not
> > require addressing in the lower 16MB of memory because they do not use
> > ISA devices with 16-bit address registers (plus one page byte register).
> > 
> > This patch allows users to disable ZONE_DMA for x86 if they know they
> > will not be using such devices with their kernel.
> > 
> > This prevents the VM from unnecessarily reserving a ratio of memory
> > (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> > for such allocations when it will never be used.
> > 
> 
> I wonder how hard it would be to do this at runtime, probably with a
> boot parameter.
> 

A "no_zone_dma" boot parameter wouldn't allow us to achieve the text or 
data savings that we see from disabling all three options in question 
here.

Hot-adding ZONE_DMA at runtime would be possible but there's no guarantee 
that memory hasn't fully been used by the time you do it, so it disregards 
lowmem_reserve_ratio unless you migrate everything, which has a dependency 
on it being movable.

> I'd be a little concerned at the effects of this on page reclaim and
> the page allocator - it might expose weird pre-existing bugs or
> inefficiencies.  But we can cross that bridge when we fall off it, I
> guess.
> 

We've run with it for a couple years, we can even undefine __GFP_DMA to 
find allocations that we compile into the kernel to ensure we don't have a 
requirement for the zone.  Perhaps only define the gfp flag when we have 
CONFIG_ZONE_DMA and break users' builds until they disable options that 
require it (or enable CONFIG_ZONE_DMA)?

It would be great if we could do "select ZONE_DMA" for all options that 
require it, though.

Re: [patch] x86: allow ZONE_DMA to be configurable

[Andrew Morton]

On Thu, 14 Oct 2010 15:15:28 -0700 (PDT)
David Rientjes <rientjes@google.com> wrote:

> On Thu, 14 Oct 2010, Andrew Morton wrote:
> 
> > > ZONE_DMA is unnecessary for a large number of machines that do not
> > > require addressing in the lower 16MB of memory because they do not use
> > > ISA devices with 16-bit address registers (plus one page byte register).
> > > 
> > > This patch allows users to disable ZONE_DMA for x86 if they know they
> > > will not be using such devices with their kernel.
> > > 
> > > This prevents the VM from unnecessarily reserving a ratio of memory
> > > (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> > > for such allocations when it will never be used.
> > > 
> > 
> > I wonder how hard it would be to do this at runtime, probably with a
> > boot parameter.
> > 
> 
> A "no_zone_dma" boot parameter wouldn't allow us to achieve the text or 
> data savings that we see from disabling all three options in question 
> here.

True, but doing it at runtime is vastly more user-friendly.  Distros
aren't doing to double the number of kernels they package for this
option!

Of course, we could do both.

> Hot-adding ZONE_DMA at runtime would be possible but there's no guarantee 
> that memory hasn't fully been used by the time you do it, so it disregards 
> lowmem_reserve_ratio unless you migrate everything, which has a dependency 
> on it being movable.
> 
> > I'd be a little concerned at the effects of this on page reclaim and
> > the page allocator - it might expose weird pre-existing bugs or
> > inefficiencies.  But we can cross that bridge when we fall off it, I
> > guess.
> > 
> 
> We've run with it for a couple years, we can even undefine __GFP_DMA to 
> find allocations that we compile into the kernel to ensure we don't have a 
> requirement for the zone.

"we" is google, I assume.

>  Perhaps only define the gfp flag when we have 
> CONFIG_ZONE_DMA and break users' builds until they disable options that 
> require it

That sounds a good idea.

What happens in the current patch if someone passes __GFP_DMA when
CONFIG_ZONE_DMA=n?  ENOMEM?  Perhaps it should WARN() as well.

> (or enable CONFIG_ZONE_DMA)?

hm.  It'd be better to make all the drivers depend on ZONE_DMA.  Good
luck with that :)

> It would be great if we could do "select ZONE_DMA" for all options that 
> require it, though.

Re: [patch] x86: allow ZONE_DMA to be configurable

[Ingo Molnar]

* H. Peter Anvin <hpa@zytor.com> wrote:

> I don't like the CONFIG_EXPERT prefix, just because it makes it hard 
> to move things into and out of this rubric, and that's bad... 

Hm, i'm not sure i agree.

> (consider: "oh, that would mean this *huge* patch) ...

If config options are present in many places in the source that's an 
independent problem already.

Typically kconfig variables are used in a few places and any rename 
patch makes it clear in all those places what happened: either a config 
variable became obscure (moved into CONFIG_EXPERT's scope), *or* a 
config variable became much less obscure (moved out of that scope) - 
both important pieces of information, which i dont mind to see happen in 
every place that gets affected.

Put differently: if we have a CONFIG_EXPERT_ kconfig variable in so many 
places in the kernel that it creates a huge patch then we have some 
other kind of problem: it either should not be an expert option (it's 
too common or too important), or it should not be an option at all (it's 
a perpetual build failure risk).

Also, historically we have not been moving kconfig variables in and out 
of CONFIG_EMBEDDED all that often for this to be an issue in the future.

> [...] not to mention that what is CONFIG_EXPERT on one platform may 
> not be for another! [...]

Dunno, that i'd consider more of a bug. If an architecture absolutely 
needs to configure in or out a kernel feature then it should not be 
under CONFIG_EXPERT on other architectures either: it only sets us up 
for cross-arch build failures if say the CONFIG_EXPERT dependency is 
present on x86.

Or, if an architecture still wants it that way, it better be explicitly 
aware that it's configuring in a little-tested kconfig switch of the 
kerenl.

I.e. something being an 'expert option' is synonymous with obscurity and 
potential weirdness on at least one arch - which i dont see as 
inherently wrong to be visible (via the name) on other architectures 
either.

Thanks,

	Ingo

Re: [patch] x86: allow ZONE_DMA to be configurable

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Thu, 14 Oct 2010 15:15:28 -0700 (PDT)
> David Rientjes <rientjes@google.com> wrote:
> 
> > On Thu, 14 Oct 2010, Andrew Morton wrote:
> > 
> > > > ZONE_DMA is unnecessary for a large number of machines that do not
> > > > require addressing in the lower 16MB of memory because they do not use
> > > > ISA devices with 16-bit address registers (plus one page byte register).
> > > > 
> > > > This patch allows users to disable ZONE_DMA for x86 if they know they
> > > > will not be using such devices with their kernel.
> > > > 
> > > > This prevents the VM from unnecessarily reserving a ratio of memory
> > > > (defaulting to 1/256th of system capacity) with lowmem_reserve_ratio
> > > > for such allocations when it will never be used.
> > > > 
> > > 
> > > I wonder how hard it would be to do this at runtime, probably with a
> > > boot parameter.
> > > 
> > 
> > A "no_zone_dma" boot parameter wouldn't allow us to achieve the text or 
> > data savings that we see from disabling all three options in question 
> > here.
> 
> True, but doing it at runtime is vastly more user-friendly.  Distros 
> aren't doing to double the number of kernels they package for this 
> option!

Strongly agreed!

Can we automate it perhaps? Drivers could express their zone needs, and 
perhaps we can free up the zone if there's no relevant driver loaded by 
the time we have booted up? If a hotplug driver arrives later on, is the 
MM good enough for us to be able to reinstate the zone dynamically?

Since we allocate top-down generally, the chance that the low zone is 
100% filled up with pinned, unmovable kmalloc buffers is small, so this 
solution could be practical - even if much of that zone is utilized 
already. These low allocations are pretty small typically.

We could also shrink the zone to an emergency amount, say 1MB, instead 
of zapping it completely, or so.

Thanks,

	Ingo

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

On 10/14/2010 08:03 PM, Ingo Molnar wrote:
> 
> Dunno, that i'd consider more of a bug. If an architecture absolutely 
> needs to configure in or out a kernel feature then it should not be 
> under CONFIG_EXPERT on other architectures either: it only sets us up 
> for cross-arch build failures if say the CONFIG_EXPERT dependency is 
> present on x86.
> 

Well, no, it really rather makes sense.

Consider something like CONFIG_SERIO_I8042 -- that is very much an
EXPERT option to remove on x86, but on a lot of other platforms it is
Just Another Interface.

So I think CONFIG_EXPERT_ will just encourage people to do strange
things and reduces flexibility.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Fri, 15 Oct 2010, Ingo Molnar wrote:

> Also, historically we have not been moving kconfig variables in and out 
> of CONFIG_EMBEDDED all that often for this to be an issue in the future.
> 

There are also kconfig options that depend on EMBEDDED that are actually 
for embedded or very small systems; the most obvious example that comes to 
mind is SLOB.  Simply doing an s/EMBEDDED/EXPERT/ conversion seems 
appropriate given the semantics for the option that Peter gave, but there 
are exceptions that will need to be identified, and I think SLOB is one of 
them.

I don't think we need a kconfig _dependency_ that exists solely to 
identify other config options for a specific field of hardware, but then 
we leave a burden for those users to identify the correct options that 
they need for their configurations that was, I'm assuming, the original 
intent of CONFIG_EMBEDDED in the first place.  So perhaps we can allow an 
option for small embedded systems that can "select" other options when 
enabled but don't have strict "depends" requirements?

I had a thread with Matt Mackall about a year and a half ago on LKML about 
the meaning of CONFIG_EMBEDDED and its use for the SLOB allocator; it 
would be much better for us to identify systems with memory limitations 
that strive to reduce footprint as much as possible to automatically 
select certain options over others.  I believe we talked about SLOB being 
the default for systems with under 16MB of memory, but we have limited 
ability to identify that for the user.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Thu, 14 Oct 2010, Andrew Morton wrote:

> > We've run with it for a couple years, we can even undefine __GFP_DMA to 
> > find allocations that we compile into the kernel to ensure we don't have a 
> > requirement for the zone.
> 
> "we" is google, I assume.
> 

Yes.

> >  Perhaps only define the gfp flag when we have 
> > CONFIG_ZONE_DMA and break users' builds until they disable options that 
> > require it
> 
> That sounds a good idea.
> 
> What happens in the current patch if someone passes __GFP_DMA when
> CONFIG_ZONE_DMA=n?  ENOMEM?  Perhaps it should WARN() as well.
> 

It'll return NULL and I suggested adding a 

#ifndef CONFIG_ZONE_DMA
	if (WARN_ON_ONCE(gfp_mask & __GFP_DMA))
		return NULL;
#endif

up front in the slowpath.

> hm.  It'd be better to make all the drivers depend on ZONE_DMA.  Good
> luck with that :)
> 

If they all truly depended on it then we'd never have been able to disable 
it.  I think breaking the build when you have CONFIG_ZONE_DMA=n and 
something is passing GFP_DMA would be the best alternative.

Re: [patch] x86: allow ZONE_DMA to be configurable

[David Rientjes]

On Fri, 15 Oct 2010, Ingo Molnar wrote:

> Can we automate it perhaps? Drivers could express their zone needs, and 
> perhaps we can free up the zone if there's no relevant driver loaded by 
> the time we have booted up? If a hotplug driver arrives later on, is the 
> MM good enough for us to be able to reinstate the zone dynamically?
> 

That's the same as setting /proc/sys/vm/lowmem_reserve_ratio to 0 for all 
zones so that no lowmem is reserved specifically for DMA allocations.  
Then, ZONE_DMA becomes like any other zone and is open to all allocations.

Re: [patch] x86: allow ZONE_DMA to be configurable

[H. Peter Anvin]

Yes, those exceptions should be factored out and put under a separate suboption... CONFIG_SMALLMEM or so.

"David Rientjes" <rientjes@google.com> wrote:

>On Fri, 15 Oct 2010, Ingo Molnar wrote:
>
>> Also, historically we have not been moving kconfig variables in and out 
>> of CONFIG_EMBEDDED all that often for this to be an issue in the future.
>> 
>
>There are also kconfig options that depend on EMBEDDED that are actually 
>for embedded or very small systems; the most obvious example that comes to 
>mind is SLOB.  Simply doing an s/EMBEDDED/EXPERT/ conversion seems 
>appropriate given the semantics for the option that Peter gave, but there 
>are exceptions that will need to be identified, and I think SLOB is one of 
>them.
>
>I don't think we need a kconfig _dependency_ that exists solely to 
>identify other config options for a specific field of hardware, but then 
>we leave a burden for those users to identify the correct options that 
>they need for their configurations that was, I'm assuming, the original 
>intent of CONFIG_EMBEDDED in the first place.  So perhaps we can allow an 
>option for small embedded systems that can "select" other options when 
>enabled but don't have strict "depends" requirements?
>
>I had a thread with Matt Mackall about a year and a half ago on LKML about 
>the meaning of CONFIG_EMBEDDED and its use for the SLOB allocator; it 
>would be much better for us to identify systems with memory limitations 
>that strive to reduce footprint as much as possible to automatically 
>select certain options over others.  I believe we talked about SLOB being 
>the default for systems with under 16MB of memory, but we have limited 
>ability to identify that for the user.

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.

Re: [patch] x86: allow ZONE_DMA to be configurable

[Christoph Lameter]

On Fri, 15 Oct 2010, David Rientjes wrote:

> > hm.  It'd be better to make all the drivers depend on ZONE_DMA.  Good
> > luck with that :)
> >
>
> If they all truly depended on it then we'd never have been able to disable
> it.  I think breaking the build when you have CONFIG_ZONE_DMA=n and
> something is passing GFP_DMA would be the best alternative.

Yes I also think that is best. Forces the issues to be finally addressed.

Building without CONFIG_ZONE_DMA will remove the need to keep dma kmalloc
caches around and avoid checking for __GFP_DMA in the hotpaths of the slab
allocators.