does non-continuous RAM means I need to select the sparse memory model?

does non-continuous RAM means I need to select the sparse memory model?

[Uwe Kleine-König]

Hello,

I'm currently working on a new arch port and my current machine has RAM
at 0x10000000 and 0x80000000. So there is a big hole between the two
banks. When selecting the sparse memory model it works, but when
selecting flat the machine runs into a BUG in mark_bootmem() called by
free_unused_memmap() to free the space between the two banks.

Is that expected (meaning I cannot use the flat model)? I currently
don't have another machine handy that has >1 memory back to test that.

This happens on 3.2-rc2 and with a port on 2.6.33 I don't need to select
sparse.

Best regards and thanks for any insight,
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

does non-continuous RAM means I need to select the sparse memory model?

[Fabio Estevam]

Hi Uwe,

2011/11/29 Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de>:
> Hello,
>
> I'm currently working on a new arch port and my current machine has RAM
> at 0x10000000 and 0x80000000. So there is a big hole between the two
> banks. When selecting the sparse memory model it works, but when
> selecting flat the machine runs into a BUG in mark_bootmem() called by
> free_unused_memmap() to free the space between the two banks.

My understanding is that you have to select ARCH_HAS_HOLES_MEMORYMODEL.

> Is that expected (meaning I cannot use the flat model)? I currently
> don't have another machine handy that has >1 memory back to test that.

In case you have access to a MX35PDK you can try on this board as it does have
the memory hole.

Regards,

Fabio Estevam

does non-continuous RAM means I need to select the sparse memory model?

[Uwe Kleine-König]

Hello Fabio,

On Tue, Nov 29, 2011 at 10:39:10PM -0200, Fabio Estevam wrote:
> 2011/11/29 Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de>:
> > Hello,
> >
> > I'm currently working on a new arch port and my current machine has RAM
> > at 0x10000000 and 0x80000000. So there is a big hole between the two
> > banks. When selecting the sparse memory model it works, but when
> > selecting flat the machine runs into a BUG in mark_bootmem() called by
> > free_unused_memmap() to free the space between the two banks.
> 
> My understanding is that you have to select ARCH_HAS_HOLES_MEMORYMODEL.
I tried that, but it didn't help.
 
> > Is that expected (meaning I cannot use the flat model)? I currently
> > don't have another machine handy that has >1 memory back to test that.
> 
> In case you have access to a MX35PDK you can try on this board as it does have
> the memory hole.
I will ask in the office later today if we have such a machine ready for
me to test.

Thanks
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

flatmem broken for nommu? [Was: Re: does non-continuous RAM means I need to select the sparse memory model?]

[Uwe Kleine-König]

On Tue, Nov 29, 2011 at 10:39:10PM -0200, Fabio Estevam wrote:
> Hi Uwe,
> 
> 2011/11/29 Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de>:
> > Hello,
> >
> > I'm currently working on a new arch port and my current machine has RAM
> > at 0x10000000 and 0x80000000. So there is a big hole between the two
> > banks. When selecting the sparse memory model it works, but when
> > selecting flat the machine runs into a BUG in mark_bootmem() called by
> > free_unused_memmap() to free the space between the two banks.
> 
> My understanding is that you have to select ARCH_HAS_HOLES_MEMORYMODEL.
I think that is not necessary.
 
> > Is that expected (meaning I cannot use the flat model)? I currently
> > don't have another machine handy that has >1 memory back to test that.
> 
> In case you have access to a MX35PDK you can try on this board as it does have
> the memory hole.
No I havn't, but I just used a 128MB machine and changed that in the
.fixup callback to 64MB + 32MB with a 32MB hole in between and it works
fine without ARCH_HAS_HOLES_MEMORYMODEL.

I debugged the problem a bit further and one symptom is that

	struct page *mem_map

is NULL for me. That looks wrong. I guess this is just broken for nommu.
I will dig into that later today.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

flatmem broken for nommu? [Was: Re: does non-continuous RAM means I need to select the sparse memory model?]

[Uwe Kleine-König]

Hello,

On Thu, Dec 01, 2011 at 11:57:18AM +0100, Uwe Kleine-K?nig wrote:
> On Tue, Nov 29, 2011 at 10:39:10PM -0200, Fabio Estevam wrote:
> > 2011/11/29 Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de>:
> > > Hello,
> > >
> > > I'm currently working on a new arch port and my current machine has RAM
> > > at 0x10000000 and 0x80000000. So there is a big hole between the two
> > > banks. When selecting the sparse memory model it works, but when
> > > selecting flat the machine runs into a BUG in mark_bootmem() called by
> > > free_unused_memmap() to free the space between the two banks.
> > 
> > My understanding is that you have to select ARCH_HAS_HOLES_MEMORYMODEL.
> I think that is not necessary.
>  
> > > Is that expected (meaning I cannot use the flat model)? I currently
> > > don't have another machine handy that has >1 memory back to test that.
> > 
> > In case you have access to a MX35PDK you can try on this board as it does have
> > the memory hole.
> No I havn't, but I just used a 128MB machine and changed that in the
> .fixup callback to 64MB + 32MB with a 32MB hole in between and it works
> fine without ARCH_HAS_HOLES_MEMORYMODEL.
> 
> I debugged the problem a bit further and one symptom is that
> 
> 	struct page *mem_map
> 
> is NULL for me. That looks wrong. I guess this is just broken for nommu.
> I will dig into that later today.
The problem is that the memory for mem_map is allocated using:

	map = alloc_bootmem_node_nopanic(pgdat, size);

without any error checking. The _nopanic was introduced by commit

	8f389a99 (mm: use alloc_bootmem_node_nopanic() on really needed path)

I don't understand the commit's log and don't really see why it should
be allowed to not panic if the allocation failes here but use a NULL
pointer instead.
I put the people involved in 8f389a99 on Cc, maybe someone can comment?

Apart from that it seems I cannot use flatmem as is on my machine. It
has only 128kiB at 0x10000000 + 1MiB at 0x80000000 and needs 14MiB to hold the
table of "struct page"s. :-(

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

flatmem broken for nommu? [Was: Re: does non-continuous RAM means I need to select the sparse memory model?]

[Stefan Hellermann]

Am 01.12.2011 16:39, schrieb Uwe Kleine-K?nig:
> Hello,
> 
> On Thu, Dec 01, 2011 at 11:57:18AM +0100, Uwe Kleine-K?nig wrote:
>> On Tue, Nov 29, 2011 at 10:39:10PM -0200, Fabio Estevam wrote:
>>> 2011/11/29 Uwe Kleine-K?nig <u.kleine-koenig@pengutronix.de>:
>>>> Hello,
>>>>
>>>> I'm currently working on a new arch port and my current machine has RAM
>>>> at 0x10000000 and 0x80000000. So there is a big hole between the two
>>>> banks. When selecting the sparse memory model it works, but when
>>>> selecting flat the machine runs into a BUG in mark_bootmem() called by
>>>> free_unused_memmap() to free the space between the two banks.
>>>
>>> My understanding is that you have to select ARCH_HAS_HOLES_MEMORYMODEL.
>> I think that is not necessary.
>>  
>>>> Is that expected (meaning I cannot use the flat model)? I currently
>>>> don't have another machine handy that has >1 memory back to test that.
>>>
>>> In case you have access to a MX35PDK you can try on this board as it does have
>>> the memory hole.
>> No I havn't, but I just used a 128MB machine and changed that in the
>> .fixup callback to 64MB + 32MB with a 32MB hole in between and it works
>> fine without ARCH_HAS_HOLES_MEMORYMODEL.
>>
>> I debugged the problem a bit further and one symptom is that
>>
>> 	struct page *mem_map
>>
>> is NULL for me. That looks wrong. I guess this is just broken for nommu.
>> I will dig into that later today.
> The problem is that the memory for mem_map is allocated using:
> 
> 	map = alloc_bootmem_node_nopanic(pgdat, size);
> 
> without any error checking. The _nopanic was introduced by commit
> 
> 	8f389a99 (mm: use alloc_bootmem_node_nopanic() on really needed path)
> 
> I don't understand the commit's log and don't really see why it should
> be allowed to not panic if the allocation failes here but use a NULL
> pointer instead.
> I put the people involved in 8f389a99 on Cc, maybe someone can comment?
> 
> Apart from that it seems I cannot use flatmem as is on my machine. It
> has only 128kiB at 0x10000000 + 1MiB at 0x80000000 and needs 14MiB to hold the
> table of "struct page"s. :-(
> 
> Best regards
> Uwe
> 
The commit was made after an bug report from me. I have an old x86
tablet pc with only 8Mb Ram. This machine fails early on bootup without
this commit. I found an archived message of the bug report here:
http://comments.gmane.org/gmane.linux.kernel/1135909

Regards,
Stefan

flatmem broken for nommu? [Was: Re: does non-continuous RAM means I need to select the sparse memory model?]

[Uwe Kleine-König]

Hello Stefan,

On Thu, Dec 01, 2011 at 05:10:23PM +0100, Stefan Hellermann wrote:
> Am 01.12.2011 16:39, schrieb Uwe Kleine-K?nig:
> > The problem is that the memory for mem_map is allocated using:
> > 
> > 	map = alloc_bootmem_node_nopanic(pgdat, size);
> > 
> > without any error checking. The _nopanic was introduced by commit
> > 
> > 	8f389a99 (mm: use alloc_bootmem_node_nopanic() on really needed path)
> > 
> > I don't understand the commit's log and don't really see why it should
> > be allowed to not panic if the allocation failes here but use a NULL
> > pointer instead.
> > I put the people involved in 8f389a99 on Cc, maybe someone can comment?
> > 
> > Apart from that it seems I cannot use flatmem as is on my machine. It
> > has only 128kiB at 0x10000000 + 1MiB at 0x80000000 and needs 14MiB to hold the
> > table of "struct page"s. :-(
> > 
> > Best regards
> > Uwe
> > 
> The commit was made after an bug report from me. I have an old x86
> tablet pc with only 8Mb Ram. This machine fails early on bootup without
> this commit. I found an archived message of the bug report here:
> http://comments.gmane.org/gmane.linux.kernel/1135909
I saw that, too, but still I think that at least the last hunk in this
patch is wrong. (I didn't check the others.) For me the allocation for
mem_map failed and instead of handling the error somehow (be it a panic
or not) just using NULL isn't nice.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

does non-continuous RAM means I need to select the sparse memory model?

[Russell King - ARM Linux]

On Tue, Nov 29, 2011 at 09:30:10PM +0100, Uwe Kleine-K?nig wrote:
> I'm currently working on a new arch port and my current machine has RAM
> at 0x10000000 and 0x80000000. So there is a big hole between the two
> banks. When selecting the sparse memory model it works, but when
> selecting flat the machine runs into a BUG in mark_bootmem() called by
> free_unused_memmap() to free the space between the two banks.

Well, it looks like you have the answer in your latest email:

> Apart from that it seems I cannot use flatmem as is on my machine. It
> has only 128kiB at 0x10000000 + 1MiB at 0x80000000 and needs 14MiB to hold the
> table of "struct page"s.

Which shows that there's no possible way to use flatmem in this platform.

Now, to use sparsemem with this, sparsemem people will tell you that
each sparsemem section must be fully populated.  So, as your minimum
memory size is 128kiB, your sparsemem section size must be 128kiB.
This makes SECTION_SIZE_BITS be 17.

You need to span to 0x80000000+, which means you need MAX_PHYSMEM_BITS
to be 32.

These two together means you're in for having an array of 2^(32 - 17) =
32768 pointers to struct section for starters.  That takes 128kiB.
Now add in the struct page structures, which for 128kiB of memory will
be 1kiB.  Now add 9 struct section's as well...

What I'm pointing out is that in the scenario you appear to have, trying
to use the small pool of memory which is soo dislocated from the rest of
memory is a waste of effort.  Adding it to the pool means that you require
more memory for the management data structures than you gain from adding
it to the system.  The net benefit from trying to use it is actually
negative.

So, forget that the 128kiB chunk of memory exists - at that size and
distance from the rest of memory its really not worth bothering with.

does non-continuous RAM means I need to select the sparse memory model?

[Uwe Kleine-König]

On Thu, Dec 01, 2011 at 06:28:52PM +0000, Russell King - ARM Linux wrote:
> On Tue, Nov 29, 2011 at 09:30:10PM +0100, Uwe Kleine-K?nig wrote:
> > I'm currently working on a new arch port and my current machine has RAM
> > at 0x10000000 and 0x80000000. So there is a big hole between the two
> > banks. When selecting the sparse memory model it works, but when
> > selecting flat the machine runs into a BUG in mark_bootmem() called by
> > free_unused_memmap() to free the space between the two banks.
> 
> Well, it looks like you have the answer in your latest email:
> 
> > Apart from that it seems I cannot use flatmem as is on my machine. It
> > has only 128kiB at 0x10000000 + 1MiB at 0x80000000 and needs 14MiB to hold the
> > table of "struct page"s.
> 
> Which shows that there's no possible way to use flatmem in this platform.
> 
> Now, to use sparsemem with this, sparsemem people will tell you that
> each sparsemem section must be fully populated.  So, as your minimum
> memory size is 128kiB, your sparsemem section size must be 128kiB.
> This makes SECTION_SIZE_BITS be 17.
> 
> You need to span to 0x80000000+, which means you need MAX_PHYSMEM_BITS
> to be 32.
> 
> These two together means you're in for having an array of 2^(32 - 17) =
> 32768 pointers to struct section for starters.  That takes 128kiB.
> Now add in the struct page structures, which for 128kiB of memory will
> be 1kiB.  Now add 9 struct section's as well...
> 
> What I'm pointing out is that in the scenario you appear to have, trying
> to use the small pool of memory which is soo dislocated from the rest of
> memory is a waste of effort.  Adding it to the pool means that you require
> more memory for the management data structures than you gain from adding
> it to the system.  The net benefit from trying to use it is actually
> negative.
> 
> So, forget that the 128kiB chunk of memory exists - at that size and
> distance from the rest of memory its really not worth bothering with.
many thanks for your description. I'll go with flatmem skipping the
128kiB bank now.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |