9a2d43b: __alloc_bootmem_core(): zero-sized request

9a2d43b: __alloc_bootmem_core(): zero-sized request

[Vegard Nossum]

Hi,

I was trying out

commit 9a2d43b7566caeeeb414aa628bc2759028897dbb
Date:   Tue Jul 15 21:21:43 2008 +0200

..as part of the debugging of a different issue, but I got this:

__alloc_bootmem_core(): zero-sized request
------------[ cut here ]------------
kernel BUG at /uio/arkimedes/s29/vegardno/git-working/linux-2.6/mm/bootmem.c:247!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
Pid: 0, comm: swapper Not tainted (2.6.26-00014-g9a2d43b #34)
EIP: 0060:[<c082be18>] EFLAGS: 00010286 CPU: 0
EIP is at __alloc_bootmem_core+0x28/0x300
EAX: 0000002e EBX: 00000080 ECX: c080e000 EDX: 00000002
ESI: 01000000 EDI: 01000000 EBP: c080ff44 ESP: c080ff00
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process swapper (pid: 0, ti=c080e000 task=c078d400 task.ti=c080e000)
Stack: c0716894 c080ff14 c0636753 0000230d 00000080 00000000 c08c2fa0 00000096
       ffffdcf3 34791ce0 00000217 c080ffac c0136894 c080ffac 00000080 00000000
       01000000 c080ff60 c082c28c 01000000 00000000 00000000 00000400 00000000
Call Trace:
 [<c0636753>] ? _spin_unlock_irqrestore+0x43/0x70
 [<c0136894>] ? vprintk+0x1c4/0x420
 [<c082c28c>] ? __alloc_bootmem_node+0x2c/0x50
 [<c082d370>] ? alloc_large_system_hash+0x1e0/0x2d0
 [<c03704f2>] ? dotest+0x382/0x400
 [<c082e40c>] ? vfs_caches_init_early+0x5c/0xb0
 [<c08159f0>] ? start_kernel+0x250/0x390
 [<c0815180>] ? unknown_bootoption+0x0/0x210
 [<c0815008>] ? __init_begin+0x8/0x10
 =======================
Code: 00 00 00 55 89 e5 57 56 53 83 ec 38 85 d2 8b 75 08 89 4d cc 8b
4d 0c 89 45 d4 89 55 d0 75 14 c7 04 24 94 68 71 c0 e8 d8 ac 90 ff <0f>
0b eb fe 8d 74 26 00 8b 7d cc 83 ef 01 85 7d cc 75 16 8b 55
EIP: [<c082be18>] __alloc_bootmem_core+0x28/0x300 SS:ESP 0068:c080ff00
---[ end trace 4eaa2a86a8e2da22 ]---

I saw some bootmem errata lately, can I cherry-pick anything to fix this?


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Vegard Nossum]

On Fri, Aug 22, 2008 at 9:24 PM, Vegard Nossum <vegard.nossum@gmail.com> wrote:
> Hi,
>
> I was trying out
>
> commit 9a2d43b7566caeeeb414aa628bc2759028897dbb
> Date:   Tue Jul 15 21:21:43 2008 +0200
>
> ..as part of the debugging of a different issue, but I got this:
>
> __alloc_bootmem_core(): zero-sized request
> ------------[ cut here ]------------
> kernel BUG at /uio/arkimedes/s29/vegardno/git-working/linux-2.6/mm/bootmem.c:247!
[...]

Don't know how much it helps, but I tried this patch:

diff --git a/mm/bootmem.c b/mm/bootmem.c
index 8d9f60e..56298cb 100644
--- a/mm/bootmem.c
+++ b/mm/bootmem.c
@@ -244,7 +244,7 @@ __alloc_bootmem_core(struct bootmem_data *bdata, unsigned lo
ng size,

        if (!size) {
                printk("__alloc_bootmem_core(): zero-sized request\n");
-               BUG();
+               return ZERO_SIZE_PTR;
        }
        BUG_ON(align & (align-1));

...and I got this (it was apparently a bad idea ;-)):

__alloc_bootmem_core(): zero-sized request
Dentry cache hash table entries: -2147483648 (order: -13, 0 bytes)
__alloc_bootmem_core(): zero-sized request
Inode-cache hash table entries: -2147483648 (o -13, 0 bytes)
Bad page state in process 'swapper'
page:000000f0 flags:0x00000000 mapping:00000000 mapcount:1 count:0
Trying to fix it up, but a reboot is needed
Backtrace:
Pid: 0, comm: swapper Not tainted 2.6.26-00014-g9a2d43b-dirty #36
 [<c01823b0>] bad_page+0x90/0xd0
 [<c018351b>] free_hot_cold_page+0x21b/0x230
 [<c018357a>] free_hot_page+0xa/0x10
 [<c01835ab>] __free_pages+0x2b/0x40
 [<c01849d2>] __free_pages_bootmem+0x82/0xa0
 [<c082baeb>] free_all_bootmem_core+0xfb/0x1d0
 [<c082bbcd>] free_all_bootmem+0xd/0x10
 [<c08268fa>] mem_init+0x7a/0x3a0
 [<c082e612>] ? inode_init_early+0x62/0xb0
 [<c082e408>] ? vfs_caches_init_early+0x18/0xb0
 [<c08159fa>] start_kernel+0x25a/0x390
 [<c0815180>] ? unknown_bootoption+0x0/0x210
 [<c0815008>] __init_begin+0x8/0x10
 =======================
list_add corruption. next->prev should be prev (0000000c), but was
00000000. (next=00000000).
------------[ cut here ]------------
kernel BUG at /uio/arkimedes/s29/vegardno/git-working/linux-2.6/lib/list_debug.c:27!


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Johannes Weiner]

"Vegard Nossum" <vegard.nossum@gmail.com> writes:

> Hi,
>
> I was trying out
>
> commit 9a2d43b7566caeeeb414aa628bc2759028897dbb
> Date:   Tue Jul 15 21:21:43 2008 +0200
>
> ..as part of the debugging of a different issue, but I got this:
>
> __alloc_bootmem_core(): zero-sized request
> ------------[ cut here ]------------
> kernel BUG at /uio/arkimedes/s29/vegardno/git-working/linux-2.6/mm/bootmem.c:247!
> invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> Pid: 0, comm: swapper Not tainted (2.6.26-00014-g9a2d43b #34)
> EIP: 0060:[<c082be18>] EFLAGS: 00010286 CPU: 0
> EIP is at __alloc_bootmem_core+0x28/0x300
> EAX: 0000002e EBX: 00000080 ECX: c080e000 EDX: 00000002
> ESI: 01000000 EDI: 01000000 EBP: c080ff44 ESP: c080ff00
>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> Process swapper (pid: 0, ti=c080e000 task=c078d400 task.ti=c080e000)
> Stack: c0716894 c080ff14 c0636753 0000230d 00000080 00000000 c08c2fa0 00000096
>        ffffdcf3 34791ce0 00000217 c080ffac c0136894 c080ffac 00000080 00000000
>        01000000 c080ff60 c082c28c 01000000 00000000 00000000 00000400 00000000
> Call Trace:
>  [<c0636753>] ? _spin_unlock_irqrestore+0x43/0x70
>  [<c0136894>] ? vprintk+0x1c4/0x420
>  [<c082c28c>] ? __alloc_bootmem_node+0x2c/0x50
>  [<c082d370>] ? alloc_large_system_hash+0x1e0/0x2d0
>  [<c03704f2>] ? dotest+0x382/0x400
>  [<c082e40c>] ? vfs_caches_init_early+0x5c/0xb0
>  [<c08159f0>] ? start_kernel+0x250/0x390
>  [<c0815180>] ? unknown_bootoption+0x0/0x210
>  [<c0815008>] ? __init_begin+0x8/0x10
>  =======================
> Code: 00 00 00 55 89 e5 57 56 53 83 ec 38 85 d2 8b 75 08 89 4d cc 8b
> 4d 0c 89 45 d4 89 55 d0 75 14 c7 04 24 94 68 71 c0 e8 d8 ac 90 ff <0f>
> 0b eb fe 8d 74 26 00 8b 7d cc 83 ef 01 85 7d cc 75 16 8b 55
> EIP: [<c082be18>] __alloc_bootmem_core+0x28/0x300 SS:ESP 0068:c080ff00
> ---[ end trace 4eaa2a86a8e2da22 ]---
>
> I saw some bootmem errata lately, can I cherry-pick anything to fix
> this?

This behaviour hasn't changed after the rewrite.

	Hannes

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Vegard Nossum]

On Sat, Aug 23, 2008 at 9:12 AM, Johannes Weiner <hannes@saeurebad.de> wrote:
> "Vegard Nossum" <vegard.nossum@gmail.com> writes:
>>
>> I was trying out
>>
>> commit 9a2d43b7566caeeeb414aa628bc2759028897dbb
>> Date:   Tue Jul 15 21:21:43 2008 +0200
>>
>> ..as part of the debugging of a different issue, but I got this:
>>
>> __alloc_bootmem_core(): zero-sized request

[...]

>> I saw some bootmem errata lately, can I cherry-pick anything to fix
>> this?
>
> This behaviour hasn't changed after the rewrite.

Yep, the error is somewhere else.

Inserted a printk("nr_kernel_pages = %llu\n", nr_kernel_pages); and
this is the output:

nr_kernel_pages = 13869392367443771392

...it looks very, very big. I think this is initialized via
free_area_init_*() functions, which come from arch code. Does x86
experts know if any of this changed recently (i.e. after July 15)?

The whole dmesg and config can be seen here:
http://userweb.kernel.org/~vegard/bugs/20080824-numa/


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Yinghai Lu]

On Sun, Aug 24, 2008 at 12:58 AM, Vegard Nossum <vegard.nossum@gmail.com> wrote:
> On Sat, Aug 23, 2008 at 9:12 AM, Johannes Weiner <hannes@saeurebad.de> wrote:
>> "Vegard Nossum" <vegard.nossum@gmail.com> writes:
>>>
>>> I was trying out
>>>
>>> commit 9a2d43b7566caeeeb414aa628bc2759028897dbb
>>> Date:   Tue Jul 15 21:21:43 2008 +0200
>>>
>>> ..as part of the debugging of a different issue, but I got this:
>>>
>>> __alloc_bootmem_core(): zero-sized request
>
> [...]
>
>>> I saw some bootmem errata lately, can I cherry-pick anything to fix
>>> this?
>>
>> This behaviour hasn't changed after the rewrite.
>
> Yep, the error is somewhere else.
>
> Inserted a printk("nr_kernel_pages = %llu\n", nr_kernel_pages); and
> this is the output:
>
> nr_kernel_pages = 13869392367443771392
>
> ...it looks very, very big. I think this is initialized via
> free_area_init_*() functions, which come from arch code. Does x86
> experts know if any of this changed recently (i.e. after July 15)?
>
> The whole dmesg and config can be seen here:
> http://userweb.kernel.org/~vegard/bugs/20080824-numa/
>

seems something wrong about apic probe...

MPTABLE: OEM ID: Intel    Product ID: Lakeport     Warning!  May not
be a NUMA-Q system!
MPTABLE: Product ID: Lakeport     <6>MPTABLE: APIC at: 0xFEE00000
Processor #0 15:6 APIC version 20 (quad 0, apic 1)
Processor #0 (Bootup-CPU)
Bus #0 is PCI    (node 0)
Bus #1 is PCI    (node 0)
Bus #2 is PCI    (node 0)
Bus #3 is ISA    (node 0)
I/O APIC #2 Version 32 at 0xFEC00000.
Enabling APIC mode:  NUMA-Q.  Using 1 I/O APICs


please check with tip/master to see if it is fixed already...

YH

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Vegard Nossum]

On Sun, Aug 24, 2008 at 10:22 AM, Yinghai Lu <yhlu.kernel@gmail.com> wrote:
>> nr_kernel_pages = 13869392367443771392
>>
>> ...it looks very, very big. I think this is initialized via
>> free_area_init_*() functions, which come from arch code. Does x86
>> experts know if any of this changed recently (i.e. after July 15)?
>>
>> The whole dmesg and config can be seen here:
>> http://userweb.kernel.org/~vegard/bugs/20080824-numa/
>>
>
> seems something wrong about apic probe...
>
> MPTABLE: OEM ID: Intel    Product ID: Lakeport     Warning!  May not
> be a NUMA-Q system!

It is not a NUMA-Q system, it's a normal desktop-class Pentium 4
machine with HT. But still, this kernel is expected to work, isn't it?
(At least expected to tell me "cannot boot" instead of giving
unexplainable errors.)

> MPTABLE: Product ID: Lakeport     <6>MPTABLE: APIC at: 0xFEE00000
> Processor #0 15:6 APIC version 20 (quad 0, apic 1)
> Processor #0 (Bootup-CPU)
> Bus #0 is PCI    (node 0)
> Bus #1 is PCI    (node 0)
> Bus #2 is PCI    (node 0)
> Bus #3 is ISA    (node 0)
> I/O APIC #2 Version 32 at 0xFEC00000.
> Enabling APIC mode:  NUMA-Q.  Using 1 I/O APICs
>
>
> please check with tip/master to see if it is fixed already...

Yes, the v2.6.27-rc4 boots fine. I was wondering if you knew that this
problem had occurred in the past (and if so, which commits I can
cherry-pick to fix it). But I guess there is not much to do. It is
also hard to test this same config with newer kernels, because
something in the Kconfig changed in an incompatible way (e.g. NUMAQ
setting changes when I change kernel versions).

Unless anybody can think of something, I think I will leave this
problem alone. It's an older version after all. Thanks for looking at
it! :-)


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Yinghai Lu]

On Sun, Aug 24, 2008 at 1:43 AM, Vegard Nossum <vegard.nossum@gmail.com> wrote:
dy...
>
> Yes, the v2.6.27-rc4 boots fine. I was wondering if you knew that this
> problem had occurred in the past (and if so, which commits I can
> cherry-pick to fix it). But I guess there is not much to do. It is
> also hard to test this same config with newer kernels, because
> something in the Kconfig changed in an incompatible way (e.g. NUMAQ
> setting changes when I change kernel versions).

a lot of changes on that section code in 2.6.27.

and according to old code, you are not supposed to use numaq enabled
kernel on non-numaq HW.
but from 2.6.27-rc1 you could do that. we made the code probe the
subarch and will fall back to default or bigsmp

YH

Re: 9a2d43b: __alloc_bootmem_core(): zero-sized request

[Vegard Nossum]

On Sun, Aug 24, 2008 at 9:58 AM, Vegard Nossum <vegard.nossum@gmail.com> wrote:
> Inserted a printk("nr_kernel_pages = %llu\n", nr_kernel_pages); and
> this is the output:
>
> nr_kernel_pages = 13869392367443771392

Also written as 0xc079fae000000000. Of which the upper 32 bits
"coincidentally" matches that of a symbol:

c079fae0 d mutex_Z2

But the significance eludes me...


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036