Fw: Re: 2.6.18-rc2-mm1

public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed

* Fw: Re: 2.6.18-rc2-mm1
       [not found] <20060727224233.7fe3724a.akpm@osdl.org>
@ 2006-07-28  8:55 ` Jan Beulich
  2006-07-28 10:35   ` Andi Kleen
  0 siblings, 1 reply; 4+ messages in thread
From: Jan Beulich @ 2006-07-28  8:55 UTC (permalink / raw)
  To: michal.k.k.piotrowski, linux-kernel
  Cc: Ingo Molnar, Andi Kleen, Andrew Morton

>>> Andrew Morton <akpm@osdl.org> 28.07.06 07:42 >>>
>
>fyi, Michael's dwarf unwinder seems to have broken.
>(please follow up on lkml).

Hmm, not being able to unwind through sysenter_entry is no surprise
(this simply cannot be properly annotated, as the return address is not
explicit), but it'd end up in user mode anyway (and the inexact backtrace
doesn't go past it either). The fallback message is a little mis-leading as
what is shown is not the left-over backtrace, but the full one (Andi
probably knows better if/when/why this is supposed to be that way).

Likewise for the more puzzling case of not being able to unwind through
error_code - the left-over trace is again more like a full one. I'm not clear
why it can't unwind through error_code here; a sufficiently large piece
of the raw stack dump would be needed to check what's going on here,
and I just again (don't know how many times I already did this) verified
that in a similar scenario I get a proper unwind through that point.

The third one, getting stuck at __down_failed, is due to the still
unresolved issue of improper (from the perspective of stack unwinding)
instruction ordering include/asm-i386/semaphore.h.

Jan

***********************************************
Begin forwarded message:

Date: Thu, 27 Jul 2006 15:32:37 +0200
From: "Michal Piotrowski" <michal.k.k.piotrowski@gmail.com>
To: "Andrew Morton" <akpm@osdl.org>
Cc: linux-kernel@vger.kernel.org 
Subject: Re: 2.6.18-rc2-mm1


Hi Andrew,

On 27/07/06, Andrew Morton <akpm@osdl.org> wrote:
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.18-rc2/2.6.18-rc2-mm1/ 
>

It appears while /sbin/start_udev

Jul 27 15:17:17 ltg01-fedora kernel: BUG: unable to handle kernel
paging request at virtual address 6b6b6c07
Jul 27 15:17:17 ltg01-fedora kernel:  printing eip:
Jul 27 15:17:17 ltg01-fedora kernel: c0138722
Jul 27 15:17:17 ltg01-fedora kernel: *pde = 00000000
Jul 27 15:17:17 ltg01-fedora kernel: Oops: 0002 [#1]
Jul 27 15:17:17 ltg01-fedora kernel: 4K_STACKS PREEMPT SMP
Jul 27 15:17:17 ltg01-fedora kernel: last sysfs file:
/devices/pci0000:00/0000:00:1d.7/uevent
Jul 27 15:17:17 ltg01-fedora kernel: Modules linked in: snd_timer snd
soundcore snd_page_alloc intel_agp agpgart ide_cd cdrom ipv6 w83627hf
hwmon_vid hwmon i2c_isa i2c_i801 skge af_packet
ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink
xt_tcpudp iptable_filter ip_tables x_tables cpufreq_userspace
p4_clockmod speedstep_lib binfmt_misc thermal processor fan container
rtc unix
Jul 27 15:17:17 ltg01-fedora kernel: CPU:    0
Jul 27 15:17:17 ltg01-fedora kernel: EIP:    0060:[<c0138722>]    Not
tainted VLI
Jul 27 15:17:17 ltg01-fedora kernel: EFLAGS: 00010046   (2.6.18-rc2-mm1 #78)
Jul 27 15:17:17 ltg01-fedora kernel: EIP is at __lock_acquire+0x362/0xaea
Jul 27 15:17:17 ltg01-fedora kernel: eax: 00000000   ebx: 6b6b6b6b
ecx: c0360358   edx: 00000000
Jul 27 15:17:17 ltg01-fedora kernel: esi: 00000000   edi: 00000000
ebp: f544ddf4   esp: f544ddc0
Jul 27 15:17:17 ltg01-fedora kernel: ds: 007b   es: 007b   ss: 0068
Jul 27 15:17:17 ltg01-fedora kernel: Process udevd (pid: 1353,
ti=f544d000 task=f6fce8f0 task.ti=f544d000)
Jul 27 15:17:17 ltg01-fedora kernel: Stack: 00000000 00000000 00000000
c7749ea4 f6fce8f0 c0138e74 000001e8 00000000
Jul 27 15:17:17 ltg01-fedora kernel:        00000000 f6653fa4 00000246
00000000 00000000 f544de1c c0139214 00000000
Jul 27 15:17:17 ltg01-fedora kernel:        00000002 00000000 c014fe3a
c7749ea4 c7749e90 f6fce8f0 f5b19b04 f544de34
Jul 27 15:17:17 ltg01-fedora kernel: Call Trace:
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0139214>] lock_acquire+0x71/0x91
Jul 27 15:17:17 ltg01-fedora kernel:  [<c02f2bfb>] _spin_lock+0x23/0x32
Jul 27 15:17:17 ltg01-fedora kernel:  [<c014fe3a>]
__delayacct_blkio_ticks+0x16/0x67
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a4f76>] do_task_stat+0x3df/0x6c1
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a5265>] proc_tgid_stat+0xd/0xf
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a29dd>] proc_info_read+0x50/0xb3
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0171cbb>] vfs_read+0xcb/0x177
Jul 27 15:17:17 ltg01-fedora kernel:  [<c017217c>] sys_read+0x3b/0x71
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0103119>] sysenter_past_esp+0x56/0x8d
Jul 27 15:17:17 ltg01-fedora kernel: DWARF2 unwinder stuck at
sysenter_past_esp+0x56/0x8d
Jul 27 15:17:17 ltg01-fedora kernel: Leftover inexact backtrace:
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0104318>] show_stack_log_lvl+0x8c/0x97
Jul 27 15:17:17 ltg01-fedora kernel:  [<c010447f>] show_registers+0x15c/0x1ed
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01046c2>] die+0x1b2/0x2b7
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0116f5f>] do_page_fault+0x410/0x4f0
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0103d1d>] error_code+0x39/0x40
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0139214>] lock_acquire+0x71/0x91
Jul 27 15:17:17 ltg01-fedora kernel:  [<c02f2bfb>] _spin_lock+0x23/0x32
Jul 27 15:17:17 ltg01-fedora kernel:  [<c014fe3a>]
__delayacct_blkio_ticks+0x16/0x67
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a4f76>] do_task_stat+0x3df/0x6c1
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a5265>] proc_tgid_stat+0xd/0xf
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01a29dd>] proc_info_read+0x50/0xb3
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0171cbb>] vfs_read+0xcb/0x177
Jul 27 15:17:17 ltg01-fedora kernel:  [<c017217c>] sys_read+0x3b/0x71
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0103119>] sysenter_past_esp+0x56/0x8d
Jul 27 15:17:17 ltg01-fedora kernel: Code: 68 4b 75 2f c0 68 d5 04 00
00 68 b9 75 31 c0 68 e3 06 31 c0 e8 ce 7e fe ff e8 87 c2 fc ff 83 c4
10 eb 08 85 db 0f 84 6b 07 00 00 <f0> ff 83 9c 00 00 00 8b 55 dc 8b 92
5c 05 00 00 89 55 e4 83 fa
Jul 27 15:17:17 ltg01-fedora kernel: EIP: [<c0138722>]
__lock_acquire+0x362/0xaea SS:ESP 0068:f544ddc0
Jul 27 15:17:17 ltg01-fedora kernel:  <3>BUG: sleeping function called
from invalid context at /usr/src/linux-mm/kernel/rwsem.c:20
Jul 27 15:17:17 ltg01-fedora kernel: in_atomic():1, irqs_disabled():1
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0104192>] show_trace_log_lvl+0x58/0x152
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:17 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0118d37>] __might_sleep+0x8d/0x95
Jul 27 15:17:17 ltg01-fedora kernel:  [<c013595d>] down_read+0x15/0x3b
Jul 27 15:17:17 ltg01-fedora kernel:  [<c012cd93>]
blocking_notifier_call_chain+0x11/0x2d
Jul 27 15:17:17 ltg01-fedora kernel:  [<c012cdc6>] notify_watchers+0x17/0x53
Jul 27 15:17:17 ltg01-fedora kernel:  [<c0122961>] do_exit+0x26/0xa4f
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01047a1>] die+0x291/0x2b7
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0116f5f>] do_page_fault+0x410/0x4f0
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103d1d>] error_code+0x39/0x40
Jul 27 15:17:18 ltg01-fedora kernel: DWARF2 unwinder stuck at
error_code+0x39/0x40
Jul 27 15:17:18 ltg01-fedora kernel: Leftover inexact backtrace:
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0118d37>] __might_sleep+0x8d/0x95
Jul 27 15:17:18 ltg01-fedora kernel:  [<c013595d>] down_read+0x15/0x3b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cd93>]
blocking_notifier_call_chain+0x11/0x2d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cdc6>] notify_watchers+0x17/0x53
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0122961>] do_exit+0x26/0xa4f
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01047a1>] die+0x291/0x2b7
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0116f5f>] do_page_fault+0x410/0x4f0
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103d1d>] error_code+0x39/0x40
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0139214>] lock_acquire+0x71/0x91
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f2bfb>] _spin_lock+0x23/0x32
Jul 27 15:17:18 ltg01-fedora kernel:  [<c014fe3a>]
__delayacct_blkio_ticks+0x16/0x67
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a4f76>] do_task_stat+0x3df/0x6c1
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a5265>] proc_tgid_stat+0xd/0xf
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a29dd>] proc_info_read+0x50/0xb3
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0171cbb>] vfs_read+0xcb/0x177
Jul 27 15:17:18 ltg01-fedora kernel:  [<c017217c>] sys_read+0x3b/0x71
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103119>] sysenter_past_esp+0x56/0x8d
Jul 27 15:17:18 ltg01-fedora kernel: note: udevd[1353] exited with
preempt_count 1
Jul 27 15:17:18 ltg01-fedora kernel: BUG: scheduling while atomic:
udevd/0x00000001/1353
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104192>] show_trace_log_lvl+0x58/0x152
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02ef76f>] __sched_text_start+0x5f/0xc95
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f2977>] __down+0xaf/0xc3
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f275e>] __down_failed+0xa/0xe
Jul 27 15:17:18 ltg01-fedora kernel: DWARF2 unwinder stuck at
__down_failed+0xa/0xe
Jul 27 15:17:18 ltg01-fedora kernel: Leftover inexact backtrace:
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02ef76f>] __sched_text_start+0x5f/0xc95
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f2977>] __down+0xaf/0xc3
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f275e>] __down_failed+0xa/0xe
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f32aa>]
.text.lock.kernel_lock+0x1b/0x3d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c023c881>] disassociate_ctty+0xd/0x16e
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0122d8d>] do_exit+0x452/0xa4f
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01047a1>] die+0x291/0x2b7
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0116f5f>] do_page_fault+0x410/0x4f0
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103d1d>] error_code+0x39/0x40
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0139214>] lock_acquire+0x71/0x91
Jul 27 15:17:18 ltg01-fedora kernel:  [<c02f2bfb>] _spin_lock+0x23/0x32
Jul 27 15:17:18 ltg01-fedora kernel:  [<c014fe3a>]
__delayacct_blkio_ticks+0x16/0x67
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a4f76>] do_task_stat+0x3df/0x6c1
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a5265>] proc_tgid_stat+0xd/0xf
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01a29dd>] proc_info_read+0x50/0xb3
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0171cbb>] vfs_read+0xcb/0x177
Jul 27 15:17:18 ltg01-fedora kernel:  [<c017217c>] sys_read+0x3b/0x71
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103119>] sysenter_past_esp+0x56/0x8d
Jul 27 15:17:18 ltg01-fedora kernel: slab error in
verify_redzone_free(): cache `delayacct_cache': double free detected
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104192>] show_trace_log_lvl+0x58/0x152
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016bdb4>] __slab_error+0x17/0x1c
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016be85>]
cache_free_debugcheck+0xcc/0x1c7
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016c74f>] kmem_cache_free+0xa0/0xff
Jul 27 15:17:18 ltg01-fedora kernel:  [<c014ff3e>]
__delayacct_tsk_exit+0x38/0x3d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0150281>]
delayacct_watch_task+0x5a/0x65
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012ca03>] notifier_call_chain+0x20/0x31
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cd9f>]
blocking_notifier_call_chain+0x1d/0x2d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cdc6>] notify_watchers+0x17/0x53
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0122bf4>] do_exit+0x2b9/0xa4f
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0123415>] sys_exit_group+0x0/0x11
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0104896>] show_trace+0xd/0x10
Jul 27 15:17:18 ltg01-fedora kernel:  [<c01049b5>] dump_stack+0x19/0x1b
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016bdb4>] __slab_error+0x17/0x1c
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016be85>]
cache_free_debugcheck+0xcc/0x1c7
Jul 27 15:17:18 ltg01-fedora kernel:  [<c016c74f>] kmem_cache_free+0xa0/0xff
Jul 27 15:17:18 ltg01-fedora kernel:  [<c014ff3e>]
__delayacct_tsk_exit+0x38/0x3d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0150281>]
delayacct_watch_task+0x5a/0x65
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012ca03>] notifier_call_chain+0x20/0x31
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cd9f>]
blocking_notifier_call_chain+0x1d/0x2d
Jul 27 15:17:18 ltg01-fedora kernel:  [<c012cdc6>] notify_watchers+0x17/0x53
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0122bf4>] do_exit+0x2b9/0xa4f
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0123415>] sys_exit_group+0x0/0x11
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0123424>] sys_exit_group+0xf/0x11
Jul 27 15:17:18 ltg01-fedora kernel:  [<c0103119>] sysenter_past_esp+0x56/0x8d

list *0xc0138722
0xc0138722 is in __lock_acquire (include2/asm/atomic.h:96).
91       *
92       * Atomically increments @v by 1.
93       */
94      static __inline__ void atomic_inc(atomic_t *v)
95      {
96              __asm__ __volatile__(
97                      LOCK_PREFIX "incl %0"
98                      :"+m" (v->counter));
99      }
100

http://www.stardust.webpages.pl/files/mm/2.6.18-rc2-mm1/mm-config 

Regards,
Michal

-- 
Michal K. K. Piotrowski
LTG - Linux Testers Group
(http://www.stardust.webpages.pl/ltg/wiki/)

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Fw: Re: 2.6.18-rc2-mm1
  2006-07-28  8:55 ` Fw: Re: 2.6.18-rc2-mm1 Jan Beulich
@ 2006-07-28 10:35   ` Andi Kleen
  0 siblings, 0 replies; 4+ messages in thread
From: Andi Kleen @ 2006-07-28 10:35 UTC (permalink / raw)
  To: Jan Beulich
  Cc: michal.k.k.piotrowski, linux-kernel, Ingo Molnar, Andrew Morton

On Fri, Jul 28, 2006 at 09:55:51AM +0100, Jan Beulich wrote:
> >>> Andrew Morton <akpm@osdl.org> 28.07.06 07:42 >>>
> >
> >fyi, Michael's dwarf unwinder seems to have broken.
> >(please follow up on lkml).
> 
> Hmm, not being able to unwind through sysenter_entry is no surprise
> (this simply cannot be properly annotated, as the return address is not
> explicit), but it'd end up in user mode anyway (and the inexact backtrace
> doesn't go past it either). The fallback message is a little mis-leading as
> what is shown is not the left-over backtrace, but the full one (Andi
> probably knows better if/when/why this is supposed to be that way).

Hmm, normally it should dump only the left over entries. On my testing
it did that.

> 
> Likewise for the more puzzling case of not being able to unwind through
> error_code - the left-over trace is again more like a full one. I'm not clear
> why it can't unwind through error_code here; a sufficiently large piece
> of the raw stack dump would be needed to check what's going on here,
> and I just again (don't know how many times I already did this) verified
> that in a similar scenario I get a proper unwind through that point.

Yes I've also seen valid stack traces through error_code

> 
> The third one, getting stuck at __down_failed, is due to the still
> unresolved issue of improper (from the perspective of stack unwinding)
> instruction ordering include/asm-i386/semaphore.h.

I'll fix that. Guess we'll just drop the lock sections.

-Andi


^ permalink raw reply	[flat|nested] 4+ messages in thread

[parent not found: <20060728011938.fc6c2d6e.akpm@osdl.org>]

* Fw: Re: 2.6.18-rc2-mm1
       [not found] <20060728011938.fc6c2d6e.akpm@osdl.org>
@ 2006-07-28  9:56 ` Jan Beulich
  2006-07-28 10:38   ` Andi Kleen
  0 siblings, 1 reply; 4+ messages in thread
From: Jan Beulich @ 2006-07-28  9:56 UTC (permalink / raw)
  To: linux-kernel; +Cc: Ingo Molnar, Andi Kleen, Andrew Morton

>>> Andrew Morton <akpm@osdl.org> 28.07.06 10:19 >>>
>
>More unwinder problems.  Seems to be specific to -mm.
>(Followups on-list are preferred, thanks).

That's another of the lose ends - while Andi and I had decided to put
a proper stack frame in place for the switch to the irq stack in the
interrupt macro, we had forgotten about doing something similar for
call_softirq() - as the code is written currently, there simply is now
way of annotating the code properly. If Andi is agreeable to this,
then I'll just change the code so that it'll have a proper stack frame.

The recursive traces appear to result from the fact that when we
did the change to the interrupt macro, we neglected the fact that
show_trace() expects to find the old stack pointer as the very first
item on the interrupt stack, which isn't the case anymore. The value
of where the subsequent stack access fails is a little strange, though.
Making this assumption be true again would, however, require adding
a push to both the interrupt macro and call_softirq. I'm afraid Andi's
not going to be too happy about that, but on the other hand I can't
see any other possible way that would get away without adding
some code to these paths.

Jan

***************************************************
Begin forwarded message:

Date: Fri, 28 Jul 2006 20:09:35 +1200
From: Reuben Farrelly <reuben-lkml@reub.net>
To: Andrew Morton <akpm@osdl.org>
Cc: linux-kernel@vger.kernel.org, Linux Netdev List <netdev@vger.kernel.org>, herbert@gondor.apana.org.au 
Subject: Re: 2.6.18-rc2-mm1




On 27/07/2006 8:56 p.m., Andrew Morton wrote:
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.18-rc2/2.6.18-rc2-mm1/ 
> 
> - git-klibc has been dropped due to very bad parallel-make problems.
> 
> - Added a new line to the boilerplate, below!
> 
> - Added Sam's lxdialog tree, as git-lxdialog.patch.  You no longer need
>   x-ray spectacles to read the menuconfig screens.
> 
> - Lots of random patches.  Many of them are bugfixes and I shall, as usual,
>   go through them all identifying 2.6.18 material.  But I can miss things, so
>   please don't be afraid to point 2.6.18 candidates out to me.
> 
>   I also have, as usual, a number of bugfixes agains the git trees.  I'll
>   send these to the maintainers until they stick and then I lose track of
>   them.  So don't be afraid to send reminders to the subsystem maintainers
>   either.

Still seeing breakage with the e1000/NAT.  Now the problem isn't so much that 
the bug exists, but that unlike in -rc1-mm2, the trace scrolls over and over and 
over (with some variation down into it) until the kernel completely panics and 
the box has to be rebooted.

http://www.ussg.iu.edu/hypermail/linux/kernel/0607.1/2689.html from Herbert Xu 
talks about the original problem.

It's a regression from the previous -mm whereby the trace would print out just once.

Starting Dovecot Imap: [  OK  ]
Starting clamd: [  OK  ]
Starting amavisd: BUG: warning at net/core/dev.c:1168/skb_checksum_help()

Call Trace:
  [<ffffffff80269c65>] show_trace+0xb5/0x324
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
DWARF2 unwinder stuck at call_softirq+0x1e/0x28
Leftover inexact backtrace:
  <IRQ> [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [1] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff806366f0  EFLAGS: 00010206
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000004
RDX: 0000000000000010 RSI: ffff81003d95c840 RDI: ffff81003d95c140
RBP: ffffffff806367f0 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000002 R11: 0000000000000001 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  ffffffff806cec48 ffffffff80636fc0 0000000000000001 ffff81002c9d78f8
  0000000000000000 0000000000000000 00007fffff571020 ffff810029e63f58
  ffffffff80636f80 0000000000000046 0000000000000000 0000000000000000
Call Trace:
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
DWARF2 unwinder stuck at call_softirq+0x1e/0x28
Leftover inexact backtrace:
  <IRQ> [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [2] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff80636378  EFLAGS: 00010006
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff880a3808
RDX: ffff81003d95c140 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffffff80636478 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8029fae0 R11: 0000000000000000 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  000000000000000b ffffffff80636fc0 0000000080632fc0 0000000000000000
  0000000000000000 0000000000000000 00007fffff571020 ffff810029e63f58
  ffffffff80636f80 0000000000000046 0000000000000001 0000000000000002
Call Trace:
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff802601ed>] error_exit+0x0/0x96
DWARF2 unwinder stuck at error_exit+0x0/0x96
Leftover inexact backtrace:
  <IRQ> [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80265999>] _spin_unlock_irqrestore+0x3f/0x47
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8024bc5d>] tso_fragment+0x5d/0x20e
  [<ffffffff80210009>] __kmalloc_track_caller+0x109/0x11b
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff8024bda6>] tso_fragment+0x1a6/0x20e
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff80228ef8>] __kfree_skb+0xfc/0x101
  [<ffffffff80236fe1>] tcp_data_queue+0x651/0xc39
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804552b0>] ip_local_deliver_finish+0x0/0x1fb
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff80439ccc>] kfree_skb+0x2c/0x31
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff8021164f>] __do_softirq+0x64/0x105
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [3] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff80635ff8  EFLAGS: 00010006
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff880a3808
RDX: ffff81003d95c140 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffffff806360f8 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8029fae0 R11: 0000000000000000 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  000000000000000b ffffffff80636fc0 0000000080632fc0 0000000000000000
  ffffffff80907000 0000000000000000 0000000000000000 ffffffff827ffff9
  000000008025fa27 0000000000000001 0000000000000000 ffffffff8029fae0
Call Trace:
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff802601ed>] error_exit+0x0/0x96
DWARF2 unwinder stuck at error_exit+0x0/0x96
Leftover inexact backtrace:
  <IRQ> [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff8029a627>] mark_held_locks+0x5d/0x96
  [<ffffffff8029a7e7>] trace_hardirqs_on+0xec/0x12c
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80265999>] _spin_unlock_irqrestore+0x3f/0x47
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8024bc5d>] tso_fragment+0x5d/0x20e
  [<ffffffff80210009>] __kmalloc_track_caller+0x109/0x11b
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff8024bda6>] tso_fragment+0x1a6/0x20e
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff80228ef8>] __kfree_skb+0xfc/0x101
  [<ffffffff80236fe1>] tcp_data_queue+0x651/0xc39
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804552b0>] ip_local_deliver_finish+0x0/0x1fb
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff80439ccc>] kfree_skb+0x2c/0x31
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff8021164f>] __do_softirq+0x64/0x105
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [4] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff80635c78  EFLAGS: 00010006
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff880a3808
RDX: ffff81003d95c140 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffffff80635d78 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8029fae0 R11: 0000000000000000 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  000000000000000b ffffffff80636fc0 0000000080632fc0 0000000000000000
  ffffffff80907000 0000000000000000 0000000000000000 ffffffff827ffff9
  000000008025fa27 0000000000000001 0000000000000000 ffffffff8029fae0
Call Trace:
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff802601ed>] error_exit+0x0/0x96
DWARF2 unwinder stuck at error_exit+0x0/0x96
Leftover inexact backtrace:
  <IRQ> [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff802a2f8a>] kallsyms_lookup+0xea/0x1b5
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff8029a627>] mark_held_locks+0x5d/0x96
  [<ffffffff8029a7e7>] trace_hardirqs_on+0xec/0x12c
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80265999>] _spin_unlock_irqrestore+0x3f/0x47
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8024bc5d>] tso_fragment+0x5d/0x20e
  [<ffffffff80210009>] __kmalloc_track_caller+0x109/0x11b
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff8024bda6>] tso_fragment+0x1a6/0x20e
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff80228ef8>] __kfree_skb+0xfc/0x101
  [<ffffffff80236fe1>] tcp_data_queue+0x651/0xc39
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804552b0>] ip_local_deliver_finish+0x0/0x1fb
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff80439ccc>] kfree_skb+0x2c/0x31
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff8021164f>] __do_softirq+0x64/0x105
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [5] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff806358f8  EFLAGS: 00010006
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff880a3808
RDX: ffff81003d95c140 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffffff806359f8 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8029fae0 R11: 0000000000000000 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  000000000000000b ffffffff80636fc0 0000000080632fc0 0000000000000000
  ffffffff80907000 0000000000000000 0000000000000000 ffffffff827ffff9
  000000008025fa27 0000000000000001 0000000000000000 ffffffff8029fae0
Call Trace:
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff802601ed>] error_exit+0x0/0x96
DWARF2 unwinder stuck at error_exit+0x0/0x96
Leftover inexact backtrace:
  <IRQ> [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff802a2f8a>] kallsyms_lookup+0xea/0x1b5
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff8029a627>] mark_held_locks+0x5d/0x96
  [<ffffffff8029a7e7>] trace_hardirqs_on+0xec/0x12c
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80265999>] _spin_unlock_irqrestore+0x3f/0x47
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8024bc5d>] tso_fragment+0x5d/0x20e
  [<ffffffff80210009>] __kmalloc_track_caller+0x109/0x11b
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff8024bda6>] tso_fragment+0x1a6/0x20e
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff80228ef8>] __kfree_skb+0xfc/0x101
  [<ffffffff80236fe1>] tcp_data_queue+0x651/0xc39
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804552b0>] ip_local_deliver_finish+0x0/0x1fb
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff80439ccc>] kfree_skb+0x2c/0x31
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff8021164f>] __do_softirq+0x64/0x105
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI><1>Unable to handle kernel paging request at ffffffff82800000 RIP:
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
PGD 203027 PUD 205027 PMD 0
Oops: 0000 [6] SMP
last sysfs file: /class/net/sit0/address
CPU 0
Modules linked in: hidp rfcomm l2cap bluetooth ipv6 ip_gre iptable_filter 
iptable_nat ip_nat ip_conntrack nfnetlink iptable_mangle

ip_tables binfmt_misc iTCO_wdt i2c_i801 serio_raw
Pid: 2127, comm: imap Not tainted 2.6.18-rc2-mm1 #1
RIP: 0010:[<ffffffff80269e6f>]  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
RSP: 0000:ffffffff80635578  EFLAGS: 00010006
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff880a3808
RDX: ffff81003d95c140 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffffff80635678 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8029fae0 R11: 0000000000000000 R12: ffffffff827ffff9
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff80907000
FS:  00002ab5ab544860(0000) GS:ffffffff808b2000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff82800000 CR3: 0000000029dea000 CR4: 00000000000006e0
Process imap (pid: 2127, threadinfo ffff810029e62000, task ffff81003d95c140)
Stack:  000000000000000b ffffffff80636fc0 0000000080632fc0 0000000000000000
  ffffffff80907000 0000000000000000 0000000000000000 ffffffff827ffff9
  000000008025fa27 0000000000000001 0000000000000000 ffffffff8029fae0
Call Trace:
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff802601ed>] error_exit+0x0/0x96
DWARF2 unwinder stuck at error_exit+0x0/0x96
Leftover inexact backtrace:
  <IRQ> [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff80285f9f>] vprintk+0x30f/0x35d
  [<ffffffff802a2f8a>] kallsyms_lookup+0xea/0x1b5
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff8029fae0>] module_text_address+0x16/0x46
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80269e7b>] show_trace+0x2cb/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269fdf>] _show_stack+0xef/0xfe
  [<ffffffff8026a07a>] show_registers+0x8c/0x101
  [<ffffffff8026a18f>] __die+0xa0/0xe3
  [<ffffffff8020af95>] do_page_fault+0x785/0x89d
  [<ffffffff8029a627>] mark_held_locks+0x5d/0x96
  [<ffffffff8029a7e7>] trace_hardirqs_on+0xec/0x12c
  [<ffffffff802601ed>] error_exit+0x0/0x96
  [<ffffffff80269e6f>] show_trace+0x2bf/0x324
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff80269ee9>] dump_stack+0x15/0x1c
  [<ffffffff8043d4e8>] skb_checksum_help+0x63/0x13b
  [<ffffffff8802f35f>] :iptable_nat:ip_nat_fn+0x5f/0x1d2
  [<ffffffff8802f71b>] :iptable_nat:ip_nat_local_fn+0x33/0xbc
  [<ffffffff80234c9e>] nf_iterate+0x5a/0x9b
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804579a6>] dst_output+0x0/0x10
  [<ffffffff80265999>] _spin_unlock_irqrestore+0x3f/0x47
  [<ffffffff80235174>] ip_queue_xmit+0x444/0x4c0
  [<ffffffff8024bc5d>] tso_fragment+0x5d/0x20e
  [<ffffffff80210009>] __kmalloc_track_caller+0x109/0x11b
  [<ffffffff8022157f>] tcp_transmit_skb+0x68f/0x6cf
  [<ffffffff8024bda6>] tso_fragment+0x1a6/0x20e
  [<ffffffff80233b67>] __tcp_push_pending_frames+0x867/0x95a
  [<ffffffff80228ef8>] __kfree_skb+0xfc/0x101
  [<ffffffff80236fe1>] tcp_data_queue+0x651/0xc39
  [<ffffffff8021b9fc>] tcp_rcv_established+0x72c/0x7c4
  [<ffffffff8023c60e>] tcp_v4_do_rcv+0x2e/0x317
  [<ffffffff8022719c>] tcp_v4_rcv+0x9fc/0xa79
  [<ffffffff80258960>] nf_hook_slow+0x60/0xcd
  [<ffffffff804552b0>] ip_local_deliver_finish+0x0/0x1fb
  [<ffffffff80235389>] ip_local_deliver+0x199/0x270
  [<ffffffff802363c3>] ip_rcv+0x4d3/0x52b
  [<ffffffff80439ccc>] kfree_skb+0x2c/0x31
  [<ffffffff8021fdcb>] netif_receive_skb+0x1eb/0x27b
  [<ffffffff803c9491>] e1000_clean_rx_irq_ps+0x5c1/0x6a0
  [<ffffffff803c7b45>] e1000_clean+0x325/0x45b
  [<ffffffff8020c9b7>] net_rx_action+0x8e/0x147
  [<ffffffff8021164f>] __do_softirq+0x64/0x105
  [<ffffffff80211663>] __do_softirq+0x78/0x105
  [<ffffffff80260716>] call_softirq+0x1e/0x28
  [<ffffffff8026b679>] do_softirq+0x39/0xa4
  [<ffffffff802888b8>] irq_exit+0x58/0x5a
  [<ffffffff8026b798>] do_IRQ+0xb4/0xbe
  [<ffffffff8025f9a1>] ret_from_intr+0x0/0xf
  <EOI>

Thanks,
Reuben

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Fw: Re: 2.6.18-rc2-mm1
  2006-07-28  9:56 ` Jan Beulich
@ 2006-07-28 10:38   ` Andi Kleen
  0 siblings, 0 replies; 4+ messages in thread
From: Andi Kleen @ 2006-07-28 10:38 UTC (permalink / raw)
  To: Jan Beulich; +Cc: linux-kernel, Ingo Molnar, Andrew Morton

On Fri, Jul 28, 2006 at 10:56:16AM +0100, Jan Beulich wrote:
> >>> Andrew Morton <akpm@osdl.org> 28.07.06 10:19 >>>
> >
> >More unwinder problems.  Seems to be specific to -mm.
> >(Followups on-list are preferred, thanks).
> 
> That's another of the lose ends - while Andi and I had decided to put
> a proper stack frame in place for the switch to the irq stack in the
> interrupt macro, we had forgotten about doing something similar for
> call_softirq() - as the code is written currently, there simply is now
> way of annotating the code properly. If Andi is agreeable to this,
> then I'll just change the code so that it'll have a proper stack frame.

Fine by me. I'll just fix it up.

> 
> The recursive traces appear to result from the fact that when we
> did the change to the interrupt macro, we neglected the fact that
> show_trace() expects to find the old stack pointer as the very first
> item on the interrupt stack, which isn't the case anymore. The value
> of where the subsequent stack access fails is a little strange, though.
> Making this assumption be true again would, however, require adding
> a push to both the interrupt macro and call_softirq. I'm afraid Andi's
> not going to be too happy about that, but on the other hand I can't
> see any other possible way that would get away without adding
> some code to these paths.

I can just add the push.

-Andi

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2006-07-28 10:38 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20060727224233.7fe3724a.akpm@osdl.org>
2006-07-28  8:55 ` Fw: Re: 2.6.18-rc2-mm1 Jan Beulich
2006-07-28 10:35   ` Andi Kleen
     [not found] <20060728011938.fc6c2d6e.akpm@osdl.org>
2006-07-28  9:56 ` Jan Beulich
2006-07-28 10:38   ` Andi Kleen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox