2.4.16 deadlock in kswapd

2.4.16 deadlock in kswapd

[FORT David]

Hi,
today i was transfering some files between two drives(reiserfs->ext3) and
suddenly everything locked up. I sys-rqed to show the executed IP and
every five times i've tryed it was showing the following stack trace:

EIP: 0010:[<c0111657>] CPU: 0 EFLAGS: 00200202    Tainted: P
Using defaults from ksymoops -t elf32-i386 -a i386
EAX: 00000002 EBX: 02000000 ECX: 00000000 EDX: 00200202
ESI: efe38120 EDI: e112fe90 EBP: efe38120 DS: 0018 ES: 0018
CR0: 8005003b CR2: 4002a009 CR3: 1f165000 CR4: 000002d0
Call Trace: [<c01117b5>] [<c012f052>] [<c01920a5>] [<c0191840>] 
[<c0191f10>]
   [<c012f4f6>] [<c012f6d2>] [<c012f72c>] [<c012f7d1>] [<c012f846>] 
[<c012f981>]
   [<c012f8e0>] [<c0105000>] [<c0105656>] [<c012f8e0>]
Warning (Oops_read): Code line not seen, dumping what data is available

 >>EIP; c0111657 <flush_tlb_others+e7/110>   <=====
Trace; c01117b5 <flush_tlb_page+75/80>
Trace; c012f052 <swap_out+312/4b0>
Trace; c01920a5 <ide_dmaproc+135/210>
Trace; c0191840 <ide_dma_intr+0/c0>
Trace; c0191f10 <dma_timer_expiry+0/60>
Trace; c012f4f6 <shrink_cache+306/390>
Trace; c012f6d2 <shrink_caches+52/80>
Trace; c012f72c <try_to_free_pages+2c/50>
Trace; c012f7d1 <kswapd_balance_pgdat+51/a0>
Trace; c012f846 <kswapd_balance+26/40>
Trace; c012f981 <kswapd+a1/c0>
Trace; c012f8e0 <kswapd+0/c0>
Trace; c0105000 <_stext+0/0>
Trace; c0105656 <kernel_thread+26/30>
Trace; c012f8e0 <kswapd+0/c0>

The interesting thing is that i don't have any swap, so i'm really 
interested
in knowing why kswapd is envolved here.
Feel free to ask additionnal informations.

PS: the kernel is tainted by lm_sensors

-- 
%--LINUX-HTTPD-PIOGENE----------------------------------------------------%
%  -datamining <-/                        |   .~.                         %
%  -networking/PHP/java/JSPs              |   /V\        L  I  N  U  X    %
%  -opensource                            |  // \\     >Fear the Penguin< %
%  -GNOME/enlightenment/GIMP              | /(   )\                       %
%           feel enlightened....          |  ^^-^^                        %
%                              HomePage: http://www.enlightened-popo.net  %
%---------- -- This was sent by Djinn running Linux 2.4.16 -- ------------%

Re: 2.4.16 deadlock in kswapd

[Andrew Morton]

FORT David wrote:
> 
> Hi,
> today i was transfering some files between two drives(reiserfs->ext3) and
> suddenly everything locked up. I sys-rqed to show the executed IP and
> every five times i've tryed it was showing the following stack trace:
> 
> ...
> 
>  >>EIP; c0111657 <flush_tlb_others+e7/110>   <=====
> Trace; c01117b5 <flush_tlb_page+75/80>
> Trace; c012f052 <swap_out+312/4b0>
> ...

Dodgy hardware, I'm afraid - it looks like a cross-CPU interrupt
was sent but not received.  Not uncommon.

> The interesting thing is that i don't have any swap, so i'm really
> interested
> in knowing why kswapd is envolved here.

Look at the swapout code: it calls flush_tlb_page() in preparation
for swapping a page out.  It then tries to allocate swap space,
finds there is none and bales out.  This can comsume quite a lot
of CPU under some circumstances.

-

Re: 2.4.16 deadlock in kswapd

[Alan Cox]

> > Trace; c01117b5 <flush_tlb_page+75/80>
> > Trace; c012f052 <swap_out+312/4b0>
> > ...
> 
> Dodgy hardware, I'm afraid - it looks like a cross-CPU interrupt
> was sent but not received.  Not uncommon.

Andrea claimed there were races in the x86 ipi code. I dont know if his
change was applied however. The x86 messaging is reliable (you'll see
showers of apic errors before it fails) but does sometimes replay a message
which is much fun