a Linux swap storm

a Linux swap storm

[Andries.Brouwer]

Earlier this evening I showed someone some pictures under X:

% display -size 300x300 *.jpg

(395 pictures, 315 MB). When display (from ImageMagick)
started to repeat, I exited the program.
At this moment the machine became unusable for twenty minutes
of solid disk activity.
No keystroke seen, not even the Ctrl-Alt-Backspace to kill X,
or Ctrl-Alt-F1 to switch consoles, no mouse movement seen,
vmstat did not produce any output for twenty minutes.

The vmstat 5 output was

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa

 0  3 315728   3164   4424   3356  646  382   792   382 2425   182  0  5  0 95
 1  1 316356   3856   4428   3572  454  366   507   366 2242   150  0  5  0 95
 0  1 317540   3784   4456   3664  530  578   590   578 2403   179  0  7  0 93
 0  1 306740   4240   4524   5372 127013 49878 129427 50061 405016 32901  0  4  0 95
 1  1 306712   3992   4536   5372   30    0    30     3  450   122  2  1 94  2
 0  0 306692   4016   4548   5372   18    0    18     3  402   134  2  2 96  1
 0  0 306692   4016   4560   5372    0    0     0     3  257    35  1  1 98  0

The machine is vanilla 2.6.14, 256MB, 550MB swap.

% rpm -qf `which X`
xorg-x11-server-6.8.2-100

I wonder what precisely happened. Is this an X bug? Or a kernel bug?
The effect is reproducible.

Andries

Re: a Linux swap storm

[Jan Engelhardt]

>% display -size 300x300 *.jpg

>I wonder what precisely happened. Is this an X bug? Or a kernel bug?
>The effect is reproducible.

Right before it starts to swap, hit Ctrl+Z to suspend 'display' and do a 
`ps u`. Does it hog memory?


Jan Engelhardt
-- 

Re: a Linux swap storm

[Jesper Juhl]

On 5/7/06, Andries.Brouwer@cwi.nl <Andries.Brouwer@cwi.nl> wrote:
[snip]
> At this moment the machine became unusable for twenty minutes
[snip]
> The machine is vanilla 2.6.14, 256MB, 550MB swap.
[snip]
> The effect is reproducible.
>
[snip]

Reproducible - nice.
Have you tried it with a newer kernel like 2.6.16.14 or
2.6.17-rc3-git13 ?  Could be interresting to see if it has already
been fixed since 2.6.14... in any case, it's one more datapoint :-)


--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: a Linux swap storm

[Mike Galbraith]

On Sun, 2006-05-07 at 21:31 +0200, Andries.Brouwer@cwi.nl wrote: 
> Earlier this evening I showed someone some pictures under X:
> 
> % display -size 300x300 *.jpg
> 
> (395 pictures, 315 MB). When display (from ImageMagick)
> started to repeat, I exited the program.
> At this moment the machine became unusable for twenty minutes
> of solid disk activity.
> No keystroke seen, not even the Ctrl-Alt-Backspace to kill X,
> or Ctrl-Alt-F1 to switch consoles, no mouse movement seen,
> vmstat did not produce any output for twenty minutes.
> 
> The vmstat 5 output was
> 
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
> 
>  0  3 315728   3164   4424   3356  646  382   792   382 2425   182  0  5  0 95
>  1  1 316356   3856   4428   3572  454  366   507   366 2242   150  0  5  0 95
>  0  1 317540   3784   4456   3664  530  578   590   578 2403   179  0  7  0 93
>  0  1 306740   4240   4524   5372 127013 49878 129427 50061 405016 32901  0  4  0 95
>  1  1 306712   3992   4536   5372   30    0    30     3  450   122  2  1 94  2
>  0  0 306692   4016   4548   5372   18    0    18     3  402   134  2  2 96  1
>  0  0 306692   4016   4560   5372    0    0     0     3  257    35  1  1 98  0

This is after ImageMagic exited?  If so, and you don't have a userland
hog sitting on that memory, I'd suggest posting /proc/meminfo and any
part of /proc/slabinfo showing large numbers of allocations.  (if you
can repeat with latest vanilla kernel that is)

> The machine is vanilla 2.6.14, 256MB, 550MB swap.
> 
> % rpm -qf `which X`
> xorg-x11-server-6.8.2-100
> 
> I wonder what precisely happened. Is this an X bug? Or a kernel bug?
> The effect is reproducible.

I'd lean toward kernel.  If it was thrashing so hard that the box became
a doorstop for 20 minutes, seems to me that's a fine description of oom,
so somebody should have been killed.  Does SysRq-M work during the
seizure?

-Mike