Re: 2.6.1 IO lockup on SMP systems

["Sergey S. Kostyliov" <rathamahata@php4.ru>]

On Tuesday 24 February 2004 00:30, Mike Fedyk wrote:
> Sergey S. Kostyliov wrote:
> > Hello Andrew,
> > 
> > Now this happens for the third time.
> > 
> > 
> >>>>I've just reproduced this lockup with 2.6.3.
> >>>>
> >>>>
> >>>>>You may need a serial console to be able to capture all the output.
> >>>>>
> >>>>>Also, it would be useful to know what sort of load the machines are
> >>>>>under, and what filesystems are in use.
> >>>>
> >>>>The machine is a http server. The main applications are:
> >>>>1) apache 1.3 which serves php pages (mod_php):
> >>>>	 15.3 requests/sec - 111.9 kB/second - 7.3 kB/request
> >>>>	 54 requests currently being processed, 19 idle servers
> >>>>2) mysql:
> >>>>	Threads: 19  Questions: 26922012  Slow queries: 9799  Opens: 64980
> >>>>	Flush tables: 1  Open tables: 630  Queries per second avg: 143.547
> >>>>
> >>>>This is an IO bound machine in general. All filesystems are reiserfs.
> >>>>
> >>>>Here is a sysrq-T output obtained from a locked box via serail console:
> >>>
> >>>OK, so everything is stuck trying to allocate memory.  Perhaps you ran out
> >>>of swapspace, or some process has gone berzerk allocating memory.
> > 
> > 
> > The memory exhaustion is indeed possible for this box. I'll double check
> > ulimit and /etc/security/limits.conf stuff. The only thing which worries
> > me that this box had been running for months without any problems with
> > 2.4.23aa1.
> > 
> > I have added another 2Gb to swap space (hope this give enough time
> > to find the memory hungry process(es)).
> 
> Also check how much memory is being used for slab in /proc/meminfo

Thanks for the hint, will do this next time. 

-- 
                   Best regards,
                   Sergey S. Kostyliov <rathamahata@php4.ru>
                   Public PGP key: http://sysadminday.org.ru/rathamahata.asc