Re: naive questions about thrashing

[Timothy Miller <miller@techsource.com>]

J.A. Magallon wrote:
> On 04.30, Timothy Miller wrote:
> 
>>I am running kernel version 2.4.18-26.7.x under Red Hat 7.2.
>>
>>I wrote a CPU-intensive program which attempts to use over 700 megs of 
>>RAM on a 512-meg box, therefore it thrashes.
>>
>>One thing I noticed was that 'top' reported that the kernel ("system") 
>>was using 68% of the CPU.  (The offending process was getting about 9%.) 
>>  How much CPU involvement is there in sending I/O requests to the drive 
>>and waiting on an interrupt?  Maybe I don't understand what's going on, 
>>but I would expect the CPU involvement in disk I/O to be practically 
>>NIL, unless it's trying to be really smart about it.  Is it?  Or maybe 
>>the kernel isn't using DMA... this is a Dell Precision 340.  I'm not 
>>sure what drive is in it, but I would be surprised if it weren't using DMA.
>>
> 
> 
> As I understand it, it is telling you that your programs spends 68% of
> its time is kernel space, ie, waiting your pages to come from disk. It
> does not mean that the CPU is doing anything, but it is locked by the
> kernel.

What would the kernel be locked while waiting on disk I/O?  Shouldn't it 
be running another process?  It's not DOING anything.  The whole idea 
behind a multitasking OS is to overlap the I/O of one process with the 
CPU usage of another whenever possible.  Swapping is an I/O operation.

And for that matter, if every runnable process has pages swapped out so 
that they cannot run, then the CPU should be IDLE.

Am I wrong?

> 
> If you can't afford to buy more memory, recode the thing. So much thrashing
> looks like you access your data very randomly. Try to process the data
> in a more sequential way, so you just fault after processing a big bunch
> of data. With 700Mb of data and a 512Mb box, at least half of your data
> fit in memory, so under an ideal sequential access you just would page
> 300Mb one time...
> 

The process got that large because of a bug in my program.  But a 
side-effect of that was kernel behavior that didn't make sense to me.  I 
decided to ask about it.