RE: scheduling oddity on 2.6.20.3 stock

["David Schwartz" <davids@webmaster.com>]

> I needed to recompress some files from .bz2 to .gz so I setup a script to
> do
>
> bunzip2 -c $file.bz2 |gzip -9 >$file.gz
>
> I expected that the two CPU heavy processes would end up on different
> cpu's and spend a little time shuffling data between the two cpu's on a
> system (dual core opteron)
>
> however, instead what I find is that each process is getting 50% of one
> cpu while the other cpu is 97% idle.

That would only be possible if the compression/decompression block size is
small compared to the maximum pipe buffer size. I suspect the reverse is the
case.

It would be interesting to write an intermediate process that basically
enlarged the pipe buffers and see if that changed anything. Basically, the
intermediate process would allocate a large buffer (16MB or so) and fill it
from 'bunzip2' while draining it to 'gzip' in a non-blocking way (unless the
buffer was full/empty, of course).

DS