Re: [Lsf] IO less throttling and cgroup aware writeback (Was: Re: Preliminary Agenda and Activities for LSF)

Vivek Goyal
2011-04-26T14:37:13Z

On Mon, Apr 25, 2011 at 02:19:54PM -0400, Vivek Goyal wrote:
> On Fri, Apr 22, 2011 at 06:28:29PM +0200, Andrea Arcangeli wrote:
>
> [..]
> > > Also it is only CFQ which provides READS so much preferrence over WRITES.
> > > deadline and noop do not which we typically use on faster storage. There
> > > we might take a bigger hit on READ latencies depending on what storage
> > > is and how effected it is with a burst of WRITES.
> > >
> > > I guess it boils down to better system control and better predictability.
> >
> > I tend to think to get even better read latency and predictability,
> > the IO scheduler could dynamically and temporarily reduce the max
> > sector size of the write dma (and also ensure any read readahead is
> > also reduced to the dynamic reduced sector size or it'd be detrimental
> > on the number of read DMA issued for each userland read).
> >
> > Maybe with tagged queuing things are better and the dma size doesn't
> > make a difference anymore, I don't know. Surely Jens knows this best
> > and can tell me if I'm wrong.
> >
> > Anyway it should be real easy to test, just a two liner reducing the
> > max sector size to scsi_lib and the max readahead, should allow you to
> > see how fast firefox starts with cfq when dd if=/dev/zero is running
> > and if there's any difference at all.
>
> I did some quick runs.
>
> - Default queue depth is 31 on my SATA disk. Reducing queue depth to 1
> helps a bit.
>
> In CFQ we already try to reduce the queue depth of WRITES if READS
> are going on.
>
> - I reduced /sys/block/sda/queue/max_sector_kb to 16. That seemed to
> help with firefox launch time.
>
> There are couple of interesting observations though.
>
> - Even after I reduced max_sector_kb to 16, I saw requests of 1024 sector
> size coming from flusher threads.
>

I realized that I had a dm device sitting on top of sda and I was
changing max_sector_kb only on sda and not on dm device hence request
size was still 1024 sector each.

I changed max_sector_kb to 16 and that seems to help. Launching and
loading first website time comes down from 1minute to roughly 30 seconds.

At dm layer no IO scheduler is running so IO scheduler really can't
do much in controlling the request size dynamically depending on what's
happening on the device.

I am not sure if one can break the requests in smaller pieces in IO
scheduler if reads are going on.

Thanks
Vivek