libata: clustering on or off?

libata: clustering on or off?

[Jeff Garzik]

The constant ATA_SHT_USE_CLUSTERING in include/linux/libata.h controls
the use of SCSI layer's use_clustering feature, for a great many libata
drivers.

The current setup has clustering disabled, which in theory causes the
block layer to do less work, at the expense of a greater number of
scatter/gather table entries used.

Any opinions WRT turning on clustering for libata?

	Jeff

Re: libata: clustering on or off?

[Arjan van de Ven]

On Sun, 2005-08-28 at 05:42 -0400, Jeff Garzik wrote:
> The constant ATA_SHT_USE_CLUSTERING in include/linux/libata.h controls
> the use of SCSI layer's use_clustering feature, for a great many libata
> drivers.
> 
> The current setup has clustering disabled, which in theory causes the
> block layer to do less work, at the expense of a greater number of
> scatter/gather table entries used.
> 
> Any opinions WRT turning on clustering for libata?

in 2.4 clustering was expensive due to a large number of checks that
were done (basically the number of fragments got recounted a gazilion
times). In 2.6 Jens fixed that afaik to make it basically free...
at which point it's a win always.

Imo clustering on the driver level should announce driver capabilities.
If clustering for some arch/kernel makes it slower, that should be
decided at a midlayer level and not in each driver; eg the midlayer
would chose to ignore the drivers capabilities.
So .. my opinion would be that libata should announce the capability (it
seems the code/hw can do it). 

Re: libata: clustering on or off?

[Jens Axboe]

On Sun, Aug 28 2005, Arjan van de Ven wrote:
> On Sun, 2005-08-28 at 05:42 -0400, Jeff Garzik wrote:
> > The constant ATA_SHT_USE_CLUSTERING in include/linux/libata.h controls
> > the use of SCSI layer's use_clustering feature, for a great many libata
> > drivers.
> > 
> > The current setup has clustering disabled, which in theory causes the
> > block layer to do less work, at the expense of a greater number of
> > scatter/gather table entries used.
> > 
> > Any opinions WRT turning on clustering for libata?
> 
> in 2.4 clustering was expensive due to a large number of checks that
> were done (basically the number of fragments got recounted a gazilion
> times). In 2.6 Jens fixed that afaik to make it basically free...
> at which point it's a win always.

Yeah, it wont cost any extra cycles, so there's no point in keeping it
turned off for that reason.

> Imo clustering on the driver level should announce driver capabilities.
> If clustering for some arch/kernel makes it slower, that should be
> decided at a midlayer level and not in each driver; eg the midlayer
> would chose to ignore the drivers capabilities.
> So .. my opinion would be that libata should announce the capability (it
> seems the code/hw can do it). 

Agree, we should just remove the ability to control clustering, as it
really overlaps with the segment settings anyways.

-- 
Jens Axboe

Re: libata: clustering on or off?

[Christoph Hellwig]

On Sun, Aug 28, 2005 at 04:20:19PM +0200, Jens Axboe wrote:
> Agree, we should just remove the ability to control clustering, as it
> really overlaps with the segment settings anyways.

What are we going to do with iscsi then?  It really doesn't like segments
over a pages size.  Best thing would probably be to switch networking to
use sg lists and dma_map_sg, but that's not a trivial task.

Re: libata: clustering on or off?

[Jens Axboe]

On Sun, Aug 28 2005, Christoph Hellwig wrote:
> On Sun, Aug 28, 2005 at 04:20:19PM +0200, Jens Axboe wrote:
> > Agree, we should just remove the ability to control clustering, as it
> > really overlaps with the segment settings anyways.
> 
> What are we going to do with iscsi then?  It really doesn't like segments
> over a pages size.  Best thing would probably be to switch networking to
> use sg lists and dma_map_sg, but that's not a trivial task.

Limit the segment size, then. There are provisions for doing both length
and boundary limits, that should suffice.

-- 
Jens Axboe

Re: libata: clustering on or off?

[Jeff Garzik]

Jens Axboe wrote:
> On Sun, Aug 28 2005, Arjan van de Ven wrote:
> 
>>On Sun, 2005-08-28 at 05:42 -0400, Jeff Garzik wrote:
>>
>>>The constant ATA_SHT_USE_CLUSTERING in include/linux/libata.h controls
>>>the use of SCSI layer's use_clustering feature, for a great many libata
>>>drivers.
>>>
>>>The current setup has clustering disabled, which in theory causes the
>>>block layer to do less work, at the expense of a greater number of
>>>scatter/gather table entries used.
>>>
>>>Any opinions WRT turning on clustering for libata?
>>
>>in 2.4 clustering was expensive due to a large number of checks that
>>were done (basically the number of fragments got recounted a gazilion
>>times). In 2.6 Jens fixed that afaik to make it basically free...
>>at which point it's a win always.

> Yeah, it wont cost any extra cycles,

A simple grep for QUEUE_FLAG_CLUSTER-related code shows that it -does- 
cost extra cycles.


>>Imo clustering on the driver level should announce driver capabilities.
>>If clustering for some arch/kernel makes it slower, that should be
>>decided at a midlayer level and not in each driver; eg the midlayer
>>would chose to ignore the drivers capabilities.
>>So .. my opinion would be that libata should announce the capability (it
>>seems the code/hw can do it). 
> 
> 
> Agree, we should just remove the ability to control clustering, as it
> really overlaps with the segment settings anyways.

OK, I guess the consensus is to use clustering :)

We'll see if anything blows up in 2.6.14...

	Jeff

Re: libata: clustering on or off?

[Jens Axboe]

On Sun, Aug 28 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >On Sun, Aug 28 2005, Arjan van de Ven wrote:
> >
> >>On Sun, 2005-08-28 at 05:42 -0400, Jeff Garzik wrote:
> >>
> >>>The constant ATA_SHT_USE_CLUSTERING in include/linux/libata.h controls
> >>>the use of SCSI layer's use_clustering feature, for a great many libata
> >>>drivers.
> >>>
> >>>The current setup has clustering disabled, which in theory causes the
> >>>block layer to do less work, at the expense of a greater number of
> >>>scatter/gather table entries used.
> >>>
> >>>Any opinions WRT turning on clustering for libata?
> >>
> >>in 2.4 clustering was expensive due to a large number of checks that
> >>were done (basically the number of fragments got recounted a gazilion
> >>times). In 2.6 Jens fixed that afaik to make it basically free...
> >>at which point it's a win always.
> 
> >Yeah, it wont cost any extra cycles,
> 
> A simple grep for QUEUE_FLAG_CLUSTER-related code shows that it -does- 
> cost extra cycles.

Well yes, none is not true of course. But it's not a lot, like extra
iterations of the request mappings like it used to. So in by far the
most cases, it should be a win overall.

> >>Imo clustering on the driver level should announce driver capabilities.
> >>If clustering for some arch/kernel makes it slower, that should be
> >>decided at a midlayer level and not in each driver; eg the midlayer
> >>would chose to ignore the drivers capabilities.
> >>So .. my opinion would be that libata should announce the capability (it
> >>seems the code/hw can do it). 
> >
> >
> >Agree, we should just remove the ability to control clustering, as it
> >really overlaps with the segment settings anyways.
> 
> OK, I guess the consensus is to use clustering :)
> 
> We'll see if anything blows up in 2.6.14...

;-)

-- 
Jens Axboe

Re: libata: clustering on or off?

[Mark Lord]

Clustering has always been a (small) win for the ATA/SATA hardware
I have worked on.  Many controllers read the scatter-gather list
"on demand", so if clustering is not used, there are a lot of
times where the controller must stop streaming data, and fetch
the next s/g entry, then resume streaming data.

Clustering reduces the number of such events, giving better bus
utilization and (slightly) faster transfers.

The more modern hardware I'm working with now will generally
read the s/g table in larger blocks, so that it only rarely
needs to interrupt data transfers to fetch s/g info.
Clustering is not as big a win with this, but still reduces
total bus cycles required.

Cheers