Re: Large Sequential Reads Being Broken Up. Why?

[James Bottomley <James.Bottomley@SteelEye.com>]

On Mon, 2006-01-30 at 11:17 -0500, James Smart wrote:
> As of 2.6.10, the kernel started paying attention to this field, which the
> emulex driver, as of that time, didn't set. The result was the kernel
> dropped back to a default max_sectors of 1024 - which results in a 512k max.
> The lpfc driver was updated in rev 8.0.29 with this change.
> 
> Caveat is : Even with this change, you must be using O_DIRECT to get high
> bandwidth. Otherwise, the upper layers will segment the requests (if I
> remember right, we had a hard time making a "normal" config exceed 256k).

Actually, please also remember that the maximum SG element list size is
128 in a normal kernel (depending on the driver ... some drivers set
lower limits as well), so on a very fragmented 4k page machine, you're
unlikely to get above 512k just because you run out of SG table entries
(obviously on 16k page machines, this goes up to 2MB, and if you're
lucky enough to have a fully functional IOMMU, this limitation won't
affect you at all).

James