chunk size must be power of two?

chunk size must be power of two?

[James Braid]

Is there any reason the chunk size has to be a power of two?

We're trying to use md devices to stripe over hardware RAID5 arrays, so 
to align the I/O correctly, I want to set the chunk size to (n - 1) * 
stripe_size for the underlying array. Which is impossible to do because 
mdadm (and the kernel) insist on the chunk size being a power of two.

I haven't tried just removing the kernel checks yet ... is there a 
reason why md is limited to power of two chunk sizes?

Failing that, does anyone know of anything else I can use to stripe 
block devices together which doesn't have the power of two limitation? 
device-mapper/LVM seems to have the same problem...

Re: chunk size must be power of two?

[Neil Brown]

On Wednesday December 6, jamesb@loreland.org wrote:
> Is there any reason the chunk size has to be a power of two?

It makes division a lot easier and faster.

> 
> We're trying to use md devices to stripe over hardware RAID5 arrays, so 
> to align the I/O correctly, I want to set the chunk size to (n - 1) * 
> stripe_size for the underlying array. Which is impossible to do because 
> mdadm (and the kernel) insist on the chunk size being a power of
> two.

Surely you want to set chunk_size to 1 * stripe_size for the
underlying array ... or maybe 1/(n-1) * stripe_size.  But certainly
not (n-1) * stripe_size.  That wouldn't make any sense at all.

> 
> I haven't tried just removing the kernel checks yet ... is there a 
> reason why md is limited to power of two chunk sizes?

You would need to check all the div and mod operations based on chunk
size and make sure they don't implicitly assume it is a power of 2.

What exactly is the geometry of the array you wan to access with md??

NeilBrown



> 
> Failing that, does anyone know of anything else I can use to stripe 
> block devices together which doesn't have the power of two limitation? 
> device-mapper/LVM seems to have the same problem...
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: chunk size must be power of two?

[James Braid]

Neil Brown wrote:
 > On Wednesday December 6, jamesb@loreland.org wrote:
 >> Is there any reason the chunk size has to be a power of two?
 >
 > It makes division a lot easier and faster.

Thanks for the reply.

 >> We're trying to use md devices to stripe over hardware RAID5 arrays, so
 >> to align the I/O correctly, I want to set the chunk size to (n - 1) *
 >> stripe_size for the underlying array. Which is impossible to do because
 >> mdadm (and the kernel) insist on the chunk size being a power of
 >> two.
 >
 > Surely you want to set chunk_size to 1 * stripe_size for the
 > underlying array ... or maybe 1/(n-1) * stripe_size.  But certainly
 > not (n-1) * stripe_size.  That wouldn't make any sense at all.

I suspect I haven't explained myself very clearly... I'm trying to 
stripe over multiple separate hardware raid5 luns using md... see below 
for details.

 >> I haven't tried just removing the kernel checks yet ... is there a
 >> reason why md is limited to power of two chunk sizes?
 >
 > You would need to check all the div and mod operations based on chunk
 > size and make sure they don't implicitly assume it is a power of 2.

OK, thanks.

 > What exactly is the geometry of the array you wan to access with md??

four 7+1 raid5 luns (from multiple controllers etc) with a stripe size 
of 128K, and I am then striping across all four luns with md.

So I want to set the md chunk_size to 896K so we write a full stripe (7 
* 128K) to each lun before moving on to the next.

Testing on other volume managers (IRIX/Linux with xvm) has shown 
aligning the I/O like this improves performance for our workload (lots 
of large files with streaming reads and writes), but I can't use xvm for 
this particular system.

Thanks, James