write barriers on raid0 and raid10

write barriers on raid0 and raid10

[Iordan Iordanov]

Hi,

We are designing a rather involved file server with an ext3 formatted 
stripe (raid0) sitting on top of raid10 devices. Each raid10 device sits 
on top of 3 iscsi targets, and has layout n3 (so it is effectively a 3 
way mirror). We chose raid10 over raid1 due to an apparent read 
performance benefit of raid10.

We are trying to decide whether to upgrade to a kernel newer than 
2.6.34, where write barriers are ostensibly supported by all possible 
raid types, because we are worried about ext3 corruption with no write 
barrier support.

However, we are also worried about whether write barriers really "make 
sense" in a multi-disk environment and are wondering whether they will 
actually make a difference in our setup. For argument's sake, let's 
assume that our drives honor write cache flushes.

Can somebody shed some light on how write barriers are implemented in 
raid0 and raid10? Also, any critical comments on the validity of our 
setup and/or assumptions is also welcome.

Many thanks!
Iordan Iordanov

Re: write barriers on raid0 and raid10

[Bill Davidsen]

Iordan Iordanov wrote:
> Hi,
>
> We are designing a rather involved file server with an ext3 formatted 
> stripe (raid0) sitting on top of raid10 devices. Each raid10 device 
> sits on top of 3 iscsi targets, and has layout n3 (so it is 
> effectively a 3 way mirror). We chose raid10 over raid1 due to an 
> apparent read performance benefit of raid10.
>
> We are trying to decide whether to upgrade to a kernel newer than 
> 2.6.34, where write barriers are ostensibly supported by all possible 
> raid types, because we are worried about ext3 corruption with no write 
> barrier support.
>
> However, we are also worried about whether write barriers really "make 
> sense" in a multi-disk environment and are wondering whether they will 
> actually make a difference in our setup. For argument's sake, let's 
> assume that our drives honor write cache flushes.
>
> Can somebody shed some light on how write barriers are implemented in 
> raid0 and raid10? Also, any critical comments on the validity of our 
> setup and/or assumptions is also welcome.
>
I'm sorry not to see an answer here, I'm interested as well.

-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

Re: write barriers on raid0 and raid10

[Neil Brown]

On Tue, 08 Jun 2010 14:23:04 -0400
Iordan Iordanov <iordan@cdf.toronto.edu> wrote:

> Hi,
> 
> We are designing a rather involved file server with an ext3 formatted 
> stripe (raid0) sitting on top of raid10 devices. Each raid10 device sits 
> on top of 3 iscsi targets, and has layout n3 (so it is effectively a 3 
> way mirror). We chose raid10 over raid1 due to an apparent read 
> performance benefit of raid10.
> 
> We are trying to decide whether to upgrade to a kernel newer than 
> 2.6.34, where write barriers are ostensibly supported by all possible 
> raid types, because we are worried about ext3 corruption with no write 
> barrier support.
> 
> However, we are also worried about whether write barriers really "make 
> sense" in a multi-disk environment and are wondering whether they will 
> actually make a difference in our setup. For argument's sake, let's 
> assume that our drives honor write cache flushes.
> 
> Can somebody shed some light on how write barriers are implemented in 
> raid0 and raid10? Also, any critical comments on the validity of our 
> setup and/or assumptions is also welcome.

Write barriers are handled as:
   - drain all outstanding requests and block new requests
   - send a zero-length barrier to each component device
   - send the data in the barrier request (if it wasn't zero length)
   - send anther zero-length barrier if there was data
   - allow new requests through.

So you may well notice a slow-down i you enable barriers, but theoretically
your data might be a bit safer.   But that is what barriers have always meant
for ext3, which is why they aren't enabled by default.

NeilBrown