Simultaneous assembly on multiple hosts

Simultaneous assembly on multiple hosts

[Neil Matai]

Hi all,

Is it dangerous have an array assembled on two hosts at the same time?
Say I have servers A and B exporting devices a and b via ATA over
Ethernet or similar to hosts C and D.  If a and b are components of an
array then both C and D automatically assemble the array as soon as
the components a and b are exported but is this safe (on the proviso
that only one of C or D is ever writing to the array at any time and
that there is no write-behind caching above the array device)?

Thank you,

- neil

Re: Simultaneous assembly on multiple hosts

[Neil Matai]

On Fri, May 03, 2013 at 08:25:59PM -0600, Marcus Sorensen wrote:
> > Is it dangerous have an array assembled on two hosts at the same time?
> > Say I have servers A and B exporting devices a and b via ATA over
> > Ethernet or similar to hosts C and D.  If a and b are components of an
> > array then both C and D automatically assemble the array as soon as
> > the components a and b are exported but is this safe (on the proviso
> > that only one of C or D is ever writing to the array at any time and
> > that there is no write-behind caching above the array device)?
> Yes, it is dangerous. There are some limited situations where it may work,
> without a bitmap and some cluster aware application in control of what
> blocks are written, but you can't for example mount it ext4 and write to
> only one side at a time. That's why people use drbd.

Sorry.  I should have said what it was for.  I realise that mounting a
typical file system twice would be dangerous (also with drdb).  The
intention would have been to live migrate guest systems between C and
D so the file system remains mounted (once) inside the guest system
but nowhere else.  Would it work in this scenario without a bitmap or
would this scenario also lead to data loss?  Would it make any
difference if 0.9 metadata were use to create the array?

Many thanks,

- neil

Re: Simultaneous assembly on multiple hosts

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

On Sat, 4 May 2013 14:01:46 +1200 Neil Matai <neil@matai.kiwi.nz> wrote:

> Hi all,
> 
> Is it dangerous have an array assembled on two hosts at the same time?
> Say I have servers A and B exporting devices a and b via ATA over
> Ethernet or similar to hosts C and D.  If a and b are components of an
> array then both C and D automatically assemble the array as soon as
> the components a and b are exported but is this safe (on the proviso
> that only one of C or D is ever writing to the array at any time and
> that there is no write-behind caching above the array device)?
> 

Yes, it is dangerous.
Both hosts will try to update the metadata and will trip over each other.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Simultaneous assembly on multiple hosts

[Sebastian Riemer]

On 06.05.2013 08:32, NeilBrown wrote:
> On Sat, 4 May 2013 14:01:46 +1200 Neil Matai <neil@matai.kiwi.nz>
> wrote:
> 
>> Hi all,
>> 
>> Is it dangerous have an array assembled on two hosts at the same
>> time? Say I have servers A and B exporting devices a and b via
>> ATA over Ethernet or similar to hosts C and D.  If a and b are
>> components of an array then both C and D automatically assemble
>> the array as soon as the components a and b are exported but is
>> this safe (on the proviso that only one of C or D is ever writing
>> to the array at any time and that there is no write-behind
>> caching above the array device)?
>> 
> 
> Yes, it is dangerous. Both hosts will try to update the metadata
> and will trip over each other.

This is why we've implemented automatic super block + bitmap reload
upon first IO. Before, the MD device acts as a read-only device with
read-only super block. This is rather complex to make it safe.
This is also difficult to merge with mainline behavior.

Get yourself a kernel developer. ;-)

Cheers,
Sebastian