Fibre Channel Shared Raid 5

Fibre Channel Shared Raid 5

[Thomas Garner]

Is it possible to take a group of fibre channel drives on a fc-al loop, 
run raid 5 on top of them, run lvm on top of that, then gfs on top of 
that, and have the entire thing readable/writable by multiple machines 
at the same time?  I have searched amongst lvm, clvm, evms, nbd, enbd, 
linux software raid, etc. but to no avail.  I assume that both clvm or 
evms would suit me to make the drives available to multiple machine 
concurrently, but not allow redundancy.  Hopefully I am wrong and this 
is in fact possible.

Thanks!
Thomas

Re: Fibre Channel Shared Raid 5

[Neil Brown]

On Saturday August 27, tlg1466@neo.tamu.edu wrote:
> Is it possible to take a group of fibre channel drives on a fc-al loop, 
> run raid 5 on top of them, run lvm on top of that, then gfs on top of 
> that, and have the entire thing readable/writable by multiple machines 
> at the same time?  I have searched amongst lvm, clvm, evms, nbd, enbd, 
> linux software raid, etc. but to no avail.  I assume that both clvm or 
> evms would suit me to make the drives available to multiple machine 
> concurrently, but not allow redundancy.  Hopefully I am wrong and this 
> is in fact possible.

I think you are not wrong, sorry (assuming I understand you
correctly).

md/raid5 cannot be distributed.  At any one time, one machine must be
in control of the array and managing parity updates etc.  The device
provided by md/raid5 can obviously be share with GFS or NFS or nbd of
whatever.  But the management of the actually raid5 array must in the
hands of just one machine.

NeilBrown

Re: Fibre Channel Shared Raid 5

[Thomas Garner]

I recently stumbled upon ddraid.  Would this work at all for what I'm 
looking for?

Thomas

Neil Brown wrote:
> I think you are not wrong, sorry (assuming I understand you
> correctly).
> 
> md/raid5 cannot be distributed.  At any one time, one machine must be
> in control of the array and managing parity updates etc.  The device
> provided by md/raid5 can obviously be share with GFS or NFS or nbd of
> whatever.  But the management of the actually raid5 array must in the
> hands of just one machine.
> 
> NeilBrown

Re: Fibre Channel Shared Raid 5

[Neil Brown]

On Sunday August 28, tlg1466@neo.tamu.edu wrote:
> I recently stumbled upon ddraid.  Would this work at all for what I'm 
> looking for?

I'm not certain, but I suspect not.

The 'dd' stands for 'distributed data'.  You have a cluster where each
node has one drive, and you use ddraid to combine all these drives
into an array which can survive as single node dying.  But I *think*
you still need a master node which runs the array.  Also the slaves
would need to access the array through the master.
But as I said, I'm not sure.  You'd need to check with the author I
guess.

NeilBrown

Re: Fibre Channel Shared Raid 5

[Dan Stromberg]

On Mon, 2005-08-29 at 13:56 +1000, Neil Brown wrote:
> On Sunday August 28, tlg1466@neo.tamu.edu wrote:
> > I recently stumbled upon ddraid.  Would this work at all for what I'm 
> > looking for?
> 
> I'm not certain, but I suspect not.
> 
> The 'dd' stands for 'distributed data'.  You have a cluster where each
> node has one drive, and you use ddraid to combine all these drives
> into an array which can survive as single node dying.  But I *think*
> you still need a master node which runs the array.  Also the slaves
> would need to access the array through the master.
> But as I said, I'm not sure.  You'd need to check with the author I
> guess.

I'm very interested in hearing more about ddraid, if there are folks on
the list who've tried it.

Thanks!

Re: Fibre Channel Shared Raid 5

[Neil Brown]

On Monday August 29, neilb@cse.unsw.edu.au wrote:
> On Sunday August 28, tlg1466@neo.tamu.edu wrote:
> > I recently stumbled upon ddraid.  Would this work at all for what I'm 
> > looking for?
> 
> I'm not certain, but I suspect not.
> 

Having reflected a bit more on what I learned at the linux.conf.au
talk on the subject, I'm beginning to suspect "so" rather than "not".

ddraid uses a modified raid3 algorithm.
Each device block (which is typically 1 or 4 k) is split into
individual (512byte) sectors and these striped over the physical devices.
Thus every write is a full-stripe-write so no pre-reading is needed
and there is no need for co-ordination with other blocks to calculate
the parity.

Because if this, multiple hosts could write to the array providing
they agree beforehand who is allowed to write to which block - no
further co-ordination is required.

This co-ordination would be needed for dlvm or gfs anyway, so it is
quite possible that it can be used in a very distributed way.

However I've never actually used it, so I cannot promise anything.

NeilBrown