Re: [linux-lvm] LVM + raid + san

["Stuart D. Gathman" <stuart@bmsi.com>]

On Sun, 7 Nov 2010, Stuart D. Gathman wrote:

> You'd have some serious locking issues.  With RAID5, each server would have to
> lock each chunk before writing to it (which involves a read/modify/write
> cycle).  This would create serious overhead.  And you were complaining about
> SAN server overhead! :-)  RAID5 *must* be centralized.  Your scheme might work
> with RAID10, but then you'd still have to ensure that writes to mirror legs
> don't get out of order, with updates from multiple servers flying over the
> wire.

Actually, this would be an interesting driver to develop.  If each
server primarily works on its own LV, then there shouldn't be much
lock contention.  You would need a lock manager, and a special network
RAID driver that uses the lock manager to coordinate updates.
Each server would hold the lock for a chunk until it is needed by another
server.  With the assumed mostly independent access, this should be
rare, and the locking should be optimized with that in mind.  I.e.,
if you already hold the lock, just go ahead an update.  If not, then
notify the holder via the lock manager, and wait until you do hold it.

You could probably avoid the lock manager by using a broadcast based protocol
(Who has chunk 12345678?)

Oh wait, this is an LVM list...

Is anything like this contemplated for devicemapper?  There is already
locking involved with shared VGs on a traditional SAN.

It is an interesting idea to avoid a traditional SAN as a single point of
failure (the switch connecting the hosts and disks would still be a single
point of failure, but a switch is simpler than a SAN server).  All hosts would
have to be trusted.

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.