Re: Rotating RAID 1

[Phil Turmel <philip@turmel.org>]

Hi Jérôme,

On 08/15/2011 03:56 PM, Jérôme Poulin wrote:
> Then what is different about a standard RAID1, I removed sdb and
> replaced it with a brand new disk, copied the partition template from
> the other one and added the new disk using mdadm -a on both arrays, it
> synced and works, then swapping the other disk back only rebuilds
> according to the bitmap, however sometimes it appears to make a full
> rebuild which is alright. However once, after a day of modifications
> and weeks after setting-up this RAID, at least 100 GB, it took seconds
> to rebuild and days later it appeared to have encountered corruption,
> the kernel complained about bad extents and fsck found errors in one
> of the file I know it had modified that day.

This is a problem.  MD only knows about two disk.  You have three.  When two disks are in place and sync'ed, the bitmaps will essentially stay cleared.

When you swap to the other disk, its bitmap is also clear, for the same reason.  I'm sure mdadm notices the different event counts, but the clear bitmap would leave mdadm little or nothing to do to resync, as far as it knows.  But lots of writes have happened in the meantime, and they won't get copied to the freshly inserted drive.  Mdadm will read from both disks in parallel when there are parallel workloads, so one workload would get current data and the other would get stale data.

If you perform a "check" pass after swapping and resyncing, I bet it finds many mismatches.  It definitely can't work as described.

I'm not sure, but this might work if you could temporarily set it up as a triple mirror, so each disk has a unique slot/role.

It would also work if you didn't use a bitmap, as a re-inserted drive would simply be overwritten completely.

> So the question is; Am I right to use md-raid to do this kind of
> stuff, rsync is too CPU heavy for what I need and I need to stay
> compatible with Windows thus choosing metadata 1.0.

How do you stay compatible with Windows?  If you let Windows write to any of these disks, you've corrupted that disk with respect to its peers.  Danger, Will Robinson!

HTH,

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html