Re: safe segmenting of conflicting changes (was: Two degraded mirror segments recombined out of sync for massive data loss)

[Phillip Susi <psusi@cfl.rr.com>]

After some more testing it seems the problems with --incremental are
more deep and general.  I have found two steps that both appear to do
the wrong thing:

mdadm /dev/md0 --fail /dev/sdb
mdadm /dev/md0 --remove /dev/sdb

At this point mdadm -E /dev/sda shows that sdb has been removed, but
mdadm -E of /dev/sdb shows both disks are active and in sync still.  The
metadata of sdb should be updated when failed or removed if possible.

mdadm --incremental /dev/sdb

This goes ahead and adds the disk back to the array, despite the fact
that it has been explicitly removed.

Whether or not the superblock on sdb is updated when it is removed,
--incremental should NOT use it as long as mdadm -D /dev/md0 says that
disk is removed, at least not use it in /dev/md0.