Re: Raid1 element stuck in (S) state

[micah <micah@riseup.net>]

NeilBrown <neilb@suse.de> writes:

> On Wed, 29 Oct 2014 10:03:16 -0400 micah <micah@riseup.net> wrote:
>
>> NeilBrown <neilb@suse.de> writes:
>> 
>> > On Mon, 27 Oct 2014 10:18:47 -0400 micah anderson <micah@debian.org> wrote:
>> >
>> >> 
>> >> Hi,
>> >> 
>> >> i've got a raid1 setup, where one drive died, it was replaced with a new
>> >> one, but its stuck in a (S) state and I can't seem to get it added into
>> >> the array, /proc/mdstat looks like this:
>> >> 
>> >> md3 : active raid1 sdc1[2](S) sdd1[1]
>> >>       976759672 blocks super 1.2 [2/1] [_U]
>> >> 
>> >> where sdc1 is the replaced drive.
>> >> 
>> >> What is the right way to get this added back?
>> >>
>> >
>> > I've a feeling this bug might have been fixed.
>> > What versions of mdadm and Linux are you using?
>> 
>> I'm using squeeze here, and had 3.1.4-1+8efb9d1+squeeze1 installed, I
>> just installed the backport, which is 3.2.5-3~bpo60+1.
>
> Is assume that is the version of mdadm.  You didn't say what version of Linux.

Yes, that is the version of mdadm. I am running squeeze, which is a
2.6.32-5 version of the kernel, and it is an amd64 machine.

>> > Are there any errors in the kernel logs when you --add the device?
>
> You didn't answer this question either.  Are there any messages in the
> kernel log: /var/log/kern.log on debian.
> Or in the output of "dmesg".

The only thing I see in the log is:

[307932.328420] mdadm: sending ioctl 1261 to a partition!
[307932.328425] mdadm: sending ioctl 1261 to a partition!
[307932.346642] mdadm: sending ioctl 1261 to a partition!
[307932.346648] mdadm: sending ioctl 1261 to a partition!
[307932.352466] mdadm: sending ioctl 1261 to a partition!
[307932.352468] mdadm: sending ioctl 1261 to a partition!
[307932.376821] mdadm: sending ioctl 1261 to a partition!
[307932.376824] mdadm: sending ioctl 1261 to a partition!
[307932.377623] mdadm: sending ioctl 1261 to a partition!
[307932.377630] mdadm: sending ioctl 1261 to a partition!
[307932.467292] md: bind<sdc1>
[307932.588154] RAID1 conf printout:
[307932.588159]  --- wd:1 rd:2
[307932.588164]  disk 0, wo:1, o:1, dev:sdc1
[307932.588167]  disk 1, wo:0, o:1, dev:sdd1
[307932.588248] md: recovery of RAID array md3
[307932.588251] md: minimum _guaranteed_  speed: 50000 KB/sec/disk.
[307932.588254] md: using maximum available idle IO bandwidth (but not more than 2000000 KB/sec) for recovery.
[307932.588260] md: using 128k window, over a total of 976759672 blocks.

but this is just when the device is added, after that it appears that
logrotation failed and I have a zero byte kern.log, and firewall spew
has filled up my dmesg ring.

>> Can I just zero the superblock of that device and re-add it in order to
>> resolve this?
>
>
> If it resyncs and the is still spare, there was almost certainly some sort of
> failure.  There really must be something in the kernel logs at that time.

It did resync, and is still a spare.... Now that I've fixed the logs,
I'm going to try it again to see if there is any error that happens
after the sync finishes.

micah