Linux RAID subsystem development
 help / color / mirror / Atom feed
* Question about recovery via mdadm
@ 2003-02-13 15:43 Cress, Andrew R
  2003-02-13 23:15 ` Neil Brown
  0 siblings, 1 reply; 7+ messages in thread
From: Cress, Andrew R @ 2003-02-13 15:43 UTC (permalink / raw)
  To: 'Neil Brown'; +Cc: linux-raid

Neil,

I've seen this issue several times, and it seems most acute with the RedHat
8.0 kernel (2.4.18-14).

Situation: 
I did a hotswap of sda, with:  mdadm -f, raidhotremove, then raidhotadd.
The md kernel messages indicate that the removal, hotadd, and bind occurred,
then it has the raid printout of the status before the recovery, but never
starts the recovery/resync.  Sometimes it will explicitly say that it is
hot-adding the disk as a spare, sometimes not.  

# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sda6[2] sdb6[1]
      289024 blocks [2/1] [U_]

md1 : active raid1 sda2[2] sdb2[1]
      64192 blocks [2/1] [U_]

md0 : active raid1 sda5[2] sdb5[1]
      5293312 blocks [2/1] [U_]

unused devices: <none>
#
The superblock mdadm --examine shows that nd=2, wd=2, but 1 disk is active
and the other (sda) is a  spare.

Question:
Solving why I got into this is another issue, but:
Is there any way, once I'm in this predicament, to force a recovery to the
spare, from userland (via mdadm)?  

Andy


^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: Question about recovery via mdadm
@ 2003-02-17 15:59 Cress, Andrew R
  0 siblings, 0 replies; 7+ messages in thread
From: Cress, Andrew R @ 2003-02-17 15:59 UTC (permalink / raw)
  To: 'Neil Brown', James Ralston; +Cc: linux-raid

The bug I'm experiencing does not produce a kernel oops, just a dump of the
RAID state output.
I'm guessing that it would not happen with the 2.5.x changes to md that
consolidated the superblock counters in one place.  

The system does start a recovery if it is rebooted, but my use case is to
avoid reboots at all cost.
Anyway, I'll go back and get the kernel messages for this problem to pursue
it.

Andy

-----Original Message-----
From: Neil Brown [mailto:neilb@cse.unsw.edu.au] 
Sent: Sunday, February 16, 2003 5:10 PM
To: James Ralston
Cc: Cress, Andrew R; linux-raid@vger.kernel.org
Subject: Re: Question about recovery via mdadm


On Sunday February 16, qralston+ml.linux-raid@andrew.cmu.edu wrote:
> On 2003-02-14 at 10:15:07+1100 Neil Brown <neilb@cse.unsw.edu.au> wrote:
> 
> > On Thursday February 13, andrew.r.cress@intel.com wrote:
> > 
> > > Solving why I got into this is another issue, but: Is there any
> > > way, once I'm in this predicament, to force a recovery to the
> > > spare, from userland (via mdadm)?
> > 
> > No.  Reconstrution should start automatically.  There is no
> > mechanism to start it from user-space.  You could try to hot-remove
> > and hot-add again, but if it didn't work the first time it is
> > unlikely to work the second time.
> > 
> > It would appear to be a kernel bug.  Are there any kernel messages?
> > An Oops or something?
> 
> I'll bet that if Andrew checks his syslog carefully, he'll find that
> the mdrecovery process generated a kernel Oops:
> 
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=82815

Thanks...
I think that bug should be fixed by the follow patch which has been
submitted and accepted and should be in 2.4.21.

NeilBrown


-----------------------------------------
Avoid races by never  releasing rdev->sb for faulty devices.

There are races relating to the superblocks being written out
just as a device has failed, and the rdev->sb getting freeing while
it is being written out.  This patch tries to avoid one of the
races by testing the faulty bit in the superblock (which gets set
early) as well as rdev->faulty (which gets set late), and does not
free rdev->sb until the rdev is fully removed, thus making the races
less critical.




 ----------- Diffstat output ------------
 ./drivers/md/md.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff ./drivers/md/md.c~current~ ./drivers/md/md.c
--- ./drivers/md/md.c~current~	2003-01-03 10:25:44.000000000 +1100
+++ ./drivers/md/md.c	2003-01-03 10:25:43.000000000 +1100
@@ -1048,7 +1048,11 @@ repeat:
 			printk("(skipping faulty ");
 		if (rdev->alias_device)
 			printk("(skipping alias ");
-
+		if (disk_faulty(&rdev->sb->this_disk)) {
+			printk("(skipping new-faulty %s )\n",
+			       partition_name(rdev->dev));
+			continue;
+		}
 		printk("%s ", partition_name(rdev->dev));
 		if (!rdev->faulty && !rdev->alias_device) {
 			printk("[events: %08lx]",
@@ -1075,7 +1079,6 @@ repeat:
  *   - the device is nonexistent (zero size)
  *   - the device has no valid superblock
  *
- * a faulty rdev _never_ has rdev->sb set.
  */
 static int md_import_device(kdev_t newdev, int on_disk)
 {
@@ -1147,8 +1150,6 @@ static int md_import_device(kdev_t newde
 	md_list_add(&rdev->all, &all_raid_disks);
 	MD_INIT_LIST_HEAD(&rdev->pending);
 
-	if (rdev->faulty && rdev->sb)
-		free_disk_sb(rdev);
 	return 0;
 
 abort_free:
@@ -3062,7 +3063,6 @@ int md_error(mddev_t *mddev, kdev_t rdev
 		return 0;
 	if (!mddev->pers->error_handler
 			|| mddev->pers->error_handler(mddev,rdev) <= 0) {
-		free_disk_sb(rrdev);
 		rrdev->faulty = 1;
 	} else
 		return 1;

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2003-02-18  5:45 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-02-13 15:43 Question about recovery via mdadm Cress, Andrew R
2003-02-13 23:15 ` Neil Brown
2003-02-16 10:32   ` James Ralston
2003-02-16 22:09     ` Neil Brown
2003-02-17  1:05       ` James Ralston
2003-02-18  5:45         ` Neil Brown
  -- strict thread matches above, loose matches on Subject: below --
2003-02-17 15:59 Cress, Andrew R

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox