Software RAID crash > /dev/md0: Invalid argument

Software RAID crash > /dev/md0: Invalid argument

[Scott Bolander]

I have a Mandrake 10 server with a 40gb boot drive and a three 200gb disk
software raid 5 array using reiserfs.  A disk went bad; so i have replaced
it.

Doing the normal things to get it back up don't work:

[root@server root]#raidhotadd /dev/md0 /dev/hdg

/dev/md0: can not hot-add disk: array not running!

[root@server root]# raidstart -a

/dev/md0: Invalid argument

I think the superblock on the virtual disk is corrupt.  Can anyone help?
This is a production server and the backup was a removable drive that
failed last week.

Thanks


# cat /etc/fstab
/dev/hde1 / reiserfs notail 1 1
none /dev/pts devpts mode=0620 0 0
# /dev/md0 /files reiserfs notail 1 2
/dev/hda /mnt/cdrom auto
umask=0,user,iocharset=iso8859-1,codepage=850,noauto,ro,exec 0 0
none /mnt/floppy supermount
dev=/dev/fd0,fs=ext2:vfat,--,umask=0,iocharset=iso8859-1,sync,codepage=850
0 0
none /proc proc defaults 0 0
/dev/hde5 swap swap defaults 0 0

# cat /etc/raidtab
raiddev       /dev/md0
raid-level    5
chunk-size    128k
persistent-superblock 1
nr-raid-disks 3
    device    /dev/hdg1
    raid-disk 0
    device    /dev/hdi1
    raid-disk 1
    device    /dev/hdk1
    raid-disk 2

dmesg:
md: invalid raid superblock magic on hdg1
md: hdg1 has invalid sb, not importing!
md: could not import hdg1!
md: autostart unknown-block(0,8705) failed!

cat /proc/modules
raid5 20512 0 - Live 0xd0a5e000

# cat /proc/mdstat
Personalities : [raid5]
md0 : inactive hdi1[1] hdk1[2]
      390716672 blocks
unused devices: <none>


# mdadm --examine /dev/md0
mdadm: /dev/md0 is too small for md


# raidhotremove /dev/md0 /dev/hdg1
/dev/md0: can not hot-remove disk: array not running!


# uname -a
2.6.3-7mdk


Scott Bolander
Computer Services of Cincinnati
http://www.cincysystems.com
scott@cincysystems.com
513-266-6656
"I said, 'government is powerless to protect you,' not powerless to punish
you." --Chief Wiggum, The Simpsons

Re: Software RAID crash > /dev/md0: Invalid argument

[Luca Berra]

On Thu, Sep 16, 2004 at 07:30:49AM -0400, Scott Bolander wrote:
>I have a Mandrake 10 server with a 40gb boot drive and a three 200gb disk
>software raid 5 array using reiserfs.  A disk went bad; so i have replaced
>it.
>
>Doing the normal things to get it back up don't work:
>
>[root@server root]#raidhotadd /dev/md0 /dev/hdg
....
># cat /etc/raidtab
>raiddev       /dev/md0
>raid-level    5
>chunk-size    128k
>persistent-superblock 1
>nr-raid-disks 3
>    device    /dev/hdg1
>    raid-disk 0
>    device    /dev/hdi1
>    raid-disk 1
>    device    /dev/hdk1
>    raid-disk 2
>
so you replaced the first disk in the array?
raidtools are not capable of handling this.
switch to mdadm
echo "DEVICE partitions" >> /etc/mdadm.conf
mdadm -Esc partitions | grep ARRAY >> /etc/mdadm.conf
mdadm -ARs
mdadm -a /dev/md0 /dev/hdg1
mv /etc/raidtab /etc/raidtab.FOOBAR (this is important)

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Software RAID crash > /dev/md0: Invalid argument

[Gordon Henderson]

On Thu, 16 Sep 2004, Luca Berra wrote:

> On Thu, Sep 16, 2004 at 07:30:49AM -0400, Scott Bolander wrote:
> >I have a Mandrake 10 server with a 40gb boot drive and a three 200gb disk
> >software raid 5 array using reiserfs.  A disk went bad; so i have replaced
> >it.
> >
> >Doing the normal things to get it back up don't work:
> >
> >[root@server root]#raidhotadd /dev/md0 /dev/hdg
> ....
> ># cat /etc/raidtab
> >raiddev       /dev/md0
> >raid-level    5
> >chunk-size    128k
> >persistent-superblock 1
> >nr-raid-disks 3
> >    device    /dev/hdg1
> >    raid-disk 0
> >    device    /dev/hdi1
> >    raid-disk 1
> >    device    /dev/hdk1
> >    raid-disk 2
> >
> so you replaced the first disk in the array?
> raidtools are not capable of handling this.
> switch to mdadm

Er - Maybe I'm missing something (like the original message in this
thread), but I've successfully used raidtools to raidhotremove and
raidhotadd the first drive (and others) in a raid5 set.., Once on a real
'hot' system, a server with SCSI drives which I didn't power cycle, and on
IDE systems where I've done a sequence of raidhotremove, power down,
change physical drive, boot up, cfdisk new drive, then raidhotadd...

Gordon

Re: Software RAID crash > /dev/md0: Invalid argument

[Luca Berra]

On Thu, Sep 16, 2004 at 02:35:45PM +0100, Gordon Henderson wrote:
>Er - Maybe I'm missing something (like the original message in this
>thread), but I've successfully used raidtools to raidhotremove and
i have not been using raidtools for a long time. but i clearly remember
that it used the drive list in /etc/raidtab to start the array and i had
issues starting an array with a missing first drive.

L.
-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

RE: Software RAID crash > /dev/md0: Invalid argument

[Guy]

Why are you using "/dev/hdg"?
Your array was using "/dev/hdg1".
Did you forget to partition the disk?

Partitioning the disk is not required, but things will be easier if you do.

mdadm is a much better product.  And supported!

I had an array that I could not start with raidstart, that is when I made
the switch to mdadm.

Guy

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Scott Bolander
Sent: Thursday, September 16, 2004 7:31 AM
To: linux-raid@vger.kernel.org
Cc: scott@bolanderhome.com
Subject: Software RAID crash > /dev/md0: Invalid argument

I have a Mandrake 10 server with a 40gb boot drive and a three 200gb disk
software raid 5 array using reiserfs.  A disk went bad; so i have replaced
it.

Doing the normal things to get it back up don't work:

[root@server root]#raidhotadd /dev/md0 /dev/hdg

/dev/md0: can not hot-add disk: array not running!

[root@server root]# raidstart -a

/dev/md0: Invalid argument

I think the superblock on the virtual disk is corrupt.  Can anyone help?
This is a production server and the backup was a removable drive that
failed last week.

Thanks


# cat /etc/fstab
/dev/hde1 / reiserfs notail 1 1
none /dev/pts devpts mode=0620 0 0
# /dev/md0 /files reiserfs notail 1 2
/dev/hda /mnt/cdrom auto
umask=0,user,iocharset=iso8859-1,codepage=850,noauto,ro,exec 0 0
none /mnt/floppy supermount
dev=/dev/fd0,fs=ext2:vfat,--,umask=0,iocharset=iso8859-1,sync,codepage=850
0 0
none /proc proc defaults 0 0
/dev/hde5 swap swap defaults 0 0

# cat /etc/raidtab
raiddev       /dev/md0
raid-level    5
chunk-size    128k
persistent-superblock 1
nr-raid-disks 3
    device    /dev/hdg1
    raid-disk 0
    device    /dev/hdi1
    raid-disk 1
    device    /dev/hdk1
    raid-disk 2

dmesg:
md: invalid raid superblock magic on hdg1
md: hdg1 has invalid sb, not importing!
md: could not import hdg1!
md: autostart unknown-block(0,8705) failed!

cat /proc/modules
raid5 20512 0 - Live 0xd0a5e000

# cat /proc/mdstat
Personalities : [raid5]
md0 : inactive hdi1[1] hdk1[2]
      390716672 blocks
unused devices: <none>


# mdadm --examine /dev/md0
mdadm: /dev/md0 is too small for md


# raidhotremove /dev/md0 /dev/hdg1
/dev/md0: can not hot-remove disk: array not running!


# uname -a
2.6.3-7mdk


Scott Bolander
Computer Services of Cincinnati
http://www.cincysystems.com
scott@cincysystems.com
513-266-6656
"I said, 'government is powerless to protect you,' not powerless to punish
you." --Chief Wiggum, The Simpsons



-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software RAID crash > /dev/md0: Invalid argument

[Neil Brown]

On Thursday September 16, gordon@drogon.net wrote:
> On Thu, 16 Sep 2004, Luca Berra wrote:
> 
> > On Thu, Sep 16, 2004 at 07:30:49AM -0400, Scott Bolander wrote:
> > >I have a Mandrake 10 server with a 40gb boot drive and a three 200gb disk
> > >software raid 5 array using reiserfs.  A disk went bad; so i have replaced
> > >it.
> > >
> > >Doing the normal things to get it back up don't work:
> > >
> > >[root@server root]#raidhotadd /dev/md0 /dev/hdg
> > ....
> > ># cat /etc/raidtab
> > >raiddev       /dev/md0
> > >raid-level    5
> > >chunk-size    128k
> > >persistent-superblock 1
> > >nr-raid-disks 3
> > >    device    /dev/hdg1
> > >    raid-disk 0
> > >    device    /dev/hdi1
> > >    raid-disk 1
> > >    device    /dev/hdk1
> > >    raid-disk 2
> > >
> > so you replaced the first disk in the array?
> > raidtools are not capable of handling this.
> > switch to mdadm
> 
> Er - Maybe I'm missing something (like the original message in this
> thread), but I've successfully used raidtools to raidhotremove and
> raidhotadd the first drive (and others) in a raid5 set.., Once on a real
> 'hot' system, a server with SCSI drives which I didn't power cycle, and on
> IDE systems where I've done a sequence of raidhotremove, power down,
> change physical drive, boot up, cfdisk new drive, then raidhotadd...

The problem is with "raidstart", not "raidhotremove".
If you use auto-detect then you won't have a problem.
If you use raidstart to start your arrays, you could have a problem.

NeilBrown

Re: Software RAID crash > /dev/md0: Invalid argument

[Scott Bolander]

Hey guys - it's been fixed.  Your help and some good work by a Tom
Glascock (www.dentar.com) got it fixed.  It is regenerating now. 
Basically mdadm and a custom kernel was involved.

Scott Bolander
Computer Services of Cincinnati
http://www.cincysystems.com
scott@bolanderhome.com
513-266-6656
"I said, 'government is powerless to protect you,' not powerless to punish
you." --Chief Wiggum, The Simpsons

Neil Brown said:
> On Thursday September 16, gordon@drogon.net wrote:
>> On Thu, 16 Sep 2004, Luca Berra wrote:
>>
>> > On Thu, Sep 16, 2004 at 07:30:49AM -0400, Scott Bolander wrote:
>> > >I have a Mandrake 10 server with a 40gb boot drive and a three 200gb
>> disk
>> > >software raid 5 array using reiserfs.  A disk went bad; so i have
>> replaced
>> > >it.
>> > >
>> > >Doing the normal things to get it back up don't work:
>> > >
>> > >[root@server root]#raidhotadd /dev/md0 /dev/hdg
>> > ....
>> > ># cat /etc/raidtab
>> > >raiddev       /dev/md0
>> > >raid-level    5
>> > >chunk-size    128k
>> > >persistent-superblock 1
>> > >nr-raid-disks 3
>> > >    device    /dev/hdg1
>> > >    raid-disk 0
>> > >    device    /dev/hdi1
>> > >    raid-disk 1
>> > >    device    /dev/hdk1
>> > >    raid-disk 2
>> > >
>> > so you replaced the first disk in the array?
>> > raidtools are not capable of handling this.
>> > switch to mdadm
>>
>> Er - Maybe I'm missing something (like the original message in this
>> thread), but I've successfully used raidtools to raidhotremove and
>> raidhotadd the first drive (and others) in a raid5 set.., Once on a real
>> 'hot' system, a server with SCSI drives which I didn't power cycle, and
>> on
>> IDE systems where I've done a sequence of raidhotremove, power down,
>> change physical drive, boot up, cfdisk new drive, then raidhotadd...
>
> The problem is with "raidstart", not "raidhotremove".
> If you use auto-detect then you won't have a problem.
> If you use raidstart to start your arrays, you could have a problem.
>
> NeilBrown
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>