Urgent SCSI I/O Error

Urgent SCSI I/O Error

[Manohar Pradhan]

Hi,

This question might have been raised before but I am stucked in
between wierd/helpless situation and wondering if someone can help me
out.

I have Red Hat Linux 6.2 (2.2.14-5.0smp) running in my HP Netserver
box. I have 2 9.1 GB HDD. The server has been up for few months and
have not had seen any problem. But today all of sudden it gave
panicking message saying following:

Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 4194368
Feb 25 18:48:12 nsdb1 kernel: EXT2-fs error (device sd(8,6)): ext2_write_inode: unable to read inode block - inode=251018, block=524296
Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 1835048
Feb 25 18:48:12 nsdb1 kernel: EXT2-fs error (device sd(8,6)): ext2_write_inode: unable to read inode block - inode=109814, block=229381
Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 40
Feb 25 18:48:12 nsdb1 kernel: EXT2-fs error (device sd(8,6)): ext2_write_inode: unable to read inode block - inode=57, block=5
Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:07: sense key Not Ready
Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:07, sector 1048832
Feb 25 18:48:12 nsdb1 kernel: EXT2-fs error (device sd(8,7)): ext2_write_inode: unable to read inode block - inode=62495, block=131104
Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 0
Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:07: sense key Not Ready
Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:07, sector 0

I have system running but cannot access files/database saved in
volumes in the second HDD sdb. Following are the partitions I have and
I haven't used any RAID while instaling, means plain hard drive spaces
in 2 drives.

/dev/sdb6               917072    732972    137516  84% /
/dev/sda1                18820      5811     12037  33% /boot
/dev/sda6              2218336    462492   1643156  22% /www
/dev/sda5              5297560    418936   4609520   8% /home
/dev/sda7              1210440    711516    437436  62% /software
/dev/sdb1              5550188     50896   5217356   1% /usr
/dev/sdb5              2016016     28572   1885032   1% /var

I can access files in all the other partitions but cannot access
files/directories in partition /www. I can see files in the
directories listing using 'ls' however accessing any file gives
Input/Output error saying:

cat check1.htm: Input/output error



Can anyone help/suggest me what should I do to make it work? I am
wondering if I reboot the system, I may fall into problem on booting
itself. Is there any thing I need to do to make this partition work?

Thanks in advance for the help.

Regards
Manohar

Re: Urgent SCSI I/O Error

[Jakob Østergaard]

On Mon, Feb 25, 2002 at 07:09:53PM +0000, Manohar Pradhan wrote:
> Hi,
> 
> This question might have been raised before but I am stucked in
> between wierd/helpless situation and wondering if someone can help me
> out.
> 
> I have Red Hat Linux 6.2 (2.2.14-5.0smp) running in my HP Netserver
> box. I have 2 9.1 GB HDD. The server has been up for few months and
> have not had seen any problem. But today all of sudden it gave
> panicking message saying following:
> 
...
> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 0
> Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:07: sense key Not Ready
> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:07, sector 0

Your disk died.  Physically.

...
> I can access files in all the other partitions but cannot access
> files/directories in partition /www. I can see files in the
> directories listing using 'ls' however accessing any file gives
> Input/Output error saying:
> 
> cat check1.htm: Input/output error
> 

Yep - can't read from bad blocks.

> Can anyone help/suggest me what should I do to make it work? I am
> wondering if I reboot the system, I may fall into problem on booting
> itself. Is there any thing I need to do to make this partition work?

Replace the harddrive, restore from backup.

You will only have a problem booting, if the boot sector / kernel 
resides on that drive - and if that part of the drive is damaged
too.   It seems like that's not the case, but it would be wise
to run an "mkbootdisk" now, just in case.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re[2]: Urgent SCSI I/O Error

[Manohar Pradhan]

Hello Jakob,

Thanks. I got another HDD read. but How can I install it, format and
make partition as earlier ( I have partition info though ..) and mount
it? Do I need to compile the Kernel or I can attach new HDD, and start
to create file systems inside.

Thanks for your enlightments.

Regards
Manohar




Monday, February 25, 2002, 8:02:39 PM, you wrote:

JØ> On Mon, Feb 25, 2002 at 07:09:53PM +0000, Manohar Pradhan wrote:
>> Hi,
>> 
>> This question might have been raised before but I am stucked in
>> between wierd/helpless situation and wondering if someone can help me
>> out.
>> 
>> I have Red Hat Linux 6.2 (2.2.14-5.0smp) running in my HP Netserver
>> box. I have 2 9.1 GB HDD. The server has been up for few months and
>> have not had seen any problem. But today all of sudden it gave
>> panicking message saying following:
>> 
JØ> ...
>> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
>> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 0
>> Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
>> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:07: sense key Not Ready
>> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:07, sector 0

JØ> Your disk died.  Physically.

JØ> ...
>> I can access files in all the other partitions but cannot access
>> files/directories in partition /www. I can see files in the
>> directories listing using 'ls' however accessing any file gives
>> Input/Output error saying:
>> 
>> cat check1.htm: Input/output error
>> 

JØ> Yep - can't read from bad blocks.

>> Can anyone help/suggest me what should I do to make it work? I am
>> wondering if I reboot the system, I may fall into problem on booting
>> itself. Is there any thing I need to do to make this partition work?

JØ> Replace the harddrive, restore from backup.

JØ> You will only have a problem booting, if the boot sector / kernel 
JØ> resides on that drive - and if that part of the drive is damaged
JØ> too.   It seems like that's not the case, but it would be wise
JØ> to run an "mkbootdisk" now, just in case.




-- 
Best regards,
 Manohar                            mailto:mpml@isp.primuseurope.com

Re: Re[2]: Urgent SCSI I/O Error

[Jakob Østergaard]

On Mon, Feb 25, 2002 at 08:36:35PM +0000, Manohar Pradhan wrote:
> Hello Jakob,
> 
> Thanks. I got another HDD read. but How can I install it, format and
> make partition as earlier ( I have partition info though ..) and mount
> it? Do I need to compile the Kernel or I can attach new HDD, and start
> to create file systems inside.

Just plug in the new disk, boot the system, fdisk, create filesystems, 
copy data / restore data,  and that's it.

You only need to compile a kernel if you are plugging in some hardware
that is not supported by your running kernel.  Given that you run a
standard RedHat kernel, it should have support for just about everything
(and a kitchen sink).

You may find sfdisk useful for creating the partition tables (man sfdisk),
or you can create them "manually" using fdisk.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re[2]: Urgent SCSI I/O Error

[Manohar Pradhan]

Hi All,

If the Hard Drive is died, means I will need to replace. I have backup
for the Data in that /www partition.

Means when I replace new HDD, I will have to partition/format using
fdisk /sda ?

>From my partition info,

/dev/sdb6               917072    732972    137516  84% /
/dev/sda1                18820      5811     12037  33% /boot
/dev/sda6              2218336    462492   1643156  22% /www
/dev/sda5              5297560    418936   4609520   8% /home
/dev/sda7              1210440    711516    437436  62% /software
/dev/sdb1              5550188     50896   5217356   1% /usr
/dev/sdb5              2016016     28572   1885032   1% /var


My problematic HDD is /sda so if I replace this HDD, how can I boot as
boot images are in /sda1 /boot partition. How can I copy this boot
images to somewhere and make it work and what will be the process?

I know if I reboot and replace the HDD, it will give problem while
booting, any Idea to struggle with this?

Thanks a lot for all your help.

Regards
Manohar



Monday, February 25, 2002, 8:02:39 PM, you wrote:

JØ> On Mon, Feb 25, 2002 at 07:09:53PM +0000, Manohar Pradhan wrote:
>> Hi,
>> 
>> This question might have been raised before but I am stucked in
>> between wierd/helpless situation and wondering if someone can help me
>> out.
>> 
>> I have Red Hat Linux 6.2 (2.2.14-5.0smp) running in my HP Netserver
>> box. I have 2 9.1 GB HDD. The server has been up for few months and
>> have not had seen any problem. But today all of sudden it gave
>> panicking message saying following:
>> 
JØ> ...
>> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:06: sense key Not Ready
>> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:06, sector 0
>> Feb 25 18:48:12 nsdb1 kernel: SCSI disk error : host 0 channel 0 id 0 lun 0 return code = 28000002
>> Feb 25 18:48:12 nsdb1 kernel: [valid=0] Info fld=0x0, Current sd08:07: sense key Not Ready
>> Feb 25 18:48:12 nsdb1 kernel: scsidisk I/O error: dev 08:07, sector 0

JØ> Your disk died.  Physically.

JØ> ...
>> I can access files in all the other partitions but cannot access
>> files/directories in partition /www. I can see files in the
>> directories listing using 'ls' however accessing any file gives
>> Input/Output error saying:
>> 
>> cat check1.htm: Input/output error
>> 

JØ> Yep - can't read from bad blocks.

>> Can anyone help/suggest me what should I do to make it work? I am
>> wondering if I reboot the system, I may fall into problem on booting
>> itself. Is there any thing I need to do to make this partition work?

JØ> Replace the harddrive, restore from backup.

JØ> You will only have a problem booting, if the boot sector / kernel 
JØ> resides on that drive - and if that part of the drive is damaged
JØ> too.   It seems like that's not the case, but it would be wise
JØ> to run an "mkbootdisk" now, just in case.




-- 
Best regards,
 Manohar                            mailto:mpml@isp.primuseurope.com

Re: Re[2]: Urgent SCSI I/O Error

[Jakob Østergaard]

On Mon, Feb 25, 2002 at 08:44:19PM +0000, Manohar Pradhan wrote:
> Hi All,
> 
> If the Hard Drive is died, means I will need to replace. I have backup
> for the Data in that /www partition.
> 
> Means when I replace new HDD, I will have to partition/format using
> fdisk /sda ?
> 
> >From my partition info,
> 
> /dev/sdb6               917072    732972    137516  84% /
> /dev/sda1                18820      5811     12037  33% /boot
> /dev/sda6              2218336    462492   1643156  22% /www
> /dev/sda5              5297560    418936   4609520   8% /home
> /dev/sda7              1210440    711516    437436  62% /software
> /dev/sdb1              5550188     50896   5217356   1% /usr
> /dev/sdb5              2016016     28572   1885032   1% /var
> 
> 
> My problematic HDD is /sda so if I replace this HDD, how can I boot as
> boot images are in /sda1 /boot partition. How can I copy this boot
> images to somewhere and make it work and what will be the process?

Plug in the new drive as well as the old one.

I assume the new drive will be /dev/sdc.  And I assume you are running
RedHat 6.2 with LILO.

If you statically set the IDs on your disks, just make sure that the
new disk has a higher ID than the two others, then it will be /dev/sdc.

> 
> I know if I reboot and replace the HDD, it will give problem while
> booting, any Idea to struggle with this?

# Create partitions on the new disk
sfdisk -d /dev/sda | sfdisk /dev/sdc

# Create filesystems
mke2fs /dev/sdc1
mke2fs /dev/sdc5
mke2fs /dev/sdc6
mke2fs /dev/sdc7

# Mount new filesystems
mkdir /mnt/boot
mount /dev/sdc1 /mnt/boot
mkdir /mnt/home
mount /dev/sdc5 /mnt/home
mkdir /mnt/www
mount /dev/sdc6 /mnt/www
mkdir /mnt/software
mount /dev/sdc7 /mnt/software

# Copy the boot filesystem
umask 0
tar cf - /boot | tar xCfp /mnt -

# Now, edit /etc/lilo.conf so that it
# contains something like:
boot=/dev/sdc
disk=0x80
map=/mnt/boot/map
# The boot and map lines are to be replaced, the
# disk line is to be inserted.

# Make the disk bootable - if this command
# complains, then I've overlooked something.
# Mail me back with the error message
lilo

# Now copy the other filesystems.  Note, these
# copy commands may fail horribly when you run
# into the bad blocks !!!  Don't worry, you have
# a backup...
tar cf - /home | tar xCfp /mnt -
tar cf - /www | tar xCfp /mnt -
tar cf - /software | tar xCfp /mnt -

# Now, your box may have died during the copying, 
# or it may still be more or less alive.  We
# don't care, it's time to lose the old drive.
# Now, power down, and unplug /dev/sda
# Move /dev/sdc so that it will be recognized as
# /dev/sda on boot.

 That should be it !  (in theory at least)

You probably want to change back your lilo.conf, and
check the files you (possibly) copied from the broken
disk.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: Re[2]: Urgent SCSI I/O Error

[Richard B. Johnson]

On Mon, 25 Feb 2002, Manohar Pradhan wrote:

> Hi All,
> 
> If the Hard Drive is died, means I will need to replace. I have backup
> for the Data in that /www partition.
> 
> Means when I replace new HDD, I will have to partition/format using
> fdisk /sda ?
> 
> >From my partition info,
> 
> /dev/sdb6               917072    732972    137516  84% /
> /dev/sda1                18820      5811     12037  33% /boot
> /dev/sda6              2218336    462492   1643156  22% /www
> /dev/sda5              5297560    418936   4609520   8% /home
> /dev/sda7              1210440    711516    437436  62% /software
> /dev/sdb1              5550188     50896   5217356   1% /usr
> /dev/sdb5              2016016     28572   1885032   1% /var
> 
> 
> My problematic HDD is /sda so if I replace this HDD, how can I boot as
> boot images are in /sda1 /boot partition. How can I copy this boot
> images to somewhere and make it work and what will be the process?
> 
> I know if I reboot and replace the HDD, it will give problem while
> booting, any Idea to struggle with this?
> 
> Thanks a lot for all your help.
> 
> Regards
> Manohar
> 
> 

Simple. This assumes your boot drive (/dev/sda) is still readable.

(1)  Comment out (in /etc/fstab) everything that mounts anything you
don't need for the basic system. This is just to simplify things
when you are working.

(2)  Make a floppy disk that boots linux and uses the current root
file-system (man rdev). If you don't need modules to boot it's just
`cp /boot/vmlinuz-whatever-rev-is /dev/fd0`
`rdev /dev/fd0 /dev/sdb6`
Boot, using your floppy, to make certain your system comes up okay.

(3)  Shut down and install your new SCSI disk as the next highest
SCSI device number. This will make it /dev/sdc because you have
/dev/sda and /dev/sdb.

(4)  Reboot. Your new SCSI disk will be /dev/sdc.

(5)  Using fdisk, partition it like /dev/sda, the one you are
replacing.

(6)  `mkfs` on each partition.
(7)  Mount each partition, one at a time, off from /mnt and tar
your files to it like:
	mount /dev/sdc1 /mnt	# will be /boot
	cd /boot
	tar -clf - . | (cd /mnt; tar -xvpf -)
	umount/mnt
Do this for each parition.

(8)  Shut down, and change the SCSI address of /dev/sdc to the address
of the drive you are replacing /dev/sda (change to device 0). Remove
the bad drive.

(9)  Reboot using the floppy.
(10) Execute lilo, this will make the new drive bootable.
(11) Un-comment /etc/fstab stuff.
(12) Execute mount -a

Done, remove floppy and reboot.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

        111,111,111 * 111,111,111 = 12,345,678,987,654,321