* RAID1 degraded
@ 2015-08-03 22:18 Hans Malissa
2015-08-03 23:48 ` Adam Goryachev
0 siblings, 1 reply; 8+ messages in thread
From: Hans Malissa @ 2015-08-03 22:18 UTC (permalink / raw)
To: linux-raid
Hi everybody,
It looks like one of my disks in my RAID1 just failed:
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc1[1](F) sdb1[0]
976629568 blocks super 1.2 [2/1] [U_]
unused devices: <none>
and
# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun May 17 15:21:30 2015
Raid Level : raid1
Array Size : 976629568 (931.39 GiB 1000.07 GB)
Used Dev Size : 976629568 (931.39 GiB 1000.07 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Mon Aug 3 16:13:56 2015
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Name : eprb21:0
UUID : 0901fe50:444a29b6:d3caff14:e45ef9cc
Events : 7619
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 0 0 1 removed
1 8 33 - faulty /dev/sdc1
Looks like there’s something wrong with /dev/sdc1:
# mdadm --examine /dev/sdb1
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 0901fe50:444a29b6:d3caff14:e45ef9cc
Name : eprb21:0
Creation Time : Sun May 17 15:21:30 2015
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 1953259520 (931.39 GiB 1000.07 GB)
Array Size : 976629568 (931.39 GiB 1000.07 GB)
Used Dev Size : 1953259136 (931.39 GiB 1000.07 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 3e6d5330:3ee6ef06:2acf46ad:44513d37
Update Time : Mon Aug 3 16:14:32 2015
Checksum : d474a441 - correct
Events : 7627
Device Role : Active device 0
Array State : A. ('A' == active, '.' == missing)
and
# mdadm --examine /dev/sdc1
mdadm: No md superblock detected on /dev/sdc1.
The file system seems to be ok for the time being:
# fsck -n /dev/md0
fsck from util-linux 2.21.2
e2fsck 1.42.6 (21-Sep-2012)
Warning! /dev/md0 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/dev/md0: clean, 218192/61046784 files, 213484777/244157392 blocks
Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
Best regards, and thanks a lot,
Hans--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: RAID1 degraded 2015-08-03 22:18 RAID1 degraded Hans Malissa @ 2015-08-03 23:48 ` Adam Goryachev 2015-08-04 4:16 ` Hans Malissa 0 siblings, 1 reply; 8+ messages in thread From: Adam Goryachev @ 2015-08-03 23:48 UTC (permalink / raw) To: Hans Malissa, linux-raid On 04/08/15 08:18, Hans Malissa wrote: > Hi everybody, > > It looks like one of my disks in my RAID1 just failed: > > [SNIP] > > Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data? > Have a look at dmesg or your system kernel logs for details. Also, use smartctl to examine what the drive itself thinks. Also, try to use dd to read/write the drive. One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RAID1 degraded 2015-08-03 23:48 ` Adam Goryachev @ 2015-08-04 4:16 ` Hans Malissa 2015-08-04 4:33 ` Adam Goryachev 0 siblings, 1 reply; 8+ messages in thread From: Hans Malissa @ 2015-08-04 4:16 UTC (permalink / raw) To: Adam Goryachev; +Cc: linux-raid Thanks a lot for your help! smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it. The kernel logs contain a lot of information; what should I be looking for? Thanks a lot, Hans # smartctl -a /dev/sdb smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.14 (AF) Device Model: ST1000DM003-1ER162 Serial Number: Z4Y6N2J3 LU WWN Device Id: 5 000c50 07afe5c18 Firmware Version: CC45 User Capacity: 1,000,204,886,016 bytes [1.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 7200 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Mon Aug 3 21:52:32 2015 MDT ==> WARNING: A firmware update for this drive may be available, see the following Seagate web pages: http://knowledge.seagate.com/articles/en_US/FAQ/207931en http://knowledge.seagate.com/articles/en_US/FAQ/223651en SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 80) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 105) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x1085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 111 100 006 Pre-fail Always - 39301104 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 20 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 063 060 030 Pre-fail Always - 2152462 9 Power_On_Hours 0x0032 098 098 000 Old_age Always - 1872 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 20 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 0 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 068 064 045 Old_age Always - 32 (Min/Max 26/35) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 0 193 Load_Cycle_Count 0x0032 093 093 000 Old_age Always - 15119 194 Temperature_Celsius 0x0022 032 040 000 Old_age Always - 32 (0 19 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 662h+04m+56.474s 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2212066311 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 4204083236 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. # smartctl -a /dev/sdc smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org Smartctl open device: /dev/sdc failed: No such device On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote: > On 04/08/15 08:18, Hans Malissa wrote: >> Hi everybody, >> >> It looks like one of my disks in my RAID1 just failed: >> >> [SNIP] >> >> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data? >> > Have a look at dmesg or your system kernel logs for details. > Also, use smartctl to examine what the drive itself thinks. > Also, try to use dd to read/write the drive. > > One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list > > Regards, > Adam > -- > Adam Goryachev Website Managers www.websitemanagers.com.au -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RAID1 degraded 2015-08-04 4:16 ` Hans Malissa @ 2015-08-04 4:33 ` Adam Goryachev 2015-08-04 5:55 ` Hans Malissa [not found] ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com> 0 siblings, 2 replies; 8+ messages in thread From: Adam Goryachev @ 2015-08-04 4:33 UTC (permalink / raw) To: Hans Malissa; +Cc: linux-raid On 04/08/15 14:16, Hans Malissa wrote: > Thanks a lot for your help! > smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it. > The kernel logs contain a lot of information; what should I be looking for? The logs should contain information on why or what happened when the disk (sdc) vanished. In your case, it does indeed look like sdc has failed, so you have a number of options depending on your preference: 1) Simply reboot (including a complete power off) the machine, and see if sdc comes back. If it does, do some tests, and then add back to the array. If it survives, then carry on as normal. 2) If you are more cautious (and more prepared to spend the money rather than risk the data), then purchase a replacement disk, and replace sdc with the new disk. Prepare the drive/partition, and add it to the raid array. Please make sure you "Research SCT/ERC on this list"!!! before purchasing the replacement drive. It is far better to buy the right drive if possible. Regards, Adam > Thanks a lot, > > Hans > > # smartctl -a /dev/sdb > smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) > Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org > > === START OF INFORMATION SECTION === > Model Family: Seagate Barracuda 7200.14 (AF) > Device Model: ST1000DM003-1ER162 > Serial Number: Z4Y6N2J3 > LU WWN Device Id: 5 000c50 07afe5c18 > Firmware Version: CC45 > User Capacity: 1,000,204,886,016 bytes [1.00 TB] > Sector Sizes: 512 bytes logical, 4096 bytes physical > Rotation Rate: 7200 rpm > Device is: In smartctl database [for details use: -P show] > ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b > SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) > Local Time is: Mon Aug 3 21:52:32 2015 MDT > > ==> WARNING: A firmware update for this drive may be available, > see the following Seagate web pages: > http://knowledge.seagate.com/articles/en_US/FAQ/207931en > http://knowledge.seagate.com/articles/en_US/FAQ/223651en > > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > > General SMART Values: > Offline data collection status: (0x00) Offline data collection activity > was never started. > Auto Offline Data Collection: Disabled. > Self-test execution status: ( 0) The previous self-test routine completed > without error or no self-test has ever > been run. > Total time to complete Offline > data collection: ( 80) seconds. > Offline data collection > capabilities: (0x73) SMART execute Offline immediate. > Auto Offline data collection on/off support. > Suspend Offline collection upon new > command. > No Offline surface scan supported. > Self-test supported. > Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > General Purpose Logging supported. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 105) minutes. > Conveyance self-test routine > recommended polling time: ( 2) minutes. > SCT capabilities: (0x1085) SCT Status supported. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 111 100 006 Pre-fail Always - 39301104 > 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 20 > 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 > 7 Seek_Error_Rate 0x000f 063 060 030 Pre-fail Always - 2152462 > 9 Power_On_Hours 0x0032 098 098 000 Old_age Always - 1872 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 > 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 20 > 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 > 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 > 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 > 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 0 0 > 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 > 190 Airflow_Temperature_Cel 0x0022 068 064 045 Old_age Always - 32 (Min/Max 26/35) > 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 > 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 0 > 193 Load_Cycle_Count 0x0032 093 093 000 Old_age Always - 15119 > 194 Temperature_Celsius 0x0022 032 040 000 Old_age Always - 32 (0 19 0 0 0) > 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 > 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 > 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 662h+04m+56.474s > 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2212066311 > 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 4204083236 > > SMART Error Log Version: 1 > No Errors Logged > > SMART Self-test log structure revision number 1 > No self-tests have been logged. [To run self-tests, use: smartctl -t] > > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. > > # smartctl -a /dev/sdc > smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) > Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org > > Smartctl open device: /dev/sdc failed: No such device > > On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote: > >> On 04/08/15 08:18, Hans Malissa wrote: >>> Hi everybody, >>> >>> It looks like one of my disks in my RAID1 just failed: >>> >>> [SNIP] >>> >>> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data? >>> >> Have a look at dmesg or your system kernel logs for details. >> Also, use smartctl to examine what the drive itself thinks. >> Also, try to use dd to read/write the drive. >> >> One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list >> >> Regards, >> Adam >> -- >> Adam Goryachev Website Managers www.websitemanagers.com.au -- Adam Goryachev Website Managers www.websitemanagers.com.au -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RAID1 degraded 2015-08-04 4:33 ` Adam Goryachev @ 2015-08-04 5:55 ` Hans Malissa 2015-08-04 14:38 ` Anthonys Lists 2015-08-04 15:56 ` Robert L Mathews [not found] ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com> 1 sibling, 2 replies; 8+ messages in thread From: Hans Malissa @ 2015-08-04 5:55 UTC (permalink / raw) To: Adam Goryachev; +Cc: linux-raid Thanks a lot for your help! Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found. So I will have to replace /dev/sdc. I tried to learn a bit about SRC/ERC from list archives, and it seems like my hard drives (1TB Seagate Barracuda’s) don’t support this option: # smartctl -l scterc /dev/sdb smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org SCT Error Recovery Control command not supported /dev/sdc is (was) of exactly the same type, so it wouldn’t support SRC/ERC either. This doesn’t seem to be the problem here, since the drive has just disappeared. But I will certainly take this into account when buying a replacement drive. Any current recommendations about what would work best in a RAID1 instead of a 1TB Seagate Barracuda? Just to make sure I understand correctly how to replace /dev/sdc and repair my RAID1, the steps to do would be: • Shutdown PC • Replace /dev/sdc • Restart computer • Partition the new /dev/sdc • Run # mdadm --manage /dev/md0 --add /dev/sdc1 • Wait for synchronization to finish Did I get this right? Am I missing anything? Are there additional steps (I am backing my data up, anyway) that I can take to maximize my chance for success? Thanks a lot, Hans On Aug 3, 2015, at 10:33 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote: > On 04/08/15 14:16, Hans Malissa wrote: >> Thanks a lot for your help! >> smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it. >> The kernel logs contain a lot of information; what should I be looking for? > > The logs should contain information on why or what happened when the disk (sdc) vanished. In your case, it does indeed look like sdc has failed, so you have a number of options depending on your preference: > 1) Simply reboot (including a complete power off) the machine, and see if sdc comes back. If it does, do some tests, and then add back to the array. If it survives, then carry on as normal. > > 2) If you are more cautious (and more prepared to spend the money rather than risk the data), then purchase a replacement disk, and replace sdc with the new disk. Prepare the drive/partition, and add it to the raid array. > > Please make sure you "Research SCT/ERC on this list"!!! before purchasing the replacement drive. It is far better to buy the right drive if possible. > > Regards, > Adam > >> Thanks a lot, >> >> Hans >> >> # smartctl -a /dev/sdb >> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) >> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org >> >> === START OF INFORMATION SECTION === >> Model Family: Seagate Barracuda 7200.14 (AF) >> Device Model: ST1000DM003-1ER162 >> Serial Number: Z4Y6N2J3 >> LU WWN Device Id: 5 000c50 07afe5c18 >> Firmware Version: CC45 >> User Capacity: 1,000,204,886,016 bytes [1.00 TB] >> Sector Sizes: 512 bytes logical, 4096 bytes physical >> Rotation Rate: 7200 rpm >> Device is: In smartctl database [for details use: -P show] >> ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b >> SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) >> Local Time is: Mon Aug 3 21:52:32 2015 MDT >> >> ==> WARNING: A firmware update for this drive may be available, >> see the following Seagate web pages: >> http://knowledge.seagate.com/articles/en_US/FAQ/207931en >> http://knowledge.seagate.com/articles/en_US/FAQ/223651en >> >> SMART support is: Available - device has SMART capability. >> SMART support is: Enabled >> >> === START OF READ SMART DATA SECTION === >> SMART overall-health self-assessment test result: PASSED >> >> General SMART Values: >> Offline data collection status: (0x00) Offline data collection activity >> was never started. >> Auto Offline Data Collection: Disabled. >> Self-test execution status: ( 0) The previous self-test routine completed >> without error or no self-test has ever >> been run. >> Total time to complete Offline >> data collection: ( 80) seconds. >> Offline data collection >> capabilities: (0x73) SMART execute Offline immediate. >> Auto Offline data collection on/off support. >> Suspend Offline collection upon new >> command. >> No Offline surface scan supported. >> Self-test supported. >> Conveyance Self-test supported. >> Selective Self-test supported. >> SMART capabilities: (0x0003) Saves SMART data before entering >> power-saving mode. >> Supports SMART auto save timer. >> Error logging capability: (0x01) Error logging supported. >> General Purpose Logging supported. >> Short self-test routine >> recommended polling time: ( 1) minutes. >> Extended self-test routine >> recommended polling time: ( 105) minutes. >> Conveyance self-test routine >> recommended polling time: ( 2) minutes. >> SCT capabilities: (0x1085) SCT Status supported. >> >> SMART Attributes Data Structure revision number: 10 >> Vendor Specific SMART Attributes with Thresholds: >> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE >> 1 Raw_Read_Error_Rate 0x000f 111 100 006 Pre-fail Always - 39301104 >> 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail Always - 0 >> 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 20 >> 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 >> 7 Seek_Error_Rate 0x000f 063 060 030 Pre-fail Always - 2152462 >> 9 Power_On_Hours 0x0032 098 098 000 Old_age Always - 1872 >> 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 >> 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 20 >> 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 >> 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 >> 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 >> 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 0 0 >> 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 >> 190 Airflow_Temperature_Cel 0x0022 068 064 045 Old_age Always - 32 (Min/Max 26/35) >> 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 >> 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 0 >> 193 Load_Cycle_Count 0x0032 093 093 000 Old_age Always - 15119 >> 194 Temperature_Celsius 0x0022 032 040 000 Old_age Always - 32 (0 19 0 0 0) >> 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 >> 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 >> 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 >> 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 662h+04m+56.474s >> 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2212066311 >> 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 4204083236 >> >> SMART Error Log Version: 1 >> No Errors Logged >> >> SMART Self-test log structure revision number 1 >> No self-tests have been logged. [To run self-tests, use: smartctl -t] >> >> >> SMART Selective self-test log data structure revision number 1 >> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS >> 1 0 0 Not_testing >> 2 0 0 Not_testing >> 3 0 0 Not_testing >> 4 0 0 Not_testing >> 5 0 0 Not_testing >> Selective self-test flags (0x0): >> After scanning selected spans, do NOT read-scan remainder of disk. >> If Selective self-test is pending on power-up, resume after 0 minute delay. >> >> # smartctl -a /dev/sdc >> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) >> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org >> >> Smartctl open device: /dev/sdc failed: No such device >> >> On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote: >> >>> On 04/08/15 08:18, Hans Malissa wrote: >>>> Hi everybody, >>>> >>>> It looks like one of my disks in my RAID1 just failed: >>>> >>>> [SNIP] >>>> >>>> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data? >>>> >>> Have a look at dmesg or your system kernel logs for details. >>> Also, use smartctl to examine what the drive itself thinks. >>> Also, try to use dd to read/write the drive. >>> >>> One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list >>> >>> Regards, >>> Adam >>> -- >>> Adam Goryachev Website Managers www.websitemanagers.com.au > > > -- > Adam Goryachev Website Managers www.websitemanagers.com.au -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RAID1 degraded 2015-08-04 5:55 ` Hans Malissa @ 2015-08-04 14:38 ` Anthonys Lists 2015-08-04 15:56 ` Robert L Mathews 1 sibling, 0 replies; 8+ messages in thread From: Anthonys Lists @ 2015-08-04 14:38 UTC (permalink / raw) To: Hans Malissa; +Cc: linux-raid On 04/08/2015 06:55, Hans Malissa wrote: > Thanks a lot for your help! > Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found. > So I will have to replace /dev/sdc. > I tried to learn a bit about SRC/ERC from list archives, and it seems like my hard drives (1TB Seagate Barracuda’s) don’t support this option: > > # smartctl -l scterc /dev/sdb > smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM) > Copyright (C) 2002-12, Bruce Allen, Christian Franke,www.smartmontools.org > > SCT Error Recovery Control command not supported I've got Seagate Barracudas (the 3TB version) and was a bit miffed to realise that they didn't support this. That said, with mirrored 1TB drives, you shouldn't have any real problems with SRC/ERC - the problem is that a drive that returns 1 error per 10TB read is within spec, if your array is over that size, then "good" drives could well cause a crashed array if you have any glitches. (It seems that most drives are well over spec, but why run the risk). If you're price-conscious, WD Reds are not much more expensive than Barracudas, and are advertised as "good for RAID". I'm looking at them for my next hard drives. Cheers, Wol -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RAID1 degraded 2015-08-04 5:55 ` Hans Malissa 2015-08-04 14:38 ` Anthonys Lists @ 2015-08-04 15:56 ` Robert L Mathews 1 sibling, 0 replies; 8+ messages in thread From: Robert L Mathews @ 2015-08-04 15:56 UTC (permalink / raw) To: linux-raid On 8/3/15 10:55 PM, Hans Malissa wrote: > Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found. Possible tip: I've seen several cases where rebooting was not enough to make a failed drive visible. In some cases, power actually needs to be removed from the server for a few seconds (or the drive needs to be removed and reinserted if it's hot-swap capable). That's because the drive software itself may be hung in a way that only gets fixed when power is removed from, then restored to, the drive. -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>]
* Re: RAID1 degraded [not found] ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com> @ 2015-08-04 6:02 ` Adam Goryachev 0 siblings, 0 replies; 8+ messages in thread From: Adam Goryachev @ 2015-08-04 6:02 UTC (permalink / raw) To: linux-raid On 04/08/15 15:51, Hans Malissa wrote: > Thanks a lot for your help! > Rebooting the system didn’t solve the problem, /dev/sdc is still > nowhere to be found. > So I will have to replace /dev/sdc. > I tried to learn a bit about SRC/ERC from list archives, and it seems > like my hard drives (1TB Seagate Barracuda’s) don’t support this option: > > # smartctl -l scterc /dev/sdb > smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE > RPM) > Copyright (C) 2002-12, Bruce Allen, Christian Franke, > www.smartmontools.org <http://www.smartmontools.org> > > SCT Error Recovery Control command not supported > > /dev/sdc is (was) of exactly the same type, so it wouldn’t support > SRC/ERC either. > This doesn’t seem to be the problem here, since the drive has just > disappeared. Nope that is true, but it is always good to learn about the issue before it becomes the problem. > But I will certainly take this into account when buying a replacement > drive. Any current recommendations about what would work best in a > RAID1 instead of a 1TB Seagate Barracuda? My personal preference was WD Black drives, or else Enterprise Black, but they were always a lot more expensive. I think WD Red are "RAID Certified" these days. (Note, I mostly use SSD now rather than any brand HDD, so not a lot of recent experiences). > Just to make sure I understand correctly how to replace /dev/sdc and > repair my RAID1, the steps to do would be: > > 1. Shutdown PC > 2. Replace /dev/sdc > 3. Restart computer > 4. Partition the new /dev/sdc > 5. Run # mdadm —manage /dev/md0 —add /dev/sdc1 > 6. Wait for synchronization to finish > > Did I get this right? Am I missing anything? Are there additional > steps (I am backing my data up, anyway) that I can take to maximize my > chance for success? Yep, all sounds good. Just make a note of the serial number for sdb before you shutdown, and ensure you are removing the correct drive. The good thing with RAID1 is that it is difficult to really screw it up, but backups are *always* a good idea :) Regards, Adam > Thanks a lot, > > Hans > > On Aug 3, 2015, at 10:33 PM, Adam Goryachev > <mailinglists@websitemanagers.com.au > <mailto:mailinglists@websitemanagers.com.au>> wrote: > >> On 04/08/15 14:16, Hans Malissa wrote: >>> Thanks a lot for your help! >>> smartctl yields the following information (details see below): >>> /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. >>> /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk >>> is physically present, but that’s about it. >>> The kernel logs contain a lot of information; what should I be >>> looking for? >> >> The logs should contain information on why or what happened when the >> disk (sdc) vanished. In your case, it does indeed look like sdc has >> failed, so you have a number of options depending on your preference: >> 1) Simply reboot (including a complete power off) the machine, and >> see if sdc comes back. If it does, do some tests, and then add back >> to the array. If it survives, then carry on as normal. >> >> 2) If you are more cautious (and more prepared to spend the money >> rather than risk the data), then purchase a replacement disk, and >> replace sdc with the new disk. Prepare the drive/partition, and add >> it to the raid array. >> >> Please make sure you "Research SCT/ERC on this list"!!! before >> purchasing the replacement drive. It is far better to buy the right >> drive if possible. >> >> Regards, >> Adam >> >>> Thanks a lot, >>> >>> Hans >>> >>> # smartctl -a /dev/sdb >>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] >>> (SUSE RPM) >>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, >>> www.smartmontools.org <http://www.smartmontools.org> >>> >>> === START OF INFORMATION SECTION === >>> Model Family: Seagate Barracuda 7200.14 (AF) >>> Device Model: ST1000DM003-1ER162 >>> Serial Number: Z4Y6N2J3 >>> LU WWN Device Id: 5 000c50 07afe5c18 >>> Firmware Version: CC45 >>> User Capacity: 1,000,204,886,016 bytes [1.00 TB] >>> Sector Sizes: 512 bytes logical, 4096 bytes physical >>> Rotation Rate: 7200 rpm >>> Device is: In smartctl database [for details use: -P show] >>> ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b >>> SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) >>> Local Time is: Mon Aug 3 21:52:32 2015 MDT >>> >>> ==> WARNING: A firmware update for this drive may be available, >>> see the following Seagate web pages: >>> http://knowledge.seagate.com/articles/en_US/FAQ/207931en >>> http://knowledge.seagate.com/articles/en_US/FAQ/223651en >>> >>> SMART support is: Available - device has SMART capability. >>> SMART support is: Enabled >>> >>> === START OF READ SMART DATA SECTION === >>> SMART overall-health self-assessment test result: PASSED >>> >>> General SMART Values: >>> Offline data collection status: (0x00) Offline data collection activity >>> was never started. >>> Auto Offline Data >>> Collection: Disabled. >>> Self-test execution status: ( 0) The previous self-test >>> routine completed >>> without error or no >>> self-test has ever >>> been run. >>> Total time to complete Offline >>> data collection: ( 80) seconds. >>> Offline data collection >>> capabilities: (0x73) SMART execute Offline immediate. >>> Auto Offline data collection >>> on/off support. >>> Suspend Offline collection >>> upon new >>> command. >>> No Offline surface scan >>> supported. >>> Self-test supported. >>> Conveyance Self-test supported. >>> Selective Self-test supported. >>> SMART capabilities: (0x0003) Saves SMART data before entering >>> power-saving mode. >>> Supports SMART auto save timer. >>> Error logging capability: (0x01) Error logging supported. >>> General Purpose Logging >>> supported. >>> Short self-test routine >>> recommended polling time: ( 1) minutes. >>> Extended self-test routine >>> recommended polling time: ( 105) minutes. >>> Conveyance self-test routine >>> recommended polling time: ( 2) minutes. >>> SCT capabilities: (0x1085) SCT Status supported. >>> >>> SMART Attributes Data Structure revision number: 10 >>> Vendor Specific SMART Attributes with Thresholds: >>> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE >>> UPDATED WHEN_FAILED RAW_VALUE >>> 1 Raw_Read_Error_Rate 0x000f 111 100 006 Pre-fail >>> Always - 39301104 >>> 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail >>> Always - 0 >>> 4 Start_Stop_Count 0x0032 100 100 020 Old_age >>> Always - 20 >>> 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail >>> Always - 0 >>> 7 Seek_Error_Rate 0x000f 063 060 030 Pre-fail >>> Always - 2152462 >>> 9 Power_On_Hours 0x0032 098 098 000 Old_age >>> Always - 1872 >>> 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail >>> Always - 0 >>> 12 Power_Cycle_Count 0x0032 100 100 020 Old_age >>> Always - 20 >>> 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age >>> Always - 0 >>> 184 End-to-End_Error 0x0032 100 100 099 Old_age >>> Always - 0 >>> 187 Reported_Uncorrect 0x0032 100 100 000 Old_age >>> Always - 0 >>> 188 Command_Timeout 0x0032 100 100 000 Old_age >>> Always - 0 0 0 >>> 189 High_Fly_Writes 0x003a 100 100 000 Old_age >>> Always - 0 >>> 190 Airflow_Temperature_Cel 0x0022 068 064 045 Old_age >>> Always - 32 (Min/Max 26/35) >>> 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age >>> Always - 0 >>> 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age >>> Always - 0 >>> 193 Load_Cycle_Count 0x0032 093 093 000 Old_age >>> Always - 15119 >>> 194 Temperature_Celsius 0x0022 032 040 000 Old_age >>> Always - 32 (0 19 0 0 0) >>> 197 Current_Pending_Sector 0x0012 100 100 000 Old_age >>> Always - 0 >>> 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age >>> Offline - 0 >>> 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age >>> Always - 0 >>> 240 Head_Flying_Hours 0x0000 100 253 000 Old_age >>> Offline - 662h+04m+56.474s >>> 241 Total_LBAs_Written 0x0000 100 253 000 Old_age >>> Offline - 2212066311 >>> 242 Total_LBAs_Read 0x0000 100 253 000 Old_age >>> Offline - 4204083236 >>> >>> SMART Error Log Version: 1 >>> No Errors Logged >>> >>> SMART Self-test log structure revision number 1 >>> No self-tests have been logged. [To run self-tests, use: smartctl -t] >>> >>> >>> SMART Selective self-test log data structure revision number 1 >>> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS >>> 1 0 0 Not_testing >>> 2 0 0 Not_testing >>> 3 0 0 Not_testing >>> 4 0 0 Not_testing >>> 5 0 0 Not_testing >>> Selective self-test flags (0x0): >>> After scanning selected spans, do NOT read-scan remainder of disk. >>> If Selective self-test is pending on power-up, resume after 0 minute >>> delay. >>> >>> # smartctl -a /dev/sdc >>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] >>> (SUSE RPM) >>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, >>> www.smartmontools.org >>> >>> Smartctl open device: /dev/sdc failed: No such device >>> >>> On Aug 3, 2015, at 5:48 PM, Adam Goryachev >>> <mailinglists@websitemanagers.com.au> wrote: >>> >>>> On 04/08/15 08:18, Hans Malissa wrote: >>>>> Hi everybody, >>>>> >>>>> It looks like one of my disks in my RAID1 just failed: >>>>> >>>>> [SNIP] >>>>> >>>>> Are there any other tests I could run in order to figure out >>>>> what’s going on? It looks like I will have to replace /dev/sdc1 >>>>> with a new hard drive. What is the correct procedure to do so >>>>> without loosing my data? >>>>> >>>> Have a look at dmesg or your system kernel logs for details. >>>> Also, use smartctl to examine what the drive itself thinks. >>>> Also, try to use dd to read/write the drive. >>>> >>>> One common scenario is that you haven't configured the timing for >>>> the drive correctly, and the drive is working perfectly, but didn't >>>> respond to the kernel quickly enough. Research SCT/ERC on this list >>>> >>>> Regards, >>>> Adam >>>> -- >>>> Adam Goryachev Website Managers www.websitemanagers.com.au >> >> >> -- >> Adam Goryachev Website Managerswww.websitemanagers.com.au >> <http://www.websitemanagers.com.au/> > -- Adam Goryachev Website Managers www.websitemanagers.com.au -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2015-08-04 15:56 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-08-03 22:18 RAID1 degraded Hans Malissa
2015-08-03 23:48 ` Adam Goryachev
2015-08-04 4:16 ` Hans Malissa
2015-08-04 4:33 ` Adam Goryachev
2015-08-04 5:55 ` Hans Malissa
2015-08-04 14:38 ` Anthonys Lists
2015-08-04 15:56 ` Robert L Mathews
[not found] ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>
2015-08-04 6:02 ` Adam Goryachev
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.