All of lore.kernel.org
 help / color / mirror / Atom feed
* RAID1 degraded
@ 2015-08-03 22:18 Hans Malissa
  2015-08-03 23:48 ` Adam Goryachev
  0 siblings, 1 reply; 8+ messages in thread
From: Hans Malissa @ 2015-08-03 22:18 UTC (permalink / raw)
  To: linux-raid

Hi everybody,

It looks like one of my disks in my RAID1 just failed:

 # cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdc1[1](F) sdb1[0]
      976629568 blocks super 1.2 [2/1] [U_]
      
unused devices: <none>

and

# mdadm --detail /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Sun May 17 15:21:30 2015
     Raid Level : raid1
     Array Size : 976629568 (931.39 GiB 1000.07 GB)
  Used Dev Size : 976629568 (931.39 GiB 1000.07 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Mon Aug  3 16:13:56 2015
          State : clean, degraded 
 Active Devices : 1
Working Devices : 1
 Failed Devices : 1
  Spare Devices : 0

           Name : eprb21:0
           UUID : 0901fe50:444a29b6:d3caff14:e45ef9cc
         Events : 7619

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       0        0        1      removed

       1       8       33        -      faulty   /dev/sdc1

Looks like there’s something wrong with /dev/sdc1:

# mdadm --examine /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 0901fe50:444a29b6:d3caff14:e45ef9cc
           Name : eprb21:0
  Creation Time : Sun May 17 15:21:30 2015
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 1953259520 (931.39 GiB 1000.07 GB)
     Array Size : 976629568 (931.39 GiB 1000.07 GB)
  Used Dev Size : 1953259136 (931.39 GiB 1000.07 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 3e6d5330:3ee6ef06:2acf46ad:44513d37

    Update Time : Mon Aug  3 16:14:32 2015
       Checksum : d474a441 - correct
         Events : 7627


   Device Role : Active device 0
   Array State : A. ('A' == active, '.' == missing)

and

# mdadm --examine /dev/sdc1
mdadm: No md superblock detected on /dev/sdc1.

The file system seems to be ok for the time being:

# fsck -n /dev/md0
fsck from util-linux 2.21.2
e2fsck 1.42.6 (21-Sep-2012)
Warning!  /dev/md0 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/dev/md0: clean, 218192/61046784 files, 213484777/244157392 blocks

Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
Best regards, and thanks a lot,

Hans--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-03 22:18 RAID1 degraded Hans Malissa
@ 2015-08-03 23:48 ` Adam Goryachev
  2015-08-04  4:16   ` Hans Malissa
  0 siblings, 1 reply; 8+ messages in thread
From: Adam Goryachev @ 2015-08-03 23:48 UTC (permalink / raw)
  To: Hans Malissa, linux-raid

On 04/08/15 08:18, Hans Malissa wrote:
> Hi everybody,
>
> It looks like one of my disks in my RAID1 just failed:
>
> [SNIP]
>
> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
>
Have a look at dmesg or your system kernel logs for details.
Also, use smartctl to examine what the drive itself thinks.
Also, try to use dd to read/write the drive.

One common scenario is that you haven't configured the timing for the 
drive correctly, and the drive is working perfectly, but didn't respond 
to the kernel quickly enough. Research SCT/ERC on this list

Regards,
Adam
-- 
Adam Goryachev Website Managers www.websitemanagers.com.au
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-03 23:48 ` Adam Goryachev
@ 2015-08-04  4:16   ` Hans Malissa
  2015-08-04  4:33     ` Adam Goryachev
  0 siblings, 1 reply; 8+ messages in thread
From: Hans Malissa @ 2015-08-04  4:16 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid

Thanks a lot for your help!
smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it.
The kernel logs contain a lot of information; what should I be looking for?
Thanks a lot,

Hans

# smartctl -a /dev/sdb
smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST1000DM003-1ER162
Serial Number:    Z4Y6N2J3
LU WWN Device Id: 5 000c50 07afe5c18
Firmware Version: CC45
User Capacity:    1,000,204,886,016 bytes [1.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Aug  3 21:52:32 2015 MDT

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (   80) seconds.
Offline data collection
capabilities:                    (0x73) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 105) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x1085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   111   100   006    Pre-fail  Always       -       39301104
  3 Spin_Up_Time            0x0003   097   097   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       20
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   063   060   030    Pre-fail  Always       -       2152462
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       1872
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       20
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   068   064   045    Old_age   Always       -       32 (Min/Max 26/35)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   093   093   000    Old_age   Always       -       15119
194 Temperature_Celsius     0x0022   032   040   000    Old_age   Always       -       32 (0 19 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       662h+04m+56.474s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       2212066311
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       4204083236

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

# smartctl -a /dev/sdc
smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org

Smartctl open device: /dev/sdc failed: No such device

On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote:

> On 04/08/15 08:18, Hans Malissa wrote:
>> Hi everybody,
>> 
>> It looks like one of my disks in my RAID1 just failed:
>> 
>> [SNIP]
>> 
>> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
>> 
> Have a look at dmesg or your system kernel logs for details.
> Also, use smartctl to examine what the drive itself thinks.
> Also, try to use dd to read/write the drive.
> 
> One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list
> 
> Regards,
> Adam
> -- 
> Adam Goryachev Website Managers www.websitemanagers.com.au

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-04  4:16   ` Hans Malissa
@ 2015-08-04  4:33     ` Adam Goryachev
  2015-08-04  5:55       ` Hans Malissa
       [not found]       ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>
  0 siblings, 2 replies; 8+ messages in thread
From: Adam Goryachev @ 2015-08-04  4:33 UTC (permalink / raw)
  To: Hans Malissa; +Cc: linux-raid

On 04/08/15 14:16, Hans Malissa wrote:
> Thanks a lot for your help!
> smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it.
> The kernel logs contain a lot of information; what should I be looking for?

The logs should contain information on why or what happened when the 
disk (sdc) vanished. In your case, it does indeed look like sdc has 
failed, so you have a number of options depending on your preference:
1) Simply reboot (including a complete power off) the machine, and see 
if sdc comes back. If it does, do some tests, and then add back to the 
array. If it survives, then carry on as normal.

2) If you are more cautious (and more prepared to spend the money rather 
than risk the data), then purchase a replacement disk, and replace sdc 
with the new disk. Prepare the drive/partition, and add it to the raid 
array.

Please make sure you "Research SCT/ERC on this list"!!! before 
purchasing the replacement drive. It is far better to buy the right 
drive if possible.

Regards,
Adam

> Thanks a lot,
>
> Hans
>
> # smartctl -a /dev/sdb
> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Model Family:     Seagate Barracuda 7200.14 (AF)
> Device Model:     ST1000DM003-1ER162
> Serial Number:    Z4Y6N2J3
> LU WWN Device Id: 5 000c50 07afe5c18
> Firmware Version: CC45
> User Capacity:    1,000,204,886,016 bytes [1.00 TB]
> Sector Sizes:     512 bytes logical, 4096 bytes physical
> Rotation Rate:    7200 rpm
> Device is:        In smartctl database [for details use: -P show]
> ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
> SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
> Local Time is:    Mon Aug  3 21:52:32 2015 MDT
>
> ==> WARNING: A firmware update for this drive may be available,
> see the following Seagate web pages:
> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
>
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
>
> General SMART Values:
> Offline data collection status:  (0x00) Offline data collection activity
>                                          was never started.
>                                          Auto Offline Data Collection: Disabled.
> Self-test execution status:      (   0) The previous self-test routine completed
>                                          without error or no self-test has ever
>                                          been run.
> Total time to complete Offline
> data collection:                (   80) seconds.
> Offline data collection
> capabilities:                    (0x73) SMART execute Offline immediate.
>                                          Auto Offline data collection on/off support.
>                                          Suspend Offline collection upon new
>                                          command.
>                                          No Offline surface scan supported.
>                                          Self-test supported.
>                                          Conveyance Self-test supported.
>                                          Selective Self-test supported.
> SMART capabilities:            (0x0003) Saves SMART data before entering
>                                          power-saving mode.
>                                          Supports SMART auto save timer.
> Error logging capability:        (0x01) Error logging supported.
>                                          General Purpose Logging supported.
> Short self-test routine
> recommended polling time:        (   1) minutes.
> Extended self-test routine
> recommended polling time:        ( 105) minutes.
> Conveyance self-test routine
> recommended polling time:        (   2) minutes.
> SCT capabilities:              (0x1085) SCT Status supported.
>
> SMART Attributes Data Structure revision number: 10
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
>    1 Raw_Read_Error_Rate     0x000f   111   100   006    Pre-fail  Always       -       39301104
>    3 Spin_Up_Time            0x0003   097   097   000    Pre-fail  Always       -       0
>    4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       20
>    5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
>    7 Seek_Error_Rate         0x000f   063   060   030    Pre-fail  Always       -       2152462
>    9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       1872
>   10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
>   12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       20
> 183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
> 184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
> 187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
> 188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
> 189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
> 190 Airflow_Temperature_Cel 0x0022   068   064   045    Old_age   Always       -       32 (Min/Max 26/35)
> 191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
> 192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
> 193 Load_Cycle_Count        0x0032   093   093   000    Old_age   Always       -       15119
> 194 Temperature_Celsius     0x0022   032   040   000    Old_age   Always       -       32 (0 19 0 0 0)
> 197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
> 198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
> 199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
> 240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       662h+04m+56.474s
> 241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       2212066311
> 242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       4204083236
>
> SMART Error Log Version: 1
> No Errors Logged
>
> SMART Self-test log structure revision number 1
> No self-tests have been logged.  [To run self-tests, use: smartctl -t]
>
>
> SMART Selective self-test log data structure revision number 1
>   SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
>      1        0        0  Not_testing
>      2        0        0  Not_testing
>      3        0        0  Not_testing
>      4        0        0  Not_testing
>      5        0        0  Not_testing
> Selective self-test flags (0x0):
>    After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
>
> # smartctl -a /dev/sdc
> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org
>
> Smartctl open device: /dev/sdc failed: No such device
>
> On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote:
>
>> On 04/08/15 08:18, Hans Malissa wrote:
>>> Hi everybody,
>>>
>>> It looks like one of my disks in my RAID1 just failed:
>>>
>>> [SNIP]
>>>
>>> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
>>>
>> Have a look at dmesg or your system kernel logs for details.
>> Also, use smartctl to examine what the drive itself thinks.
>> Also, try to use dd to read/write the drive.
>>
>> One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list
>>
>> Regards,
>> Adam
>> -- 
>> Adam Goryachev Website Managers www.websitemanagers.com.au


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-04  4:33     ` Adam Goryachev
@ 2015-08-04  5:55       ` Hans Malissa
  2015-08-04 14:38         ` Anthonys Lists
  2015-08-04 15:56         ` Robert L Mathews
       [not found]       ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>
  1 sibling, 2 replies; 8+ messages in thread
From: Hans Malissa @ 2015-08-04  5:55 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid

Thanks a lot for your help!
Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found. 
So I will have to replace /dev/sdc.
I tried to learn a bit about SRC/ERC from list archives, and it seems like my hard drives (1TB Seagate Barracuda’s) don’t support this option:

# smartctl -l scterc /dev/sdb
smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control command not supported

/dev/sdc is (was) of exactly the same type, so it wouldn’t support SRC/ERC either. 
This doesn’t seem to be the problem here, since the drive has just disappeared. 
But I will certainly take this into account when buying a replacement drive. Any current recommendations about what would work best in a RAID1 instead of a 1TB Seagate Barracuda?
Just to make sure I understand correctly how to replace /dev/sdc and repair my RAID1, the steps to do would be:
	• Shutdown PC
	• Replace /dev/sdc
	• Restart computer
	• Partition the new /dev/sdc
	• Run # mdadm --manage /dev/md0 --add /dev/sdc1
	• Wait for synchronization to finish
Did I get this right? Am I missing anything? Are there additional steps (I am backing my data up, anyway) that I can take to maximize my chance for success?
Thanks a lot,

Hans

On Aug 3, 2015, at 10:33 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote:

> On 04/08/15 14:16, Hans Malissa wrote:
>> Thanks a lot for your help!
>> smartctl yields the following information (details see below): /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk is physically present, but that’s about it.
>> The kernel logs contain a lot of information; what should I be looking for?
> 
> The logs should contain information on why or what happened when the disk (sdc) vanished. In your case, it does indeed look like sdc has failed, so you have a number of options depending on your preference:
> 1) Simply reboot (including a complete power off) the machine, and see if sdc comes back. If it does, do some tests, and then add back to the array. If it survives, then carry on as normal.
> 
> 2) If you are more cautious (and more prepared to spend the money rather than risk the data), then purchase a replacement disk, and replace sdc with the new disk. Prepare the drive/partition, and add it to the raid array.
> 
> Please make sure you "Research SCT/ERC on this list"!!! before purchasing the replacement drive. It is far better to buy the right drive if possible.
> 
> Regards,
> Adam
> 
>> Thanks a lot,
>> 
>> Hans
>> 
>> # smartctl -a /dev/sdb
>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org
>> 
>> === START OF INFORMATION SECTION ===
>> Model Family:     Seagate Barracuda 7200.14 (AF)
>> Device Model:     ST1000DM003-1ER162
>> Serial Number:    Z4Y6N2J3
>> LU WWN Device Id: 5 000c50 07afe5c18
>> Firmware Version: CC45
>> User Capacity:    1,000,204,886,016 bytes [1.00 TB]
>> Sector Sizes:     512 bytes logical, 4096 bytes physical
>> Rotation Rate:    7200 rpm
>> Device is:        In smartctl database [for details use: -P show]
>> ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
>> SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
>> Local Time is:    Mon Aug  3 21:52:32 2015 MDT
>> 
>> ==> WARNING: A firmware update for this drive may be available,
>> see the following Seagate web pages:
>> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
>> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
>> 
>> SMART support is: Available - device has SMART capability.
>> SMART support is: Enabled
>> 
>> === START OF READ SMART DATA SECTION ===
>> SMART overall-health self-assessment test result: PASSED
>> 
>> General SMART Values:
>> Offline data collection status:  (0x00) Offline data collection activity
>>                                         was never started.
>>                                         Auto Offline Data Collection: Disabled.
>> Self-test execution status:      (   0) The previous self-test routine completed
>>                                         without error or no self-test has ever
>>                                         been run.
>> Total time to complete Offline
>> data collection:                (   80) seconds.
>> Offline data collection
>> capabilities:                    (0x73) SMART execute Offline immediate.
>>                                         Auto Offline data collection on/off support.
>>                                         Suspend Offline collection upon new
>>                                         command.
>>                                         No Offline surface scan supported.
>>                                         Self-test supported.
>>                                         Conveyance Self-test supported.
>>                                         Selective Self-test supported.
>> SMART capabilities:            (0x0003) Saves SMART data before entering
>>                                         power-saving mode.
>>                                         Supports SMART auto save timer.
>> Error logging capability:        (0x01) Error logging supported.
>>                                         General Purpose Logging supported.
>> Short self-test routine
>> recommended polling time:        (   1) minutes.
>> Extended self-test routine
>> recommended polling time:        ( 105) minutes.
>> Conveyance self-test routine
>> recommended polling time:        (   2) minutes.
>> SCT capabilities:              (0x1085) SCT Status supported.
>> 
>> SMART Attributes Data Structure revision number: 10
>> Vendor Specific SMART Attributes with Thresholds:
>> ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED WHEN_FAILED RAW_VALUE
>>   1 Raw_Read_Error_Rate     0x000f   111   100   006    Pre-fail  Always       -       39301104
>>   3 Spin_Up_Time            0x0003   097   097   000    Pre-fail  Always       -       0
>>   4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       20
>>   5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
>>   7 Seek_Error_Rate         0x000f   063   060   030    Pre-fail  Always       -       2152462
>>   9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       1872
>>  10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
>>  12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       20
>> 183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
>> 184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
>> 187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
>> 188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
>> 189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
>> 190 Airflow_Temperature_Cel 0x0022   068   064   045    Old_age   Always       -       32 (Min/Max 26/35)
>> 191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
>> 192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
>> 193 Load_Cycle_Count        0x0032   093   093   000    Old_age   Always       -       15119
>> 194 Temperature_Celsius     0x0022   032   040   000    Old_age   Always       -       32 (0 19 0 0 0)
>> 197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
>> 198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
>> 199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
>> 240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       662h+04m+56.474s
>> 241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       2212066311
>> 242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       4204083236
>> 
>> SMART Error Log Version: 1
>> No Errors Logged
>> 
>> SMART Self-test log structure revision number 1
>> No self-tests have been logged.  [To run self-tests, use: smartctl -t]
>> 
>> 
>> SMART Selective self-test log data structure revision number 1
>>  SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
>>     1        0        0  Not_testing
>>     2        0        0  Not_testing
>>     3        0        0  Not_testing
>>     4        0        0  Not_testing
>>     5        0        0  Not_testing
>> Selective self-test flags (0x0):
>>   After scanning selected spans, do NOT read-scan remainder of disk.
>> If Selective self-test is pending on power-up, resume after 0 minute delay.
>> 
>> # smartctl -a /dev/sdc
>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org
>> 
>> Smartctl open device: /dev/sdc failed: No such device
>> 
>> On Aug 3, 2015, at 5:48 PM, Adam Goryachev <mailinglists@websitemanagers.com.au> wrote:
>> 
>>> On 04/08/15 08:18, Hans Malissa wrote:
>>>> Hi everybody,
>>>> 
>>>> It looks like one of my disks in my RAID1 just failed:
>>>> 
>>>> [SNIP]
>>>> 
>>>> Are there any other tests I could run in order to figure out what’s going on? It looks like I will have to replace /dev/sdc1 with a new hard drive. What is the correct procedure to do so without loosing my data?
>>>> 
>>> Have a look at dmesg or your system kernel logs for details.
>>> Also, use smartctl to examine what the drive itself thinks.
>>> Also, try to use dd to read/write the drive.
>>> 
>>> One common scenario is that you haven't configured the timing for the drive correctly, and the drive is working perfectly, but didn't respond to the kernel quickly enough. Research SCT/ERC on this list
>>> 
>>> Regards,
>>> Adam
>>> -- 
>>> Adam Goryachev Website Managers www.websitemanagers.com.au
> 
> 
> -- 
> Adam Goryachev Website Managers www.websitemanagers.com.au

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
       [not found]       ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>
@ 2015-08-04  6:02         ` Adam Goryachev
  0 siblings, 0 replies; 8+ messages in thread
From: Adam Goryachev @ 2015-08-04  6:02 UTC (permalink / raw)
  To: linux-raid

On 04/08/15 15:51, Hans Malissa wrote:
> Thanks a lot for your help!
> Rebooting the system didn’t solve the problem, /dev/sdc is still 
> nowhere to be found.
> So I will have to replace /dev/sdc.
> I tried to learn a bit about SRC/ERC from list archives, and it seems 
> like my hard drives (1TB Seagate Barracuda’s) don’t support this option:
>
> # smartctl -l scterc /dev/sdb
> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE 
> RPM)
> Copyright (C) 2002-12, Bruce Allen, Christian Franke, 
> www.smartmontools.org <http://www.smartmontools.org>
>
> SCT Error Recovery Control command not supported
>
> /dev/sdc is (was) of exactly the same type, so it wouldn’t support 
> SRC/ERC either.
> This doesn’t seem to be the problem here, since the drive has just 
> disappeared.

Nope that is true, but it is always good to learn about the issue before 
it becomes the problem.

> But I will certainly take this into account when buying a replacement 
> drive. Any current recommendations about what would work best in a 
> RAID1 instead of a 1TB Seagate Barracuda?

My personal preference was WD Black drives, or else Enterprise Black, 
but they were always a lot more expensive. I think WD Red are "RAID 
Certified" these days. (Note, I mostly use SSD now rather than any brand 
HDD, so not a lot of recent experiences).

> Just to make sure I understand correctly how to replace /dev/sdc and 
> repair my RAID1, the steps to do would be:
>
>  1. Shutdown PC
>  2. Replace /dev/sdc
>  3. Restart computer
>  4. Partition the new /dev/sdc
>  5. Run # mdadm —manage /dev/md0 —add /dev/sdc1
>  6. Wait for synchronization to finish
>
> Did I get this right? Am I missing anything? Are there additional 
> steps (I am backing my data up, anyway) that I can take to maximize my 
> chance for success?

Yep, all sounds good. Just make a note of the serial number for sdb 
before you shutdown, and ensure you are removing the correct drive. The 
good thing with RAID1 is that it is difficult to really screw it up, but 
backups are *always* a good idea :)

Regards,
Adam

> Thanks a lot,
>
> Hans
>
> On Aug 3, 2015, at 10:33 PM, Adam Goryachev 
> <mailinglists@websitemanagers.com.au 
> <mailto:mailinglists@websitemanagers.com.au>> wrote:
>
>> On 04/08/15 14:16, Hans Malissa wrote:
>>> Thanks a lot for your help!
>>> smartctl yields the following information (details see below): 
>>> /dev/sdb looks ok, but /dev/sdc seems to have quite a problem. 
>>> /dev/sdc seems nonexistent, it’s not even in /dev/ anymore. The disk 
>>> is physically present, but that’s about it.
>>> The kernel logs contain a lot of information; what should I be 
>>> looking for?
>>
>> The logs should contain information on why or what happened when the 
>> disk (sdc) vanished. In your case, it does indeed look like sdc has 
>> failed, so you have a number of options depending on your preference:
>> 1) Simply reboot (including a complete power off) the machine, and 
>> see if sdc comes back. If it does, do some tests, and then add back 
>> to the array. If it survives, then carry on as normal.
>>
>> 2) If you are more cautious (and more prepared to spend the money 
>> rather than risk the data), then purchase a replacement disk, and 
>> replace sdc with the new disk. Prepare the drive/partition, and add 
>> it to the raid array.
>>
>> Please make sure you "Research SCT/ERC on this list"!!! before 
>> purchasing the replacement drive. It is far better to buy the right 
>> drive if possible.
>>
>> Regards,
>> Adam
>>
>>> Thanks a lot,
>>>
>>> Hans
>>>
>>> # smartctl -a /dev/sdb
>>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] 
>>> (SUSE RPM)
>>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, 
>>> www.smartmontools.org <http://www.smartmontools.org>
>>>
>>> === START OF INFORMATION SECTION ===
>>> Model Family:     Seagate Barracuda 7200.14 (AF)
>>> Device Model:     ST1000DM003-1ER162
>>> Serial Number:    Z4Y6N2J3
>>> LU WWN Device Id: 5 000c50 07afe5c18
>>> Firmware Version: CC45
>>> User Capacity:    1,000,204,886,016 bytes [1.00 TB]
>>> Sector Sizes:     512 bytes logical, 4096 bytes physical
>>> Rotation Rate:    7200 rpm
>>> Device is:        In smartctl database [for details use: -P show]
>>> ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
>>> SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
>>> Local Time is:    Mon Aug  3 21:52:32 2015 MDT
>>>
>>> ==> WARNING: A firmware update for this drive may be available,
>>> see the following Seagate web pages:
>>> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
>>> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
>>>
>>> SMART support is: Available - device has SMART capability.
>>> SMART support is: Enabled
>>>
>>> === START OF READ SMART DATA SECTION ===
>>> SMART overall-health self-assessment test result: PASSED
>>>
>>> General SMART Values:
>>> Offline data collection status:  (0x00) Offline data collection activity
>>>                                         was never started.
>>>                                         Auto Offline Data 
>>> Collection: Disabled.
>>> Self-test execution status:      (   0) The previous self-test 
>>> routine completed
>>>                                         without error or no 
>>> self-test has ever
>>>                                         been run.
>>> Total time to complete Offline
>>> data collection:                (   80) seconds.
>>> Offline data collection
>>> capabilities:                    (0x73) SMART execute Offline immediate.
>>>                                         Auto Offline data collection 
>>> on/off support.
>>>                                         Suspend Offline collection 
>>> upon new
>>>                                         command.
>>>                                         No Offline surface scan 
>>> supported.
>>>                                         Self-test supported.
>>>                                         Conveyance Self-test supported.
>>>                                         Selective Self-test supported.
>>> SMART capabilities:            (0x0003) Saves SMART data before entering
>>>                                         power-saving mode.
>>>                                         Supports SMART auto save timer.
>>> Error logging capability:        (0x01) Error logging supported.
>>>                                         General Purpose Logging 
>>> supported.
>>> Short self-test routine
>>> recommended polling time:        (   1) minutes.
>>> Extended self-test routine
>>> recommended polling time:        ( 105) minutes.
>>> Conveyance self-test routine
>>> recommended polling time:        (   2) minutes.
>>> SCT capabilities:              (0x1085) SCT Status supported.
>>>
>>> SMART Attributes Data Structure revision number: 10
>>> Vendor Specific SMART Attributes with Thresholds:
>>> ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE 
>>>      UPDATED WHEN_FAILED RAW_VALUE
>>>   1 Raw_Read_Error_Rate     0x000f   111   100   006    Pre-fail 
>>>  Always       -       39301104
>>>   3 Spin_Up_Time            0x0003   097   097   000    Pre-fail 
>>>  Always       -       0
>>>   4 Start_Stop_Count        0x0032   100   100   020    Old_age 
>>>   Always       -       20
>>>   5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail 
>>>  Always       -       0
>>>   7 Seek_Error_Rate         0x000f   063   060   030    Pre-fail 
>>>  Always       -       2152462
>>>   9 Power_On_Hours          0x0032   098   098   000    Old_age 
>>>   Always       -       1872
>>>  10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail 
>>>  Always       -       0
>>>  12 Power_Cycle_Count       0x0032   100   100   020    Old_age 
>>>   Always       -       20
>>> 183 Runtime_Bad_Block       0x0032   100   100   000    Old_age 
>>>   Always       -       0
>>> 184 End-to-End_Error        0x0032   100   100   099    Old_age 
>>>   Always       -       0
>>> 187 Reported_Uncorrect      0x0032   100   100   000    Old_age 
>>>   Always       -       0
>>> 188 Command_Timeout         0x0032   100   100   000    Old_age 
>>>   Always       -       0 0 0
>>> 189 High_Fly_Writes         0x003a   100   100   000    Old_age 
>>>   Always       -       0
>>> 190 Airflow_Temperature_Cel 0x0022   068   064   045    Old_age 
>>>   Always       -       32 (Min/Max 26/35)
>>> 191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age 
>>>   Always       -       0
>>> 192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age 
>>>   Always       -       0
>>> 193 Load_Cycle_Count        0x0032   093   093   000    Old_age 
>>>   Always       -       15119
>>> 194 Temperature_Celsius     0x0022   032   040   000    Old_age 
>>>   Always       -       32 (0 19 0 0 0)
>>> 197 Current_Pending_Sector  0x0012   100   100   000    Old_age 
>>>   Always       -       0
>>> 198 Offline_Uncorrectable   0x0010   100   100   000    Old_age 
>>>   Offline      -       0
>>> 199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age 
>>>   Always       -       0
>>> 240 Head_Flying_Hours       0x0000   100   253   000    Old_age 
>>>   Offline      -       662h+04m+56.474s
>>> 241 Total_LBAs_Written      0x0000   100   253   000    Old_age 
>>>   Offline      -       2212066311
>>> 242 Total_LBAs_Read         0x0000   100   253   000    Old_age 
>>>   Offline      -       4204083236
>>>
>>> SMART Error Log Version: 1
>>> No Errors Logged
>>>
>>> SMART Self-test log structure revision number 1
>>> No self-tests have been logged.  [To run self-tests, use: smartctl -t]
>>>
>>>
>>> SMART Selective self-test log data structure revision number 1
>>>  SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
>>>     1        0        0  Not_testing
>>>     2        0        0  Not_testing
>>>     3        0        0  Not_testing
>>>     4        0        0  Not_testing
>>>     5        0        0  Not_testing
>>> Selective self-test flags (0x0):
>>>   After scanning selected spans, do NOT read-scan remainder of disk.
>>> If Selective self-test is pending on power-up, resume after 0 minute 
>>> delay.
>>>
>>> # smartctl -a /dev/sdc
>>> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] 
>>> (SUSE RPM)
>>> Copyright (C) 2002-12, Bruce Allen, Christian Franke, 
>>> www.smartmontools.org
>>>
>>> Smartctl open device: /dev/sdc failed: No such device
>>>
>>> On Aug 3, 2015, at 5:48 PM, Adam Goryachev 
>>> <mailinglists@websitemanagers.com.au> wrote:
>>>
>>>> On 04/08/15 08:18, Hans Malissa wrote:
>>>>> Hi everybody,
>>>>>
>>>>> It looks like one of my disks in my RAID1 just failed:
>>>>>
>>>>> [SNIP]
>>>>>
>>>>> Are there any other tests I could run in order to figure out 
>>>>> what’s going on? It looks like I will have to replace /dev/sdc1 
>>>>> with a new hard drive. What is the correct procedure to do so 
>>>>> without loosing my data?
>>>>>
>>>> Have a look at dmesg or your system kernel logs for details.
>>>> Also, use smartctl to examine what the drive itself thinks.
>>>> Also, try to use dd to read/write the drive.
>>>>
>>>> One common scenario is that you haven't configured the timing for 
>>>> the drive correctly, and the drive is working perfectly, but didn't 
>>>> respond to the kernel quickly enough. Research SCT/ERC on this list
>>>>
>>>> Regards,
>>>> Adam
>>>> --
>>>> Adam Goryachev Website Managers www.websitemanagers.com.au
>>
>>
>> --
>> Adam Goryachev Website Managerswww.websitemanagers.com.au 
>> <http://www.websitemanagers.com.au/>
>


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-04  5:55       ` Hans Malissa
@ 2015-08-04 14:38         ` Anthonys Lists
  2015-08-04 15:56         ` Robert L Mathews
  1 sibling, 0 replies; 8+ messages in thread
From: Anthonys Lists @ 2015-08-04 14:38 UTC (permalink / raw)
  To: Hans Malissa; +Cc: linux-raid

On 04/08/2015 06:55, Hans Malissa wrote:
> Thanks a lot for your help!
> Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found.
> So I will have to replace /dev/sdc.
> I tried to learn a bit about SRC/ERC from list archives, and it seems like my hard drives (1TB Seagate Barracuda’s) don’t support this option:
>
> # smartctl -l scterc /dev/sdb
> smartctl 6.0 2012-10-10 r3643 [x86_64-linux-3.7.10-1.45-desktop] (SUSE RPM)
> Copyright (C) 2002-12, Bruce Allen, Christian Franke,www.smartmontools.org
>
> SCT Error Recovery Control command not supported
I've got Seagate Barracudas (the 3TB version) and was a bit miffed to 
realise that they didn't support this. That said, with mirrored 1TB 
drives, you shouldn't have any real problems with SRC/ERC - the problem 
is that a drive that returns 1 error per 10TB read is within spec, if 
your array is over that size, then "good" drives could well cause a 
crashed array if you have any glitches. (It seems that most drives are 
well over spec, but why run the risk).

If you're price-conscious, WD Reds are not much more expensive than 
Barracudas, and are advertised as "good for RAID". I'm looking at them 
for my next hard drives.

Cheers,
Wol
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: RAID1 degraded
  2015-08-04  5:55       ` Hans Malissa
  2015-08-04 14:38         ` Anthonys Lists
@ 2015-08-04 15:56         ` Robert L Mathews
  1 sibling, 0 replies; 8+ messages in thread
From: Robert L Mathews @ 2015-08-04 15:56 UTC (permalink / raw)
  To: linux-raid

On 8/3/15 10:55 PM, Hans Malissa wrote:

> Rebooting the system didn’t solve the problem, /dev/sdc is still nowhere to be found. 

Possible tip: I've seen several cases where rebooting was not enough to
make a failed drive visible. In some cases, power actually needs to be
removed from the server for a few seconds (or the drive needs to be
removed and reinserted if it's hot-swap capable).

That's because the drive software itself may be hung in a way that only
gets fixed when power is removed from, then restored to, the drive.

-- 
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2015-08-04 15:56 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-08-03 22:18 RAID1 degraded Hans Malissa
2015-08-03 23:48 ` Adam Goryachev
2015-08-04  4:16   ` Hans Malissa
2015-08-04  4:33     ` Adam Goryachev
2015-08-04  5:55       ` Hans Malissa
2015-08-04 14:38         ` Anthonys Lists
2015-08-04 15:56         ` Robert L Mathews
     [not found]       ` <A5793A94-EC9B-4221-A420-9E39EF0ABEEC@me.com>
2015-08-04  6:02         ` Adam Goryachev

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.