* Re: Raid5 fails to assemble automatically
From: Rudy Zijlstra @ 2015-07-20 7:12 UTC (permalink / raw)
To: Marc Desharnais, linux-raid
In-Reply-To: <BLU437-SMTP566F1275E9035D33983D4CCE860@phx.gbl>
Hi Marc,
It seems they support SCT. But is SCT enabled?
The below snippet from one of my scripts should help you determine this:
for i in a b c d e f g h i j k l m n o p q r s t u v w x y z ; do
if [ -b /dev/sd$i ]; then
echo setting SCTERC for /dev/sd$i
if smartctl -l scterc /dev/sd$i | grep -q Disabled ; then
/usr/sbin/smartctl -l scterc,70,70 /dev/sd$i
else
if ! smartctl -l scterc /dev/sd$i | grep -q seconds ; then
echo 180 >/sys/block/sd$i/device/timeout
echo WARNING /dev/sd$i does not support SCTERC!!
fi
fi
fi
done
Note that some drives claim to support, until you tell them to use it...
If you want to know why this is important, please search for SCT support
on this list. It has been discussed many times.
Other then that, the drives seem OK. I do note you are using desktop
drives in RAID, which is not recommended.
Cheers
Rudy
Op 20-07-15 om 01:40 schreef Marc Desharnais:
> Hi Rudy,
>
> I was able to assemble the raid by stopping the array first, and
> forcing the assemble.
> I backed up most of my data now and I can breathe :)
>
> Now I want to know which drive I should replace.
>
> Here's what I got from dmesg :
>
> [ 0.284287] scsi host0: pata_marvell
> [ 0.284404] scsi host1: pata_marvell
> [ 0.284417] sata_promise 0000:07:02.0: version 2.12
> [ 0.284453] ata1: PATA max UDMA/100 cmd 0x2018 ctl 0x2024 bmdma
> 0x2000 irq 17
> [ 0.284455] ata2: DUMMY
> [ 0.285013] scsi host2: sata_promise
> [ 0.285089] scsi host3: sata_promise
> [ 0.285162] scsi host4: sata_promise
> [ 0.285238] scsi host5: sata_promise
> [ 0.285285] ata3: SATA max UDMA/133 mmio m4096@0xe3124000 ata
> 0xe3124380 irq 18
> [ 0.285288] ata4: SATA max UDMA/133 mmio m4096@0xe3124000 ata
> 0xe3124280 irq 18
> [ 0.285290] ata5: SATA max UDMA/133 mmio m4096@0xe3124000 ata
> 0xe3124200 irq 18
> [ 0.285293] ata6: SATA max UDMA/133 mmio m4096@0xe3124000 ata
> 0xe3124300 irq 18
> [ 0.308447] ata_piix 0000:00:1f.2: version 2.13
> [ 0.308556] ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
> [ 0.308956] scsi host6: ata_piix
> [ 0.309047] scsi host7: ata_piix
> [ 0.309101] ata7: SATA max UDMA/133 cmd 0x4458 ctl 0x446c bmdma
> 0x4430 irq 21
> [ 0.309105] ata8: SATA max UDMA/133 cmd 0x4450 ctl 0x4468 bmdma
> 0x4438 irq 21
> [ 0.309221] ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
> [ 0.309583] scsi host8: ata_piix
> [ 0.309672] scsi host9: ata_piix
> [ 0.309722] ata9: SATA max UDMA/133 cmd 0x4448 ctl 0x4464 bmdma
> 0x4410 irq 21
> [ 0.309725] ata10: SATA max UDMA/133 cmd 0x4440 ctl 0x4460 bmdma
> 0x4418 irq 21
> [ 0.637315] ata9: SATA link down (SStatus 0 SControl 300)
> [ 0.763382] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [ 0.776449] ata3.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max
> UDMA/133
> [ 0.776453] ata3.00: 976773168 sectors, multi 0: LBA48 NCQ (depth
> 0/32)
> [ 0.787974] ata3.00: configured for UDMA/133
> [ 0.793390] ata10: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [ 0.800314] ata10.00: ATA-8: ST3500418AS, CC38, max UDMA/133
> [ 0.800317] ata10.00: 976773168 sectors, multi 16: LBA48 NCQ (depth
> 0/32)
> [ 0.813769] ata10.00: configured for UDMA/133
> [ 0.960644] ata7.00: SATA link down (SStatus 0 SControl 300)
> [ 0.960655] ata7.01: SATA link down (SStatus 0 SControl 300)
> [ 1.113401] ata8.00: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [ 1.113413] ata8.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [ 1.167224] ata8.00: ATA-7: ST3320620AS, 3.AAD, max UDMA/133
> [ 1.167228] ata8.00: 625142448 sectors, multi 16: LBA48 NCQ (depth
> 0/32)
> [ 1.186876] ata8.01: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max
> UDMA/133
> [ 1.186879] ata8.01: 976773168 sectors, multi 16: LBA48 NCQ (depth
> 0/32)
> [ 1.233838] ata8.00: configured for UDMA/133
> [ 1.247117] ata8.01: configured for UDMA/133
> [ 1.260050] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> [ 1.284063] ata4.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max
> UDMA/133
> [ 1.284066] ata4.00: 976773168 sectors, multi 0: LBA48 NCQ (depth
> 0/32)
> [ 1.297703] ata4.00: configured for UDMA/133
> [ 1.616704] ata5: SATA link down (SStatus 0 SControl 300)
> [ 1.936703] ata6: SATA link down (SStatus 0 SControl 300)
> [ 3.788841] EXT4-fs (sdc3): mounted filesystem with ordered data
> mode. Opts: (null)
> [ 6.250846] EXT4-fs (sdc3): re-mounted. Opts: data=ordered
> [ 16.065124] REISERFS (device sdc6): using ordered data mode
> [ 16.157157] REISERFS (device sdc7): using ordered data mode
> [ 2654.314204] md: md0 stopped.
> [ 2654.314211] md: unbind<sdd>
> [ 2654.349758] md: export_rdev(sdd)
> [ 2654.349787] md: unbind<sdb>
> [ 2654.376421] md: export_rdev(sdb)
> [ 2654.376465] md: unbind<sde>
> [ 2654.403083] md: export_rdev(sde)
> [ 2654.403121] md: unbind<sda>
> [ 2654.429751] md: export_rdev(sda)
> [ 2683.902046] md: md0 stopped.
> [ 2683.906627] sda: unknown partition table
> [ 2683.908554] sdb: unknown partition table
> [ 2683.909356] sde: unknown partition table
> [ 2683.945959] md: bind<sdb>
> [ 2683.946159] md: bind<sda>
> [ 2683.946360] md: bind<sde>
> [ 2683.946386] md: md0 stopped.
> [ 2683.946390] md: unbind<sde>
> [ 2683.966426] md: export_rdev(sde)
> [ 2683.966431] md: unbind<sda>
> [ 2683.999745] md: export_rdev(sda)
> [ 2683.999749] md: unbind<sdb>
> [ 2684.019745] md: export_rdev(sdb)
> [ 2684.040292] md: bind<sde>
> [ 2684.041464] md: bind<sdb>
> [ 2684.042870] md: bind<sda>
> [ 2684.153079] raid6: sse2x1 4293 MB/s
> [ 2684.209756] raid6: sse2x2 4367 MB/s
> [ 2684.266411] raid6: sse2x4 6959 MB/s
> [ 2684.266412] raid6: using algorithm sse2x4 (6959 MB/s)
> [ 2684.266413] raid6: using ssse3x2 recovery algorithm
> [ 2684.274830] async_tx: api initialized (async)
> [ 2684.275378] xor: measuring software checksum speed
> [ 2684.306408] prefetch64-sse: 10915.200 MB/sec
> [ 2684.339741] generic_sse: 8464.800 MB/sec
> [ 2684.339742] xor: using function: prefetch64-sse (10915.200 MB/sec)
> [ 2684.371389] md: raid6 personality registered for level 6
> [ 2684.371389] md: raid5 personality registered for level 5
> [ 2684.371390] md: raid4 personality registered for level 4
> [ 2684.371740] md/raid:md0: device sda operational as raid disk 3
> [ 2684.371741] md/raid:md0: device sdb operational as raid disk 2
> [ 2684.371742] md/raid:md0: device sde operational as raid disk 0
> [ 2684.372092] md/raid:md0: allocated 0kB
> [ 2684.372115] md/raid:md0: raid level 5 active with 3 out of 4
> devices, algorithm 2
> [ 2684.372116] RAID conf printout:
> [ 2684.372117] --- level:5 rd:4 wd:3
> [ 2684.372117] disk 0, o:1, dev:sde
> [ 2684.372118] disk 2, o:1, dev:sdb
> [ 2684.372119] disk 3, o:1, dev:sda
> [ 2684.372149] md0: detected capacity change from 0 to 1500323119104
> [ 2684.372426] md0: unknown partition table
> [ 2934.325679] SGI XFS with ACLs, security attributes, realtime, no
> debug enabled
> [ 3090.249640] XFS (md0): Mounting V4 Filesystem
> [ 3090.403449] XFS (md0): Starting recovery (logdev: internal)
> [ 3090.445679] XFS (md0): Ending recovery (logdev: internal)
>
>
>
> cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md0 : active raid5 sda[3] sdb[2] sde[0]
> 1465159296 blocks level 5, 128k chunk, algorithm 2 [4/3] [U_UU]
>
> unused devices: <none>
>
>
>
> Here's the smart info
>
>
> for /dev/sda :
>
> smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
> Copyright (C) 2002-15, Bruce Allen, Christian Franke,
> www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Model Family: Western Digital Caviar Blue (SATA)
> Device Model: WDC WD5000AAKS-00TMA0
> Serial Number: WD-WCAPW1796747
> LU WWN Device Id: 5 0014ee 2003d9fca
> Firmware Version: 12.01C01
> User Capacity: 500,107,862,016 bytes [500 GB]
> Sector Size: 512 bytes logical/physical
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
> Local Time is: Sun Jul 19 15:01:42 2015 EDT
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
>
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection
> activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test
> routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (13200) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before
> entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 2) minutes.
> Extended self-test routine
> recommended polling time: ( 164) minutes.
> Conveyance self-test routine
> recommended polling time: ( 6) minutes.
> SCT capabilities: (0x303f) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
>
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
> WHEN_FAILED RAW_VALUE
> 1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
> Always - 0
> 3 Spin_Up_Time 0x0003 221 164 021 Pre-fail
> Always - 3950
> 4 Start_Stop_Count 0x0032 100 100 000 Old_age
> Always - 350
> 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
> Always - 0
> 7 Seek_Error_Rate 0x000e 200 200 051 Old_age
> Always - 0
> 9 Power_On_Hours 0x0032 024 024 000 Old_age
> Always - 55688
> 10 Spin_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 12 Power_Cycle_Count 0x0032 100 100 000 Old_age
> Always - 344
> 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
> Always - 253
> 193 Load_Cycle_Count 0x0032 200 200 000 Old_age
> Always - 364
> 194 Temperature_Celsius 0x0022 117 089 000 Old_age
> Always - 33
> 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
> Always - 0
> 197 Current_Pending_Sector 0x0012 200 200 000 Old_age
> Always - 0
> 198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
> Offline - 0
> 199 UDMA_CRC_Error_Count 0x003e 200 195 000 Old_age
> Always - 12
> 200 Multi_Zone_Error_Rate 0x0008 200 189 051 Old_age
> Offline - 0
>
> SMART Error Log Version: 1
> No Errors Logged
>
> SMART Self-test log structure revision number 1
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
>
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute
> delay.
>
>
>
> for /dev/sdb :
>
> smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
> Copyright (C) 2002-15, Bruce Allen, Christian Franke,
> www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Model Family: Western Digital Caviar Blue (SATA)
> Device Model: WDC WD5000AAKS-00TMA0
> Serial Number: WD-WCAPW1120688
> LU WWN Device Id: 5 0014ee 2aaddb938
> Firmware Version: 12.01C01
> User Capacity: 500,107,862,016 bytes [500 GB]
> Sector Size: 512 bytes logical/physical
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
> Local Time is: Sun Jul 19 15:01:46 2015 EDT
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
>
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection
> activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test
> routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (12600) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before
> entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 2) minutes.
> Extended self-test routine
> recommended polling time: ( 157) minutes.
> Conveyance self-test routine
> recommended polling time: ( 6) minutes.
> SCT capabilities: (0x303f) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
>
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
> WHEN_FAILED RAW_VALUE
> 1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
> Always - 0
> 3 Spin_Up_Time 0x0003 172 165 021 Pre-fail
> Always - 6383
> 4 Start_Stop_Count 0x0032 100 100 000 Old_age
> Always - 346
> 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
> Always - 0
> 7 Seek_Error_Rate 0x000e 200 200 051 Old_age
> Always - 0
> 9 Power_On_Hours 0x0032 025 025 000 Old_age
> Always - 55369
> 10 Spin_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 12 Power_Cycle_Count 0x0032 100 100 000 Old_age
> Always - 340
> 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
> Always - 231
> 193 Load_Cycle_Count 0x0032 200 200 000 Old_age
> Always - 356
> 194 Temperature_Celsius 0x0022 119 085 000 Old_age
> Always - 31
> 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
> Always - 0
> 197 Current_Pending_Sector 0x0012 200 200 000 Old_age
> Always - 0
> 198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
> Offline - 0
> 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
> Always - 0
> 200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age
> Offline - 0
>
> SMART Error Log Version: 1
> No Errors Logged
>
> SMART Self-test log structure revision number 1
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
>
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute
> delay.
>
>
>
> for /dev/sdd :
>
> smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
> Copyright (C) 2002-15, Bruce Allen, Christian Franke,
> www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Model Family: Western Digital Caviar Blue (SATA)
> Device Model: WDC WD5000AAKS-00TMA0
> Serial Number: WD-WCAPW1122956
> LU WWN Device Id: 5 0014ee 200331314
> Firmware Version: 12.01C01
> User Capacity: 500,107,862,016 bytes [500 GB]
> Sector Size: 512 bytes logical/physical
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
> Local Time is: Sun Jul 19 15:01:49 2015 EDT
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
>
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection
> activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test
> routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (12000) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before
> entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 2) minutes.
> Extended self-test routine
> recommended polling time: ( 150) minutes.
> Conveyance self-test routine
> recommended polling time: ( 6) minutes.
> SCT capabilities: (0x303f) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
>
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
> WHEN_FAILED RAW_VALUE
> 1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
> Always - 0
> 3 Spin_Up_Time 0x0003 171 165 021 Pre-fail
> Always - 6425
> 4 Start_Stop_Count 0x0032 100 100 000 Old_age
> Always - 482
> 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
> Always - 0
> 7 Seek_Error_Rate 0x000e 200 200 051 Old_age
> Always - 0
> 9 Power_On_Hours 0x0032 022 022 000 Old_age
> Always - 57661
> 10 Spin_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
> Always - 0
> 12 Power_Cycle_Count 0x0032 100 100 000 Old_age
> Always - 467
> 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
> Always - 528
> 193 Load_Cycle_Count 0x0032 200 200 000 Old_age
> Always - 662
> 194 Temperature_Celsius 0x0022 118 086 000 Old_age
> Always - 32
> 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
> Always - 0
> 197 Current_Pending_Sector 0x0012 200 200 000 Old_age
> Always - 0
> 198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
> Offline - 0
> 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
> Always - 0
> 200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age
> Offline - 0
>
> SMART Error Log Version: 1
> No Errors Logged
>
> SMART Self-test log structure revision number 1
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
>
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute
> delay.
>
>
>
> for /dev/sde :
>
>
> smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
> Copyright (C) 2002-15, Bruce Allen, Christian Franke,
> www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Model Family: Seagate Barracuda 7200.12
> Device Model: ST3500418AS
> Serial Number: 5VM49NSD
> LU WWN Device Id: 5 000c50 01c4cf9ab
> Firmware Version: CC38
> User Capacity: 500,107,862,016 bytes [500 GB]
> Sector Size: 512 bytes logical/physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 2.6, 3.0 Gb/s
> Local Time is: Sun Jul 19 15:01:53 2015 EDT
>
> ==> WARNING: A firmware update for this drive may be available,
> see the following Seagate web pages:
> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
> http://knowledge.seagate.com/articles/en_US/FAQ/213891en
>
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
>
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
>
> General SMART Values:
> Offline data collection status: (0x82) Offline data collection
> activity
> was completed without error.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test
> routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: ( 600) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before
> entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 87) minutes.
> Conveyance self-test routine
> recommended polling time: ( 2) minutes.
> SCT capabilities: (0x103f) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
>
> SMART Attributes Data Structure revision number: 10
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
> WHEN_FAILED RAW_VALUE
> 1 Raw_Read_Error_Rate 0x000f 112 099 006 Pre-fail
> Always - 49016311
> 3 Spin_Up_Time 0x0003 097 097 000 Pre-fail
> Always - 0
> 4 Start_Stop_Count 0x0032 100 100 020 Old_age
> Always - 398
> 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
> Always - 0
> 7 Seek_Error_Rate 0x000f 069 060 030 Pre-fail
> Always - 7542794
> 9 Power_On_Hours 0x0032 047 047 000 Old_age
> Always - 46448
> 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail
> Always - 0
> 12 Power_Cycle_Count 0x0032 100 100 020 Old_age
> Always - 199
> 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age
> Always - 0
> 184 End-to-End_Error 0x0032 100 100 099 Old_age
> Always - 0
> 187 Reported_Uncorrect 0x0032 100 100 000 Old_age
> Always - 0
> 188 Command_Timeout 0x0032 100 100 000 Old_age
> Always - 0
> 189 High_Fly_Writes 0x003a 100 100 000 Old_age
> Always - 0
> 190 Airflow_Temperature_Cel 0x0022 064 059 045 Old_age
> Always - 36 (Min/Max 36/39)
> 194 Temperature_Celsius 0x0022 036 041 000 Old_age
> Always - 36 (0 17 0 0 0)
> 195 Hardware_ECC_Recovered 0x001a 016 015 000 Old_age
> Always - 49016311
> 197 Current_Pending_Sector 0x0012 100 100 000 Old_age
> Always - 0
> 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age
> Offline - 0
> 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
> Always - 0
> 240 Head_Flying_Hours 0x0000 100 253 000 Old_age
> Offline - 46876 (84 165 0)
> 241 Total_LBAs_Written 0x0000 100 253 000 Old_age
> Offline - 166857433
> 242 Total_LBAs_Read 0x0000 100 253 000 Old_age
> Offline - 3494978241
>
> SMART Error Log Version: 1
> No Errors Logged
>
> SMART Self-test log structure revision number 1
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
>
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute
> delay.
>
>
>
>
> I'm not sure what to think of this. Which drive is really dead? Or
> were they simply synched but still usable?
>
> Let me know if more information could help you to help me :)
>
>
> Thanks!
>
>
>
>
> Le 2015-07-19 11:09, Rudy Zijlstra a écrit :
>> Hi Marc,
>>
>> On 19-07-15 16:26, Marc Desharnais wrote:
>>> Hi,
>>>
>>> I'm new to this mailing list but I subscribed because of this
>>> problem with my raid5.
>>> I really don't want to mess this up and it was clearly suggested to
>>> not rush things on the wiki. So I'm looking for help :)
>>> My raid has been running great for a long time now but I think it
>>> needs so serious maintenance.
>>> I have a 4 disks raid 5 on ArchLinux, mdadm version 3.3.1.
>>>
>>> First, the raid status :
>>>
>>> /dev/sda:
>>>
>>> 1 1 0 0 1 faulty removed
>>>
>>> /dev/sdb:
>>>
>>> 1 1 0 0 1 faulty removed
>>> 3 3 0 0 3 faulty removed
>>> /dev/sdd:
>>>
>>> Number Major Minor RaidDevice State
>>> this 1 8 16 1 active sync /dev/sdb
>>>
>>> 0 0 8 32 0 active sync /dev/sdc
>>> 1 1 8 16 1 active sync /dev/sdb
>>> 2 2 8 48 2 active sync /dev/sdd
>>> 3 3 8 64 3 active sync /dev/sde
>>> /dev/sde:
>>>
>>>
>>> Number Major Minor RaidDevice State
>>> this 0 8 64 0 active sync /dev/sde
>>>
>>> 0 0 8 64 0 active sync /dev/sde
>>> 1 1 0 0 1 faulty removed
>>> 2 2 8 16 2 active sync /dev/sdb
>>> 3 3 0 0 3 faulty removed
>>>
>>>
>>>
>>> Obvisously the event count is wrong on 2 disks. One disk has been
>>> gone for a long time now...
>>>
>>> 1 - First question : how can I safely assemble the raid to the
>>> latest sync state? I don't want to mess this up.
>>> 2 - Does this mean my hard drives are dead, or simply desync?
>>>
>>>
>>> Also I tried to force assemble :
>>> mdadm --assemble --force /dev/md0 /dev/sda /dev/sdb /dev/sde
>>>
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> First:
>> - what information is in dmesg or kern.log, or whererever your distro
>> stores disk messages? Asking only for the relevant parts about the
>> disks :)
>> - Do you have any idea when the disks were kicked out?
>> - can you give the output of smartctl -a for each disk?
>> - output of cat /proc/mdstat?
>>
>> The assemble you tried is probably failing because the raid is
>> assembled but not started. Doing a "mdadm -S /dev/md0" should take
>> care of that.
>>
>> Cheers
>>
>> Rudy
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-20 0:03 UTC (permalink / raw)
To: John Stoffel
Cc: Shaohua Li, linux-raid, songliubraving, hch, dan.j.williams,
Kernel-team
In-Reply-To: <21927.51382.8600.700720@quad.stoffel.home>
On Thu, 16 Jul 2015 11:07:34 -0400 "John Stoffel" <john@stoffel.org>
wrote:
>
> So.... how will wrapping of sequence numbers be handled in the cache
> metadata on disk? They can't grow for ever...
Can't they?
A 64bit number incremented every millisecond can grow for half a
billion years. As far as I'm concerned, that is forever.
But even if we used 32bit numbers, it is very easy to handle looping
providing you have few than half of the numbers in use at any time.
So as long as we don't have more than 2 billion active transactions in
the log at once, there is no problem.
The kernel does some internal time keeping using "jiffies" which, on
32bit systems, cycle about every fortnight. A small amount of care is
enough that this doesn't cause problem.
NeilBrown
^ permalink raw reply
* Re: Raid5 fails to assemble automatically
From: Marc Desharnais @ 2015-07-19 23:40 UTC (permalink / raw)
To: Rudy Zijlstra, linux-raid
In-Reply-To: <55ABBDC3.2070305@grumpydevil.homelinux.org>
Hi Rudy,
I was able to assemble the raid by stopping the array first, and forcing
the assemble.
I backed up most of my data now and I can breathe :)
Now I want to know which drive I should replace.
Here's what I got from dmesg :
[ 0.284287] scsi host0: pata_marvell
[ 0.284404] scsi host1: pata_marvell
[ 0.284417] sata_promise 0000:07:02.0: version 2.12
[ 0.284453] ata1: PATA max UDMA/100 cmd 0x2018 ctl 0x2024 bmdma
0x2000 irq 17
[ 0.284455] ata2: DUMMY
[ 0.285013] scsi host2: sata_promise
[ 0.285089] scsi host3: sata_promise
[ 0.285162] scsi host4: sata_promise
[ 0.285238] scsi host5: sata_promise
[ 0.285285] ata3: SATA max UDMA/133 mmio m4096@0xe3124000 ata
0xe3124380 irq 18
[ 0.285288] ata4: SATA max UDMA/133 mmio m4096@0xe3124000 ata
0xe3124280 irq 18
[ 0.285290] ata5: SATA max UDMA/133 mmio m4096@0xe3124000 ata
0xe3124200 irq 18
[ 0.285293] ata6: SATA max UDMA/133 mmio m4096@0xe3124000 ata
0xe3124300 irq 18
[ 0.308447] ata_piix 0000:00:1f.2: version 2.13
[ 0.308556] ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
[ 0.308956] scsi host6: ata_piix
[ 0.309047] scsi host7: ata_piix
[ 0.309101] ata7: SATA max UDMA/133 cmd 0x4458 ctl 0x446c bmdma
0x4430 irq 21
[ 0.309105] ata8: SATA max UDMA/133 cmd 0x4450 ctl 0x4468 bmdma
0x4438 irq 21
[ 0.309221] ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
[ 0.309583] scsi host8: ata_piix
[ 0.309672] scsi host9: ata_piix
[ 0.309722] ata9: SATA max UDMA/133 cmd 0x4448 ctl 0x4464 bmdma
0x4410 irq 21
[ 0.309725] ata10: SATA max UDMA/133 cmd 0x4440 ctl 0x4460 bmdma
0x4418 irq 21
[ 0.637315] ata9: SATA link down (SStatus 0 SControl 300)
[ 0.763382] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 0.776449] ata3.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133
[ 0.776453] ata3.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 0/32)
[ 0.787974] ata3.00: configured for UDMA/133
[ 0.793390] ata10: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 0.800314] ata10.00: ATA-8: ST3500418AS, CC38, max UDMA/133
[ 0.800317] ata10.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 0.813769] ata10.00: configured for UDMA/133
[ 0.960644] ata7.00: SATA link down (SStatus 0 SControl 300)
[ 0.960655] ata7.01: SATA link down (SStatus 0 SControl 300)
[ 1.113401] ata8.00: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.113413] ata8.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.167224] ata8.00: ATA-7: ST3320620AS, 3.AAD, max UDMA/133
[ 1.167228] ata8.00: 625142448 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 1.186876] ata8.01: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133
[ 1.186879] ata8.01: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 1.233838] ata8.00: configured for UDMA/133
[ 1.247117] ata8.01: configured for UDMA/133
[ 1.260050] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.284063] ata4.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133
[ 1.284066] ata4.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 0/32)
[ 1.297703] ata4.00: configured for UDMA/133
[ 1.616704] ata5: SATA link down (SStatus 0 SControl 300)
[ 1.936703] ata6: SATA link down (SStatus 0 SControl 300)
[ 3.788841] EXT4-fs (sdc3): mounted filesystem with ordered data
mode. Opts: (null)
[ 6.250846] EXT4-fs (sdc3): re-mounted. Opts: data=ordered
[ 16.065124] REISERFS (device sdc6): using ordered data mode
[ 16.157157] REISERFS (device sdc7): using ordered data mode
[ 2654.314204] md: md0 stopped.
[ 2654.314211] md: unbind<sdd>
[ 2654.349758] md: export_rdev(sdd)
[ 2654.349787] md: unbind<sdb>
[ 2654.376421] md: export_rdev(sdb)
[ 2654.376465] md: unbind<sde>
[ 2654.403083] md: export_rdev(sde)
[ 2654.403121] md: unbind<sda>
[ 2654.429751] md: export_rdev(sda)
[ 2683.902046] md: md0 stopped.
[ 2683.906627] sda: unknown partition table
[ 2683.908554] sdb: unknown partition table
[ 2683.909356] sde: unknown partition table
[ 2683.945959] md: bind<sdb>
[ 2683.946159] md: bind<sda>
[ 2683.946360] md: bind<sde>
[ 2683.946386] md: md0 stopped.
[ 2683.946390] md: unbind<sde>
[ 2683.966426] md: export_rdev(sde)
[ 2683.966431] md: unbind<sda>
[ 2683.999745] md: export_rdev(sda)
[ 2683.999749] md: unbind<sdb>
[ 2684.019745] md: export_rdev(sdb)
[ 2684.040292] md: bind<sde>
[ 2684.041464] md: bind<sdb>
[ 2684.042870] md: bind<sda>
[ 2684.153079] raid6: sse2x1 4293 MB/s
[ 2684.209756] raid6: sse2x2 4367 MB/s
[ 2684.266411] raid6: sse2x4 6959 MB/s
[ 2684.266412] raid6: using algorithm sse2x4 (6959 MB/s)
[ 2684.266413] raid6: using ssse3x2 recovery algorithm
[ 2684.274830] async_tx: api initialized (async)
[ 2684.275378] xor: measuring software checksum speed
[ 2684.306408] prefetch64-sse: 10915.200 MB/sec
[ 2684.339741] generic_sse: 8464.800 MB/sec
[ 2684.339742] xor: using function: prefetch64-sse (10915.200 MB/sec)
[ 2684.371389] md: raid6 personality registered for level 6
[ 2684.371389] md: raid5 personality registered for level 5
[ 2684.371390] md: raid4 personality registered for level 4
[ 2684.371740] md/raid:md0: device sda operational as raid disk 3
[ 2684.371741] md/raid:md0: device sdb operational as raid disk 2
[ 2684.371742] md/raid:md0: device sde operational as raid disk 0
[ 2684.372092] md/raid:md0: allocated 0kB
[ 2684.372115] md/raid:md0: raid level 5 active with 3 out of 4 devices,
algorithm 2
[ 2684.372116] RAID conf printout:
[ 2684.372117] --- level:5 rd:4 wd:3
[ 2684.372117] disk 0, o:1, dev:sde
[ 2684.372118] disk 2, o:1, dev:sdb
[ 2684.372119] disk 3, o:1, dev:sda
[ 2684.372149] md0: detected capacity change from 0 to 1500323119104
[ 2684.372426] md0: unknown partition table
[ 2934.325679] SGI XFS with ACLs, security attributes, realtime, no
debug enabled
[ 3090.249640] XFS (md0): Mounting V4 Filesystem
[ 3090.403449] XFS (md0): Starting recovery (logdev: internal)
[ 3090.445679] XFS (md0): Ending recovery (logdev: internal)
cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[3] sdb[2] sde[0]
1465159296 blocks level 5, 128k chunk, algorithm 2 [4/3] [U_UU]
unused devices: <none>
Here's the smart info
for /dev/sda :
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar Blue (SATA)
Device Model: WDC WD5000AAKS-00TMA0
Serial Number: WD-WCAPW1796747
LU WWN Device Id: 5 0014ee 2003d9fca
Firmware Version: 12.01C01
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
Local Time is: Sun Jul 19 15:01:42 2015 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x84) Offline data collection activity
was suspended by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (13200) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 164) minutes.
Conveyance self-test routine
recommended polling time: ( 6) minutes.
SCT capabilities: (0x303f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
Always - 0
3 Spin_Up_Time 0x0003 221 164 021 Pre-fail
Always - 3950
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 350
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000e 200 200 051 Old_age
Always - 0
9 Power_On_Hours 0x0032 024 024 000 Old_age
Always - 55688
10 Spin_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age
Always - 344
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
Always - 253
193 Load_Cycle_Count 0x0032 200 200 000 Old_age
Always - 364
194 Temperature_Celsius 0x0022 117 089 000 Old_age
Always - 33
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
Always - 0
197 Current_Pending_Sector 0x0012 200 200 000 Old_age
Always - 0
198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 195 000 Old_age
Always - 12
200 Multi_Zone_Error_Rate 0x0008 200 189 051 Old_age
Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
for /dev/sdb :
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar Blue (SATA)
Device Model: WDC WD5000AAKS-00TMA0
Serial Number: WD-WCAPW1120688
LU WWN Device Id: 5 0014ee 2aaddb938
Firmware Version: 12.01C01
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
Local Time is: Sun Jul 19 15:01:46 2015 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x84) Offline data collection activity
was suspended by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (12600) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 157) minutes.
Conveyance self-test routine
recommended polling time: ( 6) minutes.
SCT capabilities: (0x303f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
Always - 0
3 Spin_Up_Time 0x0003 172 165 021 Pre-fail
Always - 6383
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 346
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000e 200 200 051 Old_age
Always - 0
9 Power_On_Hours 0x0032 025 025 000 Old_age
Always - 55369
10 Spin_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age
Always - 340
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
Always - 231
193 Load_Cycle_Count 0x0032 200 200 000 Old_age
Always - 356
194 Temperature_Celsius 0x0022 119 085 000 Old_age
Always - 31
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
Always - 0
197 Current_Pending_Sector 0x0012 200 200 000 Old_age
Always - 0
198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age
Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
for /dev/sdd :
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar Blue (SATA)
Device Model: WDC WD5000AAKS-00TMA0
Serial Number: WD-WCAPW1122956
LU WWN Device Id: 5 0014ee 200331314
Firmware Version: 12.01C01
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA/ATAPI-7 (minor revision not indicated)
Local Time is: Sun Jul 19 15:01:49 2015 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x84) Offline data collection activity
was suspended by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (12000) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 150) minutes.
Conveyance self-test routine
recommended polling time: ( 6) minutes.
SCT capabilities: (0x303f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail
Always - 0
3 Spin_Up_Time 0x0003 171 165 021 Pre-fail
Always - 6425
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 482
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000e 200 200 051 Old_age
Always - 0
9 Power_On_Hours 0x0032 022 022 000 Old_age
Always - 57661
10 Spin_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
11 Calibration_Retry_Count 0x0012 100 100 051 Old_age
Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age
Always - 467
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age
Always - 528
193 Load_Cycle_Count 0x0032 200 200 000 Old_age
Always - 662
194 Temperature_Celsius 0x0022 118 086 000 Old_age
Always - 32
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age
Always - 0
197 Current_Pending_Sector 0x0012 200 200 000 Old_age
Always - 0
198 Offline_Uncorrectable 0x0010 200 200 000 Old_age
Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age
Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
for /dev/sde :
smartctl 6.4 2015-06-04 r4109 [x86_64-linux-3.17.4-1-ARCH] (local build)
Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 7200.12
Device Model: ST3500418AS
Serial Number: 5VM49NSD
LU WWN Device Id: 5 000c50 01c4cf9ab
Firmware Version: CC38
User Capacity: 500,107,862,016 bytes [500 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: 7200 rpm
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS T13/1699-D revision 4
SATA Version is: SATA 2.6, 3.0 Gb/s
Local Time is: Sun Jul 19 15:01:53 2015 EDT
==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/213891en
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 600) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 87) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 112 099 006 Pre-fail
Always - 49016311
3 Spin_Up_Time 0x0003 097 097 000 Pre-fail
Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age
Always - 398
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000f 069 060 030 Pre-fail
Always - 7542794
9 Power_On_Hours 0x0032 047 047 000 Old_age
Always - 46448
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail
Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age
Always - 199
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age
Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age
Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age
Always - 0
188 Command_Timeout 0x0032 100 100 000 Old_age
Always - 0
189 High_Fly_Writes 0x003a 100 100 000 Old_age
Always - 0
190 Airflow_Temperature_Cel 0x0022 064 059 045 Old_age
Always - 36 (Min/Max 36/39)
194 Temperature_Celsius 0x0022 036 041 000 Old_age
Always - 36 (0 17 0 0 0)
195 Hardware_ECC_Recovered 0x001a 016 015 000 Old_age
Always - 49016311
197 Current_Pending_Sector 0x0012 100 100 000 Old_age
Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age
Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age
Offline - 46876 (84 165 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age
Offline - 166857433
242 Total_LBAs_Read 0x0000 100 253 000 Old_age
Offline - 3494978241
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
I'm not sure what to think of this. Which drive is really dead? Or were
they simply synched but still usable?
Let me know if more information could help you to help me :)
Thanks!
Le 2015-07-19 11:09, Rudy Zijlstra a écrit :
> Hi Marc,
>
> On 19-07-15 16:26, Marc Desharnais wrote:
>> Hi,
>>
>> I'm new to this mailing list but I subscribed because of this problem
>> with my raid5.
>> I really don't want to mess this up and it was clearly suggested to
>> not rush things on the wiki. So I'm looking for help :)
>> My raid has been running great for a long time now but I think it
>> needs so serious maintenance.
>> I have a 4 disks raid 5 on ArchLinux, mdadm version 3.3.1.
>>
>> First, the raid status :
>>
>> /dev/sda:
>>
>> 1 1 0 0 1 faulty removed
>>
>> /dev/sdb:
>>
>> 1 1 0 0 1 faulty removed
>> 3 3 0 0 3 faulty removed
>> /dev/sdd:
>>
>> Number Major Minor RaidDevice State
>> this 1 8 16 1 active sync /dev/sdb
>>
>> 0 0 8 32 0 active sync /dev/sdc
>> 1 1 8 16 1 active sync /dev/sdb
>> 2 2 8 48 2 active sync /dev/sdd
>> 3 3 8 64 3 active sync /dev/sde
>> /dev/sde:
>>
>>
>> Number Major Minor RaidDevice State
>> this 0 8 64 0 active sync /dev/sde
>>
>> 0 0 8 64 0 active sync /dev/sde
>> 1 1 0 0 1 faulty removed
>> 2 2 8 16 2 active sync /dev/sdb
>> 3 3 0 0 3 faulty removed
>>
>>
>>
>> Obvisously the event count is wrong on 2 disks. One disk has been
>> gone for a long time now...
>>
>> 1 - First question : how can I safely assemble the raid to the latest
>> sync state? I don't want to mess this up.
>> 2 - Does this mean my hard drives are dead, or simply desync?
>>
>>
>> Also I tried to force assemble :
>> mdadm --assemble --force /dev/md0 /dev/sda /dev/sdb /dev/sde
>>
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> First:
> - what information is in dmesg or kern.log, or whererever your distro
> stores disk messages? Asking only for the relevant parts about the
> disks :)
> - Do you have any idea when the disks were kicked out?
> - can you give the output of smartctl -a for each disk?
> - output of cat /proc/mdstat?
>
> The assemble you tried is probably failing because the raid is
> assembled but not started. Doing a "mdadm -S /dev/md0" should take
> care of that.
>
> Cheers
>
> Rudy
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Raid5 fails to assemble automatically
From: Rudy Zijlstra @ 2015-07-19 15:09 UTC (permalink / raw)
To: Marc Desharnais, linux-raid
In-Reply-To: <BLU437-SMTP68684C478276C82E113DE6CE860@phx.gbl>
Hi Marc,
On 19-07-15 16:26, Marc Desharnais wrote:
> Hi,
>
> I'm new to this mailing list but I subscribed because of this problem
> with my raid5.
> I really don't want to mess this up and it was clearly suggested to
> not rush things on the wiki. So I'm looking for help :)
> My raid has been running great for a long time now but I think it
> needs so serious maintenance.
> I have a 4 disks raid 5 on ArchLinux, mdadm version 3.3.1.
>
> First, the raid status :
>
> /dev/sda:
>
> 1 1 0 0 1 faulty removed
>
> /dev/sdb:
>
> 1 1 0 0 1 faulty removed
> 3 3 0 0 3 faulty removed
> /dev/sdd:
>
> Number Major Minor RaidDevice State
> this 1 8 16 1 active sync /dev/sdb
>
> 0 0 8 32 0 active sync /dev/sdc
> 1 1 8 16 1 active sync /dev/sdb
> 2 2 8 48 2 active sync /dev/sdd
> 3 3 8 64 3 active sync /dev/sde
> /dev/sde:
>
>
> Number Major Minor RaidDevice State
> this 0 8 64 0 active sync /dev/sde
>
> 0 0 8 64 0 active sync /dev/sde
> 1 1 0 0 1 faulty removed
> 2 2 8 16 2 active sync /dev/sdb
> 3 3 0 0 3 faulty removed
>
>
>
> Obvisously the event count is wrong on 2 disks. One disk has been gone
> for a long time now...
>
> 1 - First question : how can I safely assemble the raid to the latest
> sync state? I don't want to mess this up.
> 2 - Does this mean my hard drives are dead, or simply desync?
>
>
> Also I tried to force assemble :
> mdadm --assemble --force /dev/md0 /dev/sda /dev/sdb /dev/sde
>
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
First:
- what information is in dmesg or kern.log, or whererever your distro
stores disk messages? Asking only for the relevant parts about the disks :)
- Do you have any idea when the disks were kicked out?
- can you give the output of smartctl -a for each disk?
- output of cat /proc/mdstat?
The assemble you tried is probably failing because the raid is assembled
but not started. Doing a "mdadm -S /dev/md0" should take care of that.
Cheers
Rudy
^ permalink raw reply
* Raid5 fails to assemble automatically
From: Marc Desharnais @ 2015-07-19 14:26 UTC (permalink / raw)
To: linux-raid
Hi,
I'm new to this mailing list but I subscribed because of this problem
with my raid5.
I really don't want to mess this up and it was clearly suggested to not
rush things on the wiki. So I'm looking for help :)
My raid has been running great for a long time now but I think it needs
so serious maintenance.
I have a 4 disks raid 5 on ArchLinux, mdadm version 3.3.1.
First, the raid status :
/dev/sda:
Magic : a92b4efc
Version : 0.90.00
UUID : 63fa3bf5:e14f2233:61302635:9841f005
Creation Time : Sat Dec 26 11:20:04 2009
Raid Level : raid5
Used Dev Size : 488386432 (465.76 GiB 500.11 GB)
Array Size : 1465159296 (1397.28 GiB 1500.32 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Fri May 22 03:08:43 2015
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 1
Spare Devices : 0
Checksum : a59a8ad1 - correct
Events : 17559
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 3 8 0 3 active sync /dev/sda
0 0 8 64 0 active sync /dev/sde
1 1 0 0 1 faulty removed
2 2 8 16 2 active sync /dev/sdb
3 3 8 0 3 active sync /dev/sda
/dev/sdb:
Magic : a92b4efc
Version : 0.90.00
UUID : 63fa3bf5:e14f2233:61302635:9841f005
Creation Time : Sat Dec 26 11:20:04 2009
Raid Level : raid5
Used Dev Size : 488386432 (465.76 GiB 500.11 GB)
Array Size : 1465159296 (1397.28 GiB 1500.32 GB)
Raid Devices : 4
Total Devices : 2
Preferred Minor : 0
Update Time : Sat May 23 07:02:59 2015
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 2
Spare Devices : 0
Checksum : a59c1346 - correct
Events : 17562
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 2 8 16 2 active sync /dev/sdb
0 0 8 64 0 active sync /dev/sde
1 1 0 0 1 faulty removed
2 2 8 16 2 active sync /dev/sdb
3 3 0 0 3 faulty removed
/dev/sdd:
Magic : a92b4efc
Version : 0.90.00
UUID : 63fa3bf5:e14f2233:61302635:9841f005
Creation Time : Sat Dec 26 11:20:04 2009
Raid Level : raid5
Used Dev Size : 488386432 (465.76 GiB 500.11 GB)
Array Size : 1465159296 (1397.28 GiB 1500.32 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Update Time : Mon Dec 24 01:09:51 2012
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : a113755e - correct
Events : 11218
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 1 8 16 1 active sync /dev/sdb
0 0 8 32 0 active sync /dev/sdc
1 1 8 16 1 active sync /dev/sdb
2 2 8 48 2 active sync /dev/sdd
3 3 8 64 3 active sync /dev/sde
/dev/sde:
Magic : a92b4efc
Version : 0.90.00
UUID : 63fa3bf5:e14f2233:61302635:9841f005
Creation Time : Sat Dec 26 11:20:04 2009
Raid Level : raid5
Used Dev Size : 488386432 (465.76 GiB 500.11 GB)
Array Size : 1465159296 (1397.28 GiB 1500.32 GB)
Raid Devices : 4
Total Devices : 2
Preferred Minor : 0
Update Time : Sat May 23 07:02:59 2015
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 2
Spare Devices : 0
Checksum : a59c1372 - correct
Events : 17562
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 0 8 64 0 active sync /dev/sde
0 0 8 64 0 active sync /dev/sde
1 1 0 0 1 faulty removed
2 2 8 16 2 active sync /dev/sdb
3 3 0 0 3 faulty removed
Obvisously the event count is wrong on 2 disks. One disk has been gone
for a long time now...
1 - First question : how can I safely assemble the raid to the latest
sync state? I don't want to mess this up.
2 - Does this mean my hard drives are dead, or simply desync?
Also I tried to force assemble :
mdadm --assemble --force /dev/md0 /dev/sda /dev/sdb /dev/sde
mdadm: /dev/sda is busy - skipping
mdadm: /dev/sdb is busy - skipping
mdadm: /dev/sde is busy - skipping
What should I do now?
Thank you!
Marc
^ permalink raw reply
* [PATCH] raid0: data corruption when using trim
From: Seunguk Shin @ 2015-07-19 3:28 UTC (permalink / raw)
To: neilb; +Cc: linux-raid
Hi,
There was report at Algolia's blog website regarding some problems using
software raid and trim in SSD.
https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/
It turns out that there is misunderstanding between raid driver and scsi/ata
driver.
The raid driver lets split bios share bio vector of source bio.
Usually, there is no problem, because the raid layer ensures
that the source bio is not freed before the split bios.
But, in case of trim, there are some problems.
The scsi/ata needs some payloads that include start address and size of
device to trim.
So, the scsi/ata driver allocates a page and stores that pointer on
bio->bi_io_vec->bv_page.
(sd_setup_discard_cmnd)
Because split bios share the source bio's bi_io_vec,
the pointer to the allocated page in scsi/ata driver is overwritten.
It leads to memory leakage and data corruption
because the overwritten pointer has wrong address and size to trim on
device.
So, we add some codes that make sure not to share bio vector if the source
bio is discard.
The linear and raid10 also have same problem
because they're using similar scheme to split the source bio.
This problem exists at the very first time when the trim is enabled on
raid0.
Before the new bio_split() (20d0189b), we can patch more easily
by modifying the condition to split bio vector in bio_split function
from (bi->bi_vec != 0) to ((bi->bi_vec != 0) || (bi->bi_rw & REQ_DISCARD)).
Thank you.
Seunguk Shin.
[“0001-PATCH-raid0-data-corruption-when-using-trim.patch”]
From ca7dbe01fcd2ef2f8cea1a38de5aca5c866c585d Mon Sep 17 00:00:00 2001
From: Seunguk Shin <seunguk.shin@samsung.com>
Date: Sat, 18 Jul 2015 20:13:44 +0900
Subject: [PATCH] [PATCH] raid0: data corruption when using trim
When we are using software raid and tirm, there is data corruption.
The raid driver lets split bios share bio vector of source bio.
The scsi/ata driver allocates a page and stores that pointer on
bio->bi_io_vec->bv_page
(sd_setup_discard_cmnd) because the scsi/ata needs some payloads
that include start address and size of device to trim.
Because split bios share the source bio's bi_io_vec,
the pointer to the allocated page in scsi/ata driver is overwritten.
This patch splits bio vector if bio is discard.
---
block/bio.c | 6 ------
drivers/md/raid0.c | 13 +++++++++++++
include/linux/bio.h | 6 ++++++
3 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 2a00d34..df7589d 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -32,12 +32,6 @@
#include <trace/events/block.h>
/*
- * Test patch to inline a certain number of bi_io_vec's inside the bio
- * itself, to shrink a bio data allocation from two mempool calls to one
- */
-#define BIO_INLINE_VECS 4
-
-/*
* if you change this list, also change bvec_alloc or things will
* break badly! cannot be bigger than what you can fit into an
* unsigned short
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index efb654e..fd1318b 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -529,6 +529,19 @@ static void raid0_make_request(struct mddev *mddev,
struct bio *bio)
if (sectors < bio_sectors(bio)) {
split = bio_split(bio, sectors, GFP_NOIO,
fs_bio_set);
+ if (unlikely(split->bi_rw & REQ_DISCARD)) {
+ struct bio_vec *bvl = NULL;
+ unsigned long idx = BIO_POOL_NONE;
+
+ bvl = bvec_alloc(GFP_NOIO,
BIO_INLINE_VECS + 1,
+ &idx,
fs_bio_set->bvec_pool);
+ split->bi_flags |= 1 <<
BIO_OWNS_VEC;
+ split->bi_flags &= ~(BIO_POOL_NONE
<<
+
BIO_POOL_OFFSET);
+ split->bi_flags |= idx <<
BIO_POOL_OFFSET;
+ split->bi_max_vecs =
BIO_INLINE_VECS + 1;
+ split->bi_io_vec = bvl;
+ }
bio_chain(split, bio);
} else {
split = bio;
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 5e963a6..8344bb5 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -40,6 +40,12 @@
#define BIO_BUG_ON
#endif
+/*
+ * Test patch to inline a certain number of bi_io_vec's inside the bio
+ * itself, to shrink a bio data allocation from two mempool calls to one
+ */
+#define BIO_INLINE_VECS 4
+
#define BIO_MAX_PAGES 256
#define BIO_MAX_SIZE (BIO_MAX_PAGES << PAGE_CACHE_SHIFT)
#define BIO_MAX_SECTORS (BIO_MAX_SIZE >> 9)
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: Reversing parity recalculation on out-of-order RAID 6 array
From: Steven Reddie @ 2015-07-19 2:26 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1507181342220.11810@uplift.swm.pp.se>
It's the same as if I had lost two disks from the array, which RAID 6
can recover. The difference is that for each stripe (right
terminology?) of the array, the overwrites didn't occur on the same
two disks due to the "Q A B C P" layout rotating from stripe to stripe
(each 64KB * 5 in my case). Therefore, for any given stripe, which
disks were overwritten is a function of the current incorrect ordering
of the disks in the array and the offset of the stripe (which tells
which rotation of "Q A B C P" is used). Including knowledge of the
original correct ordering of the disks then allows mapping to which
two components of the original data have been lost (ie. two of Q, A,
B, C, or P). Then it is a simple matter of recalculating the missing
components from the survived components.
I'm now almost done with an nbd-based translator that does this
reversal on the fly (ie. takes /dev/sdg[a-e]5 as input and presents
the translated versions as /dev/nbd[0-4]). Since I wanted to do the
remapping only once per stripe, I haven't used the nbd server and
client code as I'd have to run an instance of each for each of the 5
disks in my array. I've got my own server and client code that
interfaces with the nbd kernel module in the same way, but provides
access to all disks through the one instance.
I don't want to risk messing with any of the MD or LVM metadata on the
Synology, so once the translator is finished I will try assembling an
array from these devices across the network on another machine and
check/mount the filesystem.
I was hoping that there might be a tried and tested tool already
available for this, but if not I'll continue with mine.
Steven
On Sat, Jul 18, 2015 at 9:43 PM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Sat, 18 Jul 2015, Steven Reddie wrote:
>
>> I need to reverse these disk writes across then entire length of the
>> array, after which I can reassemble the array in the correct order.
>
>
> If you did not use --assume-clean, and had the wrong parity order and the
> array was "resynced", I don't see how you did not irrevocably overwrote
> important information and you now can't reverse this.
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: Reversing parity recalculation on out-of-order RAID 6 array
From: Mikael Abrahamsson @ 2015-07-18 11:43 UTC (permalink / raw)
To: Steven Reddie; +Cc: linux-raid
In-Reply-To: <CAFcup8G230TBec=FzM45GqUkb218yWcYiJJiHDNOkv9MQHkF0g@mail.gmail.com>
On Sat, 18 Jul 2015, Steven Reddie wrote:
> I need to reverse these disk writes across then entire length of the
> array, after which I can reassemble the array in the correct order.
If you did not use --assume-clean, and had the wrong parity order and the
array was "resynced", I don't see how you did not irrevocably overwrote
important information and you now can't reverse this.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: Reversing parity recalculation on out-of-order RAID 6 array
From: Steven Reddie @ 2015-07-18 8:35 UTC (permalink / raw)
To: Mikael Abrahamsson; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1507180731040.11810@uplift.swm.pp.se>
Hi Mikael,
I did look through the archives and also at that recovery wiki and
permute script, but I think my case is different, even though I'm sure
others must have also hit it before. My problem isn't that I don't
know the order of the disks in the array, but that the data in the
array has been scrambled. As an example, if a given extent in the
array was ordered across the disks as so:
A B C P Q
where:
P = A ^ B ^ C
Q = A ^ mul(B) ^ mul(mul(C))
Then when my array was reconfigured in the incorrect order (say, with
disk 2 moved to the end), the layout was reordered to became:
A C P Q B
Then when the array was recreated in this state, the parity
information was recalculated such that the data on disk became:
A C P P' Q'
where:
P' = A ^ C ^ P
Q' = A ^ mul(C) ^ mul(mul(P))
Thus, the new parity information is now a function of two out of three
of the original data chunks (A and C) and one of the original parity
chunks (P), with data chunk B having been overwritten.
I need to reverse these disk writes across then entire length of the
array, after which I can reassemble the array in the correct order.
Thanks,
Steven
On Sat, Jul 18, 2015 at 3:33 PM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Sat, 18 Jul 2015, Steven Reddie wrote:
>
>> I'm kind of hoping that all of this might have already been done, so
>> any pointers would be appreciated.
>
>
> https://raid.wiki.kernel.org/index.php/RAID_Recovery
> https://raid.wiki.kernel.org/index.php/Permute_array.pl
>
> There have historically been lots of threads here with people having the
> same problems, so if you search the archives for things like "assume-clean",
> "order", "mdadm", "create", "superblock", you'll find multiple examples of
> people actually successfully getting out of the problem you're also having.
>
> --
> Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: Reversing parity recalculation on out-of-order RAID 6 array
From: Mikael Abrahamsson @ 2015-07-18 5:33 UTC (permalink / raw)
To: Steven Reddie; +Cc: linux-raid
In-Reply-To: <CAFcup8FgJqu6q2BoUc7OSEL=iCFf_43wzj7dBuFXYp2R0t2kdg@mail.gmail.com>
On Sat, 18 Jul 2015, Steven Reddie wrote:
> I'm kind of hoping that all of this might have already been done, so
> any pointers would be appreciated.
https://raid.wiki.kernel.org/index.php/RAID_Recovery
https://raid.wiki.kernel.org/index.php/Permute_array.pl
There have historically been lots of threads here with people having the
same problems, so if you search the archives for things like
"assume-clean", "order", "mdadm", "create", "superblock", you'll find
multiple examples of people actually successfully getting out of the
problem you're also having.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Reversing parity recalculation on out-of-order RAID 6 array
From: Steven Reddie @ 2015-07-18 3:16 UTC (permalink / raw)
To: linux-raid
Hi,
Continuing from the "mdadm --create parameter ordering" thread, I now
want to reverse the effects of recreating the array with the disks
specified in the incorrect order.
Are there existing tools that will help with this?
I've had a quick play around with nbd and I'm confident that I can
write an nbd client that will effectively reorder the underlying disks
and recalculate the parity blocks to present 5 pseudo devices that
look like the original disks. I could then attempt to start an array
via those nbd devices and mount/check the filesystem. This approach
would also allow me to ensure that access to the physical disks is
readonly, and implement copy-on-write if necessary to support this.
I'm kind of hoping that all of this might have already been done, so
any pointers would be appreciated.
[BTW, I also expect that I'll have to mess with metadata and
superblocks, but first need to solve the problem above.]
Thanks,
Steven
^ permalink raw reply
* Re: mdadm --create parameter ordering
From: Steven Reddie @ 2015-07-18 3:08 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
In-Reply-To: <55A3BA80.50505@turmel.org>
Thanks Robin and Phil, I've verified that the parity information did
in fact get recalculated and written to different disks, though I'm
confident that I can still recover by reversing this. I expect that I
could manually reconstruct the metadata and superblocks if I needed to
but before I began I took a dd backup of the first 1GB of each
physical disk, each partition, and the MD device, and I also have the
/etc/space/*.xml files, so I expect I have enough to restore these.
I'll start a new thread to ask about the best way to proceed.
Thanks,
Steven
On Mon, Jul 13, 2015 at 11:17 PM, Phil Turmel <philip@turmel.org> wrote:
> On 07/13/2015 06:59 AM, Robin Hill wrote:
>
> [snip /]
>
> Let me elaborate on one of Robin's comments:
>
>> The --assume-clean option will prevent parity calculation (as it assumes
>> that the parity on disk is already correct), which may be what you're
>> after. It should only be used it you're absolutely certain of the data
>> though, as it can lead to large-scale corruption otherwise.
>
> It is not enough to be certain of the order to use --create
> --assume-clean on existing data. You must also be certain of superblock
> type, chunk size, data offset, and layout option.
>
> Data loss due to unwise use of --create has a history of bad outcomes.
> It destroys the previous metadata on array members, metadata that is
> almost always needed to recover damaged arrays. The --create option
> should *never* be used on an array with important data without first
> asking for advice on this list.
>
> Phil
^ permalink raw reply
* Good morning
From: fahqr @ 2015-07-18 1:54 UTC (permalink / raw)
To: linobax
Good morning
nikon,samsung product,apple....
samsung s6,280euro
www < isgayre>com
^ permalink raw reply
* Re: [PATCH v2 1/3] PM suspend/hibernate: Call notifier after freezing processes
From: Rafael J. Wysocki @ 2015-07-17 23:27 UTC (permalink / raw)
To: Pali Rohár
Cc: Alasdair Kergon, Mike Snitzer, Neil Brown, Len Brown,
Pavel Machek, dm-devel, linux-raid, linux-kernel, linux-pm
In-Reply-To: <20150716073302.GA10393@pali>
On Thursday, July 16, 2015 09:33:02 AM Pali Rohár wrote:
> On Thursday 16 July 2015 03:02:03 Rafael J. Wysocki wrote:
> > On Sunday, June 21, 2015 01:20:32 PM Pali Rohár wrote:
> > > To prevent race conditions on userspace processes with I/O some taks must be
> > > called after processes are freezed. This patch adds new events which are
> > > delivered by pm_notifier_call_chain() after freezing processes when doing
> > > suspend or hibernate action.
> > >
> > > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > > ---
> > > include/linux/suspend.h | 2 ++
> > > kernel/power/hibernate.c | 2 ++
> > > kernel/power/suspend.c | 4 +++-
> > > 3 files changed, 7 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/include/linux/suspend.h b/include/linux/suspend.h
> > > index 5efe743..bc743c8 100644
> > > --- a/include/linux/suspend.h
> > > +++ b/include/linux/suspend.h
> > > @@ -368,6 +368,8 @@ static inline bool hibernation_available(void) { return false; }
> > > #define PM_POST_SUSPEND 0x0004 /* Suspend finished */
> > > #define PM_RESTORE_PREPARE 0x0005 /* Going to restore a saved image */
> > > #define PM_POST_RESTORE 0x0006 /* Restore failed */
> > > +#define PM_HIBERNATION_AFTER_FREEZE 0x0007 /* After hibernation freeze */
> > > +#define PM_SUSPEND_AFTER_FREEZE 0x0008 /* After suspend freeze */
> > >
> > > extern struct mutex pm_mutex;
> > >
> > > diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c
> > > index 2329daa..184f7ee 100644
> > > --- a/kernel/power/hibernate.c
> > > +++ b/kernel/power/hibernate.c
> > > @@ -671,6 +671,8 @@ int hibernate(void)
> > > if (error)
> > > goto Exit;
> > >
> > > + pm_notifier_call_chain(PM_HIBERNATION_AFTER_FREEZE);
> >
> > Don't we need to check errors from these?
> >
>
> If yes, what to do in this case? Fail hibernation and goto Exit?
Yes, fail the transition in progress.
> > Also, if you're adding AFTER_FREEZE, it would be good to add BEFORE_THAW too
> > for symmetry.
> >
>
> But there is no use case for BEFORE_THAW. At least it is not needed for now.
For your use case, a single function pointer would be sufficient too.
> > > +
> > > lock_device_hotplug();
> > > /* Allocate memory management structures */
> > > error = create_basic_memory_bitmaps();
> > > diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c
> > > index 8d7a1ef..ba2a945 100644
> > > --- a/kernel/power/suspend.c
> > > +++ b/kernel/power/suspend.c
> > > @@ -277,8 +277,10 @@ static int suspend_prepare(suspend_state_t state)
> > > trace_suspend_resume(TPS("freeze_processes"), 0, true);
> > > error = suspend_freeze_processes();
> > > trace_suspend_resume(TPS("freeze_processes"), 0, false);
> > > - if (!error)
> > > + if (!error) {
> > > + pm_notifier_call_chain(PM_SUSPEND_AFTER_FREEZE);
> > > return 0;
> > > + }
> > >
> > > suspend_stats.failed_freeze++;
> > > dpm_save_failed_step(SUSPEND_FREEZE);
> > >
> >
>
>
--
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Mike Snitzer @ 2015-07-17 17:31 UTC (permalink / raw)
To: Pali Rohár
Cc: Len Brown, linux-pm, Rafael J. Wysocki, linux-kernel, linux-raid,
dm-devel, Pavel Machek, Alasdair Kergon
In-Reply-To: <201507171913.57104@pali>
On Fri, Jul 17 2015 at 1:13pm -0400,
Pali Rohár <pali.rohar@gmail.com> wrote:
> On Friday 17 July 2015 17:30:45 Mike Snitzer wrote:
> > On Fri, Jul 17 2015 at 11:22am -0400,
> >
> > Mike Snitzer <snitzer@redhat.com> wrote:
> > > On Fri, Jul 17 2015 at 10:22am -0400,
> > >
> > > Pali Rohár <pali.rohar@gmail.com> wrote:
> > > > On Friday 17 July 2015 10:04:39 Mike Snitzer wrote:
> > > > > On Sun, Jun 21 2015 at 7:20am -0400,
> > > > >
> > > > > Pali Rohár <pali.rohar@gmail.com> wrote:
> > > > > > This patch exports function dm_suspend_md() which suspend
> > > > > > mapped device so other kernel drivers can use it and could
> > > > > > suspend mapped device when needed.
> > > > > >
> > > > > > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > > > > > ---
> > > > > >
> > > > > > drivers/md/dm.c | 6 ++++++
> > > > > > drivers/md/dm.h | 5 +++++
> > > > > > 2 files changed, 11 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > > > > > index 2caf492..03298ff 100644
> > > > > > --- a/drivers/md/dm.c
> > > > > > +++ b/drivers/md/dm.c
> > > > > >
> > > > > > @@ -3343,6 +3343,12 @@ out:
> > > > > > return r;
> > > > > >
> > > > > > }
> > > > > >
> > > > > > +int dm_suspend_md(struct mapped_device *md)
> > > > > > +{
> > > > > > + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > > > > +}
> > > > > > +EXPORT_SYMBOL_GPL(dm_suspend_md);
> > > > > > +
> > > > > >
> > > > > > /*
> > > > > >
> > > > > > * Internal suspend/resume works like userspace-driven
> > > > > > suspend. It waits * until all bios finish and prevents
> > > > > > issuing new bios to the target drivers.
> > > > >
> > > > > To do this properly you should be introducing a variant of
> > > > > dm_internal_suspend. We currently have two variants:
> > > > > dm_internal_suspend_fast
> > > > > dm_internal_suspend_noflush
> > > > >
> > > > > The reason to use a dm_internal_suspend variant is this suspend
> > > > > was _not_ initiated by an upper level ioctl (from userspace).
> > > > > It was done internally from within the target.
> > > > >
> > > > > You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning
> > > > > you're interested in flushing all pending IO (in the FS
> > > > > layered on dm-crupt, if one exists).
> > > > >
> > > > > But see the comment in __dm_internal_suspend() about
> > > > > TASK_UNINTERRUPTIBLE. If you're OK with the dm-crypt initiated
> > > > > suspend being TASK_UNINTERRUPTIBLE then you could just
> > > > > introduce:
> > > > >
> > > > > void dm_internal_suspend_uninterruptible_flush(struct
> > > > > mapped_device *md) {
> > > > >
> > > > > mutex_lock(&md->suspend_lock);
> > > > > __dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > > > mutex_unlock(&md->suspend_lock);
> > > > >
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
> > > > >
> > > > > Otherwise, there is much more extensive DM core changes needed
> > > > > to __dm_internal_suspend() and .presuspend to properly support
> > > > > TASK_INTERRUPTIBLE.
> > > >
> > > > Hi! I will look at dm_internal_suspend. Anyway use case for
> > > > suspend is from dm-crypt to do both operations: suspend + key
> > > > wipe. It means that without entering key again from userspace,
> > > > resume is not possible. So my question is: It is possible to do
> > > > internal suspend and then using resume from userspace via ioctl?
> > >
> > > Good question: no, userspace resume would block waiting for
> > > internal resume.
> > >
> > > Soooo... I'll have to look at your patch 3 to understand why
> > > dm-crypt is needing to initiate the suspend internally but then
> > > userspace is initiating the resume... this imbalance is
> > > concerning.
> >
> > Why not introduce a new message that allows you to wipe the key after
> > suspend? Both initiated from userspace.
>
> There is already message for wiping key and it will success only if dm
> is suspended.
>
> But this patch series is fixing another problem: wipe key before
> suspend/hibernation action happend and to have it race free it must be
> done after userspace is freezed!
Yes, I remember now. So it isn't even userspace initiating the
suspend_and_wipe, it is the PM chain notifier code you're adding.
I'll think more about your use of dm_suspend()
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Pali Rohár @ 2015-07-17 17:13 UTC (permalink / raw)
To: Mike Snitzer
Cc: Len Brown, linux-pm, Rafael J. Wysocki, linux-kernel, linux-raid,
dm-devel, Pavel Machek, Alasdair Kergon
In-Reply-To: <20150717153045.GB1021@redhat.com>
[-- Attachment #1: Type: Text/Plain, Size: 4036 bytes --]
On Friday 17 July 2015 17:30:45 Mike Snitzer wrote:
> On Fri, Jul 17 2015 at 11:22am -0400,
>
> Mike Snitzer <snitzer@redhat.com> wrote:
> > On Fri, Jul 17 2015 at 10:22am -0400,
> >
> > Pali Rohár <pali.rohar@gmail.com> wrote:
> > > On Friday 17 July 2015 10:04:39 Mike Snitzer wrote:
> > > > On Sun, Jun 21 2015 at 7:20am -0400,
> > > >
> > > > Pali Rohár <pali.rohar@gmail.com> wrote:
> > > > > This patch exports function dm_suspend_md() which suspend
> > > > > mapped device so other kernel drivers can use it and could
> > > > > suspend mapped device when needed.
> > > > >
> > > > > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > > > > ---
> > > > >
> > > > > drivers/md/dm.c | 6 ++++++
> > > > > drivers/md/dm.h | 5 +++++
> > > > > 2 files changed, 11 insertions(+)
> > > > >
> > > > > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > > > > index 2caf492..03298ff 100644
> > > > > --- a/drivers/md/dm.c
> > > > > +++ b/drivers/md/dm.c
> > > > >
> > > > > @@ -3343,6 +3343,12 @@ out:
> > > > > return r;
> > > > >
> > > > > }
> > > > >
> > > > > +int dm_suspend_md(struct mapped_device *md)
> > > > > +{
> > > > > + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(dm_suspend_md);
> > > > > +
> > > > >
> > > > > /*
> > > > >
> > > > > * Internal suspend/resume works like userspace-driven
> > > > > suspend. It waits * until all bios finish and prevents
> > > > > issuing new bios to the target drivers.
> > > >
> > > > To do this properly you should be introducing a variant of
> > > > dm_internal_suspend. We currently have two variants:
> > > > dm_internal_suspend_fast
> > > > dm_internal_suspend_noflush
> > > >
> > > > The reason to use a dm_internal_suspend variant is this suspend
> > > > was _not_ initiated by an upper level ioctl (from userspace).
> > > > It was done internally from within the target.
> > > >
> > > > You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning
> > > > you're interested in flushing all pending IO (in the FS
> > > > layered on dm-crupt, if one exists).
> > > >
> > > > But see the comment in __dm_internal_suspend() about
> > > > TASK_UNINTERRUPTIBLE. If you're OK with the dm-crypt initiated
> > > > suspend being TASK_UNINTERRUPTIBLE then you could just
> > > > introduce:
> > > >
> > > > void dm_internal_suspend_uninterruptible_flush(struct
> > > > mapped_device *md) {
> > > >
> > > > mutex_lock(&md->suspend_lock);
> > > > __dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > > mutex_unlock(&md->suspend_lock);
> > > >
> > > > }
> > > > EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
> > > >
> > > > Otherwise, there is much more extensive DM core changes needed
> > > > to __dm_internal_suspend() and .presuspend to properly support
> > > > TASK_INTERRUPTIBLE.
> > >
> > > Hi! I will look at dm_internal_suspend. Anyway use case for
> > > suspend is from dm-crypt to do both operations: suspend + key
> > > wipe. It means that without entering key again from userspace,
> > > resume is not possible. So my question is: It is possible to do
> > > internal suspend and then using resume from userspace via ioctl?
> >
> > Good question: no, userspace resume would block waiting for
> > internal resume.
> >
> > Soooo... I'll have to look at your patch 3 to understand why
> > dm-crypt is needing to initiate the suspend internally but then
> > userspace is initiating the resume... this imbalance is
> > concerning.
>
> Why not introduce a new message that allows you to wipe the key after
> suspend? Both initiated from userspace.
There is already message for wiping key and it will success only if dm
is suspended.
But this patch series is fixing another problem: wipe key before
suspend/hibernation action happend and to have it race free it must be
done after userspace is freezed!
--
Pali Rohár
pali.rohar@gmail.com
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Mike Snitzer @ 2015-07-17 15:30 UTC (permalink / raw)
To: Pali Rohár
Cc: Len Brown, linux-pm, Rafael J. Wysocki, linux-kernel, linux-raid,
dm-devel, Pavel Machek, Alasdair Kergon
In-Reply-To: <20150717152253.GA1021@redhat.com>
On Fri, Jul 17 2015 at 11:22am -0400,
Mike Snitzer <snitzer@redhat.com> wrote:
> On Fri, Jul 17 2015 at 10:22am -0400,
> Pali Rohár <pali.rohar@gmail.com> wrote:
>
> > On Friday 17 July 2015 10:04:39 Mike Snitzer wrote:
> > > On Sun, Jun 21 2015 at 7:20am -0400,
> > > Pali Rohár <pali.rohar@gmail.com> wrote:
> > >
> > > > This patch exports function dm_suspend_md() which suspend mapped device so other
> > > > kernel drivers can use it and could suspend mapped device when needed.
> > > >
> > > > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > > > ---
> > > > drivers/md/dm.c | 6 ++++++
> > > > drivers/md/dm.h | 5 +++++
> > > > 2 files changed, 11 insertions(+)
> > > >
> > > > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > > > index 2caf492..03298ff 100644
> > > > --- a/drivers/md/dm.c
> > > > +++ b/drivers/md/dm.c
> > > > @@ -3343,6 +3343,12 @@ out:
> > > > return r;
> > > > }
> > > >
> > > > +int dm_suspend_md(struct mapped_device *md)
> > > > +{
> > > > + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(dm_suspend_md);
> > > > +
> > > > /*
> > > > * Internal suspend/resume works like userspace-driven suspend. It waits
> > > > * until all bios finish and prevents issuing new bios to the target drivers.
> > >
> > > To do this properly you should be introducing a variant of
> > > dm_internal_suspend. We currently have two variants:
> > > dm_internal_suspend_fast
> > > dm_internal_suspend_noflush
> > >
> > > The reason to use a dm_internal_suspend variant is this suspend was
> > > _not_ initiated by an upper level ioctl (from userspace). It was
> > > done internally from within the target.
> > >
> > > You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning you're
> > > interested in flushing all pending IO (in the FS layered on dm-crupt, if
> > > one exists).
> > >
> > > But see the comment in __dm_internal_suspend() about TASK_UNINTERRUPTIBLE.
> > > If you're OK with the dm-crypt initiated suspend being
> > > TASK_UNINTERRUPTIBLE then you could just introduce:
> > >
> > > void dm_internal_suspend_uninterruptible_flush(struct mapped_device *md)
> > > {
> > > mutex_lock(&md->suspend_lock);
> > > __dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > mutex_unlock(&md->suspend_lock);
> > > }
> > > EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
> > >
> > > Otherwise, there is much more extensive DM core changes needed to
> > > __dm_internal_suspend() and .presuspend to properly support
> > > TASK_INTERRUPTIBLE.
> >
> > Hi! I will look at dm_internal_suspend. Anyway use case for suspend is
> > from dm-crypt to do both operations: suspend + key wipe. It means that
> > without entering key again from userspace, resume is not possible. So my
> > question is: It is possible to do internal suspend and then using resume
> > from userspace via ioctl?
>
> Good question: no, userspace resume would block waiting for internal
> resume.
>
> Soooo... I'll have to look at your patch 3 to understand why dm-crypt is
> needing to initiate the suspend internally but then userspace is
> initiating the resume... this imbalance is concerning.
Why not introduce a new message that allows you to wipe the key after
suspend? Both initiated from userspace.
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Mike Snitzer @ 2015-07-17 15:22 UTC (permalink / raw)
To: Pali Rohár
Cc: Alasdair Kergon, Neil Brown, Rafael J. Wysocki, Len Brown,
Pavel Machek, linux-raid, dm-devel, linux-kernel, linux-pm
In-Reply-To: <20150717142251.GA12011@pali>
On Fri, Jul 17 2015 at 10:22am -0400,
Pali Rohár <pali.rohar@gmail.com> wrote:
> On Friday 17 July 2015 10:04:39 Mike Snitzer wrote:
> > On Sun, Jun 21 2015 at 7:20am -0400,
> > Pali Rohár <pali.rohar@gmail.com> wrote:
> >
> > > This patch exports function dm_suspend_md() which suspend mapped device so other
> > > kernel drivers can use it and could suspend mapped device when needed.
> > >
> > > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > > ---
> > > drivers/md/dm.c | 6 ++++++
> > > drivers/md/dm.h | 5 +++++
> > > 2 files changed, 11 insertions(+)
> > >
> > > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > > index 2caf492..03298ff 100644
> > > --- a/drivers/md/dm.c
> > > +++ b/drivers/md/dm.c
> > > @@ -3343,6 +3343,12 @@ out:
> > > return r;
> > > }
> > >
> > > +int dm_suspend_md(struct mapped_device *md)
> > > +{
> > > + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > > +}
> > > +EXPORT_SYMBOL_GPL(dm_suspend_md);
> > > +
> > > /*
> > > * Internal suspend/resume works like userspace-driven suspend. It waits
> > > * until all bios finish and prevents issuing new bios to the target drivers.
> >
> > To do this properly you should be introducing a variant of
> > dm_internal_suspend. We currently have two variants:
> > dm_internal_suspend_fast
> > dm_internal_suspend_noflush
> >
> > The reason to use a dm_internal_suspend variant is this suspend was
> > _not_ initiated by an upper level ioctl (from userspace). It was
> > done internally from within the target.
> >
> > You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning you're
> > interested in flushing all pending IO (in the FS layered on dm-crupt, if
> > one exists).
> >
> > But see the comment in __dm_internal_suspend() about TASK_UNINTERRUPTIBLE.
> > If you're OK with the dm-crypt initiated suspend being
> > TASK_UNINTERRUPTIBLE then you could just introduce:
> >
> > void dm_internal_suspend_uninterruptible_flush(struct mapped_device *md)
> > {
> > mutex_lock(&md->suspend_lock);
> > __dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > mutex_unlock(&md->suspend_lock);
> > }
> > EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
> >
> > Otherwise, there is much more extensive DM core changes needed to
> > __dm_internal_suspend() and .presuspend to properly support
> > TASK_INTERRUPTIBLE.
>
> Hi! I will look at dm_internal_suspend. Anyway use case for suspend is
> from dm-crypt to do both operations: suspend + key wipe. It means that
> without entering key again from userspace, resume is not possible. So my
> question is: It is possible to do internal suspend and then using resume
> from userspace via ioctl?
Good question: no, userspace resume would block waiting for internal
resume.
Soooo... I'll have to look at your patch 3 to understand why dm-crypt is
needing to initiate the suspend internally but then userspace is
initiating the resume... this imbalance is concerning.
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Pali Rohár @ 2015-07-17 14:22 UTC (permalink / raw)
To: Mike Snitzer
Cc: Alasdair Kergon, Neil Brown, Rafael J. Wysocki, Len Brown,
Pavel Machek, linux-raid, dm-devel, linux-kernel, linux-pm
In-Reply-To: <20150717140438.GA29704@redhat.com>
On Friday 17 July 2015 10:04:39 Mike Snitzer wrote:
> On Sun, Jun 21 2015 at 7:20am -0400,
> Pali Rohár <pali.rohar@gmail.com> wrote:
>
> > This patch exports function dm_suspend_md() which suspend mapped device so other
> > kernel drivers can use it and could suspend mapped device when needed.
> >
> > Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> > ---
> > drivers/md/dm.c | 6 ++++++
> > drivers/md/dm.h | 5 +++++
> > 2 files changed, 11 insertions(+)
> >
> > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > index 2caf492..03298ff 100644
> > --- a/drivers/md/dm.c
> > +++ b/drivers/md/dm.c
> > @@ -3343,6 +3343,12 @@ out:
> > return r;
> > }
> >
> > +int dm_suspend_md(struct mapped_device *md)
> > +{
> > + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> > +}
> > +EXPORT_SYMBOL_GPL(dm_suspend_md);
> > +
> > /*
> > * Internal suspend/resume works like userspace-driven suspend. It waits
> > * until all bios finish and prevents issuing new bios to the target drivers.
>
> To do this properly you should be introducing a variant of
> dm_internal_suspend. We currently have two variants:
> dm_internal_suspend_fast
> dm_internal_suspend_noflush
>
> The reason to use a dm_internal_suspend variant is this suspend was
> _not_ initiated by an upper level ioctl (from userspace). It was
> done internally from within the target.
>
> You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning you're
> interested in flushing all pending IO (in the FS layered on dm-crupt, if
> one exists).
>
> But see the comment in __dm_internal_suspend() about TASK_UNINTERRUPTIBLE.
> If you're OK with the dm-crypt initiated suspend being
> TASK_UNINTERRUPTIBLE then you could just introduce:
>
> void dm_internal_suspend_uninterruptible_flush(struct mapped_device *md)
> {
> mutex_lock(&md->suspend_lock);
> __dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> mutex_unlock(&md->suspend_lock);
> }
> EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
>
> Otherwise, there is much more extensive DM core changes needed to
> __dm_internal_suspend() and .presuspend to properly support
> TASK_INTERRUPTIBLE.
Hi! I will look at dm_internal_suspend. Anyway use case for suspend is
from dm-crypt to do both operations: suspend + key wipe. It means that
without entering key again from userspace, resume is not possible. So my
question is: It is possible to do internal suspend and then using resume
from userspace via ioctl?
--
Pali Rohár
pali.rohar@gmail.com
^ permalink raw reply
* Re: [PATCH v2 2/3] dm: Export function dm_suspend_md()
From: Mike Snitzer @ 2015-07-17 14:04 UTC (permalink / raw)
To: Pali Rohár
Cc: Alasdair Kergon, Neil Brown, Rafael J. Wysocki, Len Brown,
Pavel Machek, linux-raid, dm-devel, linux-kernel, linux-pm
In-Reply-To: <1434885634-19895-3-git-send-email-pali.rohar@gmail.com>
On Sun, Jun 21 2015 at 7:20am -0400,
Pali Rohár <pali.rohar@gmail.com> wrote:
> This patch exports function dm_suspend_md() which suspend mapped device so other
> kernel drivers can use it and could suspend mapped device when needed.
>
> Signed-off-by: Pali Rohár <pali.rohar@gmail.com>
> ---
> drivers/md/dm.c | 6 ++++++
> drivers/md/dm.h | 5 +++++
> 2 files changed, 11 insertions(+)
>
> diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> index 2caf492..03298ff 100644
> --- a/drivers/md/dm.c
> +++ b/drivers/md/dm.c
> @@ -3343,6 +3343,12 @@ out:
> return r;
> }
>
> +int dm_suspend_md(struct mapped_device *md)
> +{
> + return dm_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
> +}
> +EXPORT_SYMBOL_GPL(dm_suspend_md);
> +
> /*
> * Internal suspend/resume works like userspace-driven suspend. It waits
> * until all bios finish and prevents issuing new bios to the target drivers.
To do this properly you should be introducing a variant of
dm_internal_suspend. We currently have two variants:
dm_internal_suspend_fast
dm_internal_suspend_noflush
The reason to use a dm_internal_suspend variant is this suspend was
_not_ initiated by an upper level ioctl (from userspace). It was
done internally from within the target.
You're explicitly using DM_SUSPEND_LOCKFS_FLAG above.. meaning you're
interested in flushing all pending IO (in the FS layered on dm-crupt, if
one exists).
But see the comment in __dm_internal_suspend() about TASK_UNINTERRUPTIBLE.
If you're OK with the dm-crypt initiated suspend being
TASK_UNINTERRUPTIBLE then you could just introduce:
void dm_internal_suspend_uninterruptible_flush(struct mapped_device *md)
{
mutex_lock(&md->suspend_lock);
__dm_internal_suspend(md, DM_SUSPEND_LOCKFS_FLAG);
mutex_unlock(&md->suspend_lock);
}
EXPORT_SYMBOL_GPL(dm_internal_suspend_uninterruptible_flush);
Otherwise, there is much more extensive DM core changes needed to
__dm_internal_suspend() and .presuspend to properly support
TASK_INTERRUPTIBLE.
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-17 3:47 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150716174015.GA1992137@devbig257.prn2.facebook.com>
On Thu, 16 Jul 2015 10:40:17 -0700 Shaohua Li <shli@fb.com> wrote:
> On Thu, Jul 16, 2015 at 04:07:11PM +1000, NeilBrown wrote:
> > On Wed, 15 Jul 2015 21:13:06 -0700 Shaohua Li <shli@fb.com> wrote:
> >
> > It's not a show-stopper.
> > However if you keep the backwards links, then the superblock is free to
> > point to any metadata block anywhere in the log. That means we can let
> > the pointer in the superblock be updated lazily. The cache module only
> > needs to force a superblock update if there hasn't been one for any
> > other reason.
>
> superblock never need to be updated frequently. It's only updated when
> some space will be reused or to make recovery scan less. We don't need
> the backward link to achieve this. If you check my patch, superblock is
> updated very rarely.
OK.
> > I don't see that as a work-around. But then I don't really trust
> > checksums and so would rather that we always write a metadata block
> > (with FUA or FLUSH) after writing data, and using that to know that the
> > data is safe. If we did that, it would be natural to always have a
> > valid metadata block.
>
> if you don't trust checksum, how can you make sure the metadata block is
> valid?
A checksum is useful when you have a block that you have good reason to
expect to be correct, and you want to double-check to guard against
error or corruption.
So if there is a pointer in the superblock that is supposed to always
point to a valid metadata block, then I strongly expect to find a
metadata block there. If the magic number of checksum are wrong, then
that is very surprising. Still worth checking because bugs happen at
all levels, but surprising.
If you have a pointer to something that may or may no be valid, then it
isn't particularly surprising if the checksum is wrong. That is a very
different use case.
Famously reiserfs uses (or at least "used") checksums to identify
different sorts of blocks when performing an 'fsck'. If you store the
image of a reiserfs filesystem in a file in reiserfs, then fsck could
get confused and think the data is metadata - the checksum matches.
That exact scenario would not apply to our log, but it still serves as
a warning not to put too much faith in checksums. They can tell when
something if wrong, but not when it is right.
>
> You object to add a superblock. Actually there is one with your
> proposal, the 'always valid metadata block' is acting as the role of
> superblock, since we must read metadata block to find log head. Creating
> a raid array will need create such block too. In other words, we need do
> everything a real superblock need to do with your proposal. So why don't
> we have an explict superblock instead of an implicit superblock?
I accept that a metadata block can be a de-facto superblock.
But why have two things when you can make do with one?
I'm a big fan of Occam's Razor:
Entities must not be multiplied beyond necessity
NeilBrown
^ permalink raw reply
* Re: degraded raid array with bad blocks
From: Roman Mamedov @ 2015-07-17 2:09 UTC (permalink / raw)
To: Fabian Fischer; +Cc: linux-raid
In-Reply-To: <55A7F47D.1020004@fabianfischer.org>
[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]
On Thu, 16 Jul 2015 20:14:21 +0200
Fabian Fischer <raid@fabianfischer.org> wrote:
> After booting, the removed disk wasn't re added to the array (maybe
> because of different event count). --re-add doesn't work.
> So I used --add.
As to why --re-add didn't work, I *just* had the same situation, maybe you
needed to do 'mdadm --remove /dev/md127 faulty' first.
> Because of the bad blocks on one of the remaining disks, the rebuild
> stops when reaching the first bad block. The re added disk is declared
> as spare, 2 disks active and the disk with bad blocks as faulty.
One course of action is to use dd_rescue to clone the disk with bad blocks to
a new clean disk (skipping the bad blocks as you go -- you will lose some
data), then assemble the array with the new disk in place of the cloned one and
proceed with trying to rebuild. This time it will not have bad blocks, but
will have just zeroes at those locations, so rebuild should complete
successfully. After the rebuild completes you should fsck the filesystem and
check file checksums (if you saved them), to figure out where the damage
actually landed, and restore those files from backup.
--
With respect,
Roman
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* degraded raid array with bad blocks
From: Fabian Fischer @ 2015-07-16 18:14 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]
Hi,
today I had some problems with my mdadm raid5 (4disks). Firstly I try to
explaine what happened and what the result is:
One disk in my array has some bad blocks. After some hardware-changes
one of the intact disks was thrown out of the array due to a faulty
sata-cable.
I shut down the server and replaced the cable.
After booting, the removed disk wasn't re added to the array (maybe
because of different event count). --re-add doesn't work.
So I used --add.
Because of the bad blocks on one of the remaining disks, the rebuild
stops when reaching the first bad block. The re added disk is declared
as spare, 2 disks active and the disk with bad blocks as faulty.
/dev/md127:
Version : 1.2
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 1953512960 (1863.02 GiB 2000.40 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Thu Jul 16 19:02:09 2015
State : clean, FAILED
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : FiFa-Server:0
UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Events : 107223
Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 80 1 active sync /dev/sdf
5 8 32 2 active sync /dev/sdc
6 0 0 6 removed
4 8 96 - faulty /dev/sdg
6 8 64 - spare /dev/sde
In my opinion there a 3 possibilities to get the array back working. I
am not sure whether both possibilities really exist and which one is the
most promising.
- Using the 'spare'-disk as active disk. The data on the disk
should be still there.
- Ignoring the bad blocks and loose information stored in this
blocks
- force start the array without the 'spare' disk and copy the
data to backup-storage, or does the bad block will cause the
array to fail when reaching a bad block?
In the attachment you can find the output of --examine.
In can not explain why 3 disk have a Bad Block Log. According to
smart-values only sdg has Reallocated_Sector_Ct >0
Another thing I can't explain is why sdg (which is the disk with known
bad blocks) has a lower event count.
I hope I can get some great ideas how to fix my array.
Fabian
[-- Attachment #2: examine.txt --]
[-- Type: text/plain, Size: 3741 bytes --]
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 97ccd551:8820c0e3:4ab3d67d:908a2fd9
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 9d018eea - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : 6db7566d:3c709370:12634f66:f6bfd4f6
Update Time : Thu Jul 16 19:02:09 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 4b73aeab - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : spare
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1968 sectors, after=1200 sectors
State : clean
Device UUID : 27df429f:c9661838:3a8024e4:d55055e8
Update Time : Thu Jul 16 19:02:09 2015
Checksum : b4327aa4 - correct
Events : 107223
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : .AA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x8
Array UUID : 839fb405:d0b1f13a:5a55ee42:fc8a2061
Name : FiFa-Server:0
Creation Time : Tue Apr 19 08:51:32 2011
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3907027120 (1863.02 GiB 2000.40 GB)
Array Size : 5860538880 (5589.05 GiB 6001.19 GB)
Used Dev Size : 3907025920 (1863.02 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=1200 sectors
State : clean
Device UUID : b6786dff:c6be0236:10d8f95f:63afad93
Update Time : Thu Jul 16 19:02:03 2015
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 397b22c8 - correct
Events : 107192
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: Shaohua Li @ 2015-07-16 17:40 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150716160711.08c37fc1@noble>
On Thu, Jul 16, 2015 at 04:07:11PM +1000, NeilBrown wrote:
> On Wed, 15 Jul 2015 21:13:06 -0700 Shaohua Li <shli@fb.com> wrote:
>
> > On Thu, Jul 16, 2015 at 11:22:17AM +1000, NeilBrown wrote:
> > > On Wed, 15 Jul 2015 17:07:39 -0700 Shaohua Li <shli@fb.com> wrote:
> > >
> > > > On Thu, Jul 16, 2015 at 09:16:53AM +1000, NeilBrown wrote:
> > > > > On Wed, 15 Jul 2015 12:49:37 -0700 Shaohua Li <shli@fb.com> wrote:
> > > > >
> > > > > > On Wed, Jul 15, 2015 at 02:06:41PM +1000, NeilBrown wrote:
> > > > > > > On Tue, 14 Jul 2015 20:16:17 -0700 Shaohua Li <shli@fb.com> wrote:
> > > > > > >
> > > > >
> > > > > > >
> > > > > > > > I don't
> > > > > > > > understand why you object adding a superblock for cache. The advantage
> > > > > > > > is it's self contained. And there is nothing about
> > > > > > > > complexity/maintaince, as we can store the most necessary fields into
> > > > > > > > the superblock.
> > > > > > >
> > > > > > > Because there is precisely 1 number that needs to be stored in the
> > > > > > > superblock, and there seems no point having a superblock just to store
> > > > > > > one number.
> > > > > > > It isn't much extra complexity, but any extra thing is still an extra
> > > > > > > thing.
> > > > > > > Having the data section of the log device containing just a log is
> > > > > > > elegant. Elegant is good.
> > > > > > > If we decided that keeping two copies for superblocks was a good idea
> > > > > > > (which I think it is, I just haven't created a "v1.3" layout yet), then
> > > > > > > re-using the main superblock for the head-of-log pointer would instantly
> > > > > > > give us two copies of that as well.
> > > > > >
> > > > > > I think I need 2 fields to find log head/tail in recovery. Currently
> > > > > > cache superblock records checkpoint disk position (log tail) and
> > > > > > checkpoint sequence number, which can be used to find log head. Just
> > > > > > recording log tail doesn't work well (it might work, for example,
> > > > > > zeroing sectors before log head, so we can identify log head. But it's
> > > > > > really ugly and not efficient). I only found recovery_offset can be
> > > > > > overloaded. Do you have idea other fileds can be overloaded in MD
> > > > > > superblock?
> > > > >
> > > > > If each metadata block contains
> > > > > - a magic number
> > > > > - a checksum of the block
> > > > > - a sequence number
> > > > > - a pointer to the "next" metadata block (which is equivalent to
> > > > > the size of all described data)
> > > > > - a pointer to the tail (oldest active metadata block).
> > > > >
> > > > > Then given the address of any block in the log you can easily find the
> > > > > head: walk the "next" pointers forward until you find a block
> > > > > that has the wrong magic or checksum or sequence or previous pointer.
> > > > > The last block that was consistent is the head.
> > > > >
> > > > > You can then find the tail directly, and walk forward processing the
> > > > > log.
> > > > >
> > > > > Efficiency is not really an issue. On a clean shutdown (which should
> > > > > be the norm), the md superblock will contain a pointer to the head, and
> > > > > the "next" block after that can quickly be determined to be invalid.
> > > > > On an unclean shutdown it is expected that we need to do a bit more
> > > > > work, and skipping forward along the chain to find the head of the log
> > > > > is the least of our worries.
> > > >
> > > > if superblock records 2 fileds (the log tail and the seq of log tail), metadata
> > > > block doesn't need 'a pointer to the tail (oldest active metadata block)'. The
> > > > log tail/seq pair can help us find log head easily. Adding a pointer to the
> > > > tail in every metadata block is definitionly worse than adding a filed in the
> > > > superblock.
> > >
> > > I don't really follow... maybe we are confusing terms.
> > > In my mind, the "head" is where new data gets written and the "tail" is
> > > where the oldest data is - though I can see that the reverse could also
> > > make sense.
> >
> > we are in the same page about the terms.
> > > Previously you said you wanted to record "checkpoint disk position and
> > > checkpoint sequence number" which you also referred to as the "tail".
> > > So maybe you mean the "tail" to be the most recent checkpoint?
> >
> > yes
> > > In any case, wouldn't the checkpoint metadata records its own sequence
> > > number? So why do you need to start the sequence number in the
> > > superblock as well.
> >
> > The problem is you don't know if the metadata is valid/invalid (or
> > checkpointed) according to the sequence number stored in the metadata
> > itself. But if we store a sequence number in superblock, comparing the
> > sequence numebr in superblock and sequence in metadata let us know if
> > the metadata is valid.
>
> So it basically comes down to:
> You think we should store extra information in the superblock to
> validate the thing that the superblock points to.
> I think we should always have the superblock pointing to something
> that is valid.
>
> >
> > > If you record a specific end of the log in the superblock,
> > > then you need to update the superblock any time that end moves (don't
> > > you?).
> >
> > we record the tail. recovery will search from the tail
> > > If you just record some starting point for a search, then you
> > > only need to update the superblock when that location might get
> > > over-written soon.
> >
> > that's what I'm doing. we only update the superblock when the location
> > can be reused.
> >
> > > I think it makes perfect sense for all the metadata blocks in the log
> > > to be linked together. Whether they link to the previous or to the
> > > first doesn't make a lot of difference.
> > > By "first" here, I mean the block that has been in the log for the
> > > longest, but still refers to "live" data.
> >
> > I just don't find how it's useful at all.
>
> It's not a show-stopper.
> However if you keep the backwards links, then the superblock is free to
> point to any metadata block anywhere in the log. That means we can let
> the pointer in the superblock be updated lazily. The cache module only
> needs to force a superblock update if there hasn't been one for any
> other reason.
superblock never need to be updated frequently. It's only updated when
some space will be reused or to make recovery scan less. We don't need
the backward link to achieve this. If you check my patch, superblock is
updated very rarely.
> > > >
> > > > Further, how can you handle the case when log winds. For example, initially the log is
> > > > ................................
> > > > ^ superblock points to here
> > > > then we add meta and wind
> > > > |meta n-1|meta n|meta 0|meta 1|....
> > > > ^ superblock points to here
> > > >
> > > > Next time we reload log, superblock points to a valid meta. recovery will think
> > > > this is an unclean shutdown, so we rescan the whole log disk (because all metas
> > > > are valid) and apply all the changes to raid array. this is terrible. But if
> > > > superblock stores both log tail and seq. we will find the meta 0 sequence
> > > > number doesn't match with superblock, recovery stops instantly.
> > >
> > > There should always be at least one valid metadata block in the log,
> > > and the superblock should point to one of those.
> > > On restart, it is loaded and the 'next' pointers are followed until you
> > > get to the end (most recently written) of the log. You confirm you are
> > > still in the log as sequence numbers will be increasing.
> > > After a clean shutdown, the 'next' pointer won't lead any where useful.
> >
> > This is exactly the problem. Just storing tail in superblock can't
> > handle the log wind issue efficiently, because we can't know if the
> > metadata superblock pointed to is checkpointed or not. So we need
> > workaround. Here your workaround is always keeping one valid metadata
> > block in the log. If we store both checkpoint and sequence number in
> > superblock, we don't need the workaround.
>
> I don't see that as a work-around. But then I don't really trust
> checksums and so would rather that we always write a metadata block
> (with FUA or FLUSH) after writing data, and using that to know that the
> data is safe. If we did that, it would be natural to always have a
> valid metadata block.
if you don't trust checksum, how can you make sure the metadata block is
valid?
> >
> > The log will look like:
> > initially:
> > ............
> > ^super point here, super seq = 0
> >
> > we add meta:
> > |meta 0|meta 1|...
> > ^super point here, super seq = 0
> >
> > We do a checkpoint, so space of meta 0 can be reused:
> > |meta 0|meta 1|...
> > ^super point here, super seq = 1
> >
> > if log rewind, we already checkpoint meta x:
> > |meta x|meta 1|...
> > ^super point here, super seq = x + 1
> >
> > it's easy to know meta 1 should be ignored since 1 != x + 1
> > This is the classic log structure implementation.
> >
> > > From the last metadata block, you work backwards to find the first.
> > > Maybe you walk all the way following backwards links. Maybe each
> > > metadata block has a link to the earliest block worth looking at.
> > >
> > > Then you follow the log forward recording every block that is found,
> > > and discarding blocks when you find a metadata block which says
> > > something can be discarded. Once you have walked all the way from the
> > > start to the end (from the earliest block to the most recently written
> > > block) you have a complete list of all the live blocks that are in the
> > > log. Then they can be written to the RAID.
> > >
> > > I think you are suggesting that the thing stored in the superblock is
> > > the address of the oldest valid block. I don't exactly object to that,
> > > though I feel it would require updating the superblock more often
> > > than needed.
> > No, we only updating superblock in checkpoint.
>
> hm... what exactly do you mean by "checkpoint" here?
>
> I imagine there would be more checkpoints than times when the
> superblock really needs updating.
>
> >
> > > I don't think you need the seq number though.
> > > On a clean shutdown you would write out a metadata with a new seq
> > > number and store a pointer to that.
> > > On restart, that metadata block is loaded and its sequence number
> > > examined. Any other blocks in the log will have a lower sequence number
> > > and so will be ignored.
> >
> > As I said, you can workaround the issue without recording seq number in
> > superblock, writing an extra metadata as you proposed for example. But
> > recording seq number makes things much easier and cleaner.
>
> Clearly "cleaner" is in the eye of the beholder. I think it is much
> clean for the superblock to point to something that is definitely valid.
You object to add a superblock. Actually there is one with your
proposal, the 'always valid metadata block' is acting as the role of
superblock, since we must read metadata block to find log head. Creating
a raid array will need create such block too. In other words, we need do
everything a real superblock need to do with your proposal. So why don't
we have an explict superblock instead of an implicit superblock?
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: John Stoffel @ 2015-07-16 15:07 UTC (permalink / raw)
To: NeilBrown
Cc: Shaohua Li, linux-raid, songliubraving, hch, dan.j.williams,
Kernel-team
In-Reply-To: <20150716160711.08c37fc1@noble>
So.... how will wrapping of sequence numbers be handled in the cache
metadata on disk? They can't grow for ever...
John
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox