* Re: kernel refuses devices mdadm -E accepts
[not found] ` <FCB084A5-816F-4954-971E-1C69C707F213@blub.net>
@ 2015-01-21 12:37 ` Wesley W. Terpstra
0 siblings, 0 replies; 7+ messages in thread
From: Wesley W. Terpstra @ 2015-01-21 12:37 UTC (permalink / raw)
To: Valentijn Sessink; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 956 bytes --]
On Wed, Jan 21, 2015 at 12:28 AM, Valentijn Sessink <valentyn@blub.net> wrote:
> Make a snapshot using dmsetup.
Thanks for the tip. I did not know the blockdev --setro command.
I've attached the output of smartctl -A in smartctl.log, mdadm -E in
mdadm.log, and mdstat in mdstat.log.
The mdstat needs some explanation, perhaps
sd[a-d] are 3TB hard disks
sd[e-f] are SSDs
md121, md122, md125 : swap disks
md123, md124: uefi, boot
md126: PV with LV root
md125: bcache cache disk
what is missing is a raid5 on /dev/sd[a-d]4 which is the bcache backing device
Try: madam -A -o --run --force --freeze-reshape /dev/md120 /dev/sd[a-d]4
It failed the same as before. Logs in mdadm-a.log and dmesg.log
The system was in the middle of a reshape to move the data-offset from
5120 to 8192. No disks were added or removed. All disks are healthy. I
believe the system was rebooted mid-reshape.
Could this be a bug? That madam won't continue a data-offset reshape?
[-- Attachment #2: dmesg.log --]
[-- Type: application/octet-stream, Size: 669 bytes --]
[ 5.520447] md: sdc4 does not have a valid v1.2 superblock, not importing!
[ 5.520457] md: md_import_device returned -22
[ 5.536177] md: sdb4 does not have a valid v1.2 superblock, not importing!
[ 5.536203] md: md_import_device returned -22
[ 5.536328] md: sda4 does not have a valid v1.2 superblock, not importing!
[ 5.536337] md: md_import_device returned -22
[ 5.536441] md: sdd4 does not have a valid v1.2 superblock, not importing!
[ 5.536449] md: md_import_device returned -22
[ 5.544142] md: sdc4 does not have a valid v1.2 superblock, not importing!
[ 5.544155] md: md_import_device returned -22
[ 5.544176] md: md120 stopped.
[-- Attachment #3: mdadm-a.log --]
[-- Type: application/octet-stream, Size: 308 bytes --]
mdadm: failed to add /dev/sdc4 to /dev/md128: Invalid argument
mdadm: failed to add /dev/sdb4 to /dev/md128: Invalid argument
mdadm: failed to add /dev/sda4 to /dev/md128: Invalid argument
mdadm: failed to add /dev/sdd4 to /dev/md128: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md128: Invalid argument
[-- Attachment #4: mdadm.log --]
[-- Type: application/octet-stream, Size: 3746 bytes --]
/dev/sda4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 4f83a72f:c52aa18b:59d48798:d706483b
Name : pumpkin:backing (local to host pumpkin)
Creation Time : Sat Nov 30 13:48:44 2013
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 5842897920 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Used Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Data Offset : 8192 sectors
Super Offset : 8 sectors
Unused Space : before=8104 sectors, after=0 sectors
State : clean
Device UUID : 65262ece:4cc1be5b:51627b0b:89b74d71
Update Time : Mon Jan 19 20:53:07 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 5558443c - correct
Events : 795592
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 4f83a72f:c52aa18b:59d48798:d706483b
Name : pumpkin:backing (local to host pumpkin)
Creation Time : Sat Nov 30 13:48:44 2013
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 5842897920 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Used Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Data Offset : 8192 sectors
Super Offset : 8 sectors
Unused Space : before=8104 sectors, after=0 sectors
State : clean
Device UUID : 62b10c04:5207bada:02ca566e:3df26814
Update Time : Mon Jan 19 20:53:07 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 1028b7a3 - correct
Events : 795592
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 4f83a72f:c52aa18b:59d48798:d706483b
Name : pumpkin:backing (local to host pumpkin)
Creation Time : Sat Nov 30 13:48:44 2013
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 5842897920 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Used Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Data Offset : 8192 sectors
Super Offset : 8 sectors
Unused Space : before=8112 sectors, after=0 sectors
State : clean
Device UUID : 86bfac3b:c4c39510:1b75d729:8c8b8e9f
Update Time : Mon Jan 19 20:53:07 2015
Checksum : c442c656 - correct
Events : 795592
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 4f83a72f:c52aa18b:59d48798:d706483b
Name : pumpkin:backing (local to host pumpkin)
Creation Time : Sat Nov 30 13:48:44 2013
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 5842897920 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Used Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Data Offset : 8192 sectors
Super Offset : 8 sectors
Unused Space : before=8112 sectors, after=0 sectors
State : clean
Device UUID : 68fa16c1:47652504:c49b4591:9399790e
Update Time : Mon Jan 19 20:53:07 2015
Checksum : 1395d76b - correct
Events : 795592
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
[-- Attachment #5: mdstat.log --]
[-- Type: application/octet-stream, Size: 686 bytes --]
Personalities : [raid1]
md121 : active raid1 sdb3[1] sda3[0]
8388608 blocks super 1.2 [2/2] [UU]
md122 : active raid1 sdc3[0] sdd3[1]
8388608 blocks super 1.2 [2/2] [UU]
md123 : active raid1 sdc1[2] sdb1[1] sda1[0] sdd1[3]
204800 blocks super 1.0 [4/4] [UUUU]
md124 : active raid1 sdc2[2] sda2[0] sdb2[1] sdd2[3]
204800 blocks super 1.2 [4/4] [UUUU]
md125 : active raid1 sde3[0] sdf3[1]
83886080 blocks super 1.2 [2/2] [UU]
md126 : active raid1 sde2[0] sdf2[1]
20975616 blocks super 1.2 [2/2] [UU]
md127 : active raid1 sde1[1] sdf1[0]
8388608 blocks super 1.2 [2/2] [UU]
unused devices: <none>
[-- Attachment #6: smartctl.log --]
[-- Type: application/octet-stream, Size: 7661 bytes --]
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.17.7] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 146 145 021 Pre-fail Always - 9658
4 Start_Stop_Count 0x0032 056 056 000 Old_age Always - 44077
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 076 076 000 Old_age Always - 17882
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 101
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 51
193 Load_Cycle_Count 0x0032 158 158 000 Old_age Always - 127271
194 Temperature_Celsius 0x0022 121 086 000 Old_age Always - 31
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.17.7] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 148 145 021 Pre-fail Always - 9591
4 Start_Stop_Count 0x0032 055 055 000 Old_age Always - 45623
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 076 076 000 Old_age Always - 18209
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 97
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 49
193 Load_Cycle_Count 0x0032 161 161 000 Old_age Always - 117768
194 Temperature_Celsius 0x0022 120 088 000 Old_age Always - 32
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.17.7] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 176 173 021 Pre-fail Always - 6183
4 Start_Stop_Count 0x0032 095 095 000 Old_age Always - 5666
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 082 082 000 Old_age Always - 13432
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 65
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 38
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 702407
194 Temperature_Celsius 0x0022 121 094 000 Old_age Always - 29
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.17.7] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 150 147 021 Pre-fail Always - 9491
4 Start_Stop_Count 0x0032 094 094 000 Old_age Always - 6520
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 4958
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 120
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 63
193 Load_Cycle_Count 0x0032 197 197 000 Old_age Always - 9745
194 Temperature_Celsius 0x0022 121 097 000 Old_age Always - 31
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
^ permalink raw reply [flat|nested] 7+ messages in thread