* [PATCH 3/4] mdadm/test: Get rid of tests/testdev
From: Zhilong Liu @ 2017-07-19 11:11 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1500462675-6463-1-git-send-email-zlliu@suse.com>
It's time to get rid of the tests/testdev due
to it has covered by testdev() in 'test'.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
tests/testdev | 13 -------------
1 file changed, 13 deletions(-)
delete mode 100644 tests/testdev
diff --git a/tests/testdev b/tests/testdev
deleted file mode 100644
index 8b6e6f0..0000000
--- a/tests/testdev
+++ /dev/null
@@ -1,13 +0,0 @@
-dev=$1
-cnt=$2
-size=$3
-chunk=$4
-mkfs -j $dev > /dev/null 2>&1 && fsck -fn $dev >&2
-dsize=$[size/chunk]
-dsize=$[dsize*chunk]
-rasize=$[dsize*2*cnt]
-if [ $rasize -ne `/sbin/blockdev --getsize $dev` ]
-then
- echo "ERROR: size is wrong for $dev: $cnt * $size (chunk=$chunk) = $rasize, not `/sbin/blockdev --getsize $dev`"
- exit 1;
-fi
--
1.8.3.1
^ permalink raw reply related
* [PATCH 4/4] mdadm/test: Get rid of tests/check
From: Zhilong Liu @ 2017-07-19 11:11 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1500462675-6463-1-git-send-email-zlliu@suse.com>
The tests/check is not useful due to check()
in 'test' has covered various checking.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
tests/check | 35 -----------------------------------
1 file changed, 35 deletions(-)
delete mode 100644 tests/check
diff --git a/tests/check b/tests/check
deleted file mode 100644
index f4ed6d5..0000000
--- a/tests/check
+++ /dev/null
@@ -1,35 +0,0 @@
-
-case $1 in
- raid* | linear )
- grep -s "active $1 " /proc/mdstat > /dev/null || {
- echo >&2 "ERROR active $1 not found" ; cat /proc/mdstat ; exit 1;}
- ;;
- resync | recovery )
- sleep 0.1
- grep -s $1 /proc/mdstat > /dev/null || {
- echo >&2 ERROR no $1 happening; cat /proc/mdstat; exit 1; }
- ;;
-
- nosync )
- sleep 0.5
- grep -s 're[synccovery]* =' > /dev/null /proc/mdstat && {
- echo >&2 "ERROR resync or recovery is happening!"; cat /proc/mdstat ; exit 1; }
- ;;
-
- wait )
- sleep 0.1
- while grep 're[synccovery]* =' > /dev/null /proc/mdstat
- do sleep 2;
- done
- ;;
-
- state )
- grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
- echo >&2 "ERROR state $2 not found!"; cat /proc/mdstat ; exit 1; }
- sleep 0.5
- ;;
-
- * ) echo >&2 ERROR unknown check $1 ; exit 1;
-esac
-
-exit 0
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 1/4] test/00readonly: Add unit case for testing readonly/readwrite
From: Zhilong Liu @ 2017-07-19 11:37 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1500462675-6463-2-git-send-email-zlliu@suse.com>
This is a test case for testing --readonly and
--readwrite feature, it covers common metadata
versions and raid levels.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
Changing from v1:
improve checking sysfs configure.
tests/00readonly | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
create mode 100644 tests/00readonly
diff --git a/tests/00readonly b/tests/00readonly
new file mode 100644
index 0000000..28b0fa1
--- /dev/null
+++ b/tests/00readonly
@@ -0,0 +1,22 @@
+#!/bin/bash
+
+for metadata in 0.9 1.0 1.1 1.2
+do
+ for level in linear raid0 raid1 raid4 raid5 raid6 raid10
+ do
+ mdadm -CR $md0 -l $level -n 4 --metadata=$metadata \
+ $dev1 $dev2 $dev3 $dev4 --assume-clean
+ check nosync
+ check $level
+ mdadm -ro $md0
+ check readonly
+ state=$(cat /sys/block/md0/md/array_state)
+ [ "$state" == "readonly" ] ||
+ die "array_state should be 'readonly', but is $state"
+ mdadm -w $md0
+ check $level
+ mdadm -S $md0
+ done
+done
+
+exit 0
--
2.6.6
^ permalink raw reply related
* Re: 3-disk fail on raid-6, examining my options...
From: Wols Lists @ 2017-07-19 11:51 UTC (permalink / raw)
To: Wakko Warner; +Cc: Maarten, linux-raid
In-Reply-To: <20170718202550.GA2533@animx.eu.org>
On 18/07/17 21:25, Wakko Warner wrote:
> Wols Lists wrote:
>> On 18/07/17 18:20, Maarten wrote:
>>> Now from what I've gathered over the years and from earlier incidents, I
>>> have now 1 (one) chance left to rescue data off this array; by hopefully
>>> cloning the bad 3rd-failed drive with the aid of dd_rescue and
>>> re-assembling --force the fully-degraded array. (Only IF that drive is
>>> still responsive and can be cloned)
>>
>> If it clones successfully, great. If it clones, but with badblocks, I
>> keep on asking - is there any way we can work together to turn
>> dd-rescue's log into a utility that will flag failed blocks as "unreadable"?
>
> I wrote a shell script that will output a device mapper table to do this.
> It will do either zero or error targets for failed blocks. It's not
> automatic and does require a block device (loop for files). I've used this
> several times at work and works for me.
>
> I'm not sure if this is what you're talking about or not, but if you want
> the script, I'll post it.
>
I'm not sure I understand what you're saying, but I'm certainly
interested. It'll probably end up on the wiki if that's okay with you?
I'll aim to understand and document it so others will be able hopefully
to use it as a "fire and forget" tool (inasmuch as you can
fire-and-forget any recovery task :-)
What I'm thinking of is a utility that uses "hdparm --make-bad-sector".
The idea being that if you have multiple disk failures, you can at least
clone everything worth having off the broken disks, and then you can run
a "tar . > /dev/null" or do a sync or whatever, and know that if it
reads successfully off the array it isn't corrupt. Unless you're unlucky
enough to have multiple drives fail in the same stripe, you should then
recover your array no problem.
Cheers,
Wol
^ permalink raw reply
* Re: Advice please re failed Raid6
From: Peter Grandi @ 2017-07-19 12:36 UTC (permalink / raw)
To: Linux Raid
In-Reply-To: <f79a45b9-590b-4d3c-d387-81abfb02743d@gmail.com>
> Did I do it right? (See below)
> root@keruru:~# mdadm --create --assume-clean --level=6 --raid-devices=4
> --size=1953382912 /dev/md0 missing /dev/sdc /dev/sdd /dev/sde
> mdadm: /dev/sdc appears to be part of a raid array:
> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
> mdadm: /dev/sdd appears to be part of a raid array:
> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
> mdadm: /dev/sde appears to be part of a raid array:
> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
> Continue creating array? y
> mdadm: Defaulting to version 1.2 metadata
> mdadm: array /dev/md0 started.
This looks good, but is based on your original '--examine'
report as to the order of the devices, and whether they are
still bound to the same names 'sd[bcde]'.
> root@keruru:~# blkid /dev/md0
> root@keruru:~# cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md0 : active (auto-read-only) raid6 sde[3] sdd[2] sdc[1]
> 3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
> [4/3] [_UUU]
> unused devices: <none>
The 'mdstat' actually looks good, but 'blkid' should have
worked.
As I was saying, it is not clear to me whether the 'mdadm' daemon
instance triggered a 'check' or a 'repair' (bad news). I hope
that you disabled that in the meantime while you try to fix the
mistake.
Trigger a 'check' and see if the set is consistent; if it is
consistent but the content cannot be read/mounted then 'repair'
rewrote it, if it is not consistent, try a different order or
3-way subset of 'sd[bcde]'.
^ permalink raw reply
* raid5 using group_thread
From: Ofer Heifetz @ 2017-07-19 13:00 UTC (permalink / raw)
To: linux-raid@vger.kernel.org
Hi,
I have a question regarding raid5 built using group_thread and async_tx, from code (v4.4 and even v4.12) I see that only raid5d invokes async_tx_issue_pending_all, shouldn't the raid5_do_work also invoke this API to issue
all pending requests to HW?
I am assuming that there is no sync mechanism between the raid5d and the raid5_do_work, correct me if I am wrong.
Thanks,
/Ofer
^ permalink raw reply
* Re: 3-disk fail on raid-6, examining my options...
From: Wakko Warner @ 2017-07-19 17:09 UTC (permalink / raw)
To: Wols Lists; +Cc: Maarten, linux-raid
In-Reply-To: <596F47CF.6020007@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 3462 bytes --]
Wols Lists wrote:
> On 18/07/17 21:25, Wakko Warner wrote:
> > Wols Lists wrote:
> >> On 18/07/17 18:20, Maarten wrote:
> >>> Now from what I've gathered over the years and from earlier incidents, I
> >>> have now 1 (one) chance left to rescue data off this array; by hopefully
> >>> cloning the bad 3rd-failed drive with the aid of dd_rescue and
> >>> re-assembling --force the fully-degraded array. (Only IF that drive is
> >>> still responsive and can be cloned)
> >>
> >> If it clones successfully, great. If it clones, but with badblocks, I
> >> keep on asking - is there any way we can work together to turn
> >> dd-rescue's log into a utility that will flag failed blocks as "unreadable"?
> >
> > I wrote a shell script that will output a device mapper table to do this.
> > It will do either zero or error targets for failed blocks. It's not
> > automatic and does require a block device (loop for files). I've used this
> > several times at work and works for me.
> >
> > I'm not sure if this is what you're talking about or not, but if you want
> > the script, I'll post it.
> >
> I'm not sure I understand what you're saying, but I'm certainly
> interested. It'll probably end up on the wiki if that's okay with you?
That's fine.
> I'll aim to understand and document it so others will be able hopefully
> to use it as a "fire and forget" tool (inasmuch as you can
> fire-and-forget any recovery task :-)
>
> What I'm thinking of is a utility that uses "hdparm --make-bad-sector".
> The idea being that if you have multiple disk failures, you can at least
> clone everything worth having off the broken disks, and then you can run
> a "tar . > /dev/null" or do a sync or whatever, and know that if it
> reads successfully off the array it isn't corrupt. Unless you're unlucky
> enough to have multiple drives fail in the same stripe, you should then
> recover your array no problem.
That's pretty much how I use it in a way.
Here's a real ddrescue log from one that I did:
# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue -s 85900394496 /dev/sdg /path/to/image.img /path/to/image.log
# current_pos current_status
0xA078F9C00 +
# pos size status
0x00000000 0xA078F9000 +
0xA078F9000 0x00001000 -
0xA078FA000 0x9F8806000 +
0x1400100000 0x2638A2E000 ?
I use losetup to make /path/to/image.img a block device.
I run the script I wrote:
sh ddlog-to-dm.sh /dev/loop0 < /path/to/image.log
Which outputs the following:
0 84133832 linear /dev/loop0 0
84133832 8 error
84133840 83640368 linear /dev/loop0 84133840
Then I run:
dmsetup create sometarget
I paste in the output and I now have /dev/mapper/sometarget that has errors
at the location that was bad. Since it uses device mapper, the error part
doesn't retry. This will work with hard disks instead of images.
To work with a real disk, skip the losetup part and use /dev/sdX instead of
/dev/loop0. In my case above, assume I closed sdg to sdh, I would do:
sh ddlog-to-dm.sh /dev/sdh < /path/to/image.log
dmsetup create sdh
Then use /dev/mapper/sdh.
If you're familiar with device mapper, there are no partitions, you have to
create another target. I use kpartx -a for this and when I'm done, I use
kpartx -d to tear it down.
When you're done, dmsetup remove sometarget and remove the loop device.
I have attached the script.
--
Microsoft has beaten Volkswagen's world record. Volkswagen only created 22
million bugs.
[-- Attachment #2: ddlog-to-dm.sh --]
[-- Type: application/x-sh, Size: 2205 bytes --]
^ permalink raw reply
* Re: raid5 using group_thread
From: Shaohua Li @ 2017-07-19 18:36 UTC (permalink / raw)
To: Ofer Heifetz; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <721dda9c0d5f4df0838a7365a53e8e04@IL-EXCH01.marvell.com>
On Wed, Jul 19, 2017 at 01:00:45PM +0000, Ofer Heifetz wrote:
> Hi,
>
> I have a question regarding raid5 built using group_thread and async_tx, from code (v4.4 and even v4.12) I see that only raid5d invokes async_tx_issue_pending_all, shouldn't the raid5_do_work also invoke this API to issue
> all pending requests to HW?
>
> I am assuming that there is no sync mechanism between the raid5d and the raid5_do_work, correct me if I am wrong.
Can't remember why we don't call async_tx_issue_pending_all in raid5_do_work,
it shouldn't harm. In practice, I doubt calling it makes a change, because when
workers are running, raid5d are running too. Did you benchmark it?
Thanks,
Shaohua
^ permalink raw reply
* Re: Why can't I re-add my drive after partition shrink?
From: Ram Ramesh @ 2017-07-19 20:55 UTC (permalink / raw)
To: NeilBrown, Linux Raid
In-Reply-To: <8d3e1264-2f0a-6ee9-52b5-edddc1e6f185@gmail.com>
<snip>
>>> Since the problem, I did not want to leave my md in degraded state.
>>> So, I added my drive back and paid the penalty for rebuilding. I have
>>> other disks that need to be resized and *can get you want*. Please let
>>> me know if that is what you meant. If you wanted the current info
>>> after
>>> successfully rebuilding the array after a regular add, it is below.
>> I only requested the information because it might help fix, or explain,
>> your difficulty. If you don't currently have a difficulty, then I don't
>> need to look at any details.
>>
>> Thanks,
>> NeilBrown
>
> Thanks for your time. Yes, I still have the problem as I need to
> shrink other 5 disks in the array and I like to re-add rather than add
> and rebuild each time.
>
> The host with the array is currently busy, and I will get this info
> tomorrow when I attempt the process on my next hard drive.
>
> Ramesh
>
Here is my attempt to repeat the steps in my last attempt to remove,
repartition, re-add. Last time I did it on /dev/sdb. Now I am going to
do it on /dev/sdc. Note that I have not been successful as you see at
the end. I am going to keep the array degraded so that I can still get
old info from /dev/sdc1, if you need anything else. I will keep it this
way till tomorrow and then add the device for md to rebuild. Please ask
anything else before that or send me a note to keep the array degraded
so that you can examine /dev/sdc1 more.
<start>
<current-status>
> zym [rramesh] 251 > cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md0 : active raid6 sdb1[6] sdg1[11] sdd1[12] sdf1[8] sde1[9] sdc1[10]
> 12348030976 blocks super 1.2 level 6, 64k chunk, algorithm 2
> [6/6] [UUUUUU]
> bitmap: 0/23 pages [0KB], 65536KB chunk
>
> unused devices: <none>
<sdc partitions before any changes>
> zym [rramesh] 252 > sudo gdisk -l /dev/sdc
> GPT fdisk (gdisk) version 0.8.8
>
> Partition table scan:
> MBR: protective
> BSD: not present
> APM: not present
> GPT: present
>
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdc: 11721045168 sectors, 5.5 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): EF5E7965-FC30-4137-9DDC-1B2C7966B936
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 11721045134
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
>
> Number Start (sector) End (sector) Size Code Name
> 1 2048 11721045134 5.5 TiB FD00 Linux RAID
<sdc mdadm info before any changes>
> zym [rramesh] 253 > sudo mdadm --examine /dev/sdc1
> /dev/sdc1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Name : zym:0 (local to host zym)
> Creation Time : Mon Apr 22 00:08:12 2013
> Raid Level : raid6
> Raid Devices : 6
>
> Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
> Array Size : 12348030976 (11776.00 GiB 12644.38 GB)
> Used Dev Size : 6174015488 (2944.00 GiB 3161.10 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : 7e035b56:d1e1882b:e78a08ad:3ba50667
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Wed Jul 19 15:12:46 2017
> Checksum : a52ef205 - correct
> Events : 297182
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Device Role : Active device 2
> Array State : AAAAAA ('A' == active, '.' == missing)
> zym [rramesh] 256 > sudo mdadm --examine-bitmap /dev/sdc1
> Filename : /dev/sdc1
> Magic : 6d746962
> Version : 4
> UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Events : 297182
> Events Cleared : 297182
> State : OK
> Chunksize : 64 MB
> Daemon : 5s flush period
> Write Mode : Normal
> Sync Size : 3087007744 (2944.00 GiB 3161.10 GB)
> Bitmap : 47104 bits (chunks), 0 dirty (0.0%)
<removal and repartition begins>
> zym [rramesh] 254 > sudo mdadm /dev/md0 --fail /dev/sdc1
> mdadm: set /dev/sdc1 faulty in /dev/md0
>
> zym [rramesh] 255 > sudo mdadm /dev/md0 --remove /dev/sdc1
> mdadm: hot removed /dev/sdc1 from /dev/md0
>
> zym [rramesh] 261 > gdisk /dev/sdc
> <snip>
> Command (? for help): p
> <snip>
>
> Number Start (sector) End (sector) Size Code Name
> 1 2048 11721045134 5.5 TiB FD00 Linux RAID
>
> Command (? for help): d
> Using 1
>
> Command (? for help): n
> Partition number (1-128, default 1):
> First sector (34-11721045134, default = 2048) or {+-}size{KMGTP}:
> Last sector (2048-11721045134, default = 11721045134) or
> {+-}size{KMGTP}: 6442452991
> Current type is 'Linux filesystem'
> Hex code or GUID (L to show codes, Enter = 8300): FD00
> Changed type of partition to 'Linux RAID'
>
> Command (? for help): n
> Partition number (2-128, default 2):
> First sector (34-11721045134, default = 6442452992) or {+-}size{KMGTP}:
> Last sector (6442452992-11721045134, default = 11721045134) or
> {+-}size{KMGTP}:
> Current type is 'Linux filesystem'
> Hex code or GUID (L to show codes, Enter = 8300): FD00
> Changed type of partition to 'Linux RAID'
>
> Command (? for help): p
> Disk /dev/sdc: 11721045168 sectors, 5.5 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): EF5E7965-FC30-4137-9DDC-1B2C7966B936
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 11721045134
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
>
> Number Start (sector) End (sector) Size Code Name
> 1 2048 6442452991 3.0 TiB FD00 Linux RAID
> 2 6442452992 11721045134 2.5 TiB FD00 Linux RAID
>
> Command (? for help): w
>
> Final checks complete. About to write GPT data. THIS WILL OVERWRITE
> EXISTING
> PARTITIONS!!
>
> Do you want to proceed? (Y/N): Y
> OK; writing new GUID partition table (GPT) to /dev/sdc.
> The operation has completed successfully.
>
<good device that is still in md0>
> zym [rramesh] 264 > cat /proc/partitions |fgrep sdb
> 8 16 5860522584 sdb
> 8 17 3221225472 sdb1
> 8 18 2639296071 sdb2
<device just removed and repartitioned>
> zym [rramesh] 271 > cat /proc/partitions |fgrep sdc
> 8 32 5860522584 sdc
> 8 33 3221225472 sdc1
> 8 34 2639296071 sdc2
<good device still in md0>
> zym [rramesh] 265 > sudo mdadm --examine /dev/sdb1
> /dev/sdb1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Name : zym:0 (local to host zym)
> Creation Time : Mon Apr 22 00:08:12 2013
> Raid Level : raid6
> Raid Devices : 6
>
> Avail Dev Size : 6442188800 (3071.88 GiB 3298.40 GB)
> Array Size : 12348030976 (11776.00 GiB 12644.38 GB)
> Used Dev Size : 6174015488 (2944.00 GiB 3161.10 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : 702ca77d:564d69ff:e45d9679:64c314fa
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Wed Jul 19 15:15:00 2017
> Checksum : c5578b94 - correct
> Events : 297185
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Device Role : Active device 4
> Array State : AA.AAA ('A' == active, '.' == missing)
<good device still in md0>
> zym [rramesh] 266 > sudo mdadm --examine-bitmap /dev/sdb1
> Filename : /dev/sdb1
> Magic : 6d746962
> Version : 4
> UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Events : 297185
> Events Cleared : 297182
> State : OK
> Chunksize : 64 MB
> Daemon : 5s flush period
> Write Mode : Normal
> Sync Size : 3087007744 (2944.00 GiB 3161.10 GB)
> Bitmap : 47104 bits (chunks), 0 dirty (0.0%)
<device just removed and repartitioned>
> zym [rramesh] 267 > sudo mdadm --examine /dev/sdc1
> /dev/sdc1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Name : zym:0 (local to host zym)
> Creation Time : Mon Apr 22 00:08:12 2013
> Raid Level : raid6
> Raid Devices : 6
>
> Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
> Array Size : 12348030976 (11776.00 GiB 12644.38 GB)
> Used Dev Size : 6174015488 (2944.00 GiB 3161.10 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> State : clean
> Device UUID : 7e035b56:d1e1882b:e78a08ad:3ba50667
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Wed Jul 19 15:12:46 2017
> Checksum : a52ef205 - correct
> Events : 297182
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Device Role : Active device 2
> Array State : AAAAAA ('A' == active, '.' == missing)
<device just removed and repartitioned>
> zym [rramesh] 268 > sudo mdadm --examine-bitmap /dev/sdc1
> Filename : /dev/sdc1
> Magic : 6d746962
> Version : 4
> UUID : 0e9f76b5:4a89171a:a930bccd:78749144
> Events : 297182
> Events Cleared : 297182
> State : OK
> Chunksize : 64 MB
> Daemon : 5s flush period
> Write Mode : Normal
> Sync Size : 3087007744 (2944.00 GiB 3161.10 GB)
> Bitmap : 47104 bits (chunks), 0 dirty (0.0%)
> zym [rramesh] 269 > cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md0 : active raid6 sdb1[6] sdg1[11] sdd1[12] sdf1[8] sde1[9]
> 12348030976 blocks super 1.2 level 6, 64k chunk, algorithm 2
> [6/5] [UU_UUU]
> bitmap: 0/23 pages [0KB], 65536KB chunk
>
> unused devices: <none>
<Cannot re-add!!!!>
> zym [rramesh] 270 > sudo mdadm /dev/md0 --re-add /dev/sdc1
> mdadm: --re-add for /dev/sdc1 to /dev/md0 is not possible
I have not added this device yet and I am keeping the array degraded,
just in case you need anything else. I will do so till tomorrow. After
that I will simply add the device so that it will rebuild unless you ask
for delay or additional info.
Ramesh
^ permalink raw reply
* Re: Why can't I re-add my drive after partition shrink?
From: NeilBrown @ 2017-07-19 23:14 UTC (permalink / raw)
To: Ram Ramesh, Linux Raid
In-Reply-To: <ad3c5c59-dc37-2573-341a-39a381621746@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]
On Wed, Jul 19 2017, Ram Ramesh wrote:
> Here is my attempt to repeat the steps in my last attempt to remove,
> repartition, re-add. Last time I did it on /dev/sdb. Now I am going to
> do it on /dev/sdc. Note that I have not been successful as you see at
> the end. I am going to keep the array degraded so that I can still get
> old info from /dev/sdc1, if you need anything else. I will keep it this
> way till tomorrow and then add the device for md to rebuild. Please ask
> anything else before that or send me a note to keep the array degraded
> so that you can examine /dev/sdc1 more.
Thanks. I *love* getting all the details. You cannot send too many
details!
This:
> <good device still in md0>
>> zym [rramesh] 265 > sudo mdadm --examine /dev/sdb1
>> /dev/sdb1:
..
>> Avail Dev Size : 6442188800 (3071.88 GiB 3298.40 GB)
and this:
> <device just removed and repartitioned>
>> zym [rramesh] 267 > sudo mdadm --examine /dev/sdc1
>> /dev/sdc1:
...
>> Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Shows the key difference. "Avail Dev Size", aka sb->data_size, is
wrong. We can fix it.
>
> <Cannot re-add!!!!>
>> zym [rramesh] 270 > sudo mdadm /dev/md0 --re-add /dev/sdc1
>> mdadm: --re-add for /dev/sdc1 to /dev/md0 is not possible
Please try
sudo mdadm /dev/md0 --re-add /dev/sdc1 --update=devicesize
Thanks,
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: Why can't I re-add my drive after partition shrink?
From: Ram Ramesh @ 2017-07-20 0:39 UTC (permalink / raw)
To: NeilBrown, Linux Raid
In-Reply-To: <87fudsxbmz.fsf@notabene.neil.brown.name>
On 07/19/2017 06:14 PM, NeilBrown wrote:
> On Wed, Jul 19 2017, Ram Ramesh wrote:
>
>> Here is my attempt to repeat the steps in my last attempt to remove,
>> repartition, re-add. Last time I did it on /dev/sdb. Now I am going to
>> do it on /dev/sdc. Note that I have not been successful as you see at
>> the end. I am going to keep the array degraded so that I can still get
>> old info from /dev/sdc1, if you need anything else. I will keep it this
>> way till tomorrow and then add the device for md to rebuild. Please ask
>> anything else before that or send me a note to keep the array degraded
>> so that you can examine /dev/sdc1 more.
> Thanks. I *love* getting all the details. You cannot send too many
> details!
>
> This:
>> <good device still in md0>
>>> zym [rramesh] 265 > sudo mdadm --examine /dev/sdb1
>>> /dev/sdb1:
> ..
>>> Avail Dev Size : 6442188800 (3071.88 GiB 3298.40 GB)
> and this:
>
>> <device just removed and repartitioned>
>>> zym [rramesh] 267 > sudo mdadm --examine /dev/sdc1
>>> /dev/sdc1:
> ...
>>> Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
> Shows the key difference. "Avail Dev Size", aka sb->data_size, is
> wrong. We can fix it.
>
>> <Cannot re-add!!!!>
>>> zym [rramesh] 270 > sudo mdadm /dev/md0 --re-add /dev/sdc1
>>> mdadm: --re-add for /dev/sdc1 to /dev/md0 is not possible
> Please try
> sudo mdadm /dev/md0 --re-add /dev/sdc1 --update=devicesize
>
> Thanks,
> NeilBrown
Neil,
Thanks a ton. That does it. It got re-added without any issue. It is
rebuilding because the array was used to record two TV programs when it
was in degraded state. But the re-add is accepted.
> zym [rramesh] 274 > sudo mdadm /dev/md0 --re-add /dev/sdc1
> --update=devicesize
> Size was 11720780943
> Size is 6442188800
> mdadm: re-added /dev/sdc1
> zym [rramesh] 275 > cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md0 : active raid6 sdc1[10] sdb1[6] sdg1[11] sdd1[12] sdf1[8] sde1[9]
> 12348030976 blocks super 1.2 level 6, 64k chunk, algorithm 2
> [6/5] [UU_UUU]
> [========>............] recovery = 42.6%
> (1316769920/3087007744) finish=292.2min speed=100952K/sec
> bitmap: 2/23 pages [8KB], 65536KB chunk
>
> unused devices: <none>
Wol,
If you read this, this may worth a mention on wiki page.
Ramesh
^ permalink raw reply
* Re: Advice please re failed Raid6
From: Bogo Mipps @ 2017-07-20 3:59 UTC (permalink / raw)
To: Peter Grandi, Linux Raid
In-Reply-To: <22895.21096.215892.928052@tree.ty.sabi.co.uk>
On 07/20/2017 12:36 AM, Peter Grandi wrote:
>> Did I do it right? (See below)
>
>> root@keruru:~# mdadm --create --assume-clean --level=6 --raid-devices=4
>> --size=1953382912 /dev/md0 missing /dev/sdc /dev/sdd /dev/sde
>> mdadm: /dev/sdc appears to be part of a raid array:
>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>> mdadm: /dev/sdd appears to be part of a raid array:
>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>> mdadm: /dev/sde appears to be part of a raid array:
>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>> Continue creating array? y
>> mdadm: Defaulting to version 1.2 metadata
>> mdadm: array /dev/md0 started.
>
> This looks good, but is based on your original '--examine'
> report as to the order of the devices, and whether they are
> still bound to the same names 'sd[bcde]'.
>
>> root@keruru:~# blkid /dev/md0
>
>> root@keruru:~# cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4]
>> md0 : active (auto-read-only) raid6 sde[3] sdd[2] sdc[1]
>> 3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
>> [4/3] [_UUU]
>
>> unused devices: <none>
>
> The 'mdstat' actually looks good, but 'blkid' should have
> worked.
>
> As I was saying, it is not clear to me whether the 'mdadm' daemon
> instance triggered a 'check' or a 'repair' (bad news). I hope
> that you disabled that in the meantime while you try to fix the
> mistake.
>
> Trigger a 'check' and see if the set is consistent; if it is
> consistent but the content cannot be read/mounted then 'repair'
> rewrote it, if it is not consistent, try a different order or
> 3-way subset of 'sd[bcde]'.
Tried different order: sde, sdc, sdd. blkid worked. Added sdb as you
suggested. Currently rebuilding. Log below. Fingers crossed. Will
report result.
root@keruru:~# mdadm --create --assume-clean --level=6
--raid-devices=4 /dev/md0 /dev/sde /dev/sdc /dev/sdd missing
mdadm: /dev/sde appears to be part of a raid array:
level=raid6 devices=4 ctime=Wed Jul 19 13:40:04 2017
mdadm: /dev/sdc appears to be part of a raid array:
level=raid6 devices=4 ctime=Wed Jul 19 13:40:04 2017
mdadm: /dev/sdd appears to be part of a raid array:
level=raid6 devices=4 ctime=Wed Jul 19 13:40:04 2017
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@keruru:~# blkid
/dev/sda1: UUID="3a26cce9-4598-4000-a921-1cde5ba78682" TYPE="ext4"
/dev/sda5: UUID="9761c2be-fef2-4ec6-8fff-dd69aa9e4eb2" TYPE="swap"
/dev/sdd: UUID="ca038387-a44e-2bb8-c2fd-abe8e9062ffd"
UUID_SUB="8e502cd5-74ef-7b15-1bc3-6f2465e7a695" LABEL="keruru:0"
TYPE="linux_raid_member"
/dev/sdc: UUID="ca038387-a44e-2bb8-c2fd-abe8e9062ffd"
UUID_SUB="aeb6626b-3dc2-3fa8-0fd5-947081866c49" LABEL="keruru:0"
TYPE="linux_raid_member"
/dev/sde: UUID="ca038387-a44e-2bb8-c2fd-abe8e9062ffd"
UUID_SUB="bfff697b-c3ae-9e70-9ca9-ba168a30cbd2" LABEL="keruru:0"
TYPE="linux_raid_member"
root@keruru:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active (auto-read-only) raid6 sdd[2] sdc[1] sde[0]
3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
[4/3] [UUU_]
unused devices: <none>
root@keruru:~# mdadm --add /dev/md0 /dev/sdb
mdadm: added /dev/sdb
root@keruru:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdb[4] sdd[2] sdc[1] sde[0]
3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
[4/3] [UUU_]
[>....................] recovery = 0.0% (1258088/1953382912)
finish=413.7min speed=78630K/sec
unused devices: <none>
root@keruru:~#
^ permalink raw reply
* [PATCH] mdadm/bitmap: regular file descriptor is omissive in bitmap_file_open
From: Zhilong Liu @ 2017-07-20 7:11 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
External bitmap supports to store bitmap in a regular file,
bitmap_file_open() also supports to return the regular file
descriptor.
This commit is partial revert of commit 0a6bff09d416
(mdadm/util: unify fstat checking blkdev into function)
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
bitmap.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/bitmap.c b/bitmap.c
index 3653660..989cfb1 100644
--- a/bitmap.c
+++ b/bitmap.c
@@ -183,6 +183,7 @@ static int
bitmap_file_open(char *filename, struct supertype **stp, int node_num)
{
int fd;
+ struct stat stb;
struct supertype *st = *stp;
fd = open(filename, O_RDONLY|O_DIRECT);
@@ -191,8 +192,13 @@ bitmap_file_open(char *filename, struct supertype **stp, int node_num)
filename, strerror(errno));
return -1;
}
-
- if (fstat_is_blkdev(fd, filename, NULL)) {
+ if (fstat(fd, &stb) < 0) {
+ pr_err("fstat failed for %s: %s\n",
+ filename, strerror(errno));
+ close(fd);
+ return -1;
+ }
+ if ((stb.st_mode & S_IFMT) == S_IFBLK) {
/* block device, so we are probably after an internal bitmap */
if (!st)
st = guess_super(fd);
@@ -211,11 +217,7 @@ bitmap_file_open(char *filename, struct supertype **stp, int node_num)
fd = -1;
}
}
-
*stp = st;
- } else {
- close(fd);
- return -1;
}
return fd;
--
2.6.6
^ permalink raw reply related
* Re: raid5 using group_thread
From: Ofer Heifetz @ 2017-07-20 7:21 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid@vger.kernel.org
> Hi Li,
> > ----------------------------------------------------------------------
> > On Wed, Jul 19, 2017 at 01:00:45PM +0000, Ofer Heifetz wrote:
> > > Hi,
> > >
> > > I have a question regarding raid5 built using group_thread and
> > > async_tx, from code (v4.4 and even v4.12) I see that only raid5d invokes
> > async_tx_issue_pending_all, shouldn't the raid5_do_work also invoke this
> > API to issue all pending requests to HW?
> > >
> > > I am assuming that there is no sync mechanism between the raid5d and the
> > raid5_do_work, correct me if I am wrong.
> >
> > Can't remember why we don't call async_tx_issue_pending_all in
> > raid5_do_work, it shouldn't harm. In practice, I doubt calling it makes a
> > change, because when workers are running, raid5d are running too. Did you
> > benchmark it?
>
> I had a jbd2 hung issue on my system and started to debug it, I noticed that in the cases it was stuck, It had pending requests in the async_xor engine waiting to be
> issued, so basically requests were sitting in the HW ring and engine was unaware of their existence, this caused the following:
> [ 1320.280225] INFO: task jbd2/md0-8:1755 blocked for more than 120 seconds.
> [ 1320.287056] Not tainted 4.4.52-gdbc4936-dirty #45
> [ 1320.294054] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 1320.301922] jbd2/md0-8 D ffffffc000086cc0 0 1755 2 0x00000000
> [ 1320.309037] Call trace:
> [ 1320.311502] [<ffffffc000086cc0>] __switch_to+0x88/0xa0
> [ 1320.316677] [<ffffffc0008c55d0>] __schedule+0x190/0x5d8
> [ 1320.321935] [<ffffffc0008c5a5c>] schedule+0x44/0xb8
> [ 1320.326842] [<ffffffc00026f194>] jbd2_journal_commit_transaction+0x174/0x13e0
> [ 1320.334018] [<ffffffc00027378c>] kjournald2+0xc4/0x248
> [ 1320.339185] [<ffffffc0000d2bac>] kthread+0xdc/0xf0
> [ 1320.344006] [<ffffffc000085dd0>] ret_from_fork+0x10/0x40
> [ 1320.349349] INFO: task ext4lazyinit:1757 blocked for more than 120 seconds.
> [ 1320.356350] Not tainted 4.4.52-gdbc4936-dirty #45
> [ 1320.363347] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 1320.371214] ext4lazyinit D ffffffc000086cc0 0 1757 2 0x00000000
> [ 1320.378328] Call trace:
> [ 1320.380793] [<ffffffc000086cc0>] __switch_to+0x88/0xa0
> [ 1320.385964] [<ffffffc0008c55d0>] __schedule+0x190/0x5d8
> [ 1320.391218] [<ffffffc0008c5a5c>] schedule+0x44/0xb8
> [ 1320.396126] [<ffffffc0008c86f4>] schedule_timeout+0x15c/0x1b0
> [ 1320.401904] [<ffffffc0008c53c8>] io_schedule_timeout+0xb0/0x128
> [ 1320.407861] [<ffffffc0008c63e0>] bit_wait_io+0x18/0x70
> [ 1320.413033] [<ffffffc0008c6288>] __wait_on_bit_lock+0x80/0xf0
> [ 1320.418810] [<ffffffc0008c6354>] out_of_line_wait_on_bit_lock+0x5c/0x68
> [ 1320.425465] [<ffffffc0001da528>] __lock_buffer+0x38/0x48
> [ 1320.430809] [<ffffffc00026d254>] do_get_write_access+0x26c/0x540
> [ 1320.436848] [<ffffffc00026d568>] jbd2_journal_get_write_access+0x40/0x88
> [ 1320.443593] [<ffffffc00024c0bc>] __ext4_journal_get_write_access+0x34/0x88
> [ 1320.450511] [<ffffffc0002279d0>] ext4_init_inode_table+0x118/0x3c0
> [ 1320.456728] [<ffffffc000239a04>] ext4_lazyinit_thread+0x1ec/0x2b8
> [ 1320.462866] [<ffffffc0000d2bac>] kthread+0xdc/0xf0
> [ 1320.467691] [<ffffffc000085dd0>] ret_from_fork+0x10/0x40
>
>Then I went to the raid5 code and noticed that only raid5d performs the async_tx_issue_pending which seems strange, for it to work right it must be the last one calling r5l_flush_stripe_to_raid
>thus waiting for the workers to finish their r5l_flush_stripe_to_raid calls, based on the code there is no such sync point between the raid5d and raid5_do_work.
>
>I can test the performance impact but with the current code I get hung task which basically forces me to disable group_thread_cnt.
>
>/Ofer
> > Thanks,
> > Shaohua
^ permalink raw reply
* [PATCH -v4 1/1] mdadm/test: Add one test case for raid5 reshape
From: Xiao Ni @ 2017-07-20 7:58 UTC (permalink / raw)
To: linux-raid; +Cc: jes.sorensen
This case tries to allow raid5 reshape to use backwards direction.
It changes chunksize after reshape and stops the raid. Then starts
the raid again.
Signed-off-by: Xiao Ni <xni@redhat.com>
Suggested-by: Jes Sorensen <jes.sorensen@gmail.com>
Suggested-by: Zhilong Liu <zlliu@suse.com>
Suggested-by: Paul Menzel <pmenzel@molgen.mpg.de>
---
test | 7 +++++++
tests/02r5grow | 17 +++++++++++++++++
2 files changed, 24 insertions(+)
diff --git a/test b/test
index 87e2df2..98e2981 100755
--- a/test
+++ b/test
@@ -320,6 +320,13 @@ check() {
grep -sq "inactive" /proc/mdstat ||
die "array is not inactive!"
;;
+ # It only can be used when there is only one raid
+ chunk )
+ chunk_size=`awk -F',' '/chunk/{print $2}' /proc/mdstat | awk -F'[a-z]' '{print $1}'`
+ if [ "$chunk_size" -ne "$2" ] ; then
+ die "chunksize should be $2, but it's $chunk_size"
+ fi
+ ;;
* )
die "unknown check $1"
;;
diff --git a/tests/02r5grow b/tests/02r5grow
index 386e82e..ea3101b 100644
--- a/tests/02r5grow
+++ b/tests/02r5grow
@@ -34,3 +34,20 @@ check nosync
sh tests/testdev $md0 3 $[size/2] 128
mdadm -S $md0
+
+# create a raid5 array and change the chunk
+mdadm -CR $md0 --level raid5 --metadata=1.1 --chunk=32 --raid-disks 3 --size $[size/2] $dev1 $dev2 $dev3
+check wait
+check state UUU
+check chunk 32
+
+mdadm $md0 --grow --chunk=64
+check reshape
+check wait
+check chunk 64
+
+mdadm -S $md0
+mdadm -As
+check state UUU
+check chunk 64
+mdadm -Ss
--
2.7.4
^ permalink raw reply related
* Re: Why can't I re-add my drive after partition shrink?
From: Wols Lists @ 2017-07-20 10:23 UTC (permalink / raw)
To: Ram Ramesh, NeilBrown, Linux Raid
In-Reply-To: <05a2d9b7-3b6a-2b49-53ad-3436061558fc@gmail.com>
On 20/07/17 01:39, Ram Ramesh wrote:
> On 07/19/2017 06:14 PM, NeilBrown wrote:
>> On Wed, Jul 19 2017, Ram Ramesh wrote:
>>
>>> Here is my attempt to repeat the steps in my last attempt to remove,
>>> repartition, re-add. Last time I did it on /dev/sdb. Now I am going to
>>> do it on /dev/sdc. Note that I have not been successful as you see at
>>> the end. I am going to keep the array degraded so that I can still get
>>> old info from /dev/sdc1, if you need anything else. I will keep it this
>>> way till tomorrow and then add the device for md to rebuild. Please ask
>>> anything else before that or send me a note to keep the array degraded
>>> so that you can examine /dev/sdc1 more.
>> Thanks. I *love* getting all the details. You cannot send too many
>> details!
>>
>> This:
>>> <good device still in md0>
>>>> zym [rramesh] 265 > sudo mdadm --examine /dev/sdb1
>>>> /dev/sdb1:
>> ..
>>>> Avail Dev Size : 6442188800 (3071.88 GiB 3298.40 GB)
>> and this:
>>
>>> <device just removed and repartitioned>
>>>> zym [rramesh] 267 > sudo mdadm --examine /dev/sdc1
>>>> /dev/sdc1:
>> ...
>>>> Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
>> Shows the key difference. "Avail Dev Size", aka sb->data_size, is
>> wrong. We can fix it.
>>
>>> <Cannot re-add!!!!>
>>>> zym [rramesh] 270 > sudo mdadm /dev/md0 --re-add /dev/sdc1
>>>> mdadm: --re-add for /dev/sdc1 to /dev/md0 is not possible
>> Please try
>> sudo mdadm /dev/md0 --re-add /dev/sdc1 --update=devicesize
>>
>> Thanks,
>> NeilBrown
>
> Neil,
>
> Thanks a ton. That does it. It got re-added without any issue. It is
> rebuilding because the array was used to record two TV programs when it
> was in degraded state. But the re-add is accepted.
>
>> zym [rramesh] 274 > sudo mdadm /dev/md0 --re-add /dev/sdc1
>> --update=devicesize
>> Size was 11720780943
>> Size is 6442188800
>> mdadm: re-added /dev/sdc1
>> zym [rramesh] 275 > cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md0 : active raid6 sdc1[10] sdb1[6] sdg1[11] sdd1[12] sdf1[8] sde1[9]
>> 12348030976 blocks super 1.2 level 6, 64k chunk, algorithm 2
>> [6/5] [UU_UUU]
>> [========>............] recovery = 42.6%
>> (1316769920/3087007744) finish=292.2min speed=100952K/sec
>> bitmap: 2/23 pages [8KB], 65536KB chunk
>>
>> unused devices: <none>
>
> Wol,
>
> If you read this, this may worth a mention on wiki page.
>
> Ramesh
>
Got that :-)
I'll have to think how to do that - probably a section on the use of
--update to fix problems. Anyway, I've marked this email so when I work
my way through stuff I'll find it :-)
Cheers,
Wol
^ permalink raw reply
* raid 1 assembled but inactive - works from a "live" distribution...
From: Georgios Petasis @ 2017-07-20 10:36 UTC (permalink / raw)
To: linux-raid
Hi all,
I have a fedora 25 (mdadm v3.4 - 28 Jan 2016), where out of blue, it
stopped booting. The problem was because it could not mount some
filesystems that were software raid1 devices. No disk failures. cat
/proc/mdstat shows:
Personalities :
md0 : inactive sdb1[1] sda1[0]
1228797952 blocks super 1.2
md2 : inactive sdb3[2] sda3[0]
507344896 blocks super 1.2
md1 : inactive sda2[0] sdb2[2]
2047997952 blocks super 1.2
unused devices: <none>
What is strange, is that it does not say it is a raid1 array (all of
them are).
mdadm -D /dev/md0 shows:
/dev/md0:
Version : 1.2
Creation Time : Sat Feb 4 08:50:18 2012
Raid Level : raid1
Used Dev Size : 614398840 (585.94 GiB 629.14 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Wed Jul 19 22:46:24 2017
State : active, Not Started
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : server.intellitech.gr:0 (local to host
server.intellitech.gr)
UUID : 6f903cce:5f6b3df4:c865924f:b05e2cd4
Events : 447211
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
I booted my pc with a live dvd of fedora 26 (and not 25 as the system),
having mdadm v4.0, and everything works as expected. I can see the 3
raid devices, I can mount them and see their contents.
So I am puzzled why mdadm 4 can use the devices, and mdadm 3.4 cannot
any more. How can I fix this?
(I have backups of all 3 filesystems, taken with dd from the live dvd,
where the raid devices work. The backups can be mounted)
I am thinking like destroying completely the arrays and recreate them,
but I feel that there is a much simpler solution, as the raid arrays
work with mdadm v4.
Regards,
George
^ permalink raw reply
* raid 1 assembled but inactive - works from a "live" distribution...
From: Georgios Petasis @ 2017-07-20 11:04 UTC (permalink / raw)
To: linux-raid
Hi all,
I have a fedora 25 (mdadm v3.4 - 28 Jan 2016), where out of blue, it
stopped booting. The problem was because it could not mount some
filesystems that were software raid1 devices. No disk failures. cat
/proc/mdstat shows:
Personalities :
md0 : inactive sdb1[1] sda1[0]
1228797952 blocks super 1.2
md2 : inactive sdb3[2] sda3[0]
507344896 blocks super 1.2
md1 : inactive sda2[0] sdb2[2]
2047997952 blocks super 1.2
unused devices: <none>
What is strange, is that it does not say it is a raid1 array (all of
them are).
mdadm -D /dev/md0 shows:
/dev/md0:
Version : 1.2
Creation Time : Sat Feb 4 08:50:18 2012
Raid Level : raid1
Used Dev Size : 614398840 (585.94 GiB 629.14 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Wed Jul 19 22:46:24 2017
State : active, Not Started
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : server.intellitech.gr:0 (local to host
server.intellitech.gr)
UUID : 6f903cce:5f6b3df4:c865924f:b05e2cd4
Events : 447211
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
I booted my pc with a live dvd of fedora 26 (and not 25 as the system),
having mdadm v4.0, and everything works as expected. I can see the 3
raid devices, I can mount them and see their contents.
So I am puzzled why mdadm 4 can use the devices, and mdadm 3.4 cannot
any more. How can I fix this?
(I have backups of all 3 filesystems, taken with dd from the live dvd,
where the raid devices work. The backups can be mounted)
I am thinking like destroying completely the arrays and recreate them,
but I feel that there is a much simpler solution, as the raid arrays
work with mdadm v4.
Regards,
George
^ permalink raw reply
* Re: raid 1 assembled but inactive - works from a "live" distribution...
From: Georgios Petasis @ 2017-07-20 11:52 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <b9a1822b-ab01-2cb3-e417-85aafec9a3d2@yahoo.gr>
Στις 20/7/2017 14:04, ο Georgios Petasis έγραψε:
> Hi all,
>
> I have a fedora 25 (mdadm v3.4 - 28 Jan 2016), where out of blue, it
> stopped booting. The problem was because it could not mount some
> filesystems that were software raid1 devices. No disk failures. cat
> /proc/mdstat shows:
>
> Personalities :
> md0 : inactive sdb1[1] sda1[0]
> 1228797952 blocks super 1.2
>
> md2 : inactive sdb3[2] sda3[0]
> 507344896 blocks super 1.2
>
> md1 : inactive sda2[0] sdb2[2]
> 2047997952 blocks super 1.2
>
> unused devices: <none>
>
> What is strange, is that it does not say it is a raid1 array (all of
> them are).
>
> mdadm -D /dev/md0 shows:
> /dev/md0:
> Version : 1.2
> Creation Time : Sat Feb 4 08:50:18 2012
> Raid Level : raid1
> Used Dev Size : 614398840 (585.94 GiB 629.14 GB)
> Raid Devices : 2
> Total Devices : 2
> Persistence : Superblock is persistent
>
> Update Time : Wed Jul 19 22:46:24 2017
> State : active, Not Started
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 0
>
> Name : server.intellitech.gr:0 (local to host
> server.intellitech.gr)
> UUID : 6f903cce:5f6b3df4:c865924f:b05e2cd4
> Events : 447211
>
> Number Major Minor RaidDevice State
> 0 8 1 0 active sync /dev/sda1
> 1 8 17 1 active sync /dev/sdb1
>
> I booted my pc with a live dvd of fedora 26 (and not 25 as the
> system), having mdadm v4.0, and everything works as expected. I can
> see the 3 raid devices, I can mount them and see their contents.
>
> So I am puzzled why mdadm 4 can use the devices, and mdadm 3.4 cannot
> any more. How can I fix this?
>
> (I have backups of all 3 filesystems, taken with dd from the live dvd,
> where the raid devices work. The backups can be mounted)
>
> I am thinking like destroying completely the arrays and recreate them,
> but I feel that there is a much simpler solution, as the raid arrays
> work with mdadm v4.
>
> Regards,
>
> George
Another puzzling fact, is that booting fedora 25 with the previous
kernel works.
With the latest kernel does not.
I tried because I saw "md: personality for level 1 is not loaded!" in
the logs...
George
^ permalink raw reply
* Re: raid 1 assembled but inactive - works from a "live" distribution...
From: Wols Lists @ 2017-07-20 13:42 UTC (permalink / raw)
To: petasisg, linux-raid
In-Reply-To: <8504efd5-dc98-afe6-6d6c-5b1352a9c134@yahoo.gr>
On 20/07/17 12:52, Georgios Petasis wrote:
> Another puzzling fact, is that booting fedora 25 with the previous
> kernel works.
> With the latest kernel does not.
> I tried because I saw "md: personality for level 1 is not loaded!" in
> the logs...
Has the initrd or similar been changed somehow? I'll let the experts
chime in more, but this sounds like the kernel module for raid1 has
become inaccessible.
You've got superblock v1.2, which means the old trick of booting off of
one disk read-only, before assembling the raid and switching root
read/write won't work. Therefore you
(a) *MUST* have raid1 support in grub, in order to be able to read the
mirror to find the kernel, and
(b) must have raid1 support in the kernel, so the kernel can see /.
As I say, that message makes me suspect that either the grub or the
kernel raid-1 support module has disappeared ...
Cheers,
Wol
^ permalink raw reply
* Re: raid5 using group_thread
From: Shaohua Li @ 2017-07-20 16:43 UTC (permalink / raw)
To: Ofer Heifetz; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <1e734cb651e844b0bd3bc76eec9f16c9@IL-EXCH01.marvell.com>
On Thu, Jul 20, 2017 at 07:21:36AM +0000, Ofer Heifetz wrote:
> > Hi Li,
> > > ----------------------------------------------------------------------
> > > On Wed, Jul 19, 2017 at 01:00:45PM +0000, Ofer Heifetz wrote:
> > > > Hi,
> > > >
> > > > I have a question regarding raid5 built using group_thread and
> > > > async_tx, from code (v4.4 and even v4.12) I see that only raid5d invokes
> > > async_tx_issue_pending_all, shouldn't the raid5_do_work also invoke this
> > > API to issue all pending requests to HW?
> > > >
> > > > I am assuming that there is no sync mechanism between the raid5d and the
> > > raid5_do_work, correct me if I am wrong.
> > >
> > > Can't remember why we don't call async_tx_issue_pending_all in
> > > raid5_do_work, it shouldn't harm. In practice, I doubt calling it makes a
> > > change, because when workers are running, raid5d are running too. Did you
> > > benchmark it?
> >
> > I had a jbd2 hung issue on my system and started to debug it, I noticed that in the cases it was stuck, It had pending requests in the async_xor engine waiting to be
> > issued, so basically requests were sitting in the HW ring and engine was unaware of their existence, this caused the following:
> > [ 1320.280225] INFO: task jbd2/md0-8:1755 blocked for more than 120 seconds.
> > [ 1320.287056] Not tainted 4.4.52-gdbc4936-dirty #45
> > [ 1320.294054] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > [ 1320.301922] jbd2/md0-8 D ffffffc000086cc0 0 1755 2 0x00000000
> > [ 1320.309037] Call trace:
> > [ 1320.311502] [<ffffffc000086cc0>] __switch_to+0x88/0xa0
> > [ 1320.316677] [<ffffffc0008c55d0>] __schedule+0x190/0x5d8
> > [ 1320.321935] [<ffffffc0008c5a5c>] schedule+0x44/0xb8
> > [ 1320.326842] [<ffffffc00026f194>] jbd2_journal_commit_transaction+0x174/0x13e0
> > [ 1320.334018] [<ffffffc00027378c>] kjournald2+0xc4/0x248
> > [ 1320.339185] [<ffffffc0000d2bac>] kthread+0xdc/0xf0
> > [ 1320.344006] [<ffffffc000085dd0>] ret_from_fork+0x10/0x40
> > [ 1320.349349] INFO: task ext4lazyinit:1757 blocked for more than 120 seconds.
> > [ 1320.356350] Not tainted 4.4.52-gdbc4936-dirty #45
> > [ 1320.363347] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > [ 1320.371214] ext4lazyinit D ffffffc000086cc0 0 1757 2 0x00000000
> > [ 1320.378328] Call trace:
> > [ 1320.380793] [<ffffffc000086cc0>] __switch_to+0x88/0xa0
> > [ 1320.385964] [<ffffffc0008c55d0>] __schedule+0x190/0x5d8
> > [ 1320.391218] [<ffffffc0008c5a5c>] schedule+0x44/0xb8
> > [ 1320.396126] [<ffffffc0008c86f4>] schedule_timeout+0x15c/0x1b0
> > [ 1320.401904] [<ffffffc0008c53c8>] io_schedule_timeout+0xb0/0x128
> > [ 1320.407861] [<ffffffc0008c63e0>] bit_wait_io+0x18/0x70
> > [ 1320.413033] [<ffffffc0008c6288>] __wait_on_bit_lock+0x80/0xf0
> > [ 1320.418810] [<ffffffc0008c6354>] out_of_line_wait_on_bit_lock+0x5c/0x68
> > [ 1320.425465] [<ffffffc0001da528>] __lock_buffer+0x38/0x48
> > [ 1320.430809] [<ffffffc00026d254>] do_get_write_access+0x26c/0x540
> > [ 1320.436848] [<ffffffc00026d568>] jbd2_journal_get_write_access+0x40/0x88
> > [ 1320.443593] [<ffffffc00024c0bc>] __ext4_journal_get_write_access+0x34/0x88
> > [ 1320.450511] [<ffffffc0002279d0>] ext4_init_inode_table+0x118/0x3c0
> > [ 1320.456728] [<ffffffc000239a04>] ext4_lazyinit_thread+0x1ec/0x2b8
> > [ 1320.462866] [<ffffffc0000d2bac>] kthread+0xdc/0xf0
> > [ 1320.467691] [<ffffffc000085dd0>] ret_from_fork+0x10/0x40
> >
> >Then I went to the raid5 code and noticed that only raid5d performs the async_tx_issue_pending which seems strange, for it to work right it must be the last one calling r5l_flush_stripe_to_raid
> >thus waiting for the workers to finish their r5l_flush_stripe_to_raid calls, based on the code there is no such sync point between the raid5d and raid5_do_work.
> >
> >I can test the performance impact but with the current code I get hung task which basically forces me to disable group_thread_cnt.
does adding async_tx_issue_pending fix the issue? if yes, could you please
submit a patch and I will merge it. I don't have a machine with async offload
hardware.
Thanks,
Shaohua
^ permalink raw reply
* Re: Linear device of two arrays
From: Veljko @ 2017-07-20 21:40 UTC (permalink / raw)
To: linux-raid, neilb
In-Reply-To: <c4938980-5cbf-d080-3a87-af07d5414437@gmail.com>
On Tue, Jul 18, 2017 at 10:58 AM, Veljko <veljko3@gmail.com> wrote:
> On 07/17/2017 12:16 PM, Veljko wrote:
>>
>> On 07/15/2017 02:12 AM, NeilBrown wrote:
>>
>>> So command should be
>>>
>>>>> dd if=/dev/md2 of=SOMEWHERE/SAFE bs=1M skip=5626590
>>>
>>>
>>> and expect it to create a 3M file.
>>>
>>> Use this 'skip' number of the 'seek' number later.
>>>
>>> NeilBrown
>>
>>
>> Thanks, Neil, now it makes more sense.
>>
>> I tried to create new linear device, but mdadm is complaining about
>> data-offset:
>>
>> # mdadm -C /dev/md4 -l linear -n 2 --rounding=1M -e 1.0 --data-offset=0
>> /dev/md2 /dev/md3
>> mdadm: invalid data-offset: 0
>>
>> I'm using Debian 8.8 if it makes any difference.
>>
>> # mdadm -V
>> mdadm - v3.3.2 - 21st August 2014
>>
>> What could be the problem?
>
>
> I noticed that md2 and md3 use 1.2 metadata. Can that be the issue? Trying
> to create 1.0 metadata for md4?
Any advice on this?
^ permalink raw reply
* Re: Linear device of two arrays
From: NeilBrown @ 2017-07-20 22:00 UTC (permalink / raw)
To: Veljko, linux-raid
In-Reply-To: <38e015fa-a07c-8f1a-3a04-384d973966be@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1344 bytes --]
On Mon, Jul 17 2017, Veljko wrote:
> On 07/15/2017 02:12 AM, NeilBrown wrote:
>
>> So command should be
>>
>>>> dd if=/dev/md2 of=SOMEWHERE/SAFE bs=1M skip=5626590
>>
>> and expect it to create a 3M file.
>>
>> Use this 'skip' number of the 'seek' number later.
>>
>> NeilBrown
>
> Thanks, Neil, now it makes more sense.
>
> I tried to create new linear device, but mdadm is complaining about
> data-offset:
>
> # mdadm -C /dev/md4 -l linear -n 2 --rounding=1M -e 1.0 --data-offset=0
> /dev/md2 /dev/md3
> mdadm: invalid data-offset: 0
Bother.
mdadm uses "parse_size()" to parse the offset, and this rejects
"0", which makes sense for a size, but not for an offset.
Just leave the "--data-offset=0" out. I checked and that is defintely
the default for 1.0.
> I noticed that md2 and md3 use 1.2 metadata. Can that be the issue?
No the metadata on md4 is quite independent of the metadata on md2 and md3.
NeilBrown
>
> I'm using Debian 8.8 if it makes any difference.
>
> # mdadm -V
> mdadm - v3.3.2 - 21st August 2014
>
> What could be the problem?
>
> Regards,
> Veljko
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: Advice please re failed Raid6
From: Bogo Mipps @ 2017-07-21 0:44 UTC (permalink / raw)
To: Peter Grandi, Linux Raid
In-Reply-To: <cf9aac00-91b3-3cb5-bceb-df5d7113b933@gmail.com>
On 07/20/2017 03:55 PM, Bogo Mipps wrote:
> On 07/20/2017 12:36 AM, Peter Grandi wrote:
>>> Did I do it right? (See below)
>>
>>> root@keruru:~# mdadm --create --assume-clean --level=6 --raid-devices=4
>>> --size=1953382912 /dev/md0 missing /dev/sdc /dev/sdd /dev/sde
>>> mdadm: /dev/sdc appears to be part of a raid array:
>>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>>> mdadm: /dev/sdd appears to be part of a raid array:
>>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>>> mdadm: /dev/sde appears to be part of a raid array:
>>> level=raid6 devices=4 ctime=Tue Jul 11 17:33:12 2017
>>> Continue creating array? y
>>> mdadm: Defaulting to version 1.2 metadata
>>> mdadm: array /dev/md0 started.
>>
>> This looks good, but is based on your original '--examine'
>> report as to the order of the devices, and whether they are
>> still bound to the same names 'sd[bcde]'.
>>
>>> root@keruru:~# blkid /dev/md0
>>
>>> root@keruru:~# cat /proc/mdstat
>>> Personalities : [raid6] [raid5] [raid4]
>>> md0 : active (auto-read-only) raid6 sde[3] sdd[2] sdc[1]
>>> 3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
>>> [4/3] [_UUU]
>>
>>> unused devices: <none>
>>
>> The 'mdstat' actually looks good, but 'blkid' should have
>> worked.
>>
>> As I was saying, it is not clear to me whether the 'mdadm' daemon
>> instance triggered a 'check' or a 'repair' (bad news). I hope
>> that you disabled that in the meantime while you try to fix the
>> mistake.
>>
>> Trigger a 'check' and see if the set is consistent; if it is
>> consistent but the content cannot be read/mounted then 'repair'
>> rewrote it, if it is not consistent, try a different order or
>> 3-way subset of 'sd[bcde]'.
>
> Tried different order: sde, sdc, sdd and blkid worked. Added sdb as you
> suggested. Currently rebuilding. Log below. Fingers crossed. Will
> report result.
Peter, here is where I come unstuck. Where to from here? Raid6 has
rebuilt, apparently successfully, but I can't mount. I hesitate to make
another move without advice ...
root@keruru:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdb[4] sdd[2] sdc[1] sde[0]
3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
[4/3] [UUU_]
[=============>.......] recovery = 69.3% (1353992192/1953382912)
finish=162.5min speed=61440K/sec
unused devices: <none>
root@keruru:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdb[4] sdd[2] sdc[1] sde[0]
3906765824 blocks super 1.2 level 6, 512k chunk, algorithm 2
[4/4] [UUUU]
unused devices: <none>
root@keruru:/# mount /dev/md0 /mnt/md0
mount: you must specify the filesystem type
root@keruru:/# mount -t ext4 /dev/md0 /mnt/md0
mount: wrong fs type, bad option, bad superblock on /dev/md0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
root@keruru:/# dmesg | tail
[29458.547966] RAID conf printout:
[29458.547981] --- level:6 rd:4 wd:4
[29458.547989] disk 0, o:1, dev:sde
[29458.547995] disk 1, o:1, dev:sdc
[29458.548001] disk 2, o:1, dev:sdd
[29458.548007] disk 3, o:1, dev:sdb
[48138.300934] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[48138.301411] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[48138.301856] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[48155.451147] EXT4-fs (md0): VFS: Can't find ext4 filesystem
^ permalink raw reply
* [PATCH V2] md: simplify code with bio_io_error
From: Guoqing Jiang @ 2017-07-21 8:33 UTC (permalink / raw)
To: shli; +Cc: linux-raid, Guoqing Jiang
Since bio_io_error sets bi_status to BLK_STS_IOERR,
and calls bio_endio, so we can use it directly.
And as mentioned by Shaohua, there are also two
places in raid5.c can use bio_io_error either.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
V2:
No changes, just rebased on latest kernel tree.
drivers/md/raid1.c | 3 +--
drivers/md/raid10.c | 6 ++----
drivers/md/raid5.c | 9 +++------
3 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 1d235cc..f50958d 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -790,8 +790,7 @@ static void flush_bio_list(struct r1conf *conf, struct bio *bio)
bio->bi_next = NULL;
bio->bi_bdev = rdev->bdev;
if (test_bit(Faulty, &rdev->flags)) {
- bio->bi_status = BLK_STS_IOERR;
- bio_endio(bio);
+ bio_io_error(bio);
} else if (unlikely((bio_op(bio) == REQ_OP_DISCARD) &&
!blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
/* Just ignore it */
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index e2617d0..f55d4cc 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -905,8 +905,7 @@ static void flush_pending_writes(struct r10conf *conf)
bio->bi_next = NULL;
bio->bi_bdev = rdev->bdev;
if (test_bit(Faulty, &rdev->flags)) {
- bio->bi_status = BLK_STS_IOERR;
- bio_endio(bio);
+ bio_io_error(bio);
} else if (unlikely((bio_op(bio) == REQ_OP_DISCARD) &&
!blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
/* Just ignore it */
@@ -1090,8 +1089,7 @@ static void raid10_unplug(struct blk_plug_cb *cb, bool from_schedule)
bio->bi_next = NULL;
bio->bi_bdev = rdev->bdev;
if (test_bit(Faulty, &rdev->flags)) {
- bio->bi_status = BLK_STS_IOERR;
- bio_endio(bio);
+ bio_io_error(bio);
} else if (unlikely((bio_op(bio) == REQ_OP_DISCARD) &&
!blk_queue_discard(bdev_get_queue(bio->bi_bdev))))
/* Just ignore it */
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index aeeb8d6..4904dff 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3381,9 +3381,8 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh,
sh->dev[i].sector + STRIPE_SECTORS) {
struct bio *nextbi = r5_next_bio(bi, sh->dev[i].sector);
- bi->bi_status = BLK_STS_IOERR;
md_write_end(conf->mddev);
- bio_endio(bi);
+ bio_io_error(bi);
bi = nextbi;
}
if (bitmap_end)
@@ -3403,9 +3402,8 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh,
sh->dev[i].sector + STRIPE_SECTORS) {
struct bio *bi2 = r5_next_bio(bi, sh->dev[i].sector);
- bi->bi_status = BLK_STS_IOERR;
md_write_end(conf->mddev);
- bio_endio(bi);
+ bio_io_error(bi);
bi = bi2;
}
@@ -3429,8 +3427,7 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh,
struct bio *nextbi =
r5_next_bio(bi, sh->dev[i].sector);
- bi->bi_status = BLK_STS_IOERR;
- bio_endio(bi);
+ bio_io_error(bi);
bi = nextbi;
}
}
--
2.10.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox