* [PATCH v2 3/3] mdadm/test: Add '--disks=' to support testing phsical devices
From: Zhilong Liu @ 2017-06-14 13:02 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <1497445373-27985-1-git-send-email-zlliu@suse.com>
If test mode has set as '--dev=disk', then users can specify
the argument of "--disks" to test a bunch of physical devices.
For example: ./test --dev=disk --disks=/dev/sda{2..15}
could execute all test cases on physical devices.
Currently, the --dev=disk mode would confilct with testdev()
in current test cases, thus ignore testing testdev() if has
set --dev=disk mode.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
test | 37 ++++++++++++++++++++++++++++++++-----
1 file changed, 32 insertions(+), 5 deletions(-)
diff --git a/test b/test
index df2eeab..87e2df2 100755
--- a/test
+++ b/test
@@ -73,11 +73,11 @@ save_log() {
then
# not supported lvm type yet
echo
- elif [ $DEVTYPE == 'loop' ]
+ elif [ "$DEVTYPE" == 'loop' -o "$DEVTYPE" == 'disk' ]
then
if [ ! -z ${array[@]} -a ${#array[@]} -ge 1 ]
then
- md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/$DEVTYPE" | cut -d'=' -f2))
+ md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/" | cut -d'=' -f2))
cat /proc/mdstat | grep -q "bitmap"
if [ $? -eq 0 ]
then
@@ -114,6 +114,9 @@ cleanup() {
eval "lvremove --quiet -f \$dev$d"
done
;;
+ disk )
+ $mdadm --zero ${disks[@]} &> /dev/null
+ ;;
esac
}
@@ -128,6 +131,7 @@ do_setup() {
[ -d $logdir ] || mkdir -p $logdir
dmesg -c > /dev/null
+ devlist=
if [ "$DEVTYPE" == "loop" ]
then
# make sure there are no loop devices remaining.
@@ -139,8 +143,22 @@ do_setup() {
losetup -d /dev/loop[0-9]* 2> /dev/null
sleep 0.2
done
+ elif [ "$DEVTYPE" == "disk" ]
+ then
+ if [ ! -z "$disks" ]
+ then
+ for d in $(seq 0 ${#disks[@]})
+ do
+ eval "dev$d=${disks[$d]}"
+ eval devlist=\"\$devlist \$dev$d\"
+ eval devlist$d=\"\$devlist\"
+ done
+ $mdadm --zero ${disks[@]} &> /dev/null
+ else
+ echo "Forget to provide physical devices for disk mode."
+ exit 1
+ fi
fi
- devlist=
for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
do
sz=$size
@@ -320,6 +338,7 @@ no_errors() {
# basic device test
testdev() {
[ -b $1 ] || die "$1 isn't a block device."
+ [ "$DEVTYPE" == "disk" ] && return 0
udevadm settle
dev=$1
cnt=$2
@@ -383,6 +402,7 @@ do_test() {
do_help() {
cat <<-EOF
Usage: $0 [options]
+ Example for disk mode: ./test --dev=disk --disks=/dev/sda{2..15}
Options:
--tests=test1,test2,... Comma separated list of tests to run
--raidtype= raid0|linear|raid1|raid456|raid10|ddf|imsm
@@ -391,7 +411,8 @@ do_help() {
--logdir=directory Directory to save all logfiles in
--save-logs Usually use with --logdir together
--keep-going | --no-error Don't stop on error, ie. run all tests
- --dev=loop|lvm|ram Use loop devices (default), LVM, or RAM disk
+ --dev=loop|lvm|ram|disk Use loop devices (default), LVM, RAM or disk
+ --disks= Provide a bunch of physical devices for test
--volgroup=name LVM volume group for LVM test
setup Setup test environment and exit
cleanup Cleanup test environment
@@ -476,6 +497,9 @@ parse_args() {
ram )
DEVTYPE=ram
;;
+ disk )
+ DEVTYPE=disk
+ ;;
* )
echo "Unknown argument: $i"
do_help
@@ -483,6 +507,9 @@ parse_args() {
;;
esac
;;
+ --disks=* )
+ disks=(${disks[*]} ${i##*=})
+ ;;
--volgroup=* )
LVM_VOLGROUP=`expr "x$i" : 'x[^=]*=\(.*\)'`
;;
@@ -509,7 +536,7 @@ check_env() {
echo "test: please run 'make everything' before perform testing."
exit 1
}
- cmds=(mdadm lsblk df udevadm losetup mkfs.ext3 fsck)
+ cmds=(mdadm lsblk df udevadm losetup mkfs.ext3 fsck seq)
for cmd in ${cmds[@]}
do
which $cmd > /dev/null || {
--
2.6.6
^ permalink raw reply related
* Re: block: missing break in process_queued_bios()
From: Mike Snitzer @ 2017-06-14 13:18 UTC (permalink / raw)
To: Dan Carpenter, axboe
Cc: Alasdair Kergon, Christoph Hellwig, dm-devel, Shaohua Li,
linux-raid, kernel-janitors
In-Reply-To: <20170614090437.GA31017@elgon.mountain>
On Wed, Jun 14 2017 at 5:04am -0400,
Dan Carpenter <dan.carpenter@oracle.com> wrote:
> This used to be a fall through case, but we shifted code around and I
> think we want a break here now.
>
> Fixes: 4e4cbee93d56 ("block: switch bios to blk_status_t")
> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
>
> diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
> index a7d2e0840cc5..0e8ab5bb3575 100644
> --- a/drivers/md/dm-mpath.c
> +++ b/drivers/md/dm-mpath.c
> @@ -625,6 +625,7 @@ static void process_queued_bios(struct work_struct *work)
> case DM_MAPIO_KILL:
> bio->bi_status = BLK_STS_IOERR;
> bio_endio(bio);
> + break;
> case DM_MAPIO_REQUEUE:
> bio->bi_status = BLK_STS_DM_REQUEUE;
> bio_endio(bio);
Annoyed with myself for missing this during my review...
The subject should be:
"dm mpath: add missing break in process_queued_bios()"
Also, this should go through Jens and I'll rebase my linux-dm branches
accordingly. Jens, please pick this up.
Acked-by: Mike Snitzer <snitzer@redhat.com>
Thanks!
^ permalink raw reply
* Re: block: missing break in process_queued_bios()
From: Jens Axboe @ 2017-06-14 14:23 UTC (permalink / raw)
To: Mike Snitzer, Dan Carpenter
Cc: Alasdair Kergon, Christoph Hellwig, dm-devel, Shaohua Li,
linux-raid, kernel-janitors
In-Reply-To: <20170614131813.GA10647@redhat.com>
On 06/14/2017 07:18 AM, Mike Snitzer wrote:
> On Wed, Jun 14 2017 at 5:04am -0400,
> Dan Carpenter <dan.carpenter@oracle.com> wrote:
>
>> This used to be a fall through case, but we shifted code around and I
>> think we want a break here now.
>>
>> Fixes: 4e4cbee93d56 ("block: switch bios to blk_status_t")
>> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
>>
>> diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
>> index a7d2e0840cc5..0e8ab5bb3575 100644
>> --- a/drivers/md/dm-mpath.c
>> +++ b/drivers/md/dm-mpath.c
>> @@ -625,6 +625,7 @@ static void process_queued_bios(struct work_struct *work)
>> case DM_MAPIO_KILL:
>> bio->bi_status = BLK_STS_IOERR;
>> bio_endio(bio);
>> + break;
>> case DM_MAPIO_REQUEUE:
>> bio->bi_status = BLK_STS_DM_REQUEUE;
>> bio_endio(bio);
>
> Annoyed with myself for missing this during my review...
>
> The subject should be:
> "dm mpath: add missing break in process_queued_bios()"
>
> Also, this should go through Jens and I'll rebase my linux-dm branches
> accordingly. Jens, please pick this up.
>
> Acked-by: Mike Snitzer <snitzer@redhat.com>
Thanks, I've added it to the 4.13 branch.
--
Jens Axboe
^ permalink raw reply
* [RFC PATCH] md/raid1: remove unused bio in sync_request_write
From: Guoqing Jiang @ 2017-06-15 9:00 UTC (permalink / raw)
To: shli, neilb; +Cc: linux-raid, Guoqing Jiang
The "bio" is not used in sync_request_write after commit a68e58703575
("md/raid1: split out two sub-functions from sync_request_write").
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
Not sure if my understanding is right, so it is for RFC.
drivers/md/raid1.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 6701993..9c51e31 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2181,9 +2181,7 @@ static void sync_request_write(struct mddev *mddev, struct r1bio *r1_bio)
struct r1conf *conf = mddev->private;
int i;
int disks = conf->raid_disks * 2;
- struct bio *bio, *wbio;
-
- bio = r1_bio->bios[r1_bio->read_disk];
+ struct bio *wbio;
if (!test_bit(R1BIO_Uptodate, &r1_bio->state))
/* ouch - failed to read all of that. */
--
2.10.0
^ permalink raw reply related
* Re: [PATCH] md: change the initialization value for a spare device spot to MD_DISK_ROLE_SPARE
From: Shaohua Li @ 2017-06-15 17:02 UTC (permalink / raw)
To: Lidong Zhong; +Cc: jes.sorensen, linux-raid
In-Reply-To: <6e1c6747-f063-a99e-8358-9b994e2e8be2@suse.com>
On Wed, Jun 14, 2017 at 11:16:43AM +0800, Lidong Zhong wrote:
>
>
> On 06/14/2017 01:17 AM, Shaohua Li wrote:
> > On Mon, Jun 12, 2017 at 10:45:55AM +0800, Lidong Zhong wrote:
> > > The value for spare spot of sb->dev_roles is changed from
> > > MD_DISK_ROLE_FAULTY to MD_DISK_ROLE_SPARE to keep align
> > > with the value when the superblock is firstly created in
> > > userspace.
> >
> > Could you point out where mdadm does this?
> >
> Hi Shaohua,
>
> It's done in init_super1() during initialization.
> 1615memset(sb->dev_roles,0xff,
>
>
> 1616 MAX_SB_SIZE - sizeof(struct mdp_superblock_1));
>
> > I'm wondering what's the impact here. If there is a disk, we will set correct
> > role at the end of super_1_sync(), otherwise faulty sounds more reasonable.
> >
>
> Actually there is no impact if it works fine. I found the value for
> spare slot in sb->dev_roles is changed while debugging a problem.
> That's the reason I submit this patch.
ok, let's make it consistent, applied
^ permalink raw reply
* Re: [RFC PATCH] md/raid1: remove unused bio in sync_request_write
From: Shaohua Li @ 2017-06-15 17:04 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: shli, neilb, linux-raid
In-Reply-To: <20170615090025.22826-1-gqjiang@suse.com>
On Thu, Jun 15, 2017 at 05:00:25PM +0800, Guoqing Jiang wrote:
> The "bio" is not used in sync_request_write after commit a68e58703575
> ("md/raid1: split out two sub-functions from sync_request_write").
applied, wondering why gcc doesn't report this.
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> Not sure if my understanding is right, so it is for RFC.
>
> drivers/md/raid1.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> index 6701993..9c51e31 100644
> --- a/drivers/md/raid1.c
> +++ b/drivers/md/raid1.c
> @@ -2181,9 +2181,7 @@ static void sync_request_write(struct mddev *mddev, struct r1bio *r1_bio)
> struct r1conf *conf = mddev->private;
> int i;
> int disks = conf->raid_disks * 2;
> - struct bio *bio, *wbio;
> -
> - bio = r1_bio->bios[r1_bio->read_disk];
> + struct bio *wbio;
>
> if (!test_bit(R1BIO_Uptodate, &r1_bio->state))
> /* ouch - failed to read all of that. */
> --
> 2.10.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: NeilBrown @ 2017-06-16 3:26 UTC (permalink / raw)
To: Shaohua Li, linux-raid; +Cc: Shaohua Li, Mikulas Patocka
In-Reply-To: <d95473d287e196a86a83390cb9c136913643bd61.1497037210.git.shli@fb.com>
[-- Attachment #1: Type: text/plain, Size: 4431 bytes --]
On Fri, Jun 09 2017, Shaohua Li wrote:
> From: Shaohua Li <shli@fb.com>
>
> suspend range is controlled by userspace. If userspace doesn't clear suspend
> range, it's possible a thread will wait for the range forever, we can't even
> kill it. This is bad behavior. Add a timeout in the wait. If timeout happens,
> we return IO error. The app controlling suspend range looks like part of disk
> firmware, if disk isn't responded for a long time, timed out IO error is
> returned.
>
> A simple search in SCSI code shows maximum IO timeout is 120s, so I use this
> value here too.
I really don't like this. It is an API change with no really
justification. Has the current behavior caused a problem?
Both md and dm export APIs which allow IO to be suspended while
changes are made. Changing that to a timed-out period needs, I think,
to be clearly justified.
If it is changed to a timed-out period, then that should be explicit,
rather than having each request independently time out.
i.e. when the suspend is initiated, the end-time should be computed, and
any IO would block until that time, not block for some number of
seconds.
If an md device is left suspended, then the current code will block IO
indefinitely. This patch will at a 20minute times to every single
request, which will mean IO proceeds, but extremely slowly. I don't see
that as a useful improvement.
Thanks,
NeilBrown
>
> Cc: NeilBrown <neilb@suse.com>
> Cc: Mikulas Patocka <mpatocka@redhat.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> drivers/md/md.h | 1 +
> drivers/md/raid1.c | 12 +++++++++++-
> drivers/md/raid5.c | 10 +++++++++-
> 3 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/md/md.h b/drivers/md/md.h
> index 63d342d560b8..11a0ec33e79b 100644
> --- a/drivers/md/md.h
> +++ b/drivers/md/md.h
> @@ -29,6 +29,7 @@
>
> #define MaxSector (~(sector_t)0)
>
> +#define MD_SUSPEND_TIMEOUT (120 * HZ)
> /*
> * These flags should really be called "NO_RETRY" rather than
> * "FAILFAST" because they don't make any promise about time lapse,
> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
> index d9e5373444d2..bc6dee0259df 100644
> --- a/drivers/md/raid1.c
> +++ b/drivers/md/raid1.c
> @@ -1326,6 +1326,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
> (mddev_is_clustered(mddev) &&
> md_cluster_ops->area_resyncing(mddev, WRITE,
> bio->bi_iter.bi_sector, bio_end_sector(bio)))) {
> + long remaining = -1;
>
> /*
> * As the suspend_* range is controlled by userspace, we want
> @@ -1345,10 +1346,19 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
> break;
> sigfillset(&full);
> sigprocmask(SIG_BLOCK, &full, &old);
> - schedule();
> + remaining = schedule_timeout(MD_SUSPEND_TIMEOUT);
> sigprocmask(SIG_SETMASK, &old, NULL);
> + if (remaining == 0)
> + break;
> }
> finish_wait(&conf->wait_barrier, &w);
> + if (remaining == 0) {
> + pr_err("md/raid1:%s: suspend range is locked\n",
> + mdname(mddev));
> + bio->bi_error = -ETIMEDOUT;
> + bio_endio(bio);
> + return;
> + }
> }
> wait_barrier(conf, bio->bi_iter.bi_sector);
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index cf1ac2e0f4c8..24297f1530d1 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -5685,6 +5685,7 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
> if (rw == WRITE &&
> logical_sector >= mddev->suspend_lo &&
> logical_sector < mddev->suspend_hi) {
> + long remaining = -1;
> raid5_release_stripe(sh);
> /* As the suspend_* range is controlled by
> * userspace, we want an interruptible
> @@ -5697,10 +5698,17 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
> sigset_t full, old;
> sigfillset(&full);
> sigprocmask(SIG_BLOCK, &full, &old);
> - schedule();
> + remaining = schedule_timeout(
> + MD_SUSPEND_TIMEOUT);
> sigprocmask(SIG_SETMASK, &old, NULL);
> do_prepare = true;
> }
> + if (remaining == 0) {
> + pr_err("md/raid5:%s: suspend range is locked\n",
> + mdname(mddev));
> + bi->bi_error = -ETIMEDOUT;
> + break;
> + }
> goto retry;
> }
>
> --
> 2.11.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: Guoqing Jiang @ 2017-06-16 9:19 UTC (permalink / raw)
To: shli, neilb; +Cc: linux-raid, Guoqing Jiang
The device owns Bitmap_sync flag needs recovery
to become in sync, and read page from this type
device could get stale status.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
When develop for clustered raid10 feature, if add a
disk under grow mode in master node, I could get
the "bitmap superblock UUID mismatch" warning due
to the page is read from Bitmap_sync device.
drivers/md/bitmap.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index bf7419a..bf34cd8 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
rdev_for_each(rdev, mddev) {
if (! test_bit(In_sync, &rdev->flags)
- || test_bit(Faulty, &rdev->flags))
+ || test_bit(Faulty, &rdev->flags)
+ || test_bit(Bitmap_sync, &rdev->flags))
continue;
target = offset + index * (PAGE_SIZE/512);
--
2.10.0
^ permalink raw reply related
* [PATCH] Monitor: don't assume mdadm parameter is a block device
From: Tomasz Majchrzak @ 2017-06-16 14:02 UTC (permalink / raw)
To: linux-raid; +Cc: jes.sorensen, zlliu, Tomasz Majchrzak
If symlink (e.g. /dev/md/raid) is passed as a parameter to mdadm --wait,
it fails as it's not able to find a corresponding entry in /proc/mdstat
output. Get parameter file major:minor and look for block device name in
sysfs. This commit is partial revert of commit 9e04ac1c43e6
("mdadm/util: unify fstat checking blkdev into function").
Signed-off-by: Tomasz Majchrzak <tomasz.majchrzak@intel.com>
---
Monitor.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
v2:
next Zhilong Liu commit id
diff --git a/Monitor.c b/Monitor.c
index bef2f1b..c9f24bd 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -982,12 +982,21 @@ static void link_containers_with_subarrays(struct state *list)
int Wait(char *dev)
{
char devnm[32];
+ dev_t rdev;
+ char *tmp;
int rv = 1;
int frozen_remaining = 3;
- if (!stat_is_blkdev(dev, NULL))
+ if (!stat_is_blkdev(dev, &rdev))
+ return 2;
+
+ tmp = devid2devnm(rdev);
+ if (!tmp) {
+ pr_err("Cannot get md device name.\n");
return 2;
- strcpy(devnm, dev);
+ }
+
+ strcpy(devnm, tmp);
while(1) {
struct mdstat_ent *ms = mdstat_read(1, 0);
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH] md: make suspend range wait timed out
From: Shaohua Li @ 2017-06-16 15:52 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <877f0c7gvr.fsf@notabene.neil.brown.name>
On Fri, Jun 16, 2017 at 01:26:00PM +1000, Neil Brown wrote:
> On Fri, Jun 09 2017, Shaohua Li wrote:
>
> > From: Shaohua Li <shli@fb.com>
> >
> > suspend range is controlled by userspace. If userspace doesn't clear suspend
> > range, it's possible a thread will wait for the range forever, we can't even
> > kill it. This is bad behavior. Add a timeout in the wait. If timeout happens,
> > we return IO error. The app controlling suspend range looks like part of disk
> > firmware, if disk isn't responded for a long time, timed out IO error is
> > returned.
> >
> > A simple search in SCSI code shows maximum IO timeout is 120s, so I use this
> > value here too.
>
> I really don't like this. It is an API change with no really
> justification. Has the current behavior caused a problem?
This centainly causes problem. Set the suspend range will make application
stall for ever, don't you think this is a problem?
> Both md and dm export APIs which allow IO to be suspended while
> changes are made. Changing that to a timed-out period needs, I think,
> to be clearly justified.
>
> If it is changed to a timed-out period, then that should be explicit,
> rather than having each request independently time out.
> i.e. when the suspend is initiated, the end-time should be computed, and
> any IO would block until that time, not block for some number of
> seconds.
Ok, this makes sense. We can add a timeout. If it's expired, we clear suspend
range. Userspace should know what they are doing.
>
> If an md device is left suspended, then the current code will block IO
> indefinitely. This patch will at a 20minute times to every single
> request, which will mean IO proceeds, but extremely slowly. I don't see
> that as a useful improvement.
It returns error, so application will not dispatch more IO. But I agree a
timeout to clear the suspend looks a better policy.
Thanks,
Shaohua
^ permalink raw reply
* RE: mismatch_cnt > 0 during initial sync?
From: Peter Sangas @ 2017-06-16 17:28 UTC (permalink / raw)
To: 'Stephane Thiell', linux-raid
In-Reply-To: <D974501A-3A8C-438B-A043-8E44DE6D18C7@stanford.edu>
> -----Original Message-----
> From: Stephane Thiell [mailto:sthiell@stanford.edu]
> Subject: Re: mismatch_cnt > 0 during initial sync?
>
> I just did some tests and this seems normal if one of the raid disk
previously has
> some non-zero data. Perhaps there should be a note about that in md(4) or
> elsewhere? (sorry if I missed it)
Hi Stephane,
I also thought mismatch_cnt should = 0 after a sync. I am wondering if you
ran the repair command?
echo repair >> /sys/block/mdX/md/sync_action
Thanks,
Pete
^ permalink raw reply
* Re: [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: Shaohua Li @ 2017-06-16 17:36 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: neilb, linux-raid
In-Reply-To: <20170616091927.12959-1-gqjiang@suse.com>
On Fri, Jun 16, 2017 at 05:19:27PM +0800, Guoqing Jiang wrote:
> The device owns Bitmap_sync flag needs recovery
> to become in sync, and read page from this type
> device could get stale status.
>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> When develop for clustered raid10 feature, if add a
> disk under grow mode in master node, I could get
> the "bitmap superblock UUID mismatch" warning due
> to the page is read from Bitmap_sync device.
>
> drivers/md/bitmap.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
> index bf7419a..bf34cd8 100644
> --- a/drivers/md/bitmap.c
> +++ b/drivers/md/bitmap.c
> @@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
>
> rdev_for_each(rdev, mddev) {
> if (! test_bit(In_sync, &rdev->flags)
Why In_sync is set for the rdev? I think it shoudn't.
> - || test_bit(Faulty, &rdev->flags))
> + || test_bit(Faulty, &rdev->flags)
> + || test_bit(Bitmap_sync, &rdev->flags))
I didn't see code clears the Bitmap_sync bit after disk is synced, so likely
there is bug in this side.
> continue;
>
> target = offset + index * (PAGE_SIZE/512);
> --
> 2.10.0
>
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Brad Campbell @ 2017-06-16 18:06 UTC (permalink / raw)
To: Shaohua Li, NeilBrown; +Cc: linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <20170616155204.myffyxp5tuoctcoo@kernel.org>
On 16/06/17 23:52, Shaohua Li wrote:
> On Fri, Jun 16, 2017 at 01:26:00PM +1000, Neil Brown wrote:
>> If an md device is left suspended, then the current code will block IO
>> indefinitely. This patch will at a 20minute times to every single
>> request, which will mean IO proceeds, but extremely slowly. I don't see
>> that as a useful improvement.
>
> It returns error, so application will not dispatch more IO. But I agree a
> timeout to clear the suspend looks a better policy.
If you insist on doing this, make the default timeout configurable and
set it to wait forever by default. That way there is no change in
current behaviour.
There are a number of times I've tickled those values to suspend things
while doing something sinister to the array. I'd be mighty upset if I
got called away and came back to find the kernel had resumed underneath
me because it thinks it knows best.
Regards,
Brad
--
Dolphins are so intelligent that within a few weeks they can
train Americans to stand at the edge of the pool and throw them
fish.
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Shaohua Li @ 2017-06-16 19:07 UTC (permalink / raw)
To: Brad Campbell; +Cc: NeilBrown, linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <40a41d94-b930-e402-9ce4-4d4f47873e64@fnarfbargle.com>
On Sat, Jun 17, 2017 at 02:06:23AM +0800, Brad Campbell wrote:
> On 16/06/17 23:52, Shaohua Li wrote:
> > On Fri, Jun 16, 2017 at 01:26:00PM +1000, Neil Brown wrote:
> > > If an md device is left suspended, then the current code will block IO
> > > indefinitely. This patch will at a 20minute times to every single
> > > request, which will mean IO proceeds, but extremely slowly. I don't see
> > > that as a useful improvement.
> >
> > It returns error, so application will not dispatch more IO. But I agree a
> > timeout to clear the suspend looks a better policy.
>
> If you insist on doing this, make the default timeout configurable and set
> it to wait forever by default. That way there is no change in current
> behaviour.
That's meaningless if the default is wait forever
> There are a number of times I've tickled those values to suspend things
> while doing something sinister to the array. I'd be mighty upset if I got
> called away and came back to find the kernel had resumed underneath me
> because it thinks it knows best.
I'd like to know your use case, can you describe it in details? I thought only
raid6check uses this function. If there are other usage cases which depend on
current behavior, we certaily can't change it.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Brad Campbell @ 2017-06-16 19:22 UTC (permalink / raw)
To: Shaohua Li; +Cc: NeilBrown, linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <20170616190746.s4ykbmr2tcysimta@kernel.org>
> I'd like to know your use case, can you describe it in details? I thought only
> raid6check uses this function. If there are other usage cases which depend on
> current behavior, we certaily can't change it.
>
>
It's entirely manual. I have a series of test arrays I use when people
come to the list with issues and it's not uncommon to get into sysfs to
manipulate the arrays to either re-create a situation or get the array
into a state where I can try and figure out what is going on. To be
honest all the use cases for me are entirely pathological, but it's a
change to the current behavior that I have used in the past and expect
to use again in the future.
The issue being rectified is not one I've seen come up frequently (if at
all) so I thought I'd pipe up and say "I have used this in the past and
will probably use it again in the future, please don't break it of you
can avoid it".
I am currently 17,600KM away from my test stuff, so I can't elaborate
further at the moment.
Regards,
Brad
--
Dolphins are so intelligent that within a few weeks they can
train Americans to stand at the edge of the pool and throw them
fish.
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Brad Campbell @ 2017-06-16 19:37 UTC (permalink / raw)
To: Shaohua Li; +Cc: NeilBrown, linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <20170616190746.s4ykbmr2tcysimta@kernel.org>
On 17/06/17 03:07, Shaohua Li wrote:
>
> I'd like to know your use case, can you describe it in details? I thought only
> raid6check uses this function. If there are other usage cases which depend on
> current behavior, we certaily can't change it.
>
It occurs to me that my objection is based on a particular workflow and
there is probably more benefit to the general population to actually
make the change. I can adapt to whatever changes are made. Sorry for the
noise.
Regards,
Brad
--
Dolphins are so intelligent that within a few weeks they can
train Americans to stand at the edge of the pool and throw them
fish.
^ permalink raw reply
* raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: r23 @ 2017-06-18 4:14 UTC (permalink / raw)
To: linux-raid
Hi all, I have (2) 4.6 TB drives in software Raid 1
mdadm - v3.3.2 on debian jessie. A new controller
zapped something and I have reverted to the prior built in
controller. I think 1 of the 2 is salvagable but need
some help getting the array started with the 1 drive (sda2 )
that is responding with raid info. sda2 was the other
drive and does not respond to --examine.
I think i've boiled it down to this when trying to scan/run:
mdadm: failed to add /dev/sdb2 to /dev/md/debian:0: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md/debian:0: Invalid argument
Also not clear on if the name should be /dev/md0 or /dev/md/debian:0,
when I run scan/run with a name of /dev/md/debian:0 it ends up in the
/dev directory as /dev/md0
All the info follows.
I would like to get it started with the single drive / sda2 ,
in read-only mode to capture a backup.
root@mars:/home/user14# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 4.6T 0 disk
-─sda1 8:1 0 1G 0 part
|-sda2 8:2 0 4.6T 0 part
sdb 8:16 0 4.6T 0 disk
-─sdb1 8:17 0 1G 0 part
|-sdb2 8:18 0 4.6T 0 part
sdc 8:32 0 149.1G 0 disk
-─sdc1 8:33 0 78.1G 0 part
-─sdc2 8:34 0 1K 0 part
-─sdc5 8:37 0 33.6G 0 part
-─sdc6 8:38 0 18.1G 0 part
|-sdc7 8:39 0 19.2G 0 part
sdd 8:48 0 232.9G 0 disk
-─sdd1 8:49 0 8.4G 0 part /
-─sdd2 8:50 0 1K 0 part
-─sdd5 8:53 0 2.8G 0 part
-─sdd6 8:54 0 4G 0 part [SWAP]
-─sdd7 8:55 0 380M 0 part
|-sdd8 8:56 0 217.4G 0 part /home
sr0 11:0 1 1024M 0 rom
sr1 11:1 1 7.8G 0 rom
root@mars:/home/user14#
root@mars:~/raid/bin# mdadm --misc --examine /dev/sda2
mdadm: No md superblock detected on /dev/sda2.
root@mars:~/raid/bin# mdadm --misc --examine /dev/sdb2
/dev/sdb2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 2a3489a6:b430c744:2c89a792:98521913
Name : debian:0
Creation Time : Sat May 9 17:44:25 2015
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 9765179392 (4656.40 GiB 4999.77 GB)
Array Size : 4882589696 (4656.40 GiB 4999.77 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : clean
Device UUID : 4697b088:5d3b1ae5:55d30f65:516df63d
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jun 14 23:53:27 2017
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 1f245410 - expected 1f249410
Events : 129099
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
root@mars:~/raid/bin#
root@mars:~/raid/bin/util/lsdrv# ./lsdrv
PCI [sata_via] 00:0f.0 IDE interface: VIA Technologies, Inc. VT8237A SATA 2-Port Controller (rev 80)
scsi 0:0:0:0 ATA TOSHIBA MD04ACA5 {75T8K3HXFS9A}
sda 4.55t [8:0] Partitioned (gpt)
sda1 1.00g [8:1] ext2 'skip_space' {b03b8699-d492-42b1-af9e-2a68b73b733d}
sda2 4.55t [8:2] ext3 'for_raid' {5c93e223-8b7a-48b9-b4f0-6ef2990f22ae}
scsi 1:0:0:0 ATA TOSHIBA MD04ACA5 {84JHK4XKFS9A}
sdb 4.55t [8:16] Partitioned (gpt)
sdb1 1.00g [8:17] ext2 'fs_on_sdd1' {78546565-c92d-4008-b372-104c6337c11e}
sdb2 4.55t [8:18] MD raid1 (2) inactive 'debian:0' {2a3489a6-b430-c744-2c89-a79298521913}
PCI [pata_via] 00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 07)
scsi 2:0:0:0 PIONEER DVD-RW DVR-111D {PIONEER_DVD-RW_DVR-111D}
sr0 1.00g [11:0] Empty/Unknown
scsi 2:0:1:0 ATA WDC WD1600JB-00G {WD-WCAL98126376}
sdc 149.05g [8:32] Partitioned (dos)
sdc1 78.13g [8:33] ntfs {72280E84280E4795}
sdc2 1.00k [8:34] Partitioned (dos)
sdc5 33.62g [8:37] vfat {04CA-B75D}
sdc6 18.13g [8:38] ext2 'for_temp' {6cc1aa46-aaa3-4f93-8316-8924ac9ee3f2}
sdc7 19.17g [8:39] ext4 'for_var' {8d865846-5827-4f51-b826-6f672d45285d}
scsi 3:0:0:0 ATA WDC WD2500JB-00G {WD-WCAL76218162}
sdd 232.89g [8:48] Partitioned (dos)
sdd1 8.38g [8:49] ext4 {7e42bfae-8af9-4c9a-b8c2-7e70c37435a1}
Mounted as /dev/sdd1 @ /
sdd2 1.00k [8:50] Partitioned (dos)
sdd5 2.79g [8:53] ext4 {907474bc-cec7-4a80-afb0-daa963cef388}
sdd6 3.99g [8:54] swap {cbe9107e-8503-4597-8f99-5121384dfd1b}
sdd7 380.00m [8:55] ext4 {daf44546-1549-42be-b632-2be175962366}
sdd8 217.34g [8:56] ext4 {9f3bfd18-92ac-4095-b512-cc2a0b88737c}
Mounted as /dev/sdd8 @ /home
USB [usb-storage] Bus 002 Device 005: ID 0480:d010 Toshiba America Info. Systems, Inc. {20121204025996}
scsi 5:0:0:0 Toshiba External USB 3.0 {66715465}
sde 1.82t [8:64] Partitioned (dos)
sde1 1.82t [8:65] ntfs 'ELLIS' {ACAC8FDBAC8F9E88}
Mounted as /dev/sde1 @ /mnt/ellis
Other Block Devices
md0 0.00k [9:0] MD v1.2 () clear, None (None) None {None}
Empty/Unknown
root@mars:~/raid/bin/util/lsdrv#
root@mars:~/raid/bin/util/lsdrv# mdadm --manage --stop /dev/md0
mdadm: stopped /dev/md0
root@mars:~/raid/bin/util/lsdrv# mdadm --assemble /dev/md/debian:0 -v /dev/sdb2 --run
mdadm: looking for devices for /dev/md/debian:0
mdadm: /dev/sdb2 is identified as a member of /dev/md/debian:0, slot 1.
mdadm: no uptodate device for slot 0 of /dev/md/debian:0
mdadm: failed to add /dev/sdb2 to /dev/md/debian:0: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md/debian:0: Invalid argument
root@mars:~/raid/bin/util/lsdrv# ls -lR /dev | grep md
lrwxrwxrwx 1 root root 25 Jun 15 21:20 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root root 28 Jun 15 21:20 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 root root 40 Jun 15 17:20 md
brw-rw---- 1 root disk 9, 0 Jun 17 21:54 md0
lrwxrwxrwx 1 root root 6 Jun 17 21:54 9:0 -> ../md0
/dev/md:
root@mars:~/raid/bin/util/lsdrv# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Raid Level : raid0
Total Devices : 0
Persistence : Superblock is persistent
State : inactive
Number Major Minor RaidDevice
root@mars:~/raid/bin/util/lsdrv#
root@mars:~/raid/bin/util/lsdrv#
root@mars:~/raid/bin/util/lsdrv# cat /proc/mdstat
Personalities :
unused devices: <none>
root@mars:~/raid/bin/util/lsdrv#
^ permalink raw reply
* Re: raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: Wols Lists @ 2017-06-18 13:56 UTC (permalink / raw)
To: r23@anchordrop.net, linux-raid
In-Reply-To: <20170618001410.6fa96469@e20>
On 18/06/17 05:14, r23@anchordrop.net wrote:
> Hi all, I have (2) 4.6 TB drives in software Raid 1
> mdadm - v3.3.2 on debian jessie. A new controller
> zapped something and I have reverted to the prior built in
> controller. I think 1 of the 2 is salvagable but need
> some help getting the array started with the 1 drive (sda2 )
> that is responding with raid info. sda2 was the other
> drive and does not respond to --examine.
>
> I think i've boiled it down to this when trying to scan/run:
>
> mdadm: failed to add /dev/sdb2 to /dev/md/debian:0: Invalid argument
> mdadm: failed to RUN_ARRAY /dev/md/debian:0: Invalid argument
>
> Also not clear on if the name should be /dev/md0 or /dev/md/debian:0,
> when I run scan/run with a name of /dev/md/debian:0 it ends up in the
> /dev directory as /dev/md0
Probably doesn't matter. Internally, mdadm is using /dev/md/debian:0 to
indicate (if I've got this right) that this is the first raid array
created on a machine called "debian". That's meant to protect you
against trashing arrays if you move drives between machines. I'm
guessing you've said "md0" in your commands trying to get the array to
run, so mdadm is also using md0 to refer to the array. It gets even more
confusing because once things start working again, it'll call it md127
in all probability :-) Long and short of it, don't worry about this.
>
> All the info follows.
>
> I would like to get it started with the single drive / sda2 ,
> in read-only mode to capture a backup.
>
It appears from what you've mentioned below, that sda2 has had its
superblock trashed. In other words, you're not going to get that working
without some expert help. However, sdb2 is okay ...
Incidentally, have you been doing a "mdadm --stop /dev/md0" between
every one of your attempts to get things running? If you haven't, then
it's likely to have played havoc with your attempts, as a partially
assembled array usually refuses to do anything ...
>
>
> root@mars:/home/user14# lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 4.6T 0 disk
> -─sda1 8:1 0 1G 0 part
> |-sda2 8:2 0 4.6T 0 part
> sdb 8:16 0 4.6T 0 disk
> -─sdb1 8:17 0 1G 0 part
> |-sdb2 8:18 0 4.6T 0 part
> sdc 8:32 0 149.1G 0 disk
> -─sdc1 8:33 0 78.1G 0 part
> -─sdc2 8:34 0 1K 0 part
> -─sdc5 8:37 0 33.6G 0 part
> -─sdc6 8:38 0 18.1G 0 part
> |-sdc7 8:39 0 19.2G 0 part
> sdd 8:48 0 232.9G 0 disk
> -─sdd1 8:49 0 8.4G 0 part /
> -─sdd2 8:50 0 1K 0 part
> -─sdd5 8:53 0 2.8G 0 part
> -─sdd6 8:54 0 4G 0 part [SWAP]
> -─sdd7 8:55 0 380M 0 part
> |-sdd8 8:56 0 217.4G 0 part /home
> sr0 11:0 1 1024M 0 rom
> sr1 11:1 1 7.8G 0 rom
> root@mars:/home/user14#
>
>
> root@mars:~/raid/bin# mdadm --misc --examine /dev/sda2
> mdadm: No md superblock detected on /dev/sda2.
>
This is what implies sda2 has somehow been trashed ... :-(
>
> root@mars:~/raid/bin# mdadm --misc --examine /dev/sdb2
> /dev/sdb2:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 2a3489a6:b430c744:2c89a792:98521913
> Name : debian:0
> Creation Time : Sat May 9 17:44:25 2015
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 9765179392 (4656.40 GiB 4999.77 GB)
> Array Size : 4882589696 (4656.40 GiB 4999.77 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : clean
> Device UUID : 4697b088:5d3b1ae5:55d30f65:516df63d
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Wed Jun 14 23:53:27 2017
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 1f245410 - expected 1f249410
> Events : 129099
>
>
> Device Role : Active device 1
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
> root@mars:~/raid/bin#
>
This is a bit weird! An array state of AA indicates that the array is
okay! This is where I would have done a --detail on /dev/md0 or
/dev/md/debian:0
>
> root@mars:~/raid/bin/util/lsdrv# ./lsdrv
> PCI [sata_via] 00:0f.0 IDE interface: VIA Technologies, Inc. VT8237A SATA 2-Port Controller (rev 80)
> scsi 0:0:0:0 ATA TOSHIBA MD04ACA5 {75T8K3HXFS9A}
> sda 4.55t [8:0] Partitioned (gpt)
> sda1 1.00g [8:1] ext2 'skip_space' {b03b8699-d492-42b1-af9e-2a68b73b733d}
> sda2 4.55t [8:2] ext3 'for_raid' {5c93e223-8b7a-48b9-b4f0-6ef2990f22ae}
> scsi 1:0:0:0 ATA TOSHIBA MD04ACA5 {84JHK4XKFS9A}
> sdb 4.55t [8:16] Partitioned (gpt)
> sdb1 1.00g [8:17] ext2 'fs_on_sdd1' {78546565-c92d-4008-b372-104c6337c11e}
> sdb2 4.55t [8:18] MD raid1 (2) inactive 'debian:0' {2a3489a6-b430-c744-2c89-a79298521913}
> PCI [pata_via] 00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 07)
> scsi 2:0:0:0 PIONEER DVD-RW DVR-111D {PIONEER_DVD-RW_DVR-111D}
> sr0 1.00g [11:0] Empty/Unknown
> scsi 2:0:1:0 ATA WDC WD1600JB-00G {WD-WCAL98126376}
> sdc 149.05g [8:32] Partitioned (dos)
> sdc1 78.13g [8:33] ntfs {72280E84280E4795}
> sdc2 1.00k [8:34] Partitioned (dos)
> sdc5 33.62g [8:37] vfat {04CA-B75D}
> sdc6 18.13g [8:38] ext2 'for_temp' {6cc1aa46-aaa3-4f93-8316-8924ac9ee3f2}
> sdc7 19.17g [8:39] ext4 'for_var' {8d865846-5827-4f51-b826-6f672d45285d}
> scsi 3:0:0:0 ATA WDC WD2500JB-00G {WD-WCAL76218162}
> sdd 232.89g [8:48] Partitioned (dos)
> sdd1 8.38g [8:49] ext4 {7e42bfae-8af9-4c9a-b8c2-7e70c37435a1}
> Mounted as /dev/sdd1 @ /
> sdd2 1.00k [8:50] Partitioned (dos)
> sdd5 2.79g [8:53] ext4 {907474bc-cec7-4a80-afb0-daa963cef388}
> sdd6 3.99g [8:54] swap {cbe9107e-8503-4597-8f99-5121384dfd1b}
> sdd7 380.00m [8:55] ext4 {daf44546-1549-42be-b632-2be175962366}
> sdd8 217.34g [8:56] ext4 {9f3bfd18-92ac-4095-b512-cc2a0b88737c}
> Mounted as /dev/sdd8 @ /home
> USB [usb-storage] Bus 002 Device 005: ID 0480:d010 Toshiba America Info. Systems, Inc. {20121204025996}
> scsi 5:0:0:0 Toshiba External USB 3.0 {66715465}
> sde 1.82t [8:64] Partitioned (dos)
> sde1 1.82t [8:65] ntfs 'ELLIS' {ACAC8FDBAC8F9E88}
> Mounted as /dev/sde1 @ /mnt/ellis
> Other Block Devices
> md0 0.00k [9:0] MD v1.2 () clear, None (None) None {None}
> Empty/Unknown
> root@mars:~/raid/bin/util/lsdrv#
>
It looks to me like somebody's created an ext3 partition on sda2, and
thereby trashed the raid ...
>
>
> root@mars:~/raid/bin/util/lsdrv# mdadm --manage --stop /dev/md0
> mdadm: stopped /dev/md0
>
> root@mars:~/raid/bin/util/lsdrv# mdadm --assemble /dev/md/debian:0 -v /dev/sdb2 --run
> mdadm: looking for devices for /dev/md/debian:0
> mdadm: /dev/sdb2 is identified as a member of /dev/md/debian:0, slot 1.
> mdadm: no uptodate device for slot 0 of /dev/md/debian:0
> mdadm: failed to add /dev/sdb2 to /dev/md/debian:0: Invalid argument
> mdadm: failed to RUN_ARRAY /dev/md/debian:0: Invalid argument
>
> root@mars:~/raid/bin/util/lsdrv# ls -lR /dev | grep md
> lrwxrwxrwx 1 root root 25 Jun 15 21:20 initctl -> /run/systemd/initctl/fifo
> lrwxrwxrwx 1 root root 28 Jun 15 21:20 log -> /run/systemd/journal/dev-log
> drwxr-xr-x 2 root root 40 Jun 15 17:20 md
> brw-rw---- 1 root disk 9, 0 Jun 17 21:54 md0
> lrwxrwxrwx 1 root root 6 Jun 17 21:54 9:0 -> ../md0
> /dev/md:
>
> root@mars:~/raid/bin/util/lsdrv# mdadm --detail /dev/md0
> /dev/md0:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 0
> Persistence : Superblock is persistent
>
> State : inactive
>
> Number Major Minor RaidDevice
> root@mars:~/raid/bin/util/lsdrv#
> root@mars:~/raid/bin/util/lsdrv#
>
> root@mars:~/raid/bin/util/lsdrv# cat /proc/mdstat
> Personalities :
> unused devices: <none>
> root@mars:~/raid/bin/util/lsdrv#
>
Hmmm. I don't want to suggest too much, but the array is expecting two
devices, so saying just "/dev/sdb2" is probably its invalid argument.
Try changing that to "/dev/sdb2 missing".
And if that improves matters but the array won't run, you might want to
"--readonly --force". But I'd wait for someone else to chime in that
this is a good idea before trying this.
Hope this has given you some ideas.
Cheers,
Wol
^ permalink raw reply
* (unknown),
From: xa0ajutor @ 2017-06-18 14:27 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: 49828587.zip --]
[-- Type: application/zip, Size: 2024 bytes --]
^ permalink raw reply
* Re: raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: r23 @ 2017-06-18 16:08 UTC (permalink / raw)
To: Wols Lists, linux-raid
In-Reply-To: <5946867B.5010200@youngman.org.uk>
I have 2 scenarios here:
scenario 1- I tried the argument 'missing'. I think its taking that as a literal device name. I'm
probably not doing it like your asking.
root@mars:~#
root@mars:~# mdadm --manage --stop /dev/md0
mdadm: error opening /dev/md0: No such file or directory
root@mars:~# ls -lR /dev | grep md
lrwxrwxrwx 1 root root 25 Jun 18 11:52 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root root 28 Jun 18 11:52 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 root root 40 Jun 18 07:52 md
/dev/md:
root@mars:~# mdadm --assemble /dev/md/debian:0 -v --run /dev/sdb2 missing
mdadm: looking for devices for /dev/md/debian:0
mdadm: cannot open device missing: No such file or directory
mdadm: missing has no superblock - assembly aborted
root@mars:~#
Scenario 2: - trying to explicitly add the good disk:
1) examine /dev/sdb2
2) mdadm --assemble /dev/md/md0 -v --run /dev/sdb2 : result = failed to add /dev/sdb2 to /dev/md/md0: Invalid argument
3) mdadm --manage /dev/md0 -v --add /dev/sdb2 : result = mdadm: Cannot get array info for /dev/md0
4) mdadm --detail -v /dev/md0 = result = raid0 ??
detail on above 4 steps:
root@mars:~#
root@mars:~# mdadm --misc --examine /dev/sdb2
/dev/sdb2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 2a3489a6:b430c744:2c89a792:98521913
Name : debian:0
Creation Time : Sat May 9 17:44:25 2015
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 9765179392 (4656.40 GiB 4999.77 GB)
Array Size : 4882589696 (4656.40 GiB 4999.77 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : clean
Device UUID : 4697b088:5d3b1ae5:55d30f65:516df63d
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jun 14 23:53:27 2017
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 1f245410 - expected 1f249410
Events : 129099
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
root@mars:~# mdadm --assemble /dev/md/md0 -v --run /dev/sdb2
mdadm: looking for devices for /dev/md/md0
mdadm: /dev/sdb2 is identified as a member of /dev/md/md0, slot 1.
mdadm: no uptodate device for slot 0 of /dev/md/md0
mdadm: failed to add /dev/sdb2 to /dev/md/md0: Invalid argument
mdadm: failed to RUN_ARRAY /dev/md/md0: Invalid argument
root@mars:~# ls -lR /dev | grep md
lrwxrwxrwx 1 root root 25 Jun 18 11:52 initctl -> /run/systemd/initctl/fifo
lrwxrwxrwx 1 root root 28 Jun 18 11:52 log -> /run/systemd/journal/dev-log
drwxr-xr-x 2 root root 40 Jun 18 07:52 md
brw-rw---- 1 root disk 9, 0 Jun 18 11:57 md0
lrwxrwxrwx 1 root root 6 Jun 18 11:57 9:0 -> ../md0
/dev/md:
root@mars:~# mdadm --manage /dev/md0 -v --add /dev/sdb2
mdadm: Cannot get array info for /dev/md0
root@mars:~# mdadm --detail -v /dev/md0
/dev/md0:
Version : 1.2
Raid Level : raid0
Total Devices : 0
Persistence : Superblock is persistent
State : inactive
Number Major Minor RaidDevice
root@mars:~#
^ permalink raw reply
* Re: raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: NeilBrown @ 2017-06-18 20:59 UTC (permalink / raw)
To: r23@anchordrop.net, linux-raid
In-Reply-To: <20170618001410.6fa96469@e20>
[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]
On Sun, Jun 18 2017, r23@anchordrop.net wrote:
>
> root@mars:~/raid/bin# mdadm --misc --examine /dev/sdb2
> /dev/sdb2:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 2a3489a6:b430c744:2c89a792:98521913
> Name : debian:0
> Creation Time : Sat May 9 17:44:25 2015
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 9765179392 (4656.40 GiB 4999.77 GB)
> Array Size : 4882589696 (4656.40 GiB 4999.77 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : clean
> Device UUID : 4697b088:5d3b1ae5:55d30f65:516df63d
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Wed Jun 14 23:53:27 2017
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 1f245410 - expected 1f249410
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
checksum is wrong.
> root@mars:~/raid/bin/util/lsdrv# mdadm --assemble /dev/md/debian:0 -v /dev/sdb2 --run
> mdadm: looking for devices for /dev/md/debian:0
> mdadm: /dev/sdb2 is identified as a member of /dev/md/debian:0, slot 1.
> mdadm: no uptodate device for slot 0 of /dev/md/debian:0
> mdadm: failed to add /dev/sdb2 to /dev/md/debian:0: Invalid argument
Bad checksum caused kernel to reject the device.
There is no obvious way to correct the checksum, but I think you can
force it by using the --update option to --assemble
e.g.
mdadm --assemble /dev/md/debian:0 --update=name -v /dev/sdb2
When doing that, you should check that other fields all look correct
first, because something must have changed to affect the bitmap.
(I cannot see anything that looks wrong)
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: NeilBrown @ 2017-06-18 21:00 UTC (permalink / raw)
To: Wols Lists, r23@anchordrop.net, linux-raid
In-Reply-To: <5946867B.5010200@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 281 bytes --]
On Sun, Jun 18 2017, Wols Lists wrote:
> Hmmm. I don't want to suggest too much, but the array is expecting two
> devices, so saying just "/dev/sdb2" is probably its invalid argument.
> Try changing that to "/dev/sdb2 missing".
"missing" is only relevant for --create
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: NeilBrown @ 2017-06-18 21:30 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, Shaohua Li, Mikulas Patocka
In-Reply-To: <20170616155204.myffyxp5tuoctcoo@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 2939 bytes --]
On Fri, Jun 16 2017, Shaohua Li wrote:
> On Fri, Jun 16, 2017 at 01:26:00PM +1000, Neil Brown wrote:
>> On Fri, Jun 09 2017, Shaohua Li wrote:
>>
>> > From: Shaohua Li <shli@fb.com>
>> >
>> > suspend range is controlled by userspace. If userspace doesn't clear suspend
>> > range, it's possible a thread will wait for the range forever, we can't even
>> > kill it. This is bad behavior. Add a timeout in the wait. If timeout happens,
>> > we return IO error. The app controlling suspend range looks like part of disk
>> > firmware, if disk isn't responded for a long time, timed out IO error is
>> > returned.
>> >
>> > A simple search in SCSI code shows maximum IO timeout is 120s, so I use this
>> > value here too.
>>
>> I really don't like this. It is an API change with no really
>> justification. Has the current behavior caused a problem?
>
> This centainly causes problem. Set the suspend range will make application
> stall for ever, don't you think this is a problem?
I agree that it could cause a problem. I'm asking it is actually, in
practice, causes a problem. Do you have reports from people saying "the
IO to my RAID array is hanging, what is wrong?" and you look into it and
find out that suspend_hi is larger than suspend_lo?
And if that does happen, is this really the best way to fix it?
>
>> Both md and dm export APIs which allow IO to be suspended while
>> changes are made. Changing that to a timed-out period needs, I think,
>> to be clearly justified.
>>
>> If it is changed to a timed-out period, then that should be explicit,
>> rather than having each request independently time out.
>> i.e. when the suspend is initiated, the end-time should be computed, and
>> any IO would block until that time, not block for some number of
>> seconds.
>
> Ok, this makes sense. We can add a timeout. If it's expired, we clear suspend
> range. Userspace should know what they are doing.
>
>>
>> If an md device is left suspended, then the current code will block IO
>> indefinitely. This patch will at a 20minute times to every single
>> request, which will mean IO proceeds, but extremely slowly. I don't see
>> that as a useful improvement.
>
> It returns error, so application will not dispatch more IO. But I agree a
> timeout to clear the suspend looks a better policy.
Write errors only get back to the application if it calls fsync(), and
many don't do that. Write errors can easily cause a filesystem to go
read-only, and require an fsck. I think we should be very cautious
about triggering write errors.
NFS will hang indefinitely rather then return an error if the server is
not available. That can certainly be annoying, but the alternative has
been tried, and it leads to random data corruption.
The two cases are only comparable at a very high level, but I think
this result should encourage substantial caution.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: mismatch_cnt > 0 during initial sync?
From: NeilBrown @ 2017-06-18 21:34 UTC (permalink / raw)
To: Stephane Thiell, linux-raid@vger.kernel.org
In-Reply-To: <D974501A-3A8C-438B-A043-8E44DE6D18C7@stanford.edu>
[-- Attachment #1: Type: text/plain, Size: 3201 bytes --]
On Tue, Apr 25 2017, Stephane Thiell wrote:
> I just did some tests and this seems normal if one of the raid disk previously has some non-zero data. Perhaps there should be a note about that in md(4) or elsewhere? (sorry if I missed it)
From the perspective of md, the initial sync is no different from any
other sync. It will count the number of mismatches that it finds and
fixes.
Feel free to post a patch for md.4 or elsewhere to clarify this.
Thanks,
NeilBrown
>
> Thanks,
>
> Stephan
>
>> On Apr 24, 2017, at 3:01 PM, Stephane Thiell <sthiell@stanford.edu> wrote:
>>
>>
>> Hi all,
>>
>> Is it normal to get any mismatch_cnt > 0 during the initial sync? I just created a few new new raid6 volumes, and one of them is showing an increasing number of mismatch_cnt during its initial resync. Nothing is visible in the console nor after a smart short test (I also started a long test just in case).
>>
>>
>> $ mdadm --detail /dev/md7
>> /dev/md7:
>> Version : 1.2
>> Creation Time : Mon Apr 24 10:07:18 2017
>> Raid Level : raid6
>> Array Size : 62511163904 (59615.29 GiB 64011.43 GB)
>> Used Dev Size : 7813895488 (7451.91 GiB 8001.43 GB)
>> Raid Devices : 10
>> Total Devices : 10
>> Persistence : Superblock is persistent
>>
>> Intent Bitmap : Internal
>>
>> Update Time : Mon Apr 24 14:52:12 2017
>> State : clean, resyncing
>> Active Devices : 10
>> Working Devices : 10
>> Failed Devices : 0
>> Spare Devices : 0
>>
>> Layout : left-symmetric
>> Chunk Size : 64K
>>
>> Resync Status : 24% complete
>>
>> Name : oak-io1-s2:7 (local to host oak-io1-s2)
>> UUID : 6ece9d71:a97f9497:4612ed73:44dfdc0a
>> Events : 3308
>>
>> Number Major Minor RaidDevice State
>> 0 253 62 0 active sync /dev/dm-62
>> 1 253 63 1 active sync /dev/dm-63
>> 2 253 72 2 active sync /dev/dm-72
>> 3 253 75 3 active sync /dev/dm-75
>> 4 253 89 4 active sync /dev/dm-89
>> 5 253 87 5 active sync /dev/dm-87
>> 6 253 119 6 active sync /dev/dm-119
>> 7 253 97 7 active sync /dev/dm-97
>> 8 253 103 8 active sync /dev/dm-103
>> 9 253 107 9 active sync /dev/dm-107
>>
>> $ cat /sys/block/md7/md/sync_completed
>> 3795284824 / 15627790976
>>
>> $ cat /sys/block/md7/md/mismatch_cnt
>> 3424
>>
>> versions:
>> mdadm-3.4-14.el7_3.1.x86_64
>> CentOS7 (3.10.0-514.10.2.el7_lustre.x86_64)
>>
>> Thanks!
>>
>> Stephan
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: NeilBrown @ 2017-06-18 23:11 UTC (permalink / raw)
To: Shaohua Li, Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <20170616173613.c2cp6qt2hmoebgct@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 3056 bytes --]
On Fri, Jun 16 2017, Shaohua Li wrote:
> On Fri, Jun 16, 2017 at 05:19:27PM +0800, Guoqing Jiang wrote:
>> The device owns Bitmap_sync flag needs recovery
>> to become in sync, and read page from this type
>> device could get stale status.
>>
>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>> ---
>> When develop for clustered raid10 feature, if add a
>> disk under grow mode in master node, I could get
>> the "bitmap superblock UUID mismatch" warning due
>> to the page is read from Bitmap_sync device.
>>
>> drivers/md/bitmap.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
>> index bf7419a..bf34cd8 100644
>> --- a/drivers/md/bitmap.c
>> +++ b/drivers/md/bitmap.c
>> @@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
>>
>> rdev_for_each(rdev, mddev) {
>> if (! test_bit(In_sync, &rdev->flags)
>
> Why In_sync is set for the rdev? I think it shoudn't.
There are several states a device can be in.
If ->raid_disk is < 0, then the device is a spare, and doesn't contain
and valid data.
Otherwise ->raid_disk >=0 and:
In_sync is clear, which means that blocks before ->recovery_offset
contain valid data, and blocks after there don't
In_sync is set, which means ->recovery_offset is irrelevant and
should be treated as MaxSector
Bitmap_sync is set, which could apply in either of the above cases,
and means blocks corresponding to bits that are set in the bitmap
may not be up to date.
Bitmap_sync is only relevant before a device has been handed to the
personality. After it has been added, ->recovery_cp ensures that the
blocks that might be wrong are not read until until the bitmap-based
recovery has fixed them up.
Bitmap_sync is only use to stop the device from being given to the
personality if a recovery won't be started because the array is
read-only.
So it is perfectly valid for both In_sync and Bitmap_sync to be set.
If they are, it makes sense to avoid reading bitmap information from the
Bitmap_sync device, as that will be out-of-date.
I'm not quite sure why Guoqing is getting a UUID mismatch ... it
suggests that the new device wasn't initialized properly. So there
might be another bug. But I think this is definitely a bug.
>
>> - || test_bit(Faulty, &rdev->flags))
>> + || test_bit(Faulty, &rdev->flags)
>> + || test_bit(Bitmap_sync, &rdev->flags))
>
> I didn't see code clears the Bitmap_sync bit after disk is synced, so likely
> there is bug in this side.
There is no need to clear Bitmap_sync. It stays set until it becomes
irrelevant.
Thanks,
NeilBrown
>
>> continue;
>>
>> target = offset + index * (PAGE_SIZE/512);
>> --
>> 2.10.0
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox