* Re: Wrong array assembly on boot?
From: Dark Penguin @ 2017-07-24 15:27 UTC (permalink / raw)
To: Wols Lists, linux-raid
In-Reply-To: <597608BD.7080108@youngman.org.uk>
On 24/07/17 17:48, Wols Lists wrote:
> On 22/07/17 19:39, Dark Penguin wrote:
>> Greetings!
>>
>> I have a mirror RAID with two devices (sdc1 and sde1). It's not a root
>> partition, just a RAID with some data for services running on this
>> server. (I'm running Debian Jessie x86_64 with a 4.1.18 kernel.) The
>> RAID is listed in /etc/mdadm, and it has an external bitmap in /RAID .
>
> As an absolute minimum, can you please give us your version of mdadm.
Oh, right, sorry. I thought the "absolute minimum" would be the kernel
version and the distribution. :)
mdadm - v3.3.2 - 21st August 2014
> And the output of "mdadm --display" of your arrays. (I think I've got
> that right, I think --examine is the disk ...)
It's "mdadm --detail --scan" for all arrays or "mdadm --detail /dev/md0"
for md0.
I have 8 arrays on this server, and the only one that's relevant is this
one. (The rest of them are set up exactly the same way, but with
different names and UUIDs.) So, to avoid cluttering:
$ sudo mdadm --detail /dev/md/RAID
/dev/md/RAID:
Version : 1.2
Creation Time : Thu Oct 6 23:15:56 2016
Raid Level : raid1
Array Size : 244066432 (232.76 GiB 249.92 GB)
Used Dev Size : 244066432 (232.76 GiB 249.92 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : /RAID
Update Time : Mon Jul 24 17:59:53 2017
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : BAAL:RAID (local to host BAAL)
UUID : 8b5f18f0:54f655b7:8bfcc60d:4db6e6c8
Events : 5000
Number Major Minor RaidDevice State
0 8 65 0 active sync /dev/sde1
1 8 33 1 active sync writemostly /dev/sdc1
And the /etc/mdadm/mdadm.conf entry is:
ARRAY /dev/md/RAID metadata=1.2 name=BAAL:RAID bitmap=/RAID
UUID=8b5f18f0:54f655b7:8bfcc60d:4db6e6c8
I don't use the device names here because they change often in a server
with 8 arrays and 20 drives (sometimes I connect a new one or remove an
old one...). The UUID is here, the bitmap file is here, so it just looks
for all drives with this UUID and assembles the array.
As I understand, it has found the first device (/dev/sdc1, which was
outdated) and immediately added it to an array. Then it found the second
device (/dev/sde1, the up-to-date one), noticed an inconsistency and did
not add it. The question is, why did it start the array, why did it not
halt the boot process, why did it not realize that the second device is
newer (and also it already knows about the disappearance of the first
one!)...
--
darkpenguin
^ permalink raw reply
* Re: Wrong array assembly on boot?
From: Wols Lists @ 2017-07-24 19:36 UTC (permalink / raw)
To: Dark Penguin, linux-raid
In-Reply-To: <597611F6.20500@yandex.ru>
On 24/07/17 16:27, Dark Penguin wrote:
> On 24/07/17 17:48, Wols Lists wrote:
>> > On 22/07/17 19:39, Dark Penguin wrote:
>>> >> Greetings!
>>> >>
>>> >> I have a mirror RAID with two devices (sdc1 and sde1). It's not a root
>>> >> partition, just a RAID with some data for services running on this
>>> >> server. (I'm running Debian Jessie x86_64 with a 4.1.18 kernel.) The
>>> >> RAID is listed in /etc/mdadm, and it has an external bitmap in /RAID .
>> >
>> > As an absolute minimum, can you please give us your version of mdadm.
> Oh, right, sorry. I thought the "absolute minimum" would be the kernel
> version and the distribution. :)
>
> mdadm - v3.3.2 - 21st August 2014
>
>
I was afraid it might be that ...
You've hit a known bug in mdadm. It doesn't always successfully assemble
a mirror. I had exactly that problem - I created one mirror and when I
rebooted I had two ...
Can't offer any advice about how to fix your damaged mirror, but you
need to upgrade mdadm! That's two minor versions out of date - 3.4 and 4.0.
Cheers,
Wol
^ permalink raw reply
* Re: Wrong array assembly on boot?
From: Dark Penguin @ 2017-07-24 19:58 UTC (permalink / raw)
To: Wols Lists, linux-raid
In-Reply-To: <59764C40.4020503@youngman.org.uk>
On 24/07/17 22:36, Wols Lists wrote:
> On 24/07/17 16:27, Dark Penguin wrote:
>> On 24/07/17 17:48, Wols Lists wrote:
>>>> On 22/07/17 19:39, Dark Penguin wrote:
>>>>>> Greetings!
>>>>>>
>>>>>> I have a mirror RAID with two devices (sdc1 and sde1). It's not a root
>>>>>> partition, just a RAID with some data for services running on this
>>>>>> server. (I'm running Debian Jessie x86_64 with a 4.1.18 kernel.) The
>>>>>> RAID is listed in /etc/mdadm, and it has an external bitmap in /RAID .
>>>>
>>>> As an absolute minimum, can you please give us your version of mdadm.
>> Oh, right, sorry. I thought the "absolute minimum" would be the kernel
>> version and the distribution. :)
>>
>> mdadm - v3.3.2 - 21st August 2014
>>
>>
> I was afraid it might be that ...
>
> You've hit a known bug in mdadm. It doesn't always successfully assemble
> a mirror. I had exactly that problem - I created one mirror and when I
> rebooted I had two ...
>
> Can't offer any advice about how to fix your damaged mirror, but you
> need to upgrade mdadm! That's two minor versions out of date - 3.4 and 4.0.
>
> Cheers,
> Wol
My mirror is not damaged anymore - it's quite healthy and cleanly
missing some information I've overwritten. :) Of course, there's no way
to help that now - that's what backups are for. I just wanted to learn
how to avoid this situation in the future. And learn how is it really
supposed to handle such things.
Is this bug fixed in the newer mdadm? Or is it "known, but not fixed yet"?
--
darkpenguin
^ permalink raw reply
* Re: Wrong array assembly on boot?
From: Wols Lists @ 2017-07-24 20:20 UTC (permalink / raw)
To: Dark Penguin, linux-raid
In-Reply-To: <59765183.1030508@yandex.ru>
On 24/07/17 20:58, Dark Penguin wrote:
> On 24/07/17 22:36, Wols Lists wrote:
>> On 24/07/17 16:27, Dark Penguin wrote:
>>> On 24/07/17 17:48, Wols Lists wrote:
>>>>> On 22/07/17 19:39, Dark Penguin wrote:
>>>>>>> Greetings!
>>>>>>>
>>>>>>> I have a mirror RAID with two devices (sdc1 and sde1). It's not a root
>>>>>>> partition, just a RAID with some data for services running on this
>>>>>>> server. (I'm running Debian Jessie x86_64 with a 4.1.18 kernel.) The
>>>>>>> RAID is listed in /etc/mdadm, and it has an external bitmap in /RAID .
>>>>>
>>>>> As an absolute minimum, can you please give us your version of mdadm.
>>> Oh, right, sorry. I thought the "absolute minimum" would be the kernel
>>> version and the distribution. :)
>>>
>>> mdadm - v3.3.2 - 21st August 2014
>>>
>>>
>> I was afraid it might be that ...
>>
>> You've hit a known bug in mdadm. It doesn't always successfully assemble
>> a mirror. I had exactly that problem - I created one mirror and when I
>> rebooted I had two ...
>>
>> Can't offer any advice about how to fix your damaged mirror, but you
>> need to upgrade mdadm! That's two minor versions out of date - 3.4 and 4.0.
>>
>> Cheers,
>> Wol
>
> My mirror is not damaged anymore - it's quite healthy and cleanly
> missing some information I've overwritten. :) Of course, there's no way
> to help that now - that's what backups are for. I just wanted to learn
> how to avoid this situation in the future. And learn how is it really
> supposed to handle such things.
>
> Is this bug fixed in the newer mdadm? Or is it "known, but not fixed yet"?
>
>
Long fixed :-)
Cheers,
Wol
^ permalink raw reply
* (unknown),
From: hp @ 2017-07-25 20:01 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_89826583725_linux-raid.zip --]
[-- Type: application/zip, Size: 5777 bytes --]
^ permalink raw reply
* Re: Triggering WARN_ON_ONCE in drivers/md/md.c::set_in_sync()
From: Shaohua Li @ 2017-07-25 22:13 UTC (permalink / raw)
To: Joshua Kinard, neilb; +Cc: linux-raid
In-Reply-To: <1e68d380-762c-fbb4-73f9-842bd0fa7093@gentoo.org>
On Sun, Jul 23, 2017 at 09:11:39PM -0400, Joshua Kinard wrote:
> Hi,
>
> I'm testing out a netboot installer image on an old SGI MIPS machine,
> which has two disks (/dev/sda, /dev/sdb) in an md raid1 setup, all
> filesystems using XFS V5. root filesystem is on /dev/md0 and /dev/md2
> is where /usr will mount, but /usr is in the middle of a resync. The
> remaining md devices are synced and have bitmaps enabled.
>
> If I attempt to mount the root filesystem, I trigger these messages on
> the console:
> [ 147.156932] XFS (md0): Mounting V5 Filesystem
> [ 148.545726] ------------[ cut here ]------------
> [ 148.550522] WARNING: CPU: 0 PID: 258 at drivers/md/md.c:2273 set_in_sync+0x38/0xfc
> [ 148.558265] CPU: 0 PID: 258 Comm: md0_raid1 Not tainted 4.12.3-mipsgit-20170703 #1
> [ 148.565915] Stack : 0000000000000046 0000000000000000 0000000000000000 ffffffff9401fce1
> [ 148.574021] 0000000000000000 0000000000000000 0000000000000005 ffffffff8005a03c
> [ 148.582100] ffffffff80726e57 ffffffff806b3060 980000005318d800 0000000000000102
> [ 148.590198] ffffffff80b91f90 00000000000008e1 ffffffff806b0000 ffffffff80b70000
> [ 148.598298] 0000000000000000 ffffffff80096b5c 980000005355fbc8 ffffffff8002d170
> [ 148.606395] ffffffff8046c974 ffffffff8005b03c 0000000000000007 ffffffff806b3060
> [ 148.614495] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 148.622576] 0000000000000000 980000005355fb10 0000000000000000 ffffffff8002d3e0
> [ 148.630673] 0000000000000000 0000000000000000 ffffffff8046c974 0000000000000000
> [ 148.638773] 0000000000000000 ffffffff8000e81c 0000000000000000 ffffffff8002d3e0
> [ 148.646869] ...
> [ 148.649354] Call Trace:
> [ 148.651878] [<ffffffff8000e81c>] show_stack+0x70/0x8c
> [ 148.657012] [<ffffffff8002d3e0>] __warn+0x108/0x110
> [ 148.661935] [<ffffffff8046c974>] set_in_sync+0x38/0xfc
> [ 148.667157] [<ffffffff80476990>] md_check_recovery+0x2fc/0x5c0
> [ 148.673080] [<ffffffff8044bba8>] raid1d+0x48/0x1298
> [ 148.678032] [<ffffffff8046c934>] md_thread+0x178/0x180
> [ 148.683235] [<ffffffff80047650>] kthread+0x140/0x148
> [ 148.688271] [<ffffffff80009260>] ret_from_kernel_thread+0x14/0x1c
> [ 148.694438] ---[ end trace d27f806e939dc049 ]---
> [ 149.210292] XFS (md0): Ending clean mount
>
> Checking *(set_in_sync+0x38) in gdb yields:
> (gdb) l *(set_in_sync+0x38)
> 0xffffffff8046c974 is in set_in_sync (drivers/md/md.c:2274).
> 2269 }
> 2270
> 2271 static bool set_in_sync(struct mddev *mddev)
> 2272 {
> 2273 WARN_ON_ONCE(!spin_is_locked(&mddev->lock));
> 2274 if (!mddev->in_sync) {
> 2275 mddev->sync_checkers++;
> 2276 spin_unlock(&mddev->lock);
> 2277 percpu_ref_switch_to_atomic_sync(&mddev->writes_pending);
> 2278 spin_lock(&mddev->lock);
>
> Everything is still usable after this point, but attempting to untar a
> large file onto the /usr mount (/dev/md2) will crash/panic the kernel,
> but those panic messages are marked as "tainted". I'm currently
> waiting for the resync to finish now before proceeding further. I'll
> add that this machine only has one CPU, so my understanding was all
> spinlocks compile out in that case (if PREEMPT is not enabled, which it
> isn't). Thus I am a bit stumped why this is being triggered, especially
> when mounting an unrelated md device that is already fully resynced.
This isn't a big problem. spin_is_locked always returns 0, if you don't enable
CONFIG_SMP. We probably should change the code as:
WARN_ON_ONCE(!spin_is_locked(&mddev->lock) && defined(CONFIG_SMP));
Interesting is if I disable CONFIG_SMP, there are several bugs exposed, I can't
even boot my machine. Looks nobody tests UP case these days.
Thanks,
Shaohua
^ permalink raw reply
* *3gравсmвуйme! Bас uнmeрeсуюm клueнmскue 6азы gанныx?
From: ftqirqxoktupinfo@gorenje-rus.ru @ 2017-07-26 9:51 UTC (permalink / raw)
To: .
*Зgрaвсmвуйme! Вaс инmeрeсуюm kлиeнmсkиe бaзы gaнныx?
^ permalink raw reply
* Re: Triggering WARN_ON_ONCE in drivers/md/md.c::set_in_sync()
From: Wols Lists @ 2017-07-26 10:40 UTC (permalink / raw)
To: Shaohua Li, Joshua Kinard, neilb; +Cc: linux-raid
In-Reply-To: <20170725221325.z6ozo6vmos3edwse@kernel.org>
On 25/07/17 23:13, Shaohua Li wrote:
> Interesting is if I disable CONFIG_SMP, there are several bugs exposed, I can't
> even boot my machine. Looks nobody tests UP case these days.
That's not good :-( Although I doubt there are many single-core machines
left these days, gentoo would almost certainly disable it even today if
appropriate. Although I can't imagine anyone running gentoo on a
single-core now :-) My old Thunderbird K8 based system used to struggle
before I retired it a few years back ...
Cheers,
Wol
^ permalink raw reply
* Re: Triggering WARN_ON_ONCE in drivers/md/md.c::set_in_sync()
From: Joshua Kinard @ 2017-07-26 13:59 UTC (permalink / raw)
To: Wols Lists, Shaohua Li, neilb; +Cc: linux-raid
In-Reply-To: <59787190.1000802@youngman.org.uk>
On 07/26/2017 06:40, Wols Lists wrote:
> On 25/07/17 23:13, Shaohua Li wrote:
>> Interesting is if I disable CONFIG_SMP, there are several bugs exposed, I can't
>> even boot my machine. Looks nobody tests UP case these days.
>
> That's not good :-( Although I doubt there are many single-core machines
> left these days, gentoo would almost certainly disable it even today if
> appropriate. Although I can't imagine anyone running gentoo on a
> single-core now :-) My old Thunderbird K8 based system used to struggle
> before I retired it a few years back ...
>
> Cheers,
> Wol
The machine I'm running the raid1 array on is an SGI O2 (a.k.a., "IP32"),
purchased back in ~2005 and upgraded at random. It's got a single RM7000 CPU @
350MHz, with 256KB L2, 1MB L3 and 1GB RAM. About as maxed out as this machine
can get in Linux. So, no, not fast, especially for compiling code, but such is
the way in Gentoo. I'm actually re-installing it with a uClibc-based
filesystem, as glibc-linked binaries/libs shuffle too large of code around and
this thing's memory subsystem is rather slow.
For reference, anyone whose read the "UserFriendly" comic series, Erwin is an
SGI O2 machine.
--
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
"The past tempts us, the present confuses us, the future frightens us. And our
lives slip away, moment by moment, lost in that vast, terrible in-between."
--Emperor Turhan, Centauri Republic
^ permalink raw reply
* [PATCH 1/1] super1: Always round data offset to 1M
From: Pawel Baldysiak @ 2017-07-26 14:41 UTC (permalink / raw)
To: jes.sorensen; +Cc: linux-raid, Pawel Baldysiak
Currently if metadata requires more then 1M,
data offset will be rounded down to closest MB.
This is not correct, since less then required space is reserved.
Always round data offset up to multiple of 1M.
Signed-off-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
---
super1.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/super1.c b/super1.c
index 86ec850d..b15a1c7a 100644
--- a/super1.c
+++ b/super1.c
@@ -2796,8 +2796,7 @@ static int validate_geometry1(struct supertype *st, int level,
headroom >>= 1;
data_offset = 12*2 + bmspace + headroom;
#define ONE_MEG (2*1024)
- if (data_offset > ONE_MEG)
- data_offset = (data_offset / ONE_MEG) * ONE_MEG;
+ data_offset = ROUND_UP(data_offset, ONE_MEG);
break;
}
if (st->data_offset == INVALID_SECTORS)
--
2.13.0
^ permalink raw reply related
* (unknown),
From: een @ 2017-07-26 20:45 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_87861780008_linux-raid.zip --]
[-- Type: application/zip, Size: 2705 bytes --]
^ permalink raw reply
* [PATCH 0/9] Nowait feature for stacked block devices
From: Goldwyn Rodrigues @ 2017-07-26 23:57 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel
This is a continuation of the nowait support which was incorporated
a while back. We introduced REQ_NOWAIT which would return immediately
if the call would block at the block layer. Request based-devices
do not wait. However, bio based devices (the ones which exclusively
call make_request_fn) need to be trained to handle REQ_NOWAIT.
This effort covers the devices under MD and DM which would block
for any reason. If there should be more devices or situations
which need to be covered, please let me know.
--
Goldwyn
^ permalink raw reply
* [PATCH 1/9] QUEUE_FLAG_NOWAIT to indicate device supports nowait
From: Goldwyn Rodrigues @ 2017-07-26 23:57 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
Nowait is a feature of direct AIO, where users can request
to return immediately if the I/O is going to block. This translates
to REQ_NOWAIT in bio.bi_opf flags. While request based devices
don't wait, stacked devices such as md/dm will.
In order to explicitly mark stacked devices as supported, we
set the QUEUE_FLAG_NOWAIT in the queue_flags and return -EAGAIN
whenever the device would block.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
block/blk-core.c | 3 ++-
include/linux/blkdev.h | 2 ++
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 970b9c9638c5..1c9a981d88e5 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -2025,7 +2025,8 @@ generic_make_request_checks(struct bio *bio)
* if queue is not a request based queue.
*/
- if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_rq_based(q))
+ if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_rq_based(q) &&
+ !blk_queue_supports_nowait(q))
goto not_supported;
part = bio->bi_bdev->bd_part;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 25f6a0cb27d3..fae021ebec1b 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -633,6 +633,7 @@ struct request_queue {
#define QUEUE_FLAG_REGISTERED 29 /* queue has been registered to a disk */
#define QUEUE_FLAG_SCSI_PASSTHROUGH 30 /* queue supports SCSI commands */
#define QUEUE_FLAG_QUIESCED 31 /* queue has been quiesced */
+#define QUEUE_FLAG_NOWAIT 32 /* stack device driver supports REQ_NOWAIT */
#define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \
(1 << QUEUE_FLAG_STACKABLE) | \
@@ -732,6 +733,7 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q)
#define blk_queue_dax(q) test_bit(QUEUE_FLAG_DAX, &(q)->queue_flags)
#define blk_queue_scsi_passthrough(q) \
test_bit(QUEUE_FLAG_SCSI_PASSTHROUGH, &(q)->queue_flags)
+#define blk_queue_supports_nowait(q) test_bit(QUEUE_FLAG_NOWAIT, &(q)->queue_flags)
#define blk_noretry_request(rq) \
((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \
--
2.12.3
^ permalink raw reply related
* [PATCH 2/9] md: Add nowait support to md
From: Goldwyn Rodrigues @ 2017-07-26 23:57 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
Set queue flags to QUEUE_FLAG_NOWAIT to indicate REQ_NOWAIT
will be handled.
If an I/O on the md will be delayed, it would bail by calling
bio_wouldblock_error(). The conditions when this could happen are:
+ MD is suspended
+ There is a change pending on the SB, and current I/O would
block until that is complete.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/md.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 8cdca0296749..d96c27d16841 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -285,6 +285,13 @@ static blk_qc_t md_make_request(struct request_queue *q, struct bio *bio)
bio_endio(bio);
return BLK_QC_T_NONE;
}
+
+ if (mddev->suspended && (bio->bi_opf & REQ_NOWAIT)) {
+ bio_wouldblock_error(bio);
+ rcu_read_unlock();
+ return BLK_QC_T_NONE;
+ }
+
check_suspended:
rcu_read_lock();
if (mddev->suspended) {
@@ -5274,6 +5281,10 @@ static int md_alloc(dev_t dev, char *name)
mddev->queue = NULL;
goto abort;
}
+
+ /* Set the NOWAIT flags to show support */
+ queue_flag_set_unlocked(QUEUE_FLAG_NOWAIT, mddev->queue);
+
disk->major = MAJOR(mddev->unit);
disk->first_minor = unit << shift;
if (name)
@@ -8010,8 +8021,20 @@ bool md_write_start(struct mddev *mddev, struct bio *bi)
rcu_read_unlock();
if (did_change)
sysfs_notify_dirent_safe(mddev->sysfs_state);
- wait_event(mddev->sb_wait,
- !test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags) && !mddev->suspended);
+
+ /* Don't wait for sb writes if marked with REQ_NOWAIT */
+ if (test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags) ||
+ mddev->suspended) {
+ if (bi->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bi);
+ percpu_ref_put(&mddev->writes_pending);
+ return false;
+ }
+
+ wait_event(mddev->sb_wait,
+ !test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags) && !mddev->suspended);
+ }
+
if (test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags)) {
percpu_ref_put(&mddev->writes_pending);
return false;
--
2.12.3
^ permalink raw reply related
* [PATCH 3/9] md: raid1 nowait support
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
The RAID1 driver would bail with EAGAIN in case of:
+ I/O has to wait for a barrier
+ array is frozen
+ Area is suspended
+ There are too many pending I/O that it will be queued.
To facilitate error for wait barriers, wait_barrier() is
returning bool. True in case if there was a wait (or is not
required). False in case a wait was required, but was not performed.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/raid1.c | 74 +++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 57 insertions(+), 17 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 3febfc8391fb..66ca4288e3e8 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -903,8 +903,9 @@ static void lower_barrier(struct r1conf *conf, sector_t sector_nr)
wake_up(&conf->wait_barrier);
}
-static void _wait_barrier(struct r1conf *conf, int idx)
+static bool _wait_barrier(struct r1conf *conf, int idx, bool nowait)
{
+ bool ret = true;
/*
* We need to increase conf->nr_pending[idx] very early here,
* then raise_barrier() can be blocked when it waits for
@@ -935,7 +936,7 @@ static void _wait_barrier(struct r1conf *conf, int idx)
*/
if (!READ_ONCE(conf->array_frozen) &&
!atomic_read(&conf->barrier[idx]))
- return;
+ return ret;
/*
* After holding conf->resync_lock, conf->nr_pending[idx]
@@ -953,18 +954,26 @@ static void _wait_barrier(struct r1conf *conf, int idx)
*/
wake_up(&conf->wait_barrier);
/* Wait for the barrier in same barrier unit bucket to drop. */
- wait_event_lock_irq(conf->wait_barrier,
- !conf->array_frozen &&
- !atomic_read(&conf->barrier[idx]),
- conf->resync_lock);
+ if (conf->array_frozen || atomic_read(&conf->barrier[idx])) {
+ if (nowait)
+ ret = false;
+ else
+ wait_event_lock_irq(conf->wait_barrier,
+ !conf->array_frozen &&
+ !atomic_read(&conf->barrier[idx]),
+ conf->resync_lock);
+ }
atomic_inc(&conf->nr_pending[idx]);
atomic_dec(&conf->nr_waiting[idx]);
spin_unlock_irq(&conf->resync_lock);
+ return ret;
}
-static void wait_read_barrier(struct r1conf *conf, sector_t sector_nr)
+static bool wait_read_barrier(struct r1conf *conf, sector_t sector_nr,
+ bool nowait)
{
int idx = sector_to_idx(sector_nr);
+ bool ret = true;
/*
* Very similar to _wait_barrier(). The difference is, for read
@@ -976,7 +985,7 @@ static void wait_read_barrier(struct r1conf *conf, sector_t sector_nr)
atomic_inc(&conf->nr_pending[idx]);
if (!READ_ONCE(conf->array_frozen))
- return;
+ return ret;
spin_lock_irq(&conf->resync_lock);
atomic_inc(&conf->nr_waiting[idx]);
@@ -987,19 +996,28 @@ static void wait_read_barrier(struct r1conf *conf, sector_t sector_nr)
*/
wake_up(&conf->wait_barrier);
/* Wait for array to be unfrozen */
- wait_event_lock_irq(conf->wait_barrier,
- !conf->array_frozen,
- conf->resync_lock);
+ if (conf->array_frozen) {
+ /* If nowait flag is set, return false to
+ * show we did not wait
+ */
+ if (nowait)
+ ret = false;
+ else
+ wait_event_lock_irq(conf->wait_barrier,
+ !conf->array_frozen,
+ conf->resync_lock);
+ }
atomic_inc(&conf->nr_pending[idx]);
atomic_dec(&conf->nr_waiting[idx]);
spin_unlock_irq(&conf->resync_lock);
+ return ret;
}
-static void wait_barrier(struct r1conf *conf, sector_t sector_nr)
+static bool wait_barrier(struct r1conf *conf, sector_t sector_nr, bool nowait)
{
int idx = sector_to_idx(sector_nr);
- _wait_barrier(conf, idx);
+ return _wait_barrier(conf, idx, nowait);
}
static void wait_all_barriers(struct r1conf *conf)
@@ -1007,7 +1025,7 @@ static void wait_all_barriers(struct r1conf *conf)
int idx;
for (idx = 0; idx < BARRIER_BUCKETS_NR; idx++)
- _wait_barrier(conf, idx);
+ _wait_barrier(conf, idx, false);
}
static void _allow_barrier(struct r1conf *conf, int idx)
@@ -1223,7 +1241,11 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio,
* Still need barrier for READ in case that whole
* array is frozen.
*/
- wait_read_barrier(conf, bio->bi_iter.bi_sector);
+ if (!wait_read_barrier(conf, bio->bi_iter.bi_sector,
+ bio->bi_opf & REQ_NOWAIT)) {
+ bio_wouldblock_error(bio);
+ return;
+ }
if (!r1_bio)
r1_bio = alloc_r1bio(mddev, bio);
@@ -1333,6 +1355,11 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
* an interruptible wait.
*/
DEFINE_WAIT(w);
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
+
for (;;) {
sigset_t full, old;
prepare_to_wait(&conf->wait_barrier,
@@ -1351,7 +1378,11 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
}
finish_wait(&conf->wait_barrier, &w);
}
- wait_barrier(conf, bio->bi_iter.bi_sector);
+ if (!wait_barrier(conf, bio->bi_iter.bi_sector,
+ bio->bi_opf & REQ_NOWAIT)) {
+ bio_wouldblock_error(bio);
+ return;
+ }
r1_bio = alloc_r1bio(mddev, bio);
r1_bio->sectors = max_write_sectors;
@@ -1359,6 +1390,10 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
if (conf->pending_count >= max_queued_requests) {
md_wakeup_thread(mddev->thread);
raid1_log(mddev, "wait queued");
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
wait_event(conf->wait_barrier,
conf->pending_count < max_queued_requests);
}
@@ -1442,6 +1477,11 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
/* Wait for this device to become unblocked */
int j;
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
+
for (j = 0; j < i; j++)
if (r1_bio->bios[j])
rdev_dec_pending(conf->mirrors[j].rdev, mddev);
@@ -1449,7 +1489,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
allow_barrier(conf, bio->bi_iter.bi_sector);
raid1_log(mddev, "wait rdev %d blocked", blocked_rdev->raid_disk);
md_wait_for_blocked_rdev(blocked_rdev, mddev);
- wait_barrier(conf, bio->bi_iter.bi_sector);
+ wait_barrier(conf, bio->bi_iter.bi_sector, false);
goto retry_write;
}
--
2.12.3
^ permalink raw reply related
* [PATCH 4/9] md: raid5 nowait support
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
Return EAGAIN in case RAID5 would block because of waiting due to:
+ Reshaping
+ Suspension
+ Stripe Expansion
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/raid5.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index aeeb8d6854e2..d1b3bcf26d29 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -5635,6 +5635,11 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
? logical_sector < conf->reshape_safe
: logical_sector >= conf->reshape_safe) {
spin_unlock_irq(&conf->device_lock);
+ if (bi->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bi);
+ finish_wait(&conf->wait_for_overlap, &w);
+ return true;
+ }
schedule();
do_prepare = true;
goto retry;
@@ -5672,6 +5677,11 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
spin_unlock_irq(&conf->device_lock);
if (must_retry) {
raid5_release_stripe(sh);
+ if (bi->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bi);
+ finish_wait(&conf->wait_for_overlap, &w);
+ return true;
+ }
schedule();
do_prepare = true;
goto retry;
@@ -5700,6 +5710,11 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
sigset_t full, old;
sigfillset(&full);
sigprocmask(SIG_BLOCK, &full, &old);
+ if (bi->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bi);
+ finish_wait(&conf->wait_for_overlap, &w);
+ return true;
+ }
schedule();
sigprocmask(SIG_SETMASK, &old, NULL);
do_prepare = true;
@@ -5715,6 +5730,11 @@ static bool raid5_make_request(struct mddev *mddev, struct bio * bi)
*/
md_wakeup_thread(mddev->thread);
raid5_release_stripe(sh);
+ if (bi->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bi);
+ finish_wait(&conf->wait_for_overlap, &w);
+ return true;
+ }
schedule();
do_prepare = true;
goto retry;
--
2.12.3
^ permalink raw reply related
* [PATCH 5/9] md: raid10 nowait support
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
Bail and status to EAGAIN if raid10 is going to wait for:
+ barriers
+ reshape operation
+ Too many queued requests
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/raid10.c | 62 ++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 47 insertions(+), 15 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 5026e7ad51d3..6d80438c5040 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -978,8 +978,9 @@ static void lower_barrier(struct r10conf *conf)
wake_up(&conf->wait_barrier);
}
-static void wait_barrier(struct r10conf *conf)
+static bool wait_barrier(struct r10conf *conf, bool nowait)
{
+ bool ret = true;
spin_lock_irq(&conf->resync_lock);
if (conf->barrier) {
conf->nr_waiting++;
@@ -993,19 +994,23 @@ static void wait_barrier(struct r10conf *conf)
* count down.
*/
raid10_log(conf->mddev, "wait barrier");
- wait_event_lock_irq(conf->wait_barrier,
- !conf->barrier ||
- (atomic_read(&conf->nr_pending) &&
- current->bio_list &&
- (!bio_list_empty(¤t->bio_list[0]) ||
- !bio_list_empty(¤t->bio_list[1]))),
- conf->resync_lock);
+ if (!nowait)
+ wait_event_lock_irq(conf->wait_barrier,
+ !conf->barrier ||
+ (atomic_read(&conf->nr_pending) &&
+ current->bio_list &&
+ (!bio_list_empty(¤t->bio_list[0]) ||
+ !bio_list_empty(¤t->bio_list[1]))),
+ conf->resync_lock);
+ else
+ ret = false;
conf->nr_waiting--;
if (!conf->nr_waiting)
wake_up(&conf->wait_barrier);
}
atomic_inc(&conf->nr_pending);
spin_unlock_irq(&conf->resync_lock);
+ return ret;
}
static void allow_barrier(struct r10conf *conf)
@@ -1158,7 +1163,10 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,
* thread has put up a bar for new requests.
* Continue immediately if no resync is active currently.
*/
- wait_barrier(conf);
+ if (!wait_barrier(conf, bio->bi_opf & REQ_NOWAIT)) {
+ bio_wouldblock_error(bio);
+ return;
+ }
sectors = r10_bio->sectors;
while (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
@@ -1169,12 +1177,16 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,
* pass
*/
raid10_log(conf->mddev, "wait reshape");
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
allow_barrier(conf);
wait_event(conf->wait_barrier,
conf->reshape_progress <= bio->bi_iter.bi_sector ||
conf->reshape_progress >= bio->bi_iter.bi_sector +
sectors);
- wait_barrier(conf);
+ wait_barrier(conf, false);
}
rdev = read_balance(conf, r10_bio, &max_sectors);
@@ -1308,7 +1320,10 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
* thread has put up a bar for new requests.
* Continue immediately if no resync is active currently.
*/
- wait_barrier(conf);
+ if (!wait_barrier(conf, bio->bi_opf & REQ_NOWAIT)) {
+ bio_wouldblock_error(bio);
+ return;
+ }
sectors = r10_bio->sectors;
while (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
@@ -1319,12 +1334,16 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
* pass
*/
raid10_log(conf->mddev, "wait reshape");
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
allow_barrier(conf);
wait_event(conf->wait_barrier,
conf->reshape_progress <= bio->bi_iter.bi_sector ||
conf->reshape_progress >= bio->bi_iter.bi_sector +
sectors);
- wait_barrier(conf);
+ wait_barrier(conf, false);
}
if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
@@ -1339,6 +1358,10 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
BIT(MD_SB_CHANGE_DEVS) | BIT(MD_SB_CHANGE_PENDING));
md_wakeup_thread(mddev->thread);
raid10_log(conf->mddev, "wait reshape metadata");
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
wait_event(mddev->sb_wait,
!test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags));
@@ -1348,6 +1371,10 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
if (conf->pending_count >= max_queued_requests) {
md_wakeup_thread(mddev->thread);
raid10_log(mddev, "wait queued");
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
wait_event(conf->wait_barrier,
conf->pending_count < max_queued_requests);
}
@@ -1454,6 +1481,11 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
int j;
int d;
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return;
+ }
+
for (j = 0; j < i; j++) {
if (r10_bio->devs[j].bio) {
d = r10_bio->devs[j].devnum;
@@ -1474,7 +1506,7 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
allow_barrier(conf);
raid10_log(conf->mddev, "wait rdev %d blocked", blocked_rdev->raid_disk);
md_wait_for_blocked_rdev(blocked_rdev, mddev);
- wait_barrier(conf);
+ wait_barrier(conf, false);
goto retry_write;
}
@@ -1703,7 +1735,7 @@ static void print_conf(struct r10conf *conf)
static void close_sync(struct r10conf *conf)
{
- wait_barrier(conf);
+ wait_barrier(conf, false);
allow_barrier(conf);
mempool_destroy(conf->r10buf_pool);
@@ -4347,7 +4379,7 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
if (need_flush ||
time_after(jiffies, conf->reshape_checkpoint + 10*HZ)) {
/* Need to update reshape_position in metadata */
- wait_barrier(conf);
+ wait_barrier(conf, false);
mddev->reshape_position = conf->reshape_progress;
if (mddev->reshape_backwards)
mddev->curr_resync_completed = raid10_size(mddev, 0, 0)
--
2.12.3
^ permalink raw reply related
* [PATCH 6/9] dm: add nowait support
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
Add support for bio based dm devices, which exclusively
sets a make_request_fn(). Request based devices are supported
by default.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/dm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 2edbcc2d7d3f..aa9c1a5f2966 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1550,6 +1550,8 @@ static blk_qc_t dm_make_request(struct request_queue *q, struct bio *bio)
if (!(bio->bi_opf & REQ_RAHEAD))
queue_io(md, bio);
+ else if (bio->bi_opf & REQ_NOWAIT)
+ bio_wouldblock_error(bio);
else
bio_io_error(bio);
return BLK_QC_T_NONE;
@@ -2066,6 +2068,7 @@ int dm_setup_md_queue(struct mapped_device *md, struct dm_table *t)
case DM_TYPE_DAX_BIO_BASED:
dm_init_normal_md_queue(md);
blk_queue_make_request(md->queue, dm_make_request);
+ queue_flag_set_unlocked(QUEUE_FLAG_NOWAIT, md->queue);
/*
* DM handles splitting bios as needed. Free the bio_split bioset
* since it won't be used (saves 1 process per bio-based DM device).
--
2.12.3
^ permalink raw reply related
* [PATCH 7/9] dm: Add nowait support to raid1
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
If the I/O would block because the devices are syncing, bail.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/dm-raid1.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/md/dm-raid1.c b/drivers/md/dm-raid1.c
index a4fbd911d566..446ac581627f 100644
--- a/drivers/md/dm-raid1.c
+++ b/drivers/md/dm-raid1.c
@@ -1219,6 +1219,11 @@ static int mirror_map(struct dm_target *ti, struct bio *bio)
if (bio->bi_opf & REQ_RAHEAD)
return DM_MAPIO_KILL;
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return DM_MAPIO_SUBMITTED;
+ }
+
queue_bio(ms, bio, rw);
return DM_MAPIO_SUBMITTED;
}
--
2.12.3
^ permalink raw reply related
* [PATCH 8/9] dm: Add nowait support to dm-delay
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
I/O should bail out if any value for delay is set.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/dm-delay.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/md/dm-delay.c b/drivers/md/dm-delay.c
index ae3158795d26..97da97c3c039 100644
--- a/drivers/md/dm-delay.c
+++ b/drivers/md/dm-delay.c
@@ -240,6 +240,10 @@ static int delay_bio(struct delay_c *dc, int delay, struct bio *bio)
if (!delay || !atomic_read(&dc->may_delay))
return DM_MAPIO_REMAPPED;
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return DM_MAPIO_SUBMITTED;
+ }
delayed = dm_per_bio_data(bio, sizeof(struct dm_delay_info));
delayed->context = dc;
--
2.12.3
^ permalink raw reply related
* [PATCH 9/9] dm-mpath: Add nowait support
From: Goldwyn Rodrigues @ 2017-07-26 23:58 UTC (permalink / raw)
To: linux-block; +Cc: hch, jack, linux-raid, dm-devel, Goldwyn Rodrigues
In-Reply-To: <20170726235806.12148-1-rgoldwyn@suse.de>
From: Goldwyn Rodrigues <rgoldwyn@suse.com>
If there are no queues, bail if REQ_NOWAIT is set instead
of queueing up I/O.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
---
drivers/md/dm-mpath.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
index 0e8ab5bb3575..c6572a9967dc 100644
--- a/drivers/md/dm-mpath.c
+++ b/drivers/md/dm-mpath.c
@@ -543,6 +543,11 @@ static int __multipath_map_bio(struct multipath *m, struct bio *bio, struct dm_m
if ((pgpath && queue_io) ||
(!pgpath && test_bit(MPATHF_QUEUE_IF_NO_PATH, &m->flags))) {
+ /* Bail if nowait is set */
+ if (bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return DM_MAPIO_SUBMITTED;
+ }
/* Queue for the daemon to resubmit */
spin_lock_irqsave(&m->lock, flags);
bio_list_add(&m->queued_bios, bio);
--
2.12.3
^ permalink raw reply related
* Re: Triggering WARN_ON_ONCE in drivers/md/md.c::set_in_sync()
From: NeilBrown @ 2017-07-27 3:07 UTC (permalink / raw)
To: Shaohua Li, Joshua Kinard; +Cc: linux-raid
In-Reply-To: <20170725221325.z6ozo6vmos3edwse@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 4556 bytes --]
On Tue, Jul 25 2017, Shaohua Li wrote:
> On Sun, Jul 23, 2017 at 09:11:39PM -0400, Joshua Kinard wrote:
>> Hi,
>>
>> I'm testing out a netboot installer image on an old SGI MIPS machine,
>> which has two disks (/dev/sda, /dev/sdb) in an md raid1 setup, all
>> filesystems using XFS V5. root filesystem is on /dev/md0 and /dev/md2
>> is where /usr will mount, but /usr is in the middle of a resync. The
>> remaining md devices are synced and have bitmaps enabled.
>>
>> If I attempt to mount the root filesystem, I trigger these messages on
>> the console:
>> [ 147.156932] XFS (md0): Mounting V5 Filesystem
>> [ 148.545726] ------------[ cut here ]------------
>> [ 148.550522] WARNING: CPU: 0 PID: 258 at drivers/md/md.c:2273 set_in_sync+0x38/0xfc
>> [ 148.558265] CPU: 0 PID: 258 Comm: md0_raid1 Not tainted 4.12.3-mipsgit-20170703 #1
>> [ 148.565915] Stack : 0000000000000046 0000000000000000 0000000000000000 ffffffff9401fce1
>> [ 148.574021] 0000000000000000 0000000000000000 0000000000000005 ffffffff8005a03c
>> [ 148.582100] ffffffff80726e57 ffffffff806b3060 980000005318d800 0000000000000102
>> [ 148.590198] ffffffff80b91f90 00000000000008e1 ffffffff806b0000 ffffffff80b70000
>> [ 148.598298] 0000000000000000 ffffffff80096b5c 980000005355fbc8 ffffffff8002d170
>> [ 148.606395] ffffffff8046c974 ffffffff8005b03c 0000000000000007 ffffffff806b3060
>> [ 148.614495] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 148.622576] 0000000000000000 980000005355fb10 0000000000000000 ffffffff8002d3e0
>> [ 148.630673] 0000000000000000 0000000000000000 ffffffff8046c974 0000000000000000
>> [ 148.638773] 0000000000000000 ffffffff8000e81c 0000000000000000 ffffffff8002d3e0
>> [ 148.646869] ...
>> [ 148.649354] Call Trace:
>> [ 148.651878] [<ffffffff8000e81c>] show_stack+0x70/0x8c
>> [ 148.657012] [<ffffffff8002d3e0>] __warn+0x108/0x110
>> [ 148.661935] [<ffffffff8046c974>] set_in_sync+0x38/0xfc
>> [ 148.667157] [<ffffffff80476990>] md_check_recovery+0x2fc/0x5c0
>> [ 148.673080] [<ffffffff8044bba8>] raid1d+0x48/0x1298
>> [ 148.678032] [<ffffffff8046c934>] md_thread+0x178/0x180
>> [ 148.683235] [<ffffffff80047650>] kthread+0x140/0x148
>> [ 148.688271] [<ffffffff80009260>] ret_from_kernel_thread+0x14/0x1c
>> [ 148.694438] ---[ end trace d27f806e939dc049 ]---
>> [ 149.210292] XFS (md0): Ending clean mount
>>
>> Checking *(set_in_sync+0x38) in gdb yields:
>> (gdb) l *(set_in_sync+0x38)
>> 0xffffffff8046c974 is in set_in_sync (drivers/md/md.c:2274).
>> 2269 }
>> 2270
>> 2271 static bool set_in_sync(struct mddev *mddev)
>> 2272 {
>> 2273 WARN_ON_ONCE(!spin_is_locked(&mddev->lock));
>> 2274 if (!mddev->in_sync) {
>> 2275 mddev->sync_checkers++;
>> 2276 spin_unlock(&mddev->lock);
>> 2277 percpu_ref_switch_to_atomic_sync(&mddev->writes_pending);
>> 2278 spin_lock(&mddev->lock);
>>
>> Everything is still usable after this point, but attempting to untar a
>> large file onto the /usr mount (/dev/md2) will crash/panic the kernel,
>> but those panic messages are marked as "tainted". I'm currently
>> waiting for the resync to finish now before proceeding further. I'll
>> add that this machine only has one CPU, so my understanding was all
>> spinlocks compile out in that case (if PREEMPT is not enabled, which it
>> isn't). Thus I am a bit stumped why this is being triggered, especially
>> when mounting an unrelated md device that is already fully resynced.
>
> This isn't a big problem. spin_is_locked always returns 0, if you don't enable
> CONFIG_SMP. We probably should change the code as:
> WARN_ON_ONCE(!spin_is_locked(&mddev->lock) && defined(CONFIG_SMP));
Or WARN_ON_SMP (from kernel/futex.c)
or WARN_ON_ONCE(NR_CPUS != 1 && !spin_is_locked....) (from
mm/khugepage.c)
I'd probably go for lockdep_assert_held_once() as they is definitely
safe, and should provide enough warnings.
Do you want me to send a patch, or will you fix it up?
>
> Interesting is if I disable CONFIG_SMP, there are several bugs exposed, I can't
> even boot my machine. Looks nobody tests UP case these days.
Yes, that is sad.
Thanks,
NeilBrown
>
> Thanks,
> Shaohua
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: Triggering WARN_ON_ONCE in drivers/md/md.c::set_in_sync()
From: Shaohua Li @ 2017-07-27 3:51 UTC (permalink / raw)
To: NeilBrown; +Cc: Joshua Kinard, linux-raid
In-Reply-To: <87k22uwpaz.fsf@notabene.neil.brown.name>
On Thu, Jul 27, 2017 at 01:07:16PM +1000, Neil Brown wrote:
> On Tue, Jul 25 2017, Shaohua Li wrote:
>
> > On Sun, Jul 23, 2017 at 09:11:39PM -0400, Joshua Kinard wrote:
> >> Hi,
> >>
> >> I'm testing out a netboot installer image on an old SGI MIPS machine,
> >> which has two disks (/dev/sda, /dev/sdb) in an md raid1 setup, all
> >> filesystems using XFS V5. root filesystem is on /dev/md0 and /dev/md2
> >> is where /usr will mount, but /usr is in the middle of a resync. The
> >> remaining md devices are synced and have bitmaps enabled.
> >>
> >> If I attempt to mount the root filesystem, I trigger these messages on
> >> the console:
> >> [ 147.156932] XFS (md0): Mounting V5 Filesystem
> >> [ 148.545726] ------------[ cut here ]------------
> >> [ 148.550522] WARNING: CPU: 0 PID: 258 at drivers/md/md.c:2273 set_in_sync+0x38/0xfc
> >> [ 148.558265] CPU: 0 PID: 258 Comm: md0_raid1 Not tainted 4.12.3-mipsgit-20170703 #1
> >> [ 148.565915] Stack : 0000000000000046 0000000000000000 0000000000000000 ffffffff9401fce1
> >> [ 148.574021] 0000000000000000 0000000000000000 0000000000000005 ffffffff8005a03c
> >> [ 148.582100] ffffffff80726e57 ffffffff806b3060 980000005318d800 0000000000000102
> >> [ 148.590198] ffffffff80b91f90 00000000000008e1 ffffffff806b0000 ffffffff80b70000
> >> [ 148.598298] 0000000000000000 ffffffff80096b5c 980000005355fbc8 ffffffff8002d170
> >> [ 148.606395] ffffffff8046c974 ffffffff8005b03c 0000000000000007 ffffffff806b3060
> >> [ 148.614495] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> >> [ 148.622576] 0000000000000000 980000005355fb10 0000000000000000 ffffffff8002d3e0
> >> [ 148.630673] 0000000000000000 0000000000000000 ffffffff8046c974 0000000000000000
> >> [ 148.638773] 0000000000000000 ffffffff8000e81c 0000000000000000 ffffffff8002d3e0
> >> [ 148.646869] ...
> >> [ 148.649354] Call Trace:
> >> [ 148.651878] [<ffffffff8000e81c>] show_stack+0x70/0x8c
> >> [ 148.657012] [<ffffffff8002d3e0>] __warn+0x108/0x110
> >> [ 148.661935] [<ffffffff8046c974>] set_in_sync+0x38/0xfc
> >> [ 148.667157] [<ffffffff80476990>] md_check_recovery+0x2fc/0x5c0
> >> [ 148.673080] [<ffffffff8044bba8>] raid1d+0x48/0x1298
> >> [ 148.678032] [<ffffffff8046c934>] md_thread+0x178/0x180
> >> [ 148.683235] [<ffffffff80047650>] kthread+0x140/0x148
> >> [ 148.688271] [<ffffffff80009260>] ret_from_kernel_thread+0x14/0x1c
> >> [ 148.694438] ---[ end trace d27f806e939dc049 ]---
> >> [ 149.210292] XFS (md0): Ending clean mount
> >>
> >> Checking *(set_in_sync+0x38) in gdb yields:
> >> (gdb) l *(set_in_sync+0x38)
> >> 0xffffffff8046c974 is in set_in_sync (drivers/md/md.c:2274).
> >> 2269 }
> >> 2270
> >> 2271 static bool set_in_sync(struct mddev *mddev)
> >> 2272 {
> >> 2273 WARN_ON_ONCE(!spin_is_locked(&mddev->lock));
> >> 2274 if (!mddev->in_sync) {
> >> 2275 mddev->sync_checkers++;
> >> 2276 spin_unlock(&mddev->lock);
> >> 2277 percpu_ref_switch_to_atomic_sync(&mddev->writes_pending);
> >> 2278 spin_lock(&mddev->lock);
> >>
> >> Everything is still usable after this point, but attempting to untar a
> >> large file onto the /usr mount (/dev/md2) will crash/panic the kernel,
> >> but those panic messages are marked as "tainted". I'm currently
> >> waiting for the resync to finish now before proceeding further. I'll
> >> add that this machine only has one CPU, so my understanding was all
> >> spinlocks compile out in that case (if PREEMPT is not enabled, which it
> >> isn't). Thus I am a bit stumped why this is being triggered, especially
> >> when mounting an unrelated md device that is already fully resynced.
> >
> > This isn't a big problem. spin_is_locked always returns 0, if you don't enable
> > CONFIG_SMP. We probably should change the code as:
> > WARN_ON_ONCE(!spin_is_locked(&mddev->lock) && defined(CONFIG_SMP));
>
> Or WARN_ON_SMP (from kernel/futex.c)
> or WARN_ON_ONCE(NR_CPUS != 1 && !spin_is_locked....) (from
> mm/khugepage.c)
>
> I'd probably go for lockdep_assert_held_once() as they is definitely
> safe, and should provide enough warnings.
>
> Do you want me to send a patch, or will you fix it up?
Already fixed. I added NR_CPUS != 1 there
> >
> > Interesting is if I disable CONFIG_SMP, there are several bugs exposed, I can't
> > even boot my machine. Looks nobody tests UP case these days.
>
> Yes, that is sad.
That's a pci/nvme bug
Thanks,
Shaohua
^ permalink raw reply
* (unknown),
From: hp @ 2017-07-27 5:01 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: EMAIL_86618341708_linux-raid.zip --]
[-- Type: application/zip, Size: 2744 bytes --]
^ permalink raw reply
* Re: [PATCH 1/1] super1: Always round data offset to 1M
From: Jes Sorensen @ 2017-07-27 9:32 UTC (permalink / raw)
To: Pawel Baldysiak; +Cc: linux-raid
In-Reply-To: <20170726144154.31798-1-pawel.baldysiak@intel.com>
On 07/26/2017 10:41 AM, Pawel Baldysiak wrote:
> Currently if metadata requires more then 1M,
> data offset will be rounded down to closest MB.
> This is not correct, since less then required space is reserved.
> Always round data offset up to multiple of 1M.
>
> Signed-off-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
> ---
> super1.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
Applied
Thanks,
Jes
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox