* Re: [PATCH RESEND] Monitor: fix for regression with container devices
From: Artur Paszkiewicz @ 2015-02-13 15:29 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, pawel.baldysiak
In-Reply-To: <20150211153814.333cb17a@notabene.brown>
On 02/11/2015 05:38 AM, NeilBrown wrote:
> On Mon, 9 Feb 2015 11:13:50 +0100 Artur Paszkiewicz
> <artur.paszkiewicz@intel.com> wrote:
>
> > This patch fixes 2 problems introduced by commit 9a518d8: not closing a
> > file descriptor and ignoring container devices. Array state is always
> > "inactive" for containers, so we make sure that the device is not a
> > container by reading also the "level" sysfs entry.
> >
> > Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
> > Reviewed-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
> > ---
> > Monitor.c | 14 ++++++++++----
> > 1 file changed, 10 insertions(+), 4 deletions(-)
> >
> > diff --git a/Monitor.c b/Monitor.c
> > index 971d2ec..66d67ba 100644
> > --- a/Monitor.c
> > +++ b/Monitor.c
> > @@ -483,11 +483,17 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
> > strncmp(buf,"inact",5) == 0) {
> > if (fd >= 0)
> > close(fd);
> > - if (!st->err)
> > - alert("DeviceDisappeared", dev, NULL, ainfo);
> > - st->err++;
> > - return 0;
> > + fd = sysfs_open(st->devnm, NULL, "level");
> > + if (fd < 0 || read(fd, buf, 10) != 0) {
> > + if (fd >= 0)
> > + close(fd);
> > + if (!st->err)
> > + alert("DeviceDisappeared", dev, NULL, ainfo);
> > + st->err++;
> > + return 0;
> > + }
> > }
> > + close(fd);
> > }
> > fd = open(dev, O_RDONLY);
> > if (fd < 0) {
>
> Thanks for the patch.
>
> I don't think I agree with the logic of using 'level' though.
> For the sort of arrays that I need to ignore here, 'level' will be empty.
>
> It would make sense to test 'metadata' though. If that starts 'external:',
> then we don't want to ignore the array.
>
> Could you confirm that this works please?
>
Hi Neil,
I tested your patch. I assume you wanted to use 'metadata_version',
because there is no 'metadata' attribute, right? I had also thought
about that, but simply looking for 'external:' is not enough to
determine that the array is a container - for volumes inside the
container it looks like this: 'external:/md127/0'. But I think that the
arrays you want to ignore will just have 'none' there, so maybe it can
be done like this?
diff --git a/Monitor.c b/Monitor.c
index 971d2ec..83daf3b 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -483,11 +483,18 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
strncmp(buf,"inact",5) == 0) {
if (fd >= 0)
close(fd);
- if (!st->err)
- alert("DeviceDisappeared", dev, NULL, ainfo);
- st->err++;
- return 0;
+ fd = sysfs_open(st->devnm, NULL, "metadata_version");
+ if (fd < 0 || read(fd, buf, 4) < 0 ||
+ strncmp(buf, "none", 4) == 0) {
+ if (fd >= 0)
+ close(fd);
+ if (!st->err)
+ alert("DeviceDisappeared", dev, NULL, ainfo);
+ st->err++;
+ return 0;
+ }
}
+ close(fd);
}
fd = open(dev, O_RDONLY);
if (fd < 0) {
Thanks,
Artur
> Thanks,
> NeilBrown
>
> diff --git a/Monitor.c b/Monitor.c
> index 971d2ecbea72..6e085cb24993 100644
> --- a/Monitor.c
> +++ b/Monitor.c
> @@ -483,11 +483,18 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
> strncmp(buf,"inact",5) == 0) {
> if (fd >= 0)
> close(fd);
> - if (!st->err)
> - alert("DeviceDisappeared", dev, NULL, ainfo);
> - st->err++;
> - return 0;
> + fd = sysfs_open(st->devnm, NULL, "metadata");
> + if (fd < 0 || read(fd, buf, 9) != 9 ||
> + strncmp(buf, "external:", 9) != 0) {
> + if (fd >= 0)
> + close(fd);
> + if (!st->err)
> + alert("DeviceDisappeared", dev, NULL, ainfo);
> + st->err++;
> + return 0;
> + }
> }
> + close(fd);
> }
> fd = open(dev, O_RDONLY);
> if (fd < 0) {
>
^ permalink raw reply related
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 14:48 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213102746.GO2896@worktop.programming.kicks-ass.net>
On Fri, Feb 13, 2015 at 11:27:46AM +0100, Peter Zijlstra wrote:
> > I've moved blk_flush_plug to the beginning of the function.
>
> > I wondered if it really make sense to call blk_flush_plug with nr_iowait
> > elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
> > for non-"io" reasons and maybe that could upset stuff???
>
> Yeah, good question that. Lemme ponder that a bit.
Yes, I thikn your version makes most sense as, you say, even regular
schedule() call nested in my version would go towards blk delayacct --
and I doubt that was the intent; even though the current kernel works
that way.
I'll move the now rudimentary io_schedule() into sched.h as an inline.
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 10:27 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213194953.0368355d@notabene.brown>
On Fri, Feb 13, 2015 at 07:49:53PM +1100, NeilBrown wrote:
> > Like said, that will still recursive call delayacct_blkio_*() and would
> > increase nr_iowait for a second time; while arguably its still the same
> > one io-wait instance.
>
> No it doesn't. There is no "blk_flush_plug" call between the
> delayacct_blkio_*() calls.
Duh, clearly I needed to still wake up :/
> I've moved blk_flush_plug to the beginning of the function.
> I wondered if it really make sense to call blk_flush_plug with nr_iowait
> elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
> for non-"io" reasons and maybe that could upset stuff???
Yeah, good question that. Lemme ponder that a bit.
> I don't really know. I'm happy with your version. I don't suppose anyone
> else is paying attention and could give a third opinion....
:-)
^ permalink raw reply
* mdadm failed to remove internal bitmap
From: gary @ 2015-02-13 9:46 UTC (permalink / raw)
To: linux-raid
Hi,
I used v3.3.1 mdadm to do some test for bitmap, but when switch bitmap from
internal to none, the output shows fail info about remove internal
bitmap, is it
just a warning? Since the bitmap seems to be cleared, and it doesn't
show with
v3.2.6 mdadm with the same steps.
linux:~ # mdadm --create md0 --raid-devices=2 --level=mirror
--assume-clean /dev/vdb /dev/vdc
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md0 started.
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
unused devices: <none>
linux:~ # mdadm --grow --bitmap=internal /dev/md127
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
unused devices: <none>
linux:~ # mdadm --grow --bitmap=none /dev/md127
mdadm: failed to remove internal bitmap.
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
unused devices: <none>
Thanks,
Gary
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: NeilBrown @ 2015-02-13 8:49 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213083250.GN2896@worktop.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 3876 bytes --]
On Fri, 13 Feb 2015 09:32:50 +0100 Peter Zijlstra <peterz@infradead.org>
wrote:
> On Fri, Feb 13, 2015 at 04:26:00PM +1100, NeilBrown wrote:
> > I choose ... Buzz Lightyear !!!
>
> Great choice!
>
> > From: NeilBrown <neilb@suse.de>
> > Date: Fri, 13 Feb 2015 15:49:17 +1100
> > Subject: [PATCH] sched: prevent recursion in io_schedule()
> >
> > io_schedule() calls blk_flush_plug() which, depending on the
> > contents of current->plug, can initiate arbitrary blk-io requests.
> >
> > Note that this contrasts with blk_schedule_flush_plug() which requires
> > all non-trivial work to be handed off to a separate thread.
> >
> > This makes it possible for io_schedule() to recurse, and initiating
> > block requests could possibly call mempool_alloc() which, in times of
> > memory pressure, uses io_schedule().
> >
> > Apart from any stack usage issues, io_schedule() will not behave
> > correctly when called recursively as delayacct_blkio_start() does
> > not allow for repeated calls.
>
> Which seems to still be an issue with this patch.
>
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 1f37fe7f77a4..90f3de8bc7ca 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
> > */
> > void __sched io_schedule(void)
> > {
> > + io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
> > }
> > EXPORT_SYMBOL(io_schedule);
>
> Might as well move it to sched.h as an inline or so..
>
> > long __sched io_schedule_timeout(long timeout)
> > {
> > + struct rq *rq;
> > long ret;
> > + int old_iowait = current->in_iowait;
> > +
> > + current->in_iowait = 1;
> > + if (old_iowait)
> > + blk_schedule_flush_plug(current);
> > + else
> > + blk_flush_plug(current);
> >
> > delayacct_blkio_start();
> > + rq = raw_rq();
> > atomic_inc(&rq->nr_iowait);
> > ret = schedule_timeout(timeout);
> > + current->in_iowait = old_iowait;
> > atomic_dec(&rq->nr_iowait);
> > delayacct_blkio_end();
> > return ret;
>
> Like said, that will still recursive call delayacct_blkio_*() and would
> increase nr_iowait for a second time; while arguably its still the same
> one io-wait instance.
No it doesn't. There is no "blk_flush_plug" call between the
delayacct_blkio_*() calls.
I've moved blk_flush_plug to the beginning of the function.
>
> So would a little something like:
>
> long __sched io_schedule_timeout(long timeout)
> {
> struct rq *rq;
> long ret;
>
> /*
> * Recursive io_schedule() call; make sure to not recurse
> * on the blk_flush_plug() stuff again.
> */
> if (unlikely(current->in_iowait)) {
> /*
> * Our parent io_schedule() call will already have done
> * all the required io-wait accounting.
> */
> blk_schedule_flush_plug(current);
> return schedule_timeout(timeout);
> }
>
> current->in_iowait = 1;
> delayacct_blkio_start();
> rq = raw_rq();
> atomic_inc(&rq->nr_iowait);
> blk_flush_plug(current);
> ret = schedule_timeout(timeout);
> atomic_dec(&rq->nr_iowait);
> delayacct_blkio_end();
> current->in_iowait = 0;
>
> return ret;
> }
>
> not make more sense?
That does make a similar amount of sense at least....
I wondered if it really make sense to call blk_flush_plug with nr_iowait
elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
for non-"io" reasons and maybe that could upset stuff???
I don't really know. I'm happy with your version. I don't suppose anyone
else is paying attention and could give a third opinion....
Thanks,
NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 8:32 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213162600.059fffb2@notabene.brown>
On Fri, Feb 13, 2015 at 04:26:00PM +1100, NeilBrown wrote:
> I choose ... Buzz Lightyear !!!
Great choice!
> From: NeilBrown <neilb@suse.de>
> Date: Fri, 13 Feb 2015 15:49:17 +1100
> Subject: [PATCH] sched: prevent recursion in io_schedule()
>
> io_schedule() calls blk_flush_plug() which, depending on the
> contents of current->plug, can initiate arbitrary blk-io requests.
>
> Note that this contrasts with blk_schedule_flush_plug() which requires
> all non-trivial work to be handed off to a separate thread.
>
> This makes it possible for io_schedule() to recurse, and initiating
> block requests could possibly call mempool_alloc() which, in times of
> memory pressure, uses io_schedule().
>
> Apart from any stack usage issues, io_schedule() will not behave
> correctly when called recursively as delayacct_blkio_start() does
> not allow for repeated calls.
Which seems to still be an issue with this patch.
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 1f37fe7f77a4..90f3de8bc7ca 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
> */
> void __sched io_schedule(void)
> {
> + io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
> }
> EXPORT_SYMBOL(io_schedule);
Might as well move it to sched.h as an inline or so..
> long __sched io_schedule_timeout(long timeout)
> {
> + struct rq *rq;
> long ret;
> + int old_iowait = current->in_iowait;
> +
> + current->in_iowait = 1;
> + if (old_iowait)
> + blk_schedule_flush_plug(current);
> + else
> + blk_flush_plug(current);
>
> delayacct_blkio_start();
> + rq = raw_rq();
> atomic_inc(&rq->nr_iowait);
> ret = schedule_timeout(timeout);
> + current->in_iowait = old_iowait;
> atomic_dec(&rq->nr_iowait);
> delayacct_blkio_end();
> return ret;
Like said, that will still recursive call delayacct_blkio_*() and would
increase nr_iowait for a second time; while arguably its still the same
one io-wait instance.
So would a little something like:
long __sched io_schedule_timeout(long timeout)
{
struct rq *rq;
long ret;
/*
* Recursive io_schedule() call; make sure to not recurse
* on the blk_flush_plug() stuff again.
*/
if (unlikely(current->in_iowait)) {
/*
* Our parent io_schedule() call will already have done
* all the required io-wait accounting.
*/
blk_schedule_flush_plug(current);
return schedule_timeout(timeout);
}
current->in_iowait = 1;
delayacct_blkio_start();
rq = raw_rq();
atomic_inc(&rq->nr_iowait);
blk_flush_plug(current);
ret = schedule_timeout(timeout);
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
current->in_iowait = 0;
return ret;
}
not make more sense?
^ permalink raw reply
* Re: raid1 narrow_write_error with 4K disks, sd "bad block number requested" messages
From: NeilBrown @ 2015-02-13 6:01 UTC (permalink / raw)
To: Nate Dailey; +Cc: linux-raid, linux-scsi
In-Reply-To: <54DCD8DD.7080103@stratus.com>
[-- Attachment #1: Type: text/plain, Size: 3813 bytes --]
On Thu, 12 Feb 2015 11:46:21 -0500 Nate Dailey <nate.dailey@stratus.com>
wrote:
> On 02/04/2015 11:59 PM, NeilBrown wrote:
> > On Wed, 28 Jan 2015 10:29:46 -0500 Nate Dailey <nate.dailey@stratus.com>
> > wrote:
> >
> >> I'm writing about something that appears to be an issue with raid1's
> >> narrow_write_error, particular to non-512-byte-sector disks. Here's what
> >> I'm doing:
> >>
> >> - 2 disk raid1, 4K disks, each connected to a different SAS HBA
> >> - mount a filesystem on the raid1, run a test that writes to it
> >> - remove one of the SAS HBAs (echo 1 >
> >> /sys/bus/pci/devices/0000\:45\:00.0/remove)
> >>
> >> At this point, writes fail and narrow_write_error breaks them up and
> >> retries, one sector at a time. But these are 512-byte sectors, and sd
> >> doesn't like it:
> >>
> >> [ 2645.310517] sd 3:0:1:0: [sde] Bad block number requested
> >> [ 2645.310610] sd 3:0:1:0: [sde] Bad block number requested
> >> [ 2645.310690] sd 3:0:1:0: [sde] Bad block number requested
> >> ...
> >>
> >> There appears to be no real harm done, but there can be a huge number of
> >> these messages in the log.
> >>
> >> I can avoid this by disabling bad block tracking, but it looks like
> >> maybe the superblock's bblog_shift is intended to address this exact
> >> issue. However, I don't see a way to change it. Presumably this is
> >> something mdadm should be setting up? I don't see bblog_shift ever set
> >> to anything other than 0.
> >>
> >> This is on a RHEL 7.1 kernel, version 3.10.0-221.el7. I took a look at
> >> upstream sd and md changes and nothing jumps out at me that would have
> >> affected this (but I have not tested to see if the bad block messages do
> >> or do not happen on an upstream kernel).
> >>
> >> I'd appreciate any advice re: how to handle this. Thanks!
> >
> > Thanks for the report.
> >
> > narrow_write_error() should use bdev_logical_block_size() and round up to
> > that.
> > Possibly mdadm should get the same information and set bblog_shift
> > accordingly when creating a bad block log.
> >
> > I've made a note to fix that, but I'm happy to review patches too :-)
> >
> > thanks,
> > NeilBrown
> >
>
> I will post a narrow_write_error patch shortly.
>
> I did some experimentation with setting the bblog_shift in mdadm, but it
> didn't work out the way I expected. It turns out that the value is only
> loaded from the superblock if:
>
> 1453 if ((le32_to_cpu(sb->feature_map) & MD_FEATURE_BAD_BLOCKS) &&
> 1454 rdev->badblocks.count == 0) {
> ...
> 1473 rdev->badblocks.shift = sb->bblog_shift;
>
> And this feature bit is only set if any bad blocks have actually been
> recorded.
>
> It also appears to me that the shift is used when loading the bad blocks
> from the superblock, but not when storing the bad block list in the
> superblock.
>
> Seems like these are bugs, but I'm not certain how the code is supposed
> to work (and am getting in a bit over my head with this).
Yes, that's probably a bug.
The
} else if (sb->bblog_offset != 0)
rdev->badblocks.shift = 0;
should be
} else if (sb->bblog_offset != 0)
rdev->badblocks.shift = sb->bblog_shift;
>
> In any case, it doesn't appear to me that there's any harm in having the
> bblog_shift not match the disk's block size (right?).
Having the bblog_shift larger than the disk's block size certainly should not
be a problem. Having it small only causes the problem that you have already
discovered.
NeilBrown
>
> Nate Dailey
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: NeilBrown @ 2015-02-13 5:26 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150210092936.GW21418@twins.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 4838 bytes --]
On Tue, 10 Feb 2015 10:29:36 +0100 Peter Zijlstra <peterz@infradead.org>
wrote:
> On Tue, Feb 10, 2015 at 01:50:17PM +1100, NeilBrown wrote:
> > On Mon, 9 Feb 2015 10:10:00 +0100 Peter Zijlstra <peterz@infradead.org> wrote:
> > > > However, when io_schedule() explicitly calls blk_flush_plug(), then
> > > > @from_schedule=false variant is used, and the unplug functions are allowed to
> > > > allocate memory and block and maybe even call mempool_alloc() which might
> > > > call io_schedule().
> > > >
> > > > This shouldn't be a problem as blk_flush_plug() spliced out the plug list, so
> > > > any recursive call will find an empty list and do nothing.
> > >
> > > Unless, something along the way stuck something back on, right? So
> > > should we stick an:
> > >
> > > WARN_ON(current->in_iowait);
> > >
> > > somewhere near where things are added to this plug list? (and move the
> > > blk_flush_plug() call inside of where that's actually true of course).
> >
> > No, I don't think so.
> >
> > It is certainly possible that some request on plug->cb_list could add
> > something to plug->list - which is processed after ->cb_list.
> >
> > I think the best way to think about this is that the *problem* was that a
> > wait_event loop could spin without making any progress. So any time that
> > clear forward progress is made it is safe sleep without necessitating the
> > warning. Hence sched_annotate_sleep() is reasonable.
> > blk_flush_plug() with definitely have dispatched some requests if it
> > might_sleep(), so the sleep is OK.
>
> Well, yes, but you forget that this gets us back into recursion land.
> io_schedule() calling io_schedule() calling io_schedule() and *boom*
> stack overflow -> dead machine.
>
> We must either guarantee io_schedule() will never call io_schedule() or
> that io_schedule() itself will not add new work to the current plug such
> that calling io_schedule() itself will not recurse on the blk stuff.
>
> Pick either option, but pick one.
I choose ... Buzz Lightyear !!!
Sorry, go carried away there. Uhhmm. I think I pick a/ (But I expect I'll
find a goat... ho hum).
Does this look credible?
Thanks,
NeilBrown
From: NeilBrown <neilb@suse.de>
Date: Fri, 13 Feb 2015 15:49:17 +1100
Subject: [PATCH] sched: prevent recursion in io_schedule()
io_schedule() calls blk_flush_plug() which, depending on the
contents of current->plug, can initiate arbitrary blk-io requests.
Note that this contrasts with blk_schedule_flush_plug() which requires
all non-trivial work to be handed off to a separate thread.
This makes it possible for io_schedule() to recurse, and initiating
block requests could possibly call mempool_alloc() which, in times of
memory pressure, uses io_schedule().
Apart from any stack usage issues, io_schedule() will not behave
correctly when called recursively as delayacct_blkio_start() does
not allow for repeated calls.
So:
- use in_iowait to detect recursion. Set it earlier, and restore
it to the old value.
- move the call to "raw_rq" after the call to blk_flush_plug().
As this is some sort of per-cpu thing, we want some chance that
we are on the right CPU
- When io_schedule() is called recurively, use blk_schedule_flush_plug()
which cannot further recurse.
- as this makes io_schedule() a lot more complex and as io_schedule()
must match io_schedule_timeout(), but all the changes in io_schedule_timeout()
and make io_schedule a simple wrapper for that.
Signed-off-by: NeilBrown <neilb@suse.de>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Peter Zijlstra <peterz@infradead.org>
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1f37fe7f77a4..90f3de8bc7ca 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
*/
void __sched io_schedule(void)
{
- struct rq *rq = raw_rq();
-
- delayacct_blkio_start();
- atomic_inc(&rq->nr_iowait);
- blk_flush_plug(current);
- current->in_iowait = 1;
- schedule();
- current->in_iowait = 0;
- atomic_dec(&rq->nr_iowait);
- delayacct_blkio_end();
+ io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
}
EXPORT_SYMBOL(io_schedule);
long __sched io_schedule_timeout(long timeout)
{
- struct rq *rq = raw_rq();
+ struct rq *rq;
long ret;
+ int old_iowait = current->in_iowait;
+
+ current->in_iowait = 1;
+ if (old_iowait)
+ blk_schedule_flush_plug(current);
+ else
+ blk_flush_plug(current);
delayacct_blkio_start();
+ rq = raw_rq();
atomic_inc(&rq->nr_iowait);
- blk_flush_plug(current);
- current->in_iowait = 1;
ret = schedule_timeout(timeout);
- current->in_iowait = 0;
+ current->in_iowait = old_iowait;
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
return ret;
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* [md PATCH] md/raid1: round up to bdev_logical_block_size in narrow_write_error
From: Nate Dailey @ 2015-02-12 17:02 UTC (permalink / raw)
To: linux-raid
This modifies raid1's narrow_write_error to round up block_sectors to the
device's logical block size.
This prevents sd complaining about "Bad block number requested" for non-512-byte
sector disks.
Signed-off-by: Nate Dailey <nate.dailey@stratus.com>
---
diff -Nupr a/drivers/md/raid1.c b/drivers/md/raid1.c
--- a/drivers/md/raid1.c 2015-02-10 15:29:02.000000000 -0500
+++ b/drivers/md/raid1.c 2015-02-10 15:29:45.000000000 -0500
@@ -2206,7 +2206,8 @@ static int narrow_write_error(struct r1b
if (rdev->badblocks.shift < 0)
return 0;
- block_sectors = 1 << rdev->badblocks.shift;
+ block_sectors = roundup(1 << rdev->badblocks.shift,
+ bdev_logical_block_size(rdev->bdev) >> 9);
sector = r1_bio->sector;
sectors = ((sector + block_sectors)
& ~(sector_t)(block_sectors - 1))
^ permalink raw reply
* Re: raid1 narrow_write_error with 4K disks, sd "bad block number requested" messages
From: Nate Dailey @ 2015-02-12 16:46 UTC (permalink / raw)
To: linux-raid; +Cc: linux-scsi
In-Reply-To: <20150205155953.64e9b1e4@notabene.brown>
On 02/04/2015 11:59 PM, NeilBrown wrote:
> On Wed, 28 Jan 2015 10:29:46 -0500 Nate Dailey <nate.dailey@stratus.com>
> wrote:
>
>> I'm writing about something that appears to be an issue with raid1's
>> narrow_write_error, particular to non-512-byte-sector disks. Here's what
>> I'm doing:
>>
>> - 2 disk raid1, 4K disks, each connected to a different SAS HBA
>> - mount a filesystem on the raid1, run a test that writes to it
>> - remove one of the SAS HBAs (echo 1 >
>> /sys/bus/pci/devices/0000\:45\:00.0/remove)
>>
>> At this point, writes fail and narrow_write_error breaks them up and
>> retries, one sector at a time. But these are 512-byte sectors, and sd
>> doesn't like it:
>>
>> [ 2645.310517] sd 3:0:1:0: [sde] Bad block number requested
>> [ 2645.310610] sd 3:0:1:0: [sde] Bad block number requested
>> [ 2645.310690] sd 3:0:1:0: [sde] Bad block number requested
>> ...
>>
>> There appears to be no real harm done, but there can be a huge number of
>> these messages in the log.
>>
>> I can avoid this by disabling bad block tracking, but it looks like
>> maybe the superblock's bblog_shift is intended to address this exact
>> issue. However, I don't see a way to change it. Presumably this is
>> something mdadm should be setting up? I don't see bblog_shift ever set
>> to anything other than 0.
>>
>> This is on a RHEL 7.1 kernel, version 3.10.0-221.el7. I took a look at
>> upstream sd and md changes and nothing jumps out at me that would have
>> affected this (but I have not tested to see if the bad block messages do
>> or do not happen on an upstream kernel).
>>
>> I'd appreciate any advice re: how to handle this. Thanks!
>
> Thanks for the report.
>
> narrow_write_error() should use bdev_logical_block_size() and round up to
> that.
> Possibly mdadm should get the same information and set bblog_shift
> accordingly when creating a bad block log.
>
> I've made a note to fix that, but I'm happy to review patches too :-)
>
> thanks,
> NeilBrown
>
I will post a narrow_write_error patch shortly.
I did some experimentation with setting the bblog_shift in mdadm, but it
didn't work out the way I expected. It turns out that the value is only
loaded from the superblock if:
1453 if ((le32_to_cpu(sb->feature_map) & MD_FEATURE_BAD_BLOCKS) &&
1454 rdev->badblocks.count == 0) {
...
1473 rdev->badblocks.shift = sb->bblog_shift;
And this feature bit is only set if any bad blocks have actually been
recorded.
It also appears to me that the shift is used when loading the bad blocks
from the superblock, but not when storing the bad block list in the
superblock.
Seems like these are bugs, but I'm not certain how the code is supposed
to work (and am getting in a bit over my head with this).
In any case, it doesn't appear to me that there's any harm in having the
bblog_shift not match the disk's block size (right?).
Nate Dailey
^ permalink raw reply
* RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
From: Manibalan P @ 2015-02-12 13:56 UTC (permalink / raw)
To: NeilBrown, linux-raid; +Cc: Pasi Kärkkäinen
In-Reply-To: <20150203093040.569aa5e1@notabene.brown>
Dear All,
Gentle reminder !.. Any update on this issue..
I have tested with all the live lock related patch.. and I hope this is some specific scenario..
Sync thread become busy, and this happens only when RAID array is initializing and heavy IO is happening in parallel
Thanks,
Manibalan.
-----Original Message-----
From: Manibalan P
Sent: Wednesday, February 4, 2015 11:27 AM
To: 'NeilBrown'; linux-raid
Cc: Pasi Kärkkäinen
Subject: RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
>> Dear All,
>> Any updates on this issue.
>Probably the same as:
> http://marc.info/?l=linux-raid&m=142283560704091&w=2
Dear Neil
This patch is not fixing this issue.
This issue happens only if a drive removed from a RAID5 array, which is "initializing" and "heavy IO" is performed on the array.
In such case, as soon as the drive removed, the array state changed to resync=PENDING and md0_raid5 thread using 100% of CPU.
Thanks,
Manibalan.
>which follows on from
> http://marc.info/?t=142221642300001&r=1&w=2
>and
> http://marc.info/?t=142172432500001&r=1&w=2
>NeilBrown
-----Original Message-----
From: NeilBrown [mailto:neilb@suse.de]
Sent: Tuesday, February 3, 2015 4:01 AM
To: Manibalan P
Cc: Pasi Kärkkäinen; linux-raid
Subject: Re: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
On Mon, 2 Feb 2015 07:10:14 +0000 Manibalan P <pmanibalan@amiindia.co.in>
wrote:
> Dear All,
> Any updates on this issue.
Probably the same as:
http://marc.info/?l=linux-raid&m=142283560704091&w=2
which follows on from
http://marc.info/?t=142221642300001&r=1&w=2
and
http://marc.info/?t=142172432500001&r=1&w=2
NeilBrown
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Wednesday, January 14, 2015 3:55 PM
> To: 'Pasi Kärkkäinen'
> Cc: 'neilb@suse.de'; 'linux-raid'
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
> Could you able to find something on this issue.
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Friday, January 2, 2015 12:08 PM
> To: 'Pasi Kärkkäinen'
> Cc: neilb@suse.de; linux-raid
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
>
> I have add the bug in
> https://bugzilla.redhat.com/show_bug.cgi?id=1178080
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Wednesday, December 31, 2014 10:18 PM
> To: Manibalan P
> Cc: neilb@suse.de; linux-raid
> Subject: Re: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> On Tue, Dec 30, 2014 at 11:06:47AM +0000, Manibalan P wrote:
> > Dear Neil,
> >
>
> Hello,
>
> > Few this for you kind attention,
> > 1. I tried the same test with FC11 (2.6.32 kernel before MD code
> > change). And the issue is not there 2. But with Centos 6.4 (2.6.32 kernel after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Also, a bug has been raise with RHEL regarding this issue. Please find the bug link "https://access.redhat.com/support/cases/#/case/01320319"
> >
>
> That support case URL can only be accessed by you and Redhat. Do you happen to have a public bugzilla link?
>
>
> Thanks,
>
> -- Pasi
>
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 24, 2014 12:15 PM
> > To: neilb@suse.de; 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> >
> > Dear Neil,
> >
> > Few this for you kind attention,
> > 1. I tried the same tesst with FC11 (2.6 kernel before MD code change). And the issue is not there 2. But with Centos 6.4 (2.6 after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Thursday, December 18, 2014 11:38 AM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear neil,
> >
> > I also compiled the latest 3.18 kernel on CentOS 6.4 with GIT MD pull patches form 3.19, that also ran in to the same issue after removing a drive during resync.
> >
> > Dec 17 19:07:32 ITX002590129362 kernel: Linux version 3.18.0 (root@mycentos6) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Wed Dec 17 15:59:09 EST 2014 Dec 17 19:07:32 ITX002590129362 kernel: Command line: ro root=/dev/md255 rd_NO_LVM rd_NO_DM rhgb quiet md_mod.start_ro=1 nmi_watchdog=1 md_mod.start_dirty_degraded=1 ??? Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sda6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdb6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdc6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdh6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdi6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdj6> Dec 17 19:10:15 ITX002590129362 kernel: async_tx: api initialized (async) Dec 17 19:10:15 ITX002590129362 kernel: xor: measuring software checksum speed
> > Dec 17 19:10:15 ITX002590129362 kernel: prefetch64-sse: 10048.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: generic_sse: 8824.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: xor: using function: prefetch64-sse (10048.000 MB/sec)
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x1 5921 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x2 6933 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x4 7476 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: using algorithm
> > sse2x4
> > (7476 MB/s) Dec 17 19:10:15 ITX002590129362 kernel: raid6: using
> > ssse3x2 recovery algorithm Dec 17 19:10:15 ITX002590129362 kernel: md:
> > raid6 personality registered for level 6 Dec 17 19:10:15
> > ITX002590129362 kernel: md: raid5 personality registered for level 5
> > Dec 17 19:10:15 ITX002590129362 kernel: md: raid4 personality
> > registered for level 4 Dec 17 19:10:15 ITX002590129362 kernel:
> > md/raid:md0: not clean -- starting background reconstruction Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdj6
> > operational as raid disk 5 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdi6 operational as raid disk 4 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/raid:md0: device sdh6 operational as raid disk 3 Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdc6
> > operational as raid disk 2 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdb6 operational as raid disk 1 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/ra
> > id:md0: device sda6 operational as raid disk 0 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: allocated 0kB Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: raid level 5 active with 6 out of 6 devices, algorithm 2 Dec 17 19:10:15 ITX002590129362 kernel: md0: detected capacity change from 0 to 2361059573760 Dec 17 19:10:15 ITX002590129362 kernel: md0: unknown partition table Dec 17 19:10:35 ITX002590129362 kernel: md: md0 switched to read-write mode.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: resync of RAID array md0 Dec 17 19:10:35 ITX002590129362 kernel: md: minimum _guaranteed_ speed: 10000 KB/sec/disk.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using maximum available idle IO bandwidth (but not more than 30000 KB/sec) for resync.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using 128k window, over a total of 461144448k.
> > ???
> > Started IOs using fio tool.
> >
> > ./fio --name=md0 --filename=/dev/md0 --thread --numjobs=10
> > --direct=1 --group_reporting --unlink=0 --loops=1 --offset=0
> > --randrepeat=1 --norandommap --scramble_buffers=1 --stonewall
> > --ioengine=libaio --rw=randwrite --bs=8704 --iodepth=4000
> > --runtime=3000
> > --blockalign=512
> >
> > ???
> > Removed a drive form the system..
> >
> > Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) ..
> > Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 02 69 03 70 00 00 10 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 40436592 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 51 b3 d0 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 206681040 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 3a f3 40 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 205189952 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK ??? Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb 00 00 00 08 00 Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:25 ITX002590129362 kernel: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb f0 00 00 10 00 Dec 17 19:13:25 ITX002590129362 aghswap: devpath [0:0:7:0] action [remove] devtype [scsi_disk] Dec 17 19:13:25 ITX002590129362 aghswap: MHSA: Sent event 0 0 7 0 remove scsi_disk Dec 17 19:13:25 ITX002590129362 kernel: mpt2sas0: removing handle(0x0011), sas_addr(0x500605ba0101e305) Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Disk failure on sdh6, disabling device.
> > Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Operation continuing on 5 devices.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: md0: resync interrupted.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: checkpointing resync of md0.
> > ..
> > Log messages after enabling debufgs on raid5.c, it is getting repeated continuously.
> >
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0 handling stripe 273480328, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x18 read (null) write ffff8808029b6b00 written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=273480328 for sector 273480328, rmw=2 rcw=1 handling stripe 65238568, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081a956b00 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=65238568 for sector 65238568, rmw=2 rcw=1 handling stripe 713868672, state=0x2041 cnt=1, pd_idx=4, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x10 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081f020100 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=713868672 for sector 713868672, rmw=2 rcw=1 handling stripe 729622496, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622496 for sector 729622496, rmw=2 rcw=1 handling stripe 729622504, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622504 for sector 729622504, rmw=2 rcw=1 handling stripe 245773680, state=0x2041 cnt=1, pd_idx=0, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081cab7a00 written (null)
> > check 0: state 0x10 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=245773680 for sector 245773680, rmw=2 rcw=1 handling stripe 867965560, state=0x2041 cnt=1, pd_idx=1, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff880802b2bf00 written (null)
> > check 1: state 0x10 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=867965560 for sector 867965560, rmw=2 rcw=1 handling stripe 550162280, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x18 read (null) write ffff880802b08800 written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1
> > force RCW max_degraded=1, recovery_cp=7036944 sh->sector=550162280
> > for sector 550162280, rmw=2 rcw=1
> >
> >
> > Thanks,
> > Manibalan
> >
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:11 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > The same Issue is reproducible in the latest upstream kernel also.
> >
> > Tested in "3.17.6" latest stable upstream kernel and find the same issue.
> >
> > [root@root ~]# modinfo raid456
> > filename: /lib/modules/3.17.6/kernel/drivers/md/raid456.ko
> > alias: raid6
> > alias: raid5
> > alias: md-level-6
> > alias: md-raid6
> > alias: md-personality-8
> > alias: md-level-4
> > alias: md-level-5
> > alias: md-raid4
> > alias: md-raid5
> > alias: md-personality-4
> > description: RAID4/5/6 (striping with parity) personality for MD
> > license: GPL
> > srcversion: 0EEF680023FDC7410F7989A
> > depends: async_raid6_recov,async_pq,async_tx,async_memcpy,async_xor
> > intree: Y
> > vermagic: 3.17.6 SMP mod_unload modversions
> > parm: devices_handle_discard_safely:Set to Y if all devices in each array reliably return zeroes on reads from discarded regions (bool)
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:01 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > We are facing IO struck issue with raid5 in the following scenario.
> > (please see the attachment for the complete information) In RAID5
> > array, if a drive is removed while initialization and the same time
> > if IO is happening to that md. Then IO is getting struck, and
> > md_raid5 thread is using 100 % of CPU. Also the md state showing as
> > resync=PENDING
> >
> > Kernel : Issue found in the following kernels RHEL 6.5
> > (2.6.32-431.el6.x86_64) CentOS 7 (kernel-3.10.0-123.13.1.el7.x86_64)
> >
> > Steps to Reproduce the issue:
> >
> > 1. Created a raid 5 md with 4 drives using the below mdadm command.
> > mdadm -C /dev/md0 -c 64 -l 5 -f -n 4 -e 1.2 /dev/sdb6 /dev/sdc6
> > /dev/sdd6 /dev/sde6
> >
> > 2. Make the md writable
> > mdadm ???readwrite /dev/md0
> >
> > 3. Now md will start initialization
> >
> > 4. Run FIO Tool, the the below said configuration /usr/bin/fio
> > --name=md0 --filename=/dev/md0 --thread --numjobs=10 --direct=1
> > --group_reporting --unlink=0 --loops=1 --offset=0 --randrepeat=1
> > --norandommap --scramble_buffers=1 --stonewall --ioengine=libaio
> > --rw=randwrite --bs=8704 --iodepth=4000 --runtime=3000
> > --blockalign=512
> >
> > 4. During MD initialzing, remove a drive(either using MDADM set
> > faulty/remove or remove manually)
> >
> > 5. Now the IO will struck, and cat /proc/mdstat shows states with
> > resync=PENDING
> > --------------------------------------------------------------------
> > --
> > ----------------------- top - output show, md_raid5 using 100% cpu
> >
> > top - 17:55:06 up 1:09, 3 users, load average: 11.98, 8.53, 3.99
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 2690 root 20 0 0 0 0 R 100.0 0.0 6:44.41 md0_raid5
> > --------------------------------------------------------------------
> > --
> > -----------------------
> > dmesg - show the stack trace
> >
> > INFO: task fio:2715 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000a 0 2715 2654 0x00000080
> > ffff88043b623598 0000000000000082 0000000000000000 ffffffff81058d53
> > ffff88043b623548 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b40b098 ffff88043b623fd8 000000000000fbc8 ffff88043b40b098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff8140fa39>] ?
> > md_wakeup_thread+0x39/0x70 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffffa0308f66>] ?
> > make_request+0x306/0xc6c [raid456] [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81122283>] ?
> > mempool_alloc+0x63/0x140 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c767a>] do_direct_IO+0x7ca/0xfa0 [<ffffffff811c8196>]
> > __blockdev_direct_IO_newtrunc+0x346/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2717 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000004 0 2717 2654 0x00000080
> > ffff880439e97698 0000000000000082 ffff880439e97628 ffffffff81058d53
> > ffff880439e97648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0adab8 ffff880439e97fd8 000000000000fbc8 ffff88043b0adab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2718 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000005 0 2718 2654 0x00000080
> > ffff88043bc13698 0000000000000082 ffff88043bc13628 ffffffff81058d53
> > ffff88043bc13648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ad058 ffff88043bc13fd8 000000000000fbc8 ffff88043b0ad058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2719 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000001 0 2719 2654 0x00000080
> > ffff880439ebb698 0000000000000082 ffff880439ebb628 ffffffff81058d53
> > ffff880439ebb648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ac5f8 ffff880439ebbfd8 000000000000fbc8 ffff88043b0ac5f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2720 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000008 0 2720 2654 0x00000080
> > ffff88043b8cf698 0000000000000082 ffff88043b8cf628 ffffffff81058d53
> > ffff88043b8cf648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89af8 ffff88043b8cffd8 000000000000fbc8 ffff880439e89af8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2721 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2721 2654 0x00000080
> > ffff88043b047698 0000000000000082 ffff88043b047628 ffffffff81058d53
> > ffff88043b047648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89098 ffff88043b047fd8 000000000000fbc8 ffff880439e89098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2722 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2722 2654 0x00000080
> > ffff880439ea3698 0000000000000082 ffff880439ea3628 ffffffff81058d53
> > ffff880439ea3648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e88638 ffff880439ea3fd8 000000000000fbc8 ffff880439e88638 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2723 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000006 0 2723 2654 0x00000080
> > ffff88043bf5f698 0000000000000082 ffff88043bf5f628 ffffffff81058d53
> > ffff88043bf5f648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183ab8 ffff88043bf5ffd8 000000000000fbc8 ffff88043a183ab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2724 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000b 0 2724 2654 0x00000080
> > ffff88043be05698 0000000000000082 ffff88043be05628 ffffffff81058d53
> > ffff88043be05648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183058 ffff88043be05fd8 000000000000fbc8 ffff88043a183058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2725 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000003 0 2725 2654 0x00000080
> > ffff88043be07698 0000000000000082 ffff88043be07628 ffffffff81058d53
> > ffff88043be07648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a1825f8 ffff88043be07fd8 000000000000fbc8 ffff88043a1825f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> >
> > [root@root ~]# cat /proc/2690/stack
> > [<ffffffff810686da>] __cond_resched+0x2a/0x40 [<ffffffffa030361c>]
> > ops_run_io+0x2c/0x920 [raid456] [<ffffffffa03052cc>]
> > handle_stripe+0x9cc/0x2980 [raid456] [<ffffffffa03078a4>]
> > raid5d+0x624/0x850 [raid456] [<ffffffff81416f05>]
> > md_thread+0x115/0x150 [<ffffffff8109aef6>] kthread+0x96/0xa0
> > [<ffffffff8100c20a>] child_rip+0xa/0x20 [<ffffffffffffffff>]
> > 0xffffffffffffffff
> >
> > [root@root ~]# cat /proc/2690/stat
> > 2690 (md0_raid5) R 2 0 0 0 -1 2149613632 0 0 0 0 0 68495 0 0 20 0 1
> > 0
> > 350990 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483391 256 0 0 0
> > 17
> > 2 0 0 6855 0 0 [root@root ~]# cat /proc/2690/statm
> > 0 0 0 0 0 0 0
> > [root@root ~]# cat /proc/2690/stat
> > stat statm status
> > [root@root ~]# cat /proc/2690/status
> > Name: md0_raid5
> > State: R (running)
> > Tgid: 2690
> > Pid: 2690
> > PPid: 2
> > TracerPid: 0
> > Uid: 0 0 0 0
> > Gid: 0 0 0 0
> > Utrace: 0
> > FDSize: 64
> > Groups:
> > Threads: 1
> > SigQ: 2/128402
> > SigPnd: 0000000000000000
> > ShdPnd: 0000000000000000
> > SigBlk: 0000000000000000
> > SigIgn: fffffffffffffeff
> > SigCgt: 0000000000000100
> > CapInh: 0000000000000000
> > CapPrm: ffffffffffffffff
> > CapEff: fffffffffffffeff
> > CapBnd: ffffffffffffffff
> > Cpus_allowed: ffffff
> > Cpus_allowed_list: 0-23
> > Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
> > Mems_allowed_list: 0-1
> > voluntary_ctxt_switches: 5411612
> > nonvoluntary_ctxt_switches: 257032
> >
> >
> > Thanks,
> > Manibalan.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: RAID 1 metadata - keep separate from mirror disks ?
From: Suresh Babu Kandukuru @ 2015-02-12 11:42 UTC (permalink / raw)
To: Phil Turmel, linux-raid
In-Reply-To: <54DB6BF0.3040309@turmel.org>
Thanks Phil . This helps.
/Suresh
-----Original Message-----
From: Phil Turmel [mailto:philip@turmel.org]
Sent: Wednesday, February 11, 2015 8:19 PM
To: Suresh Babu Kandukuru; linux-raid@vger.kernel.org
Subject: Re: RAID 1 metadata - keep separate from mirror disks ?
Good morning Suresh,
On 02/11/2015 07:14 AM, Suresh Babu Kandukuru wrote:
> Hi There,
>
> On the RAID 1 metadata: is there any way to keep the metadata
> separate from the mirror disks? Could you guide us on this ?,
> please. In general, we need to keep all metadata off the device
> itself, leaving all the device available for user data. This is
> particularly important in the migration case, where we want to take an
> existing LUN and add a second leg to it to create the mirror device
> without changing any of the data or metadata on the LUN.
If you look at "man 4 md" you'll see some options. If a legacy array type meets your needs, you can operate without metadata at all. Use "mdadm --build" to assemble your raid at each boot.
Or, if your storage server can insert a leg ahead of you current LUN, you can then create the array with an explicit data offset matching the size of the inserted leg. Create it degraded with the existing LUN, then add (a) LUN(s) to start mirroring. This process will leave you the option to resize with more legs later.
Or you can add a leg to the end and create your array with version 1.0 metadata, which is placed at the end of the device.
Finally, you could write your own metadata container service for use with mdmon. (That's a bit beyond my ability, sorry.)
Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: please help - raid 1 degraded
From: Roaming @ 2015-02-12 10:11 UTC (permalink / raw)
To: sunruh; +Cc: linux-raid
In-Reply-To: <20150212000940.GA49579@eris.prismnet.com>
On 12/02/2015 00:09, sunruh@prismnet.com wrote:
> i dont seem to be seeing the partition sizes or im stupid.
> couldnt i just dd if=/dev/sdb of=/dev/sdc bs=1G count=240 and then do the
> mdadm?
NOT a good idea. I don't know what it would do in your case, where you
are using the entire disk, but if you're using a partition table you
would suddenly end up with a bunch of duplicate GUIDs. Bearing in mind
the "GU" stands for "globally unique", your management tools are likely
to get confused ... not a good idea especially wrt raid.
Cheers,
Wol
^ permalink raw reply
* [GIT PULL REQUEST] md updates for 3.20
From: NeilBrown @ 2015-02-12 3:20 UTC (permalink / raw)
To: Linus Torvalds
Cc: lkml, linux RAID, Xiao Ni, Hannes Reinecke, Jes Sorensen,
Jan Beulich, Dan Carpenter
[-- Attachment #1: Type: text/plain, Size: 3577 bytes --]
Hi Linus,
20 is a big number ... is it time for 4.0 yet :-)
anyway, herewith is my pull request for 3.20. It has been sitting in
-next for a while, and Dan's scripts only found 2 bugs - now fixed.
Thanks,
NeilBrown
The following changes since commit d95901433436aeb921eac58bfd8a2aa77f110384:
md/bitmap: fix a might_sleep() warning. (2015-02-02 17:08:03 +1100)
are available in the git repository at:
git://neil.brown.name/md tags/md/3.20
for you to fetch changes up to 53a6ab4d3f6d6dc87ec8f14998b4b5536ee2968c:
md/raid10: fix conversion from RAID0 to RAID10 (2015-02-12 14:09:57 +1100)
----------------------------------------------------------------
md updates for 3.20
- assorted locking changes so that access to /proc/mdstat
and much of /sys/block/mdXX/md/* is protected by a spinlock
rather than a mutex and will never block indefinitely.
- Make an 'if' condition in RAID5 - which has been implicated
in recent bugs - more readable.
- misc minor fixes
----------------------------------------------------------------
Hannes Reinecke (1):
md: wakeup thread upon rdev_dec_pending()
Jan Beulich (1):
x86/raid6: correctly check for assembler capabilities
Jes Sorensen (1):
md: do_release_stripe(): No need to call md_wakeup_thread() twice
NeilBrown (25):
md/raid5: separate large if clause out of fetch_block().
md/raid5: separate out the easy conditions in need_this_block.
md/raid5: need_this_block: start simplifying the last two conditions.
md/raid5: need_this_block: tidy/fix last condition.
md: rename mddev->write_lock to mddev->lock
md: make ->congested robust against personality changes.
md: make merge_bvec_fn more robust in face of personality changes.
md/linear: remove rcu protections in favour of suspend/resume
md: split detach operation out from ->stop.
md: rename ->stop to ->free
md: level_store: group all important changes into one place.
md: protect ->pers changes with mddev->lock
md/bitmap: protect clearing of ->bitmap by mddev->lock
md: remove need for mddev_lock() in md_seq_show()
md/raid5: use ->lock to protect accessing raid5 sysfs attributes.
md: remove mddev_lock() from md_attr_show()
md: remove mddev_lock from rdev_attr_show()
md: remove unnecessary 'buf' from get_bitmap_file.
md: tidy up set_bitmap_file
md: move GET_BITMAP_FILE ioctl out from mddev_lock.
md: minor cleanup in safe_delay_store.
md: use mddev->lock to protect updates to resync_{min,max}.
md: move mddev_lock and related to md.h
md: make reconfig_mutex optional for writes to md sysfs files.
md/raid10: fix conversion from RAID0 to RAID10
arch/x86/Makefile | 1 +
drivers/md/bitmap.c | 15 +-
drivers/md/dm-raid.c | 8 +-
drivers/md/faulty.c | 8 +-
drivers/md/linear.c | 67 ++--
drivers/md/md.c | 816 ++++++++++++++++++++++++++++++------------------
drivers/md/md.h | 57 +++-
drivers/md/multipath.c | 22 +-
drivers/md/raid0.c | 29 +-
drivers/md/raid1.c | 52 +--
drivers/md/raid1.h | 3 -
drivers/md/raid10.c | 49 +--
drivers/md/raid10.h | 3 -
drivers/md/raid5.c | 334 ++++++++++++--------
drivers/md/raid5.h | 1 -
lib/raid6/algos.c | 2 +-
lib/raid6/recov_avx2.c | 2 +-
lib/raid6/recov_ssse3.c | 6 +
18 files changed, 867 insertions(+), 608 deletions(-)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: please help - raid 1 degraded
From: Eyal Lebedinsky @ 2015-02-12 3:12 UTC (permalink / raw)
Cc: linux-raid
In-Reply-To: <54DBFD91.7080507@websitemanagers.com.au>
On 12/02/15 12:10, Adam Goryachev wrote:
> On 12/02/15 12:02, sunruh@prismnet.com wrote:
>> ok, so now the really important questions: once done, what files/stats do i need to save off for the next time it craters?
>
> I think the usual information requested is the following:
> fdisk -lu /dev/sd?
> mdadm --manage --query /dev/sd?
> mdadm --manage --detail /dev/md*
> mdadm --manage --examine /dev/sd?
Maybe
mdadm --misc --query /dev/md*
mdadm --misc --detail /dev/md*
mdadm --misc --examine /dev/sd*
> cat /proc/mdstat
> ls -l /dev/disk/by-id/
>
> If you can keep a copy of all those things, then you will be much further ahead than many people. Of course, RAID1 is just so much easier/simpler than RAID5/RAID6, so usually you won't need any of that. RAID1 is simple mirror, so if you have two disks, one with data, one without, then you just need to decide which disk has the data, and start with that.
> It is even possible to start two MD arrays, one from each disk, and then compare the contents to decide which one you want to keep.
> Or, you can simply mount the device directly (skipping any MD data at the beginning if needed).
>
> Like I said, RAID1 is by far the simplest type of RAID if you want redundancy and can fit your dataset onto a single device.
>
> Glad you had a successful recovery :)
>
> Regards,
> Adam
>
--
Eyal Lebedinsky (eyal@eyal.emu.id.au)
^ permalink raw reply
* Re: [PATCH RESEND] Change way of printing name of a process
From: NeilBrown @ 2015-02-12 2:54 UTC (permalink / raw)
To: Pawel Baldysiak; +Cc: linux-raid, artur.paszkiewicz
In-Reply-To: <20150211212401.14402.5228.stgit@gklab-154-222.intel.com>
[-- Attachment #1: Type: text/plain, Size: 10493 bytes --]
On Wed, 11 Feb 2015 22:25:03 +0100 Pawel Baldysiak
<pawel.baldysiak@intel.com> wrote:
> Sometimes mdadm prints messages with wrong name "mdmon",
> and vice versa.
> This patch solves this problem by changing method of determining
> process name.
> Now "Name" will be set in const at start of a program,
> previously was hardcoded as #define.
>
> Signed-off-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
> Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
> ---
> Grow.c | 18 +++++++++---------
> Incremental.c | 4 ++--
> Monitor.c | 4 ++--
> ReadMe.c | 2 +-
> mdadm.c | 7 ++++---
> mdadm.h | 6 +++---
> mdmon.c | 2 ++
> mdmon.h | 3 +--
> super-intel.c | 2 +-
> sysfs.c | 8 ++++----
> xmalloc.c | 12 ++++++++----
> 11 files changed, 37 insertions(+), 31 deletions(-)
>
> diff --git a/Grow.c b/Grow.c
> index 6ff225a..4a01a31 100644
> --- a/Grow.c
> +++ b/Grow.c
> @@ -1919,7 +1919,7 @@ size_change_error:
> int err;
> err = remove_disks_for_takeover(st, sra, array.layout);
> if (err) {
> - dprintf(Name": Array cannot be reshaped\n");
> + dprintf("%s: Array cannot be reshaped\n", Name);
> if (cfd > -1)
> close(cfd);
> rv = 1;
> @@ -2133,7 +2133,7 @@ static int verify_reshape_position(struct mdinfo *info, int level)
> char *ep;
> unsigned long long position = strtoull(buf, &ep, 0);
>
> - dprintf(Name": Read sync_max sysfs entry is: %s\n", buf);
> + dprintf("%s: Read sync_max sysfs entry is: %s\n", Name, buf);
> if (!(ep == buf || (*ep != 0 && *ep != '\n' && *ep != ' '))) {
> position *= get_data_disks(level,
> info->new_layout,
> @@ -3494,8 +3494,8 @@ int reshape_container(char *container, char *devname,
> return 1;
> default: /* parent */
> if (!freeze_reshape)
> - printf(Name ": multi-array reshape continues"
> - " in background\n");
> + printf("%s: multi-array reshape continues"
> + " in background\n", Name);
> return 0;
> case 0: /* child */
> map_fork();
> @@ -3557,8 +3557,8 @@ int reshape_container(char *container, char *devname,
>
> fd = open_dev(mdstat->devnm);
> if (fd < 0) {
> - printf(Name ": Device %s cannot be opened for reshape.",
> - adev);
> + printf("%s: Device %s cannot be opened for reshape.",
> + Name, adev);
> break;
> }
>
> @@ -3573,8 +3573,8 @@ int reshape_container(char *container, char *devname,
> * This is possibly interim until the behaviour of
> * reshape_array is resolved().
> */
> - printf(Name ": Multiple reshape execution detected for "
> - "device %s.", adev);
> + printf("%s: Multiple reshape execution detected for "
> + "device %s.", Name, adev);
> close(fd);
> break;
> }
> @@ -4611,7 +4611,7 @@ int Grow_restart(struct supertype *st, struct mdinfo *info, int *fdlist, int cnt
> st->ss->free_super(st);
> offsets[j] = dinfo.data_offset * 512;
> }
> - printf(Name ": restoring critical section\n");
> + printf("%s: restoring critical section\n", Name);
>
> if (restore_stripes(fdlist, offsets,
> info->array.raid_disks,
> diff --git a/Incremental.c b/Incremental.c
> index 13b68bc..38c9989 100644
> --- a/Incremental.c
> +++ b/Incremental.c
> @@ -1710,9 +1710,9 @@ int IncrementalRemove(char *devname, char *id_path, int verbose)
> char buf[32];
>
> if (!id_path)
> - dprintf(Name ": incremental removal without --path <id_path> "
> + dprintf("%s: incremental removal without --path <id_path> "
> "lacks the possibility to re-add new device in this "
> - "port\n");
> + "port\n", Name);
>
> if (strchr(devname, '/')) {
> pr_err("incremental removal requires a "
> diff --git a/Monitor.c b/Monitor.c
> index 971d2ec..7a5203c 100644
> --- a/Monitor.c
> +++ b/Monitor.c
> @@ -381,14 +381,14 @@ static void alert(char *event, char *dev, char *disc, struct alert_info *info)
> if (info->mailfrom)
> fprintf(mp, "From: %s\n", info->mailfrom);
> else
> - fprintf(mp, "From: " Name " monitoring <root>\n");
> + fprintf(mp, "From: %s monitoring <root>\n", Name);
> fprintf(mp, "To: %s\n", info->mailaddr);
> fprintf(mp, "Subject: %s event on %s:%s\n\n",
> event, dev, hname);
>
> fprintf(mp,
> "This is an automatically generated"
> - " mail message from " Name "\n");
> + " mail message from %s\n", Name);
> fprintf(mp, "running on %s\n\n", hname);
>
> fprintf(mp,
> diff --git a/ReadMe.c b/ReadMe.c
> index 445c388..87a4916 100644
> --- a/ReadMe.c
> +++ b/ReadMe.c
> @@ -30,7 +30,7 @@
> #ifndef VERS_DATE
> #define VERS_DATE "21st August 2014"
> #endif
> -char Version[] = Name " - v" VERSION " - " VERS_DATE "\n";
> +char Version[] = "mdadm - v" VERSION " - " VERS_DATE "\n";
>
> /*
> * File: ReadMe.c
> diff --git a/mdadm.c b/mdadm.c
> index c856fcd..475c71c 100644
> --- a/mdadm.c
> +++ b/mdadm.c
> @@ -38,6 +38,7 @@ static int misc_list(struct mddev_dev *devlist,
> struct mddev_ident *ident,
> char *dump_directory,
> struct supertype *ss, struct context *c);
> +const char Name[] = "mdadm";
>
> int main(int argc, char *argv[])
> {
> @@ -771,12 +772,12 @@ int main(int argc, char *argv[])
> if (strcmp(c.update,"?") == 0 ||
> strcmp(c.update, "help") == 0) {
> outf = stdout;
> - fprintf(outf, Name ": ");
> + fprintf(outf, "%s: ", Name);
> } else {
> outf = stderr;
> fprintf(outf,
> - Name ": '--update=%s' is invalid. ",
> - c.update);
> + "%s: '--update=%s' is invalid. ",
> + Name, c.update);
> }
> fprintf(outf, "Valid --update options are:\n"
> " 'sparc2.2', 'super-minor', 'uuid', 'name', 'resync',\n"
> diff --git a/mdadm.h b/mdadm.h
> index fc1fd31..c8d079c 100644
> --- a/mdadm.h
> +++ b/mdadm.h
> @@ -180,6 +180,8 @@ extern __off64_t lseek64 __P ((int __fd, __off64_t __offset, int __whence));
>
> #define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
>
> +extern const char Name[];
> +
> /* general information that might be extracted from a superblock */
> struct mdinfo {
> mdu_array_info_t array;
> @@ -262,8 +264,6 @@ struct createinfo {
> struct supertype *supertype;
> };
>
> -#define Name "mdadm"
> -
> enum mode {
> ASSEMBLE=1,
> BUILD,
> @@ -1457,7 +1457,7 @@ static inline int xasprintf(char **strp, const char *fmt, ...) {
> return ret;
> }
>
> -#define pr_err(fmt ...) fprintf(stderr, Name ": " fmt)
> +#define pr_err(fmt, args...) fprintf(stderr, "%s: "fmt, Name, ##args)
> #define cont_err(fmt ...) fprintf(stderr, " " fmt)
>
> void *xmalloc(size_t len);
> diff --git a/mdmon.c b/mdmon.c
> index 27045a1..ee12b7c 100644
> --- a/mdmon.c
> +++ b/mdmon.c
> @@ -67,6 +67,8 @@
> #include "mdadm.h"
> #include "mdmon.h"
>
> +char const Name[] = "mdmon";
> +
> struct active_array *discard_this;
> struct active_array *pending_discard;
>
> diff --git a/mdmon.h b/mdmon.h
> index 5a8e120..aa750c6 100644
> --- a/mdmon.h
> +++ b/mdmon.h
> @@ -18,8 +18,7 @@
> * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
> */
>
> -#undef pr_err
> -#define pr_err(fmt ...) fprintf(stderr, "mdmon: " fmt)
> +extern const char Name[];
>
> enum array_state { clear, inactive, suspended, readonly, read_auto,
> clean, active, write_pending, active_idle, bad_word};
> diff --git a/super-intel.c b/super-intel.c
> index 4b23b9a..d900339 100644
> --- a/super-intel.c
> +++ b/super-intel.c
> @@ -1386,7 +1386,7 @@ static int imsm_check_attributes(__u32 attributes)
> }
>
> if (not_supported)
> - dprintf(Name "(IMSM): Unknown attributes : %x\n", not_supported);
> + dprintf("%s (IMSM): Unknown attributes : %x\n", Name, not_supported);
>
> ret_val = 0;
> }
> diff --git a/sysfs.c b/sysfs.c
> index 9a1d856..ceab27f 100644
> --- a/sysfs.c
> +++ b/sysfs.c
> @@ -413,8 +413,8 @@ int sysfs_set_str(struct mdinfo *sra, struct mdinfo *dev,
> n = write(fd, val, strlen(val));
> close(fd);
> if (n != strlen(val)) {
> - dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
> - val, fname, strerror(errno));
> + dprintf("%s: failed to write '%s' to '%s' (%s)\n",
> + Name, val, fname, strerror(errno));
> return -1;
> }
> return 0;
> @@ -450,8 +450,8 @@ int sysfs_uevent(struct mdinfo *sra, char *event)
> n = write(fd, event, strlen(event));
> close(fd);
> if (n != (int)strlen(event)) {
> - dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
> - event, fname, strerror(errno));
> + dprintf("%s: failed to write '%s' to '%s' (%s)\n",
> + Name, event, fname, strerror(errno));
> return -1;
> }
> return 0;
> diff --git a/xmalloc.c b/xmalloc.c
> index 8d42a7c..75ae4e2 100644
> --- a/xmalloc.c
> +++ b/xmalloc.c
> @@ -37,7 +37,8 @@ void *xmalloc(size_t len)
> char *msg;
> if (rv)
> return rv;
> - msg = Name ": memory allocation failure - aborting\n";
> + msg = ": memory allocation failure - aborting\n";
> + write(2, Name, strlen(Name));
> exit(4+!!write(2, msg, strlen(msg)));
> }
>
> @@ -47,7 +48,8 @@ void *xrealloc(void *ptr, size_t len)
> char *msg;
> if (rv)
> return rv;
> - msg = Name ": memory allocation failure - aborting\n";
> + msg = ": memory allocation failure - aborting\n";
> + write(2, Name, strlen(Name));
> exit(4+!!write(2, msg, strlen(msg)));
> }
>
> @@ -57,7 +59,8 @@ void *xcalloc(size_t num, size_t size)
> char *msg;
> if (rv)
> return rv;
> - msg = Name ": memory allocation failure - aborting\n";
> + msg = ": memory allocation failure - aborting\n";
> + write(2, Name, strlen(Name));
> exit(4+!!write(2, msg, strlen(msg)));
> }
>
> @@ -67,6 +70,7 @@ char *xstrdup(const char *str)
> char *msg;
> if (rv)
> return rv;
> - msg = Name ": memory allocation failure - aborting\n";
> + msg = ": memory allocation failure - aborting\n";
> + write(2, Name, strlen(Name));
> exit(4+!!write(2, msg, strlen(msg)));
> }
Thanks.
I applied this patch, then made a number of other changes so some of it isn't
needed.
In particular, dprintf now always includes the Name and __func__, and pr_err
now always includes the Name.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: All drive in Raid 5 are in 'spare' mode
From: Phil Turmel @ 2015-02-12 1:41 UTC (permalink / raw)
To: Dush, linux-raid@vger.kernel.org
In-Reply-To: <CAL7hTOctZ4tAoPSbO3iFEohRyXuvcY4=6yQj8U-FpoAhV6sT0w@mail.gmail.com>
Hi Dush,
On 02/11/2015 02:56 PM, Dush wrote:
> Hi,
>
> I have a RAID 5 composed by 4x 500Go hdd but for some days, it's 'inactive'.
>
> I'm not raid expert and I prefer asking before doing an unrecoverable mistake...
>
> Is it possible to fix this raid (md126)?
> Is it possible to recover data on it?
Probably. Very good report, btw.
> Do I have a disk to change or it's "just" a desynchronization between disks?
One disk is now truly a spare (/dev/sdd3), which suggests you already
tried to '--add' it and didn't get anywhere.
Step one: collect some forensics for later. syslog or dmesg containing
your failure events. Can be trimmed to just device and md stuff.
"smartctl -x /dev/sdX" for each drive involved in the arrays.
Then, we'll try the simple stuff.
Make sure the array is stopped with:
mdadm --stop /dev/md126
Then, force assemble it without sdd:
mdadm --assemble --force --verbose --run /dev/md126 /dev/sd[bce]3
If that works, mount it and catch a backup of critical files.
Then add your /dev/sdd3 back to the array and let it rebuild:
mdadm --add /dev/md126 /dev/sdd3
It may not make it through the rebuild if you have the common timeout
mismatch problem.[1] Show the dmesg and smartctl data (pasted inline is
preferred) and we'll see.
Phil
Recent typical case:
[1] http://marc.info/?l=linux-raid&m=142353387024935&w=1
^ permalink raw reply
* Re: please help - raid 1 degraded
From: Adam Goryachev @ 2015-02-12 1:10 UTC (permalink / raw)
To: sunruh; +Cc: linux-raid
In-Reply-To: <20150212010200.GA51009@eris.prismnet.com>
On 12/02/15 12:02, sunruh@prismnet.com wrote:
> ok, so now the really important questions: once done, what files/stats
> do i need to save off for the next time it craters?
I think the usual information requested is the following:
fdisk -lu /dev/sd?
mdadm --manage --query /dev/sd?
mdadm --manage --detail /dev/md*
mdadm --manage --examine /dev/sd?
cat /proc/mdstat
ls -l /dev/disk/by-id/
If you can keep a copy of all those things, then you will be much
further ahead than many people. Of course, RAID1 is just so much
easier/simpler than RAID5/RAID6, so usually you won't need any of that.
RAID1 is simple mirror, so if you have two disks, one with data, one
without, then you just need to decide which disk has the data, and start
with that.
It is even possible to start two MD arrays, one from each disk, and then
compare the contents to decide which one you want to keep.
Or, you can simply mount the device directly (skipping any MD data at
the beginning if needed).
Like I said, RAID1 is by far the simplest type of RAID if you want
redundancy and can fit your dataset onto a single device.
Glad you had a successful recovery :)
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: please help - raid 1 degraded
From: sunruh @ 2015-02-12 1:02 UTC (permalink / raw)
To: Adam Goryachev; +Cc: sunruh, linux-raid
In-Reply-To: <54DBF587.6050004@websitemanagers.com.au>
On Thu, Feb 12, 2015 at 11:36:23AM +1100, Adam Goryachev wrote:
> On 12/02/15 11:09, sunruh@prismnet.com wrote:
> > On Thu, Feb 12, 2015 at 09:12:50AM +1100, Adam Goryachev wrote:
> >> On 12/02/15 05:04, sunruh@prismnet.com wrote:
> >>> centos 6.6
> >>> 2x 240gig ssd in raid1
> >>> this is a live running production machine and the raid1 is for /u of
> >>> users home dirs.
> >>>
> >>> 1 ssd went totally offline and i replaced it after noticing the firmware
> >>> levels are not the same. the new ssd has the same level firmware.
> >>>
> >>> /dev/sdb is the good ssd
> >>> /dev/sdc is the new blank ssd
> >>>
> >>> when working it was /u1 from /dev/md127p1 and /u2 from /dev/md127p2
> >>> p1 is 80gig and p2 is 160gig for the full 240gig size of the ssd
> >>>
> >>>> ls -al /dev/md*
> >>> brw-rw---- 1 root disk 9, 127 Feb 11 11:09 /dev/md127
> >>> brw-rw---- 1 root disk 259, 0 Feb 10 20:23 /dev/md127p1
> >>> brw-rw---- 1 root disk 259, 1 Feb 10 20:23 /dev/md127p2
> >>>
> >>> /dev/md:
> >>> total 8
> >>> drwxr-xr-x 2 root root 140 Feb 10 20:24 .
> >>> drwxr-xr-x 20 root root 3980 Feb 10 20:24 ..
> >>> lrwxrwxrwx 1 root root 8 Feb 11 11:09 240ssd_0 -> ../md127
> >>> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p1 -> ../md127p1
> >>> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p2 -> ../md127p2
> >>> -rw-r--r-- 1 root root 5 Feb 10 20:24 autorebuild.pid
> >>> -rw------- 1 root root 63 Feb 10 20:23 md-device-map
> >>>
> >>>> ps -eaf | grep mdadm
> >>> root 2188 1 0 Feb10 ? 00:00:00 mdadm --monitor --scan -f --pid-file=/var/run/mdadm/mdadm.pid
> >>>
> >>> how do i rebuild /dev/sdc into the mirror of /dev/sdb?
> >>>
> >> Please send the output of fdisk -lu /dev/sd[bc] and cat /proc/mdstat
> >> (preferably both when it was working and current).
> >>
> >> In general, when replacing a failed RAID1 disk, and assuming you
> >> configured it the way I think you did:
> >> 1) fdisk -lu /dev/sdb
> >> Find out the exact partition sizes
> >> 2) fdisk /dev/sdc
> >> Create the new partitions exactly the same as /dev/sdb
> >> 3) mdadm --manage /dev/md127 --add /dev/sdb1
> >> Add the partition to the array
> >> 4) cat /proc/mdstat
> >> Watch the rebuild progress, once it is complete, relax.
> >>
> >> PS, steps 1 and 2 may not be needed if you are using the full block
> >> device instead of a partition. Also, change the command in step 3 to
> >> "mdadm --manage /dev/md127 --add /dev/sdb"
> >>
> >> PPS, if this is a bootable disk, you will probably also need to do
> >> something with your boot manager to get that installed onto the new disk
> >> as well.
> >>
> >> Hope this helps, otherwise, please provide more information.
> >>
> >>
> >> Regards,
> >> Adam
> >>
> >> --
> >> Adam Goryachev Website Managers www.websitemanagers.com.au
> > Adam (and anybody else that can help),
> > after issue i do not have before. and no they are not bootable.
> >
> > [root@shell ~]# fdisk -lu /dev/sd[bc]
> >
> > Disk /dev/sdb: 240.1 GB, 240057409536 bytes
> > 255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk identifier: 0x0001a740
> >
> >
> > Disk /dev/sdc: 240.1 GB, 240057409536 bytes
> > 255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk identifier: 0x00000000
> >
> > [root@shell ~]# cat /proc/mdstat
> > Personalities : [raid1]
> > md127 : active raid1 sdb[2]
> > 234299840 blocks super 1.2 [2/1] [U_]
> >
> > unused devices: <none>
>
> > i dont seem to be seeing the partition sizes or im stupid.
> > couldnt i just dd if=/dev/sdb of=/dev/sdc bs=1G count=240 and then do the
> > mdadm?
> OK, so you aren't using partitioned disks, so it is as simple as what I
> said above (with one minor correction):
>
> "mdadm --manage /dev/md127 --add /dev/sdc"
>
>
> /dev/sdc is the new blank ssd, so that is the one to add, the above
> command with /dev/sdb wouldn't have done anything at all .... So just
> run that command, and then do "watch cat /proc/mdstat" until the good
> stuff is completed.
>
> Regards,
> Adam
>
> --
> Adam Goryachev Website Managers www.websitemanagers.com.au
awesome sauce!
it is recovering and at a fast pace too. says it will be done in 16mins.
ok, so now the really important questions:
once done, what files/stats do i need to save off for the next time it
craters?
^ permalink raw reply
* Re: please help - raid 1 degraded
From: Adam Goryachev @ 2015-02-12 0:36 UTC (permalink / raw)
To: sunruh; +Cc: linux-raid
In-Reply-To: <20150212000940.GA49579@eris.prismnet.com>
On 12/02/15 11:09, sunruh@prismnet.com wrote:
> On Thu, Feb 12, 2015 at 09:12:50AM +1100, Adam Goryachev wrote:
>> On 12/02/15 05:04, sunruh@prismnet.com wrote:
>>> centos 6.6
>>> 2x 240gig ssd in raid1
>>> this is a live running production machine and the raid1 is for /u of
>>> users home dirs.
>>>
>>> 1 ssd went totally offline and i replaced it after noticing the firmware
>>> levels are not the same. the new ssd has the same level firmware.
>>>
>>> /dev/sdb is the good ssd
>>> /dev/sdc is the new blank ssd
>>>
>>> when working it was /u1 from /dev/md127p1 and /u2 from /dev/md127p2
>>> p1 is 80gig and p2 is 160gig for the full 240gig size of the ssd
>>>
>>>> ls -al /dev/md*
>>> brw-rw---- 1 root disk 9, 127 Feb 11 11:09 /dev/md127
>>> brw-rw---- 1 root disk 259, 0 Feb 10 20:23 /dev/md127p1
>>> brw-rw---- 1 root disk 259, 1 Feb 10 20:23 /dev/md127p2
>>>
>>> /dev/md:
>>> total 8
>>> drwxr-xr-x 2 root root 140 Feb 10 20:24 .
>>> drwxr-xr-x 20 root root 3980 Feb 10 20:24 ..
>>> lrwxrwxrwx 1 root root 8 Feb 11 11:09 240ssd_0 -> ../md127
>>> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p1 -> ../md127p1
>>> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p2 -> ../md127p2
>>> -rw-r--r-- 1 root root 5 Feb 10 20:24 autorebuild.pid
>>> -rw------- 1 root root 63 Feb 10 20:23 md-device-map
>>>
>>>> ps -eaf | grep mdadm
>>> root 2188 1 0 Feb10 ? 00:00:00 mdadm --monitor --scan -f --pid-file=/var/run/mdadm/mdadm.pid
>>>
>>> how do i rebuild /dev/sdc into the mirror of /dev/sdb?
>>>
>> Please send the output of fdisk -lu /dev/sd[bc] and cat /proc/mdstat
>> (preferably both when it was working and current).
>>
>> In general, when replacing a failed RAID1 disk, and assuming you
>> configured it the way I think you did:
>> 1) fdisk -lu /dev/sdb
>> Find out the exact partition sizes
>> 2) fdisk /dev/sdc
>> Create the new partitions exactly the same as /dev/sdb
>> 3) mdadm --manage /dev/md127 --add /dev/sdb1
>> Add the partition to the array
>> 4) cat /proc/mdstat
>> Watch the rebuild progress, once it is complete, relax.
>>
>> PS, steps 1 and 2 may not be needed if you are using the full block
>> device instead of a partition. Also, change the command in step 3 to
>> "mdadm --manage /dev/md127 --add /dev/sdb"
>>
>> PPS, if this is a bootable disk, you will probably also need to do
>> something with your boot manager to get that installed onto the new disk
>> as well.
>>
>> Hope this helps, otherwise, please provide more information.
>>
>>
>> Regards,
>> Adam
>>
>> --
>> Adam Goryachev Website Managers www.websitemanagers.com.au
> Adam (and anybody else that can help),
> after issue i do not have before. and no they are not bootable.
>
> [root@shell ~]# fdisk -lu /dev/sd[bc]
>
> Disk /dev/sdb: 240.1 GB, 240057409536 bytes
> 255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0x0001a740
>
>
> Disk /dev/sdc: 240.1 GB, 240057409536 bytes
> 255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0x00000000
>
> [root@shell ~]# cat /proc/mdstat
> Personalities : [raid1]
> md127 : active raid1 sdb[2]
> 234299840 blocks super 1.2 [2/1] [U_]
>
> unused devices: <none>
> i dont seem to be seeing the partition sizes or im stupid.
> couldnt i just dd if=/dev/sdb of=/dev/sdc bs=1G count=240 and then do the
> mdadm?
OK, so you aren't using partitioned disks, so it is as simple as what I
said above (with one minor correction):
"mdadm --manage /dev/md127 --add /dev/sdc"
/dev/sdc is the new blank ssd, so that is the one to add, the above
command with /dev/sdb wouldn't have done anything at all .... So just
run that command, and then do "watch cat /proc/mdstat" until the good
stuff is completed.
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Phil Turmel @ 2015-02-12 0:15 UTC (permalink / raw)
To: Kyle Logue; +Cc: linux-raid
In-Reply-To: <CAP7a4UR1HzJ9AueEr1e=Zh5+p0=QgODA4=_Bp7bfQ=f2CwPhYg@mail.gmail.com>
On 02/11/2015 05:12 PM, Kyle Logue wrote:
> Good news phil. Under the hypothesis that the new disk that I added
> didn't fully replace my sde I omitted it from my assemble. The array
> went full UUUUU, then I echo'd check > /sys/block/md0/md/sync_action
>
> Much later it kicked out the faulty disk (previously sdc) and now i
> have a _UUUU.
>
> So hopefully this is the final question, but should I just evacuate as
> much data as possible immediately? Or try to add another spare and
> rebuild?
So long as you haven't mounted it yet, I suggest you do another forced
assembly to get back to UUUUU, then kick off another check. When many
UREs are allowed to accumulate, mdadm can hit its read error rate limit
and kick the drive. If it hasn't been mounted, you can keep doing it
until you get through the entire check.
But, you also had misaligned partitions. If sdcN is one of them, the
above won't work, and you should get your backups ASAP. And then make a
new array from scratch.
If you do succeed in completing a check scrub, you can use --replace to
put the array on properly aligned partitions.
Phil
^ permalink raw reply
* Re: please help - raid 1 degraded
From: sunruh @ 2015-02-12 0:09 UTC (permalink / raw)
To: Adam Goryachev; +Cc: sunruh, linux-raid
In-Reply-To: <54DBD3E2.80701@websitemanagers.com.au>
On Thu, Feb 12, 2015 at 09:12:50AM +1100, Adam Goryachev wrote:
> On 12/02/15 05:04, sunruh@prismnet.com wrote:
> > centos 6.6
> > 2x 240gig ssd in raid1
> > this is a live running production machine and the raid1 is for /u of
> > users home dirs.
> >
> > 1 ssd went totally offline and i replaced it after noticing the firmware
> > levels are not the same. the new ssd has the same level firmware.
> >
> > /dev/sdb is the good ssd
> > /dev/sdc is the new blank ssd
> >
> > when working it was /u1 from /dev/md127p1 and /u2 from /dev/md127p2
> > p1 is 80gig and p2 is 160gig for the full 240gig size of the ssd
> >
> >> ls -al /dev/md*
> > brw-rw---- 1 root disk 9, 127 Feb 11 11:09 /dev/md127
> > brw-rw---- 1 root disk 259, 0 Feb 10 20:23 /dev/md127p1
> > brw-rw---- 1 root disk 259, 1 Feb 10 20:23 /dev/md127p2
> >
> > /dev/md:
> > total 8
> > drwxr-xr-x 2 root root 140 Feb 10 20:24 .
> > drwxr-xr-x 20 root root 3980 Feb 10 20:24 ..
> > lrwxrwxrwx 1 root root 8 Feb 11 11:09 240ssd_0 -> ../md127
> > lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p1 -> ../md127p1
> > lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p2 -> ../md127p2
> > -rw-r--r-- 1 root root 5 Feb 10 20:24 autorebuild.pid
> > -rw------- 1 root root 63 Feb 10 20:23 md-device-map
> >
> >> ps -eaf | grep mdadm
> > root 2188 1 0 Feb10 ? 00:00:00 mdadm --monitor --scan -f --pid-file=/var/run/mdadm/mdadm.pid
> >
> > how do i rebuild /dev/sdc into the mirror of /dev/sdb?
> >
>
> Please send the output of fdisk -lu /dev/sd[bc] and cat /proc/mdstat
> (preferably both when it was working and current).
>
> In general, when replacing a failed RAID1 disk, and assuming you
> configured it the way I think you did:
> 1) fdisk -lu /dev/sdb
> Find out the exact partition sizes
> 2) fdisk /dev/sdc
> Create the new partitions exactly the same as /dev/sdb
> 3) mdadm --manage /dev/md127 --add /dev/sdb1
> Add the partition to the array
> 4) cat /proc/mdstat
> Watch the rebuild progress, once it is complete, relax.
>
> PS, steps 1 and 2 may not be needed if you are using the full block
> device instead of a partition. Also, change the command in step 3 to
> "mdadm --manage /dev/md127 --add /dev/sdb"
>
> PPS, if this is a bootable disk, you will probably also need to do
> something with your boot manager to get that installed onto the new disk
> as well.
>
> Hope this helps, otherwise, please provide more information.
>
>
> Regards,
> Adam
>
> --
> Adam Goryachev Website Managers www.websitemanagers.com.au
Adam (and anybody else that can help),
after issue i do not have before. and no they are not bootable.
[root@shell ~]# fdisk -lu /dev/sd[bc]
Disk /dev/sdb: 240.1 GB, 240057409536 bytes
255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0001a740
Disk /dev/sdc: 240.1 GB, 240057409536 bytes
255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
[root@shell ~]# cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 sdb[2]
234299840 blocks super 1.2 [2/1] [U_]
unused devices: <none>
[root@shell ~]# fdisk -lu /dev/sdb
Disk /dev/sdb: 240.1 GB, 240057409536 bytes
255 heads, 63 sectors/track, 29185 cylinders, total 468862128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0001a740
i dont seem to be seeing the partition sizes or im stupid.
couldnt i just dd if=/dev/sdb of=/dev/sdc bs=1G count=240 and then do the
mdadm?
^ permalink raw reply
* Re: please help - raid 1 degraded
From: Adam Goryachev @ 2015-02-11 22:12 UTC (permalink / raw)
To: sunruh, linux-raid
In-Reply-To: <20150211180403.GA14805@fnord.prismnet.com>
On 12/02/15 05:04, sunruh@prismnet.com wrote:
> centos 6.6
> 2x 240gig ssd in raid1
> this is a live running production machine and the raid1 is for /u of
> users home dirs.
>
> 1 ssd went totally offline and i replaced it after noticing the firmware
> levels are not the same. the new ssd has the same level firmware.
>
> /dev/sdb is the good ssd
> /dev/sdc is the new blank ssd
>
> when working it was /u1 from /dev/md127p1 and /u2 from /dev/md127p2
> p1 is 80gig and p2 is 160gig for the full 240gig size of the ssd
>
>> ls -al /dev/md*
> brw-rw---- 1 root disk 9, 127 Feb 11 11:09 /dev/md127
> brw-rw---- 1 root disk 259, 0 Feb 10 20:23 /dev/md127p1
> brw-rw---- 1 root disk 259, 1 Feb 10 20:23 /dev/md127p2
>
> /dev/md:
> total 8
> drwxr-xr-x 2 root root 140 Feb 10 20:24 .
> drwxr-xr-x 20 root root 3980 Feb 10 20:24 ..
> lrwxrwxrwx 1 root root 8 Feb 11 11:09 240ssd_0 -> ../md127
> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p1 -> ../md127p1
> lrwxrwxrwx 1 root root 10 Feb 10 20:23 240ssd_0p2 -> ../md127p2
> -rw-r--r-- 1 root root 5 Feb 10 20:24 autorebuild.pid
> -rw------- 1 root root 63 Feb 10 20:23 md-device-map
>
>> ps -eaf | grep mdadm
> root 2188 1 0 Feb10 ? 00:00:00 mdadm --monitor --scan -f --pid-file=/var/run/mdadm/mdadm.pid
>
> how do i rebuild /dev/sdc into the mirror of /dev/sdb?
>
Please send the output of fdisk -lu /dev/sd[bc] and cat /proc/mdstat
(preferably both when it was working and current).
In general, when replacing a failed RAID1 disk, and assuming you
configured it the way I think you did:
1) fdisk -lu /dev/sdb
Find out the exact partition sizes
2) fdisk /dev/sdc
Create the new partitions exactly the same as /dev/sdb
3) mdadm --manage /dev/md127 --add /dev/sdb1
Add the partition to the array
4) cat /proc/mdstat
Watch the rebuild progress, once it is complete, relax.
PS, steps 1 and 2 may not be needed if you are using the full block
device instead of a partition. Also, change the command in step 3 to
"mdadm --manage /dev/md127 --add /dev/sdb"
PPS, if this is a bootable disk, you will probably also need to do
something with your boot manager to get that installed onto the new disk
as well.
Hope this helps, otherwise, please provide more information.
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Kyle Logue @ 2015-02-11 22:12 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
In-Reply-To: <54DB6707.5030901@turmel.org>
Good news phil. Under the hypothesis that the new disk that I added
didn't fully replace my sde I omitted it from my assemble. The array
went full UUUUU, then I echo'd check > /sys/block/md0/md/sync_action
Much later it kicked out the faulty disk (previously sdc) and now i
have a _UUUU.
So hopefully this is the final question, but should I just evacuate as
much data as possible immediately? Or try to add another spare and
rebuild?
Thanks for the help,
Kyle L
^ permalink raw reply
* [PATCH RESEND] Change way of printing name of a process
From: Pawel Baldysiak @ 2015-02-11 21:25 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, pawel.baldysiak, artur.paszkiewicz
Sometimes mdadm prints messages with wrong name "mdmon",
and vice versa.
This patch solves this problem by changing method of determining
process name.
Now "Name" will be set in const at start of a program,
previously was hardcoded as #define.
Signed-off-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
---
Grow.c | 18 +++++++++---------
Incremental.c | 4 ++--
Monitor.c | 4 ++--
ReadMe.c | 2 +-
mdadm.c | 7 ++++---
mdadm.h | 6 +++---
mdmon.c | 2 ++
mdmon.h | 3 +--
super-intel.c | 2 +-
sysfs.c | 8 ++++----
xmalloc.c | 12 ++++++++----
11 files changed, 37 insertions(+), 31 deletions(-)
diff --git a/Grow.c b/Grow.c
index 6ff225a..4a01a31 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1919,7 +1919,7 @@ size_change_error:
int err;
err = remove_disks_for_takeover(st, sra, array.layout);
if (err) {
- dprintf(Name": Array cannot be reshaped\n");
+ dprintf("%s: Array cannot be reshaped\n", Name);
if (cfd > -1)
close(cfd);
rv = 1;
@@ -2133,7 +2133,7 @@ static int verify_reshape_position(struct mdinfo *info, int level)
char *ep;
unsigned long long position = strtoull(buf, &ep, 0);
- dprintf(Name": Read sync_max sysfs entry is: %s\n", buf);
+ dprintf("%s: Read sync_max sysfs entry is: %s\n", Name, buf);
if (!(ep == buf || (*ep != 0 && *ep != '\n' && *ep != ' '))) {
position *= get_data_disks(level,
info->new_layout,
@@ -3494,8 +3494,8 @@ int reshape_container(char *container, char *devname,
return 1;
default: /* parent */
if (!freeze_reshape)
- printf(Name ": multi-array reshape continues"
- " in background\n");
+ printf("%s: multi-array reshape continues"
+ " in background\n", Name);
return 0;
case 0: /* child */
map_fork();
@@ -3557,8 +3557,8 @@ int reshape_container(char *container, char *devname,
fd = open_dev(mdstat->devnm);
if (fd < 0) {
- printf(Name ": Device %s cannot be opened for reshape.",
- adev);
+ printf("%s: Device %s cannot be opened for reshape.",
+ Name, adev);
break;
}
@@ -3573,8 +3573,8 @@ int reshape_container(char *container, char *devname,
* This is possibly interim until the behaviour of
* reshape_array is resolved().
*/
- printf(Name ": Multiple reshape execution detected for "
- "device %s.", adev);
+ printf("%s: Multiple reshape execution detected for "
+ "device %s.", Name, adev);
close(fd);
break;
}
@@ -4611,7 +4611,7 @@ int Grow_restart(struct supertype *st, struct mdinfo *info, int *fdlist, int cnt
st->ss->free_super(st);
offsets[j] = dinfo.data_offset * 512;
}
- printf(Name ": restoring critical section\n");
+ printf("%s: restoring critical section\n", Name);
if (restore_stripes(fdlist, offsets,
info->array.raid_disks,
diff --git a/Incremental.c b/Incremental.c
index 13b68bc..38c9989 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -1710,9 +1710,9 @@ int IncrementalRemove(char *devname, char *id_path, int verbose)
char buf[32];
if (!id_path)
- dprintf(Name ": incremental removal without --path <id_path> "
+ dprintf("%s: incremental removal without --path <id_path> "
"lacks the possibility to re-add new device in this "
- "port\n");
+ "port\n", Name);
if (strchr(devname, '/')) {
pr_err("incremental removal requires a "
diff --git a/Monitor.c b/Monitor.c
index 971d2ec..7a5203c 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -381,14 +381,14 @@ static void alert(char *event, char *dev, char *disc, struct alert_info *info)
if (info->mailfrom)
fprintf(mp, "From: %s\n", info->mailfrom);
else
- fprintf(mp, "From: " Name " monitoring <root>\n");
+ fprintf(mp, "From: %s monitoring <root>\n", Name);
fprintf(mp, "To: %s\n", info->mailaddr);
fprintf(mp, "Subject: %s event on %s:%s\n\n",
event, dev, hname);
fprintf(mp,
"This is an automatically generated"
- " mail message from " Name "\n");
+ " mail message from %s\n", Name);
fprintf(mp, "running on %s\n\n", hname);
fprintf(mp,
diff --git a/ReadMe.c b/ReadMe.c
index 445c388..87a4916 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -30,7 +30,7 @@
#ifndef VERS_DATE
#define VERS_DATE "21st August 2014"
#endif
-char Version[] = Name " - v" VERSION " - " VERS_DATE "\n";
+char Version[] = "mdadm - v" VERSION " - " VERS_DATE "\n";
/*
* File: ReadMe.c
diff --git a/mdadm.c b/mdadm.c
index c856fcd..475c71c 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -38,6 +38,7 @@ static int misc_list(struct mddev_dev *devlist,
struct mddev_ident *ident,
char *dump_directory,
struct supertype *ss, struct context *c);
+const char Name[] = "mdadm";
int main(int argc, char *argv[])
{
@@ -771,12 +772,12 @@ int main(int argc, char *argv[])
if (strcmp(c.update,"?") == 0 ||
strcmp(c.update, "help") == 0) {
outf = stdout;
- fprintf(outf, Name ": ");
+ fprintf(outf, "%s: ", Name);
} else {
outf = stderr;
fprintf(outf,
- Name ": '--update=%s' is invalid. ",
- c.update);
+ "%s: '--update=%s' is invalid. ",
+ Name, c.update);
}
fprintf(outf, "Valid --update options are:\n"
" 'sparc2.2', 'super-minor', 'uuid', 'name', 'resync',\n"
diff --git a/mdadm.h b/mdadm.h
index fc1fd31..c8d079c 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -180,6 +180,8 @@ extern __off64_t lseek64 __P ((int __fd, __off64_t __offset, int __whence));
#define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
+extern const char Name[];
+
/* general information that might be extracted from a superblock */
struct mdinfo {
mdu_array_info_t array;
@@ -262,8 +264,6 @@ struct createinfo {
struct supertype *supertype;
};
-#define Name "mdadm"
-
enum mode {
ASSEMBLE=1,
BUILD,
@@ -1457,7 +1457,7 @@ static inline int xasprintf(char **strp, const char *fmt, ...) {
return ret;
}
-#define pr_err(fmt ...) fprintf(stderr, Name ": " fmt)
+#define pr_err(fmt, args...) fprintf(stderr, "%s: "fmt, Name, ##args)
#define cont_err(fmt ...) fprintf(stderr, " " fmt)
void *xmalloc(size_t len);
diff --git a/mdmon.c b/mdmon.c
index 27045a1..ee12b7c 100644
--- a/mdmon.c
+++ b/mdmon.c
@@ -67,6 +67,8 @@
#include "mdadm.h"
#include "mdmon.h"
+char const Name[] = "mdmon";
+
struct active_array *discard_this;
struct active_array *pending_discard;
diff --git a/mdmon.h b/mdmon.h
index 5a8e120..aa750c6 100644
--- a/mdmon.h
+++ b/mdmon.h
@@ -18,8 +18,7 @@
* 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
*/
-#undef pr_err
-#define pr_err(fmt ...) fprintf(stderr, "mdmon: " fmt)
+extern const char Name[];
enum array_state { clear, inactive, suspended, readonly, read_auto,
clean, active, write_pending, active_idle, bad_word};
diff --git a/super-intel.c b/super-intel.c
index 4b23b9a..d900339 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1386,7 +1386,7 @@ static int imsm_check_attributes(__u32 attributes)
}
if (not_supported)
- dprintf(Name "(IMSM): Unknown attributes : %x\n", not_supported);
+ dprintf("%s (IMSM): Unknown attributes : %x\n", Name, not_supported);
ret_val = 0;
}
diff --git a/sysfs.c b/sysfs.c
index 9a1d856..ceab27f 100644
--- a/sysfs.c
+++ b/sysfs.c
@@ -413,8 +413,8 @@ int sysfs_set_str(struct mdinfo *sra, struct mdinfo *dev,
n = write(fd, val, strlen(val));
close(fd);
if (n != strlen(val)) {
- dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
- val, fname, strerror(errno));
+ dprintf("%s: failed to write '%s' to '%s' (%s)\n",
+ Name, val, fname, strerror(errno));
return -1;
}
return 0;
@@ -450,8 +450,8 @@ int sysfs_uevent(struct mdinfo *sra, char *event)
n = write(fd, event, strlen(event));
close(fd);
if (n != (int)strlen(event)) {
- dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
- event, fname, strerror(errno));
+ dprintf("%s: failed to write '%s' to '%s' (%s)\n",
+ Name, event, fname, strerror(errno));
return -1;
}
return 0;
diff --git a/xmalloc.c b/xmalloc.c
index 8d42a7c..75ae4e2 100644
--- a/xmalloc.c
+++ b/xmalloc.c
@@ -37,7 +37,8 @@ void *xmalloc(size_t len)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -47,7 +48,8 @@ void *xrealloc(void *ptr, size_t len)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -57,7 +59,8 @@ void *xcalloc(size_t num, size_t size)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -67,6 +70,7 @@ char *xstrdup(const char *str)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox