* Re: Can't reshape raid0 to raid10
From: Xiao Ni @ 2015-02-05 8:44 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20150204173921.23df9d79@notabene.brown>
----- Original Message -----
> From: "NeilBrown" <neilb@suse.de>
> To: "Xiao Ni" <xni@redhat.com>
> Cc: linux-raid@vger.kernel.org
> Sent: Wednesday, February 4, 2015 2:39:21 PM
> Subject: Re: Can't reshape raid0 to raid10
>
> On Mon, 29 Dec 2014 22:13:41 -0500 (EST) Xiao Ni <xni@redhat.com> wrote:
>
> > Hi Neil
> >
> > When I try to reshape a raid0 to raid10, it'll fail like this:
> >
> > [root@dhcp-12-133 mdadm-3.3.2]# lsblk
> > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> > sda 8:0 0 111.8G 0 disk
> > ├─sda1 8:1 0 1000M 0 part /boot
> > ├─sda2 8:2 0 29.3G 0 part /
> > ├─sda3 8:3 0 512M 0 part [SWAP]
> > ├─sda4 8:4 0 1K 0 part
> > ├─sda5 8:5 0 102M 0 part
> > └─sda6 8:6 0 10.1G 0 part
> > └─VolGroup00-LogVol00 254:0 0 9.9G 0 lvm
> > sdb 8:16 0 111.8G 0 disk
> > ├─sdb1 8:17 0 2G 0 part
> > └─sdb2 8:18 0 10G 0 part
> > sdc 8:32 0 186.3G 0 disk
> > ├─sdc1 8:33 0 2G 0 part
> > └─sdc2 8:34 0 10G 0 part
> > sdd 8:48 0 111.8G 0 disk
> > ├─sdd1 8:49 0 2G 0 part
> > └─sdd2 8:50 0 10G 0 part
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1
> > /dev/sdc1 /dev/sdd1
> > mdadm: Defaulting to version 1.2 metadata
> > mdadm: array /dev/md0 started.
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2
> > /dev/sdc2 /dev/sdd2
> > mdadm: level of /dev/md0 changed to raid10
> > mdadm: add new device failed for /dev/sdb2 as 6: No space left on device
> >
> >
> > But if I reshape the raid0 to raid5, reshape raid5 to raid0, then
> > reshape raid0 to raid10 use
> > the same command it'll succeed.
> >
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1
> > /dev/sdc1 /dev/sdd1
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l5
> > [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat
> > Personalities : [raid6] [raid5] [raid4] [raid0] [raid10]
> > md0 : active raid5 sdd1[2] sdc1[1] sdb1[0]
> > 6285312 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3]
> > [UUU_]
> >
> > unused devices: <none>
> >
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l0
> > [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat
> > Personalities : [raid6] [raid5] [raid4] [raid0] [raid10]
> > md0 : active raid0 sdd1[2] sdc1[1] sdb1[0]
> > 6285312 blocks super 1.2 512k chunks
> >
> > unused devices: <none>
> > [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2
> > /dev/sdc2 /dev/sdd2
> > mdadm: level of /dev/md0 changed to raid10
> > mdadm: added /dev/sdb2
> > mdadm: added /dev/sdc2
> > mdadm: added /dev/sdd2
> >
> > So I guess it's the problem add the disk to raid10 after the reshaping.
> > In the function
> > super_1_validate, it'll set the mddev->dev_sectors using the superblock
> > read from disks.
> > If it's raid0, the le64_to_cpu(sb-size) is 0. So when add disk to raid10
> > bind_rdev_to_array
> > return -ENOSPC.
> >
> > When create raid0, it doesn't write give the value to s->size. So the
> > sb-size is 0.
> > I modify the code about Create.c. I'm not sure whether it's right to do so.
> > But it can resolve
> > the problem.
>
> Thanks for the report.
>
> I think this need to be fixing in the md driver, when the reshape happens.
>
> Something like the following.
>
> NeilBrown
>
> diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
> index d1203cddb024..b8d76b1fba64 100644
> --- a/drivers/md/raid10.c
> +++ b/drivers/md/raid10.c
> @@ -3872,7 +3872,7 @@ static int raid10_resize(struct mddev *mddev, sector_t
> sectors)
> return 0;
> }
>
> -static void *raid10_takeover_raid0(struct mddev *mddev)
> +static void *raid10_takeover_raid0(struct mddev *mddev, sector_t size, int
> devs)
> {
> struct md_rdev *rdev;
> struct r10conf *conf;
> @@ -3882,6 +3882,7 @@ static void *raid10_takeover_raid0(struct mddev *mddev)
> mdname(mddev));
> return ERR_PTR(-EINVAL);
> }
> + sector_div(size, devs);
>
> /* Set new parameters */
> mddev->new_level = 10;
> @@ -3892,12 +3893,15 @@ static void *raid10_takeover_raid0(struct mddev
> *mddev)
> mddev->raid_disks *= 2;
> /* make sure it will be not marked as dirty */
> mddev->recovery_cp = MaxSector;
> + mddev->dev_sectors = size;
>
> conf = setup_conf(mddev);
> if (!IS_ERR(conf)) {
> rdev_for_each(rdev, mddev)
> - if (rdev->raid_disk >= 0)
> + if (rdev->raid_disk >= 0) {
> rdev->new_raid_disk = rdev->raid_disk * 2;
> + rdev->sectors = size;
> + }
> conf->barrier = 1;
> }
>
> @@ -3920,7 +3924,9 @@ static void *raid10_takeover(struct mddev *mddev)
> mdname(mddev));
> return ERR_PTR(-EINVAL);
> }
> - return raid10_takeover_raid0(mddev);
> + return raid10_takeover_raid0(mddev,
> + raid0_conf->strip_zone->zone_end,
> + raid0_conf->strip_zone->nb_dev);
> }
> return ERR_PTR(-EINVAL);
> }
>
Thanks, the problem is fixed.
Best Regards
Xiao
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: BUG drivers/md/md.c: data-offset reshape renders array unloadable
From: Wesley W. Terpstra @ 2015-02-05 8:17 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20150205171402.7dc2f394@notabene.brown>
On Thu, Feb 5, 2015 at 7:14 AM, NeilBrown <neilb@suse.de> wrote:
> A v1.2 array normally has data from the data_offset all the way to the end of
> the device (maybe rounded down to chunk size).
Right. So in my case there were 5120 sectors unused at the start.
Unfortunately, I do not know whether the array initially had a 'size'
that included the trailing 3072 sectors or not.
What I do know: before reshape I had a chunk size of 512K and 4 disks
in raid5. When I created the array, tools at the time did not support
--data-offset, but I probably specified all the options I could find
saying I wanted 4M alignment. I reshaped the array from 1 disk to 2 to
3 to 4 as I added disks.
Unfortunately, I do not have a record of what 'mdadm -E' said before I
started the reshape.
Before I fixed the superblock:
Avail Dev Size : 5842897920 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Used Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Data Offset : 8192 sectors
After I fixed the superblock:
Avail Dev Size : 5842894848 (2786.11 GiB 2991.56 GB)
Array Size : 8764342272 (8358.33 GiB 8974.69 GB)
Data Offset : 8192 sectors
> So you normally wouldn't be able to increase the data_offset from 5120 to
> 8192 unless you first reduced the --size of the array (having reduced the
> size of any filesystem first).
Well, it let me do it.
> Did you do that? That should have reduced sb->data_size appropriately.
No. I just went ahead and increased the data offset.
> If you did --grow the --size first, that should have reduced sb->data_size.
I did not do this step.
> If you didn't the --grow --data-offset should have failed.
I did this directly, and it let me. I had a slightly older kernel when
I started the reshape. 3.17.7 and mdadm 3.3.2.
^ permalink raw reply
* Re: BUG drivers/md/md.c: data-offset reshape renders array unloadable
From: NeilBrown @ 2015-02-05 6:14 UTC (permalink / raw)
To: Wesley W. Terpstra; +Cc: linux-raid
In-Reply-To: <CAA-O0Xjw8H1nPsi+MsQnrLxv=gC2tj3=W=t3aMnCE6ESUHMF8w@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]
On Sun, 25 Jan 2015 17:46:20 +0100 "Wesley W. Terpstra" <wesley@terpstra.ca>
wrote:
> On Sat, Jan 24, 2015 at 11:23 PM, Wesley W. Terpstra <wesley@terpstra.ca> wrote:
> >> First, it is obviously the last test in super_1_load that is rejecting
> >> the array. The superblock reports more sectors than are calculated, so
> >> the check
> >> if (sectors < le64_to_cpu(sb->data_size)) {
> >> fails.
> >
> > Thus, an alternative explanation could be that the the sb->data_size
> > was not updated after the reshape completed.
>
> I can confirm that this was the problem.
>
> I manually modified my super block using the attached quick hack.
> Thereafter I was able to reassemble the array and fsck everything
> successfully.
>
> I will try and see if I can reproduce the problem tomorrow. It's a
> pretty nasty bug to have a reshape complete and render your array
> unassemblable.
Thanks for all the details and analysis!!!
A v1.2 array normally has data from the data_offset all the way to the end of
the device (maybe rounded down to chunk size).
So you normally wouldn't be able to increase the data_offset from 5120 to
8192 unless you first reduced the --size of the array (having reduced the
size of any filesystem first).
Did you do that? That should have reduced sb->data_size appropriately.
If you did --grow the --size first, that should have reduced sb->data_size.
If you didn't the --grow --data-offset should have failed.
Obviously one of those 'should's didn't...
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: raid5 Unplug the disk
From: NeilBrown @ 2015-02-05 5:03 UTC (permalink / raw)
To: John Drescher; +Cc: lilofile, stan, Linux RAID
In-Reply-To: <CAEhu1-4krpTbF-0mLdgoTKrkdrWo8J2k80BC0t48rv3WkG3UBA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 934 bytes --]
On Wed, 28 Jan 2015 11:59:30 -0500 John Drescher <drescherjm@gmail.com> wrote:
> On Wed, Jan 28, 2015 at 1:44 AM, lilofile <lilofile@aliyun.com> wrote:
> > kernel linux 3.10, centos
> > when I use 5 disk to create raid5,after sync complete, I unplug a disk,and wait a moment, then replug the same disk.
> >
> > the original 4 disk are still a raid5 which is degraded.
> > I find the single disk become a new raid5,but inactive,what the reason?
>
> stop whatever new array that shows up then use
>
> mdadm --manage /dev/mymddevice --re-add /dev/disk5
>
Also, you probably need a newer version of mdadm. Creating a new inactive
raid5 was a bug.
If you have udev configured to run "mdadm -I" on newly added devices (which I
think most distros do), then adding
POLICY action=re-add
to /etc/mdadm.conf should cause the device to automatically be added back to
the array when you plug it in.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: raid1 narrow_write_error with 4K disks, sd "bad block number requested" messages
From: NeilBrown @ 2015-02-05 4:59 UTC (permalink / raw)
To: Nate Dailey; +Cc: linux-raid, linux-scsi
In-Reply-To: <54C9006A.2030807@stratus.com>
[-- Attachment #1: Type: text/plain, Size: 1942 bytes --]
On Wed, 28 Jan 2015 10:29:46 -0500 Nate Dailey <nate.dailey@stratus.com>
wrote:
> I'm writing about something that appears to be an issue with raid1's
> narrow_write_error, particular to non-512-byte-sector disks. Here's what
> I'm doing:
>
> - 2 disk raid1, 4K disks, each connected to a different SAS HBA
> - mount a filesystem on the raid1, run a test that writes to it
> - remove one of the SAS HBAs (echo 1 >
> /sys/bus/pci/devices/0000\:45\:00.0/remove)
>
> At this point, writes fail and narrow_write_error breaks them up and
> retries, one sector at a time. But these are 512-byte sectors, and sd
> doesn't like it:
>
> [ 2645.310517] sd 3:0:1:0: [sde] Bad block number requested
> [ 2645.310610] sd 3:0:1:0: [sde] Bad block number requested
> [ 2645.310690] sd 3:0:1:0: [sde] Bad block number requested
> ...
>
> There appears to be no real harm done, but there can be a huge number of
> these messages in the log.
>
> I can avoid this by disabling bad block tracking, but it looks like
> maybe the superblock's bblog_shift is intended to address this exact
> issue. However, I don't see a way to change it. Presumably this is
> something mdadm should be setting up? I don't see bblog_shift ever set
> to anything other than 0.
>
> This is on a RHEL 7.1 kernel, version 3.10.0-221.el7. I took a look at
> upstream sd and md changes and nothing jumps out at me that would have
> affected this (but I have not tested to see if the bad block messages do
> or do not happen on an upstream kernel).
>
> I'd appreciate any advice re: how to handle this. Thanks!
Thanks for the report.
narrow_write_error() should use bdev_logical_block_size() and round up to
that.
Possibly mdadm should get the same information and set bblog_shift
accordingly when creating a bad block log.
I've made a note to fix that, but I'm happy to review patches too :-)
thanks,
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: unsynchronized raid10 with different events count
From: NeilBrown @ 2015-02-05 4:52 UTC (permalink / raw)
To: LuVar; +Cc: linux-raid
In-Reply-To: <850189010.36671422720846038.JavaMail.root@shiva>
[-- Attachment #1: Type: text/plain, Size: 1344 bytes --]
On Sat, 31 Jan 2015 17:14:06 +0100 (GMT+01:00) LuVar <luvar@plaintext.sk>
wrote:
> Hi,
> finally,
> I have run --re-add thing. It seems that it is what I wanted. My array is now OK.
>
> So only one question remains. Is there any possibility to make improvements and make array autoassemble if there are clear that in raid10, there is consistent highest events count for at least one drive for raid1 subparts? I have layout=n3 for situations when one drive fails, to be able to correctly recover from others two same copies and to know that given data are probably ok if they are same.
It appears from the mdstat you posted:
Personalities : [raid10]
md3 : active raid10 sdj[9] sdl[13] sdm[6] sdi[3] sdb[10]
937709952 blocks super 1.2 128K chunks 3 near-copies [9/5] [U_UU__UU_]
bitmap: 4/7 pages [16KB], 65536KB chunk
unused devices: <none>
That it *did* autoassemble and provide you with a working array. It just
didn't include all the devices - because some of them looked old and mdadm
couldn't be sure they were reliable.
You did the correct thing to --re-add them. That is how you tell mdadm
"these devices are reliable, even though someone went wrong before and they
didn't get updated with the latest event count".
So it appears to me that everything is working correctly.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Off-Topic Write cache disabling?
From: NeilBrown @ 2015-02-05 4:47 UTC (permalink / raw)
To: Weedy; +Cc: linux-raid
In-Reply-To: <CAFE24U1G9+bqTzK4eD-cenyc5CGQQdrcu7S9Ks4CsHPS9wJ4zA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2389 bytes --]
On Tue, 3 Feb 2015 12:45:44 -0500 Weedy <weedy2887@gmail.com> wrote:
> Is there a kernel option or sysfs toggle that disables write caching?
> Or forces the kernel to commit everything constantly.
>
> ------
> I don't really want to join another ML, especially a higher traffic
> one just to ask this when it only bugs me sometimes. But I'll shut up
> if this is unwanted.
> ------
>
> I use a similar kernel config with respect to selected options on all
> my systems but this only effect my laptop.
Maybe it has something to do with "laptop_mode"
/proc/sys/vm/laptop_mode
https://www.kernel.org/doc/Documentation/laptops/laptop-mode.txt
> On any of my personal system or system I have remote access too
> nr_dirty drifts up and down and nr_writeback stays around 0 (assuming
> the system isn't working hard).
> On my laptop both nr_dirty and nr_writeback stay at 0. I can make them
> go up to 10ish if I untar something but then almost immediately go
> back to 0. If I didn't know better I would swear dirty_*_centisecs or
> something was set to a near instant commit interval but I haven't
> found evidence of that. The hard drive light blinks almost constantly
> once a second, even if I'm at a X login screen.
That doesn't quite sound like laptop mode...
laptop mode syncs things more aggressively when the disk is spinning, and
more lazily when it isn't ... or something like that.
So I can imagine nr_dirty staying low - any read will spin the disk, and that
will quickly flush out any dirty pages.
The once-a-second blink is harder to explain.
Still, it would be worth checking if laptop mode is enabled, and if so
disable it and see if anything changes.
NeilBrown
> As I said this doesn't bug me most of the time but if I let my FF
> session get too large or start multiple VMs anything that might make
> me swap a little, the machine pretty much dies from IOWAIT. Which I'm
> guessing is because it's trying to flush (syncfs?) imediately and
> constantly.
>
> You guys spend all day in the IO subsystem, any idea where I can keep
> looking? It has persisted across reboots and kernel updates.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Array died during grow; now resync stopped
From: NeilBrown @ 2015-02-05 4:40 UTC (permalink / raw)
To: Jörg Habenicht; +Cc: linux-raid
In-Reply-To: <trinity-85824e10-6eeb-490e-91e9-29cac6bb884e-1423051086472@3capp-gmx-bs24>
[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]
On Wed, 4 Feb 2015 12:58:06 +0100 "Jörg Habenicht" <j.habenicht@gmx.de> wrote:
> It works! :-)) The array syncs!
Excellent!
>
>
> > > >What does
> > > >cat /proc/1671/stack
> > > >cat /proc/1672/stack
> > > >show?
> > >
> > > $ cat /proc/1671/stack
> > > cat: /proc/1671/stack: No such file or directory
> >
> > I guess you don't have that feature compiled into your kernel.
>
> Guess so. I'm going to look to the missing CONFIG_ line.
CONFIG_STACKTRACE ... in case you haven't found it already.
> >
> > And this is stuck in the same place.... what what is consuming all the
> > stripes I wonder....
>
> Do you like me to collect more information?
No thanks, the required information was further down. They were all consumed
by the reshape (which needs at least twice the largest chunk size).
> > If you double it, the problem should go away.
> >
> > mdadm should do that for you .... I wonder why it didn't.
> >
>
> Do you like to have more test results?
I had a look at the code and it was fairly obvious. mdadm adjusts the stripe
cache based on the 'old' chunk size. It never includes the 'new' chunk size
in those calculations. I've made a note to fix that.
Thanks,
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Off-Topic Write cache disabling?
From: Bernd Schubert @ 2015-02-04 12:46 UTC (permalink / raw)
To: Weedy, linux-raid
In-Reply-To: <CAFE24U1G9+bqTzK4eD-cenyc5CGQQdrcu7S9Ks4CsHPS9wJ4zA@mail.gmail.com>
On 02/03/2015 06:45 PM, Weedy wrote:
> Is there a kernel option or sysfs toggle that disables write caching?
> Or forces the kernel to commit everything constantly.
>
> ------
> I don't really want to join another ML, especially a higher traffic
> one just to ask this when it only bugs me sometimes. But I'll shut up
> if this is unwanted.
> ------
>
> I use a similar kernel config with respect to selected options on all
> my systems but this only effect my laptop.
> On any of my personal system or system I have remote access too
> nr_dirty drifts up and down and nr_writeback stays around 0 (assuming
> the system isn't working hard).
> On my laptop both nr_dirty and nr_writeback stay at 0. I can make them
> go up to 10ish if I untar something but then almost immediately go
> back to 0. If I didn't know better I would swear dirty_*_centisecs or
> something was set to a near instant commit interval but I haven't
> found evidence of that. The hard drive light blinks almost constantly
> once a second, even if I'm at a X login screen.
> As I said this doesn't bug me most of the time but if I let my FF
> session get too large or start multiple VMs anything that might make
> me swap a little, the machine pretty much dies from IOWAIT. Which I'm
> guessing is because it's trying to flush (syncfs?) imediately and
> constantly.
>
> You guys spend all day in the IO subsystem, any idea where I can keep
> looking? It has persisted across reboots and kernel updates.
Try mount -osync or force your application to write with direct IO.
Cheers,
Bernd
^ permalink raw reply
* Re: Off-Topic Write cache disabling?
From: Bernd Schubert @ 2015-02-04 12:45 UTC (permalink / raw)
To: Roger Heflin, Weedy; +Cc: Linux RAID
In-Reply-To: <CAAMCDedvF+Nv9LYqM7Ddgg2xMQKtgQDntOsG0krxWCV4H+zQmA@mail.gmail.com>
On 02/03/2015 07:49 PM, Roger Heflin wrote:
> In general if you completely disable write caching things will likely
> be unusable. When I have benchmarked things attempting to turn off
> write cache performance has been mostly unusable.
>
> I usually change /etc/sysctl.conf and change these entries to control
> the amount of dirty cache.
>
> vm.dirty_background_bytes = 25000000
> vm.dirty_bytes = 50000000
>
> That says max dirty allowed is 50mb, and it syncs down to 25mb when it syncs.
Er, where do you have this from?
dirty_background_bytes and dirty_bytes are more about non-blocking and
blocking dirty writes. And about the amount of dirty data that triggers
it (i.e. there is no need to flush temporary file data to disk if files
get immediately deleted after writing them)
linux/Documentation/sysctl/vm.txt
==============================================================
dirty_background_bytes
Contains the amount of dirty memory at which the background kernel
flusher threads will start writeback.
Note: dirty_background_bytes is the counterpart of dirty_background_ratio. Only
one of them may be specified at a time. When one sysctl is written it is
immediately taken into account to evaluate the dirty memory limits and the
other appears as 0 when read.
==============================================================
==============================================================
dirty_bytes
Contains the amount of dirty memory at which a process generating disk writes
will itself start writeback.
Note: dirty_bytes is the counterpart of dirty_ratio. Only one of them may be
specified at a time. When one sysctl is written it is immediately taken into
account to evaluate the dirty memory limits and the other appears as 0 when
read.
Note: the minimum value allowed for dirty_bytes is two pages (in bytes); any
value lower than this limit will be ignored and the old configuration will be
retained.
==============================================================
Bernd
^ permalink raw reply
* Aw: Re: Array died during grow; now resync stopped
From: "Jörg Habenicht" @ 2015-02-04 11:58 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150204174556.426eb865@notabene.brown>
It works! :-)) The array syncs!
> > >What does
> > >cat /proc/1671/stack
> > >cat /proc/1672/stack
> > >show?
> >
> > $ cat /proc/1671/stack
> > cat: /proc/1671/stack: No such file or directory
>
> I guess you don't have that feature compiled into your kernel.
Guess so. I'm going to look to the missing CONFIG_ line.
> > >Alternatively,
> > >echo w > /proc/sysrq-trigger
> > >and see what appears in 'dmesg'.
> >
> > No good:
>
> Quite the reverse, this is exactly what I wanted. It shows the stack trace
> of pid 1671 and 1672..
Ahh, OK. I thought it being a crash dump.
> >
> > [99166.625796] SysRq : Show Blocked State
> > [99166.625829] task PC stack pid father
> > [99166.625845] md0_reshape D ffff88006cb81e08 0 1671 2 0x00000000
> > [99166.625854] ffff88006a17fb30 0000000000000046 000000000000a000 ffff88006cc9b7e0
> > [99166.625861] ffff88006a17ffd8 ffff88006cc9b7e0 ffff88006fc11830 ffff88006fc11830
> > [99166.625866] 0000000000000001 ffffffff81068670 ffff88006ca56848 ffff88006fc11830
> > [99166.625871] Call Trace:
> > [99166.625884] [<ffffffff81068670>] ? __dequeue_entity+0x40/0x50
> > [99166.625891] [<ffffffff8106b966>] ? pick_next_task_fair+0x56/0x1b0
> > [99166.625898] [<ffffffff813f4a50>] ? __schedule+0x2a0/0x820
> > [99166.625905] [<ffffffff8106273d>] ? ttwu_do_wakeup+0xd/0x80
> > [99166.625914] [<ffffffffa027b4c5>] ? get_active_stripe+0x185/0x5c0 [raid456]
> > [99166.625922] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> > [99166.625929] [<ffffffffa027e83a>] ? reshape_request+0x21a/0x860 [raid456]
> > [99166.625935] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> > [99166.625942] [<ffffffffa02744f6>] ? sync_request+0x236/0x380 [raid456]
> > [99166.625955] [<ffffffffa01557ad>] ? md_do_sync+0x82d/0xd00 [md_mod]
> > [99166.625961] [<ffffffff810684b4>] ? update_curr+0x64/0xe0
> > [99166.625971] [<ffffffffa0152197>] ? md_thread+0xf7/0x110 [md_mod]
> > [99166.625977] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> > [99166.625985] [<ffffffffa01520a0>] ? md_register_thread+0xf0/0xf0 [md_mod]
> > [99166.625991] [<ffffffff81059de8>] ? kthread+0xb8/0xd0
> > [99166.625997] [<ffffffff81059d30>] ? kthread_create_on_node+0x180/0x180
> > [99166.626003] [<ffffffff813f837c>] ? ret_from_fork+0x7c/0xb0
> > [99166.626008] [<ffffffff81059d30>] ? kthread_create_on_node+0x180/0x180
>
> That's not surprise. Whenever anything goes wrong in raid5, something gets
> stuck in get_active_stripe()...
>
>
> > [99166.626012] udevd D ffff88006cb81e08 0 1672 1289 0x00000004
> > [99166.626017] ffff88006a1819e8 0000000000000086 000000000000a000 ffff88006c4967a0
> > [99166.626022] ffff88006a181fd8 ffff88006c4967a0 0000000000000000 0000000000000000
> > [99166.626027] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> > [99166.626032] Call Trace:
> > [99166.626039] [<ffffffff810c24ed>] ? zone_statistics+0x9d/0xa0
> > [99166.626044] [<ffffffff810c24ed>] ? zone_statistics+0x9d/0xa0
> > [99166.626050] [<ffffffff810b13e7>] ? get_page_from_freelist+0x507/0x850
> > [99166.626057] [<ffffffffa027b4c5>] ? get_active_stripe+0x185/0x5c0 [raid456]
> > [99166.626063] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> > [99166.626069] [<ffffffffa027f627>] ? make_request+0x7a7/0xa00 [raid456]
> > [99166.626075] [<ffffffff81080afd>] ? ktime_get_ts+0x3d/0xd0
> > [99166.626080] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> > [99166.626089] [<ffffffffa014ea12>] ? md_make_request+0xd2/0x210 [md_mod]
> > [99166.626096] [<ffffffff811e649d>] ? generic_make_request_checks+0x23d/0x270
> > [99166.626100] [<ffffffff810acc68>] ? mempool_alloc+0x58/0x140
> > [99166.626106] [<ffffffff811e7238>] ? generic_make_request+0xa8/0xf0
> > [99166.626111] [<ffffffff811e72e7>] ? submit_bio+0x67/0x130
> > [99166.626117] [<ffffffff8112a638>] ? bio_alloc_bioset+0x1b8/0x2a0
> > [99166.626123] [<ffffffff81126a57>] ? _submit_bh+0x127/0x200
> > [99166.626129] [<ffffffff8112815d>] ? block_read_full_page+0x1fd/0x290
> > [99166.626133] [<ffffffff8112b680>] ? I_BDEV+0x10/0x10
> > [99166.626140] [<ffffffff810aad2b>] ? add_to_page_cache_locked+0x6b/0xc0
> > [99166.626146] [<ffffffff810b5520>] ? __do_page_cache_readahead+0x1b0/0x220
> > [99166.626152] [<ffffffff810b5812>] ? force_page_cache_readahead+0x62/0xa0
> > [99166.626159] [<ffffffff810ac936>] ? generic_file_aio_read+0x4b6/0x6c0
> > [99166.626166] [<ffffffff810f9f87>] ? do_sync_read+0x57/0x90
> > [99166.626172] [<ffffffff810fa571>] ? vfs_read+0xa1/0x180
> > [99166.626178] [<ffffffff810fb0ab>] ? SyS_read+0x4b/0xc0
> > [99166.626183] [<ffffffff813f7f72>] ? page_fault+0x22/0x30
> > [99166.626190] [<ffffffff813f8422>] ? system_call_fastpath+0x16/0x1b
>
> And this is stuck in the same place.... what what is consuming all the
> stripes I wonder....
Do you like me to collect more information?
> > >
> > > The system got 2G RAM and 2G swap. Is this sufficient to complete?
> >
> > >Memory shouldn't be a problem.
> > >However it wouldn't hurt to see what value is in
> > >/sys/block/md0/md/stripe_cache_size
> > >and double it.
> >
> > $ cat /sys/block/md0/md/stripe_cache_size
> > 256
>
>
> You are setting the chunk size to 1M, which is 256 4K pages.
> So this stripe_cache only just has enough space to store one full stripe at
> the new chunk size. That isn't enough.
>
> If you double it, the problem should go away.
>
> mdadm should do that for you .... I wonder why it didn't.
>
Do you like to have more test results?
> >
> > I did not change it due to the crash in md_reshape
>
> What crash is that? The above stack traces that you said "No good" about?
> That isn't a crash. That is the kernel showing you stack traces because you
> asked for them.
Ok, Thanks, learned something more :-)
>
> echo 1024 > /sys/block/md0/md/stripe_cache_size
>
> should make it work.
Yes it did!
$ echo 1024 > /sys/block/md0/md/stripe_cache_size
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[0] sdg1[9] sdc1[6] sdb1[7] sdd1[8] sdf1[5]
5860548608 blocks super 1.0 level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]
[=====>...............] reshape = 28.0% (410341376/1465137152) finish=29602912.5min speed=0K/sec
unused devices: <none>
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[0] sdg1[9] sdc1[6] sdb1[7] sdd1[8] sdf1[5]
5860548608 blocks super 1.0 level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]
[=====>...............] reshape = 28.0% (410656252/1465137152) finish=7746625.6min speed=2K/sec
unused devices: <none>
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[0] sdg1[9] sdc1[6] sdb1[7] sdd1[8] sdf1[5]
5860548608 blocks super 1.0 level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]
[=====>...............] reshape = 28.0% (410851328/1465137152) finish=5314609.8min speed=3K/sec
unused devices: <none>
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[0] sdg1[9] sdc1[6] sdb1[7] sdd1[8] sdf1[5]
5860548608 blocks super 1.0 level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]
[=====>...............] reshape = 28.4% (416577276/1465137152) finish=870.3min speed=20079K/sec
unused devices: <none>
Immediatly it starts to sync.
I wonder why it got stuck at 27% and not at 0%? Shouldn't it get stuck at the beginning if the cache size is not sufficient?
Or is it because of a reboot which happened at 27% sync status?
Anyways,
Thank you for your help.
>
> NeilBrown
>
>
cu,
Joerg
^ permalink raw reply
* Re: Array died during grow; now resync stopped
From: NeilBrown @ 2015-02-04 6:45 UTC (permalink / raw)
To: Jörg Habenicht; +Cc: linux-raid
In-Reply-To: <trinity-d6d8d6b9-e19b-4ff6-a39f-3748b5fb2e13-1422959753855@3capp-gmx-bs42>
[-- Attachment #1: Type: text/plain, Size: 7305 bytes --]
On Tue, 3 Feb 2015 11:35:53 +0100 "Jörg Habenicht" <j.habenicht@gmx.de> wrote:
> Hello Neil,
>
> thank you for caring.
>
> (And sorry for the malformed structure, I have to use webmail.. )
>
>
> On Mon, 2 Feb 2015 09:41:02 +0000 (UTC) Jörg Habenicht <j.habenicht@gmx.de>
> wrote:
>
> > Hi all,
> >
> > I had a server crash during an array grow.
> > Commandline was "mdadm --grow /dev/md0 --raid-devices=6 --chunk=1M"
> >
> >
> >
> > Could this be caused by a software lock?
>
> >Some sort of software problem I suspect.
> >What does
> >cat /proc/1671/stack
> >cat /proc/1672/stack
> >show?
>
> $ cat /proc/1671/stack
> cat: /proc/1671/stack: No such file or directory
I guess you don't have that feature compiled into your kernel.
>
> Huch?
> $ ls /proc/1671
> ls: cannot read symbolic link /proc/1671/exe: No such file or directory
> attr comm fdinfo mounts oom_score stat
> autogroup coredump_filter io mountstats oom_score_adj statm
> auxv cwd limits net pagemap status
> cgroup environ maps ns personality syscall
> clear_refs exe mem numa_maps root task
> cmdline fd mountinfo oom_adj smaps wchan
> $ id
> uid=0(root) gid=0(root) groups=0(root), ...
>
> $ cat /proc/1672/stack
> cat: /proc/1672/stack: No such file or directory
>
>
> >Alternatively,
> >echo w > /proc/sysrq-trigger
> >and see what appears in 'dmesg'.
>
> No good:
Quite the reverse, this is exactly what I wanted. It shows the stack trace
of pid 1671 and 1672..
>
> [99166.625796] SysRq : Show Blocked State
> [99166.625829] task PC stack pid father
> [99166.625845] md0_reshape D ffff88006cb81e08 0 1671 2 0x00000000
> [99166.625854] ffff88006a17fb30 0000000000000046 000000000000a000 ffff88006cc9b7e0
> [99166.625861] ffff88006a17ffd8 ffff88006cc9b7e0 ffff88006fc11830 ffff88006fc11830
> [99166.625866] 0000000000000001 ffffffff81068670 ffff88006ca56848 ffff88006fc11830
> [99166.625871] Call Trace:
> [99166.625884] [<ffffffff81068670>] ? __dequeue_entity+0x40/0x50
> [99166.625891] [<ffffffff8106b966>] ? pick_next_task_fair+0x56/0x1b0
> [99166.625898] [<ffffffff813f4a50>] ? __schedule+0x2a0/0x820
> [99166.625905] [<ffffffff8106273d>] ? ttwu_do_wakeup+0xd/0x80
> [99166.625914] [<ffffffffa027b4c5>] ? get_active_stripe+0x185/0x5c0 [raid456]
> [99166.625922] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> [99166.625929] [<ffffffffa027e83a>] ? reshape_request+0x21a/0x860 [raid456]
> [99166.625935] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> [99166.625942] [<ffffffffa02744f6>] ? sync_request+0x236/0x380 [raid456]
> [99166.625955] [<ffffffffa01557ad>] ? md_do_sync+0x82d/0xd00 [md_mod]
> [99166.625961] [<ffffffff810684b4>] ? update_curr+0x64/0xe0
> [99166.625971] [<ffffffffa0152197>] ? md_thread+0xf7/0x110 [md_mod]
> [99166.625977] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> [99166.625985] [<ffffffffa01520a0>] ? md_register_thread+0xf0/0xf0 [md_mod]
> [99166.625991] [<ffffffff81059de8>] ? kthread+0xb8/0xd0
> [99166.625997] [<ffffffff81059d30>] ? kthread_create_on_node+0x180/0x180
> [99166.626003] [<ffffffff813f837c>] ? ret_from_fork+0x7c/0xb0
> [99166.626008] [<ffffffff81059d30>] ? kthread_create_on_node+0x180/0x180
That's not surprise. Whenever anything goes wrong in raid5, something gets
stuck in get_active_stripe()...
> [99166.626012] udevd D ffff88006cb81e08 0 1672 1289 0x00000004
> [99166.626017] ffff88006a1819e8 0000000000000086 000000000000a000 ffff88006c4967a0
> [99166.626022] ffff88006a181fd8 ffff88006c4967a0 0000000000000000 0000000000000000
> [99166.626027] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [99166.626032] Call Trace:
> [99166.626039] [<ffffffff810c24ed>] ? zone_statistics+0x9d/0xa0
> [99166.626044] [<ffffffff810c24ed>] ? zone_statistics+0x9d/0xa0
> [99166.626050] [<ffffffff810b13e7>] ? get_page_from_freelist+0x507/0x850
> [99166.626057] [<ffffffffa027b4c5>] ? get_active_stripe+0x185/0x5c0 [raid456]
> [99166.626063] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> [99166.626069] [<ffffffffa027f627>] ? make_request+0x7a7/0xa00 [raid456]
> [99166.626075] [<ffffffff81080afd>] ? ktime_get_ts+0x3d/0xd0
> [99166.626080] [<ffffffff81072110>] ? __wake_up_sync+0x10/0x10
> [99166.626089] [<ffffffffa014ea12>] ? md_make_request+0xd2/0x210 [md_mod]
> [99166.626096] [<ffffffff811e649d>] ? generic_make_request_checks+0x23d/0x270
> [99166.626100] [<ffffffff810acc68>] ? mempool_alloc+0x58/0x140
> [99166.626106] [<ffffffff811e7238>] ? generic_make_request+0xa8/0xf0
> [99166.626111] [<ffffffff811e72e7>] ? submit_bio+0x67/0x130
> [99166.626117] [<ffffffff8112a638>] ? bio_alloc_bioset+0x1b8/0x2a0
> [99166.626123] [<ffffffff81126a57>] ? _submit_bh+0x127/0x200
> [99166.626129] [<ffffffff8112815d>] ? block_read_full_page+0x1fd/0x290
> [99166.626133] [<ffffffff8112b680>] ? I_BDEV+0x10/0x10
> [99166.626140] [<ffffffff810aad2b>] ? add_to_page_cache_locked+0x6b/0xc0
> [99166.626146] [<ffffffff810b5520>] ? __do_page_cache_readahead+0x1b0/0x220
> [99166.626152] [<ffffffff810b5812>] ? force_page_cache_readahead+0x62/0xa0
> [99166.626159] [<ffffffff810ac936>] ? generic_file_aio_read+0x4b6/0x6c0
> [99166.626166] [<ffffffff810f9f87>] ? do_sync_read+0x57/0x90
> [99166.626172] [<ffffffff810fa571>] ? vfs_read+0xa1/0x180
> [99166.626178] [<ffffffff810fb0ab>] ? SyS_read+0x4b/0xc0
> [99166.626183] [<ffffffff813f7f72>] ? page_fault+0x22/0x30
> [99166.626190] [<ffffffff813f8422>] ? system_call_fastpath+0x16/0x1b
And this is stuck in the same place.... what what is consuming all the
stripes I wonder....
>
>
> >
> > The system got 2G RAM and 2G swap. Is this sufficient to complete?
>
> >Memory shouldn't be a problem.
> >However it wouldn't hurt to see what value is in
> >/sys/block/md0/md/stripe_cache_size
> >and double it.
>
> $ cat /sys/block/md0/md/stripe_cache_size
> 256
You are setting the chunk size to 1M, which is 256 4K pages.
So this stripe_cache only just has enough space to store one full stripe at
the new chunk size. That isn't enough.
If you double it, the problem should go away.
mdadm should do that for you .... I wonder why it didn't.
>
> I did not change it due to the crash in md_reshape
What crash is that? The above stack traces that you said "No good" about?
That isn't a crash. That is the kernel showing you stack traces because you
asked for them.
echo 1024 > /sys/block/md0/md/stripe_cache_size
should make it work.
NeilBrown
>
>
> >If all else fails a reboot should be safe and will probably start the reshape
> >properly. md is very careful about surviving reboots.
>
> I already did reboot twice before I wrote to the list. Same result.
>
>
> >NeilBrown
>
>
> cu,
> Joerg
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: Can't reshape raid0 to raid10
From: NeilBrown @ 2015-02-04 6:39 UTC (permalink / raw)
To: Xiao Ni; +Cc: linux-raid
In-Reply-To: <484995116.1735572.1419909221275.JavaMail.zimbra@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 4916 bytes --]
On Mon, 29 Dec 2014 22:13:41 -0500 (EST) Xiao Ni <xni@redhat.com> wrote:
> Hi Neil
>
> When I try to reshape a raid0 to raid10, it'll fail like this:
>
> [root@dhcp-12-133 mdadm-3.3.2]# lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 111.8G 0 disk
> ├─sda1 8:1 0 1000M 0 part /boot
> ├─sda2 8:2 0 29.3G 0 part /
> ├─sda3 8:3 0 512M 0 part [SWAP]
> ├─sda4 8:4 0 1K 0 part
> ├─sda5 8:5 0 102M 0 part
> └─sda6 8:6 0 10.1G 0 part
> └─VolGroup00-LogVol00 254:0 0 9.9G 0 lvm
> sdb 8:16 0 111.8G 0 disk
> ├─sdb1 8:17 0 2G 0 part
> └─sdb2 8:18 0 10G 0 part
> sdc 8:32 0 186.3G 0 disk
> ├─sdc1 8:33 0 2G 0 part
> └─sdc2 8:34 0 10G 0 part
> sdd 8:48 0 111.8G 0 disk
> ├─sdd1 8:49 0 2G 0 part
> └─sdd2 8:50 0 10G 0 part
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1
> mdadm: Defaulting to version 1.2 metadata
> mdadm: array /dev/md0 started.
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2 /dev/sdc2 /dev/sdd2
> mdadm: level of /dev/md0 changed to raid10
> mdadm: add new device failed for /dev/sdb2 as 6: No space left on device
>
>
> But if I reshape the raid0 to raid5, reshape raid5 to raid0, then reshape raid0 to raid10 use
> the same command it'll succeed.
>
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm -CR /dev/md0 -l0 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l5
> [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid0] [raid10]
> md0 : active raid5 sdd1[2] sdc1[1] sdb1[0]
> 6285312 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
>
> unused devices: <none>
>
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l0
> [root@dhcp-12-133 mdadm-3.3.2]# cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid0] [raid10]
> md0 : active raid0 sdd1[2] sdc1[1] sdb1[0]
> 6285312 blocks super 1.2 512k chunks
>
> unused devices: <none>
> [root@dhcp-12-133 mdadm-3.3.2]# mdadm --grow /dev/md0 -l10 -a /dev/sdb2 /dev/sdc2 /dev/sdd2
> mdadm: level of /dev/md0 changed to raid10
> mdadm: added /dev/sdb2
> mdadm: added /dev/sdc2
> mdadm: added /dev/sdd2
>
> So I guess it's the problem add the disk to raid10 after the reshaping. In the function
> super_1_validate, it'll set the mddev->dev_sectors using the superblock read from disks.
> If it's raid0, the le64_to_cpu(sb-size) is 0. So when add disk to raid10 bind_rdev_to_array
> return -ENOSPC.
>
> When create raid0, it doesn't write give the value to s->size. So the sb-size is 0.
> I modify the code about Create.c. I'm not sure whether it's right to do so. But it can resolve
> the problem.
Thanks for the report.
I think this need to be fixing in the md driver, when the reshape happens.
Something like the following.
NeilBrown
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index d1203cddb024..b8d76b1fba64 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -3872,7 +3872,7 @@ static int raid10_resize(struct mddev *mddev, sector_t sectors)
return 0;
}
-static void *raid10_takeover_raid0(struct mddev *mddev)
+static void *raid10_takeover_raid0(struct mddev *mddev, sector_t size, int devs)
{
struct md_rdev *rdev;
struct r10conf *conf;
@@ -3882,6 +3882,7 @@ static void *raid10_takeover_raid0(struct mddev *mddev)
mdname(mddev));
return ERR_PTR(-EINVAL);
}
+ sector_div(size, devs);
/* Set new parameters */
mddev->new_level = 10;
@@ -3892,12 +3893,15 @@ static void *raid10_takeover_raid0(struct mddev *mddev)
mddev->raid_disks *= 2;
/* make sure it will be not marked as dirty */
mddev->recovery_cp = MaxSector;
+ mddev->dev_sectors = size;
conf = setup_conf(mddev);
if (!IS_ERR(conf)) {
rdev_for_each(rdev, mddev)
- if (rdev->raid_disk >= 0)
+ if (rdev->raid_disk >= 0) {
rdev->new_raid_disk = rdev->raid_disk * 2;
+ rdev->sectors = size;
+ }
conf->barrier = 1;
}
@@ -3920,7 +3924,9 @@ static void *raid10_takeover(struct mddev *mddev)
mdname(mddev));
return ERR_PTR(-EINVAL);
}
- return raid10_takeover_raid0(mddev);
+ return raid10_takeover_raid0(mddev,
+ raid0_conf->strip_zone->zone_end,
+ raid0_conf->strip_zone->nb_dev);
}
return ERR_PTR(-EINVAL);
}
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
From: Manibalan P @ 2015-02-04 5:56 UTC (permalink / raw)
To: NeilBrown, linux-raid; +Cc: Pasi Kärkkäinen
In-Reply-To: <20150203093040.569aa5e1@notabene.brown>
>> Dear All,
>> Any updates on this issue.
>Probably the same as:
> http://marc.info/?l=linux-raid&m=142283560704091&w=2
Dear Neil
This patch is not fixing this issue.
This issue happens only if a drive removed from a RAID5 array, which is "initializing" and "heavy IO" is performed on the array.
In such case, as soon as the drive removed, the array state changed to resync=PENDING and md0_raid5 thread using 100% of CPU.
Thanks,
Manibalan.
>which follows on from
> http://marc.info/?t=142221642300001&r=1&w=2
>and
> http://marc.info/?t=142172432500001&r=1&w=2
>NeilBrown
-----Original Message-----
From: NeilBrown [mailto:neilb@suse.de]
Sent: Tuesday, February 3, 2015 4:01 AM
To: Manibalan P
Cc: Pasi Kärkkäinen; linux-raid
Subject: Re: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
On Mon, 2 Feb 2015 07:10:14 +0000 Manibalan P <pmanibalan@amiindia.co.in>
wrote:
> Dear All,
> Any updates on this issue.
Probably the same as:
http://marc.info/?l=linux-raid&m=142283560704091&w=2
which follows on from
http://marc.info/?t=142221642300001&r=1&w=2
and
http://marc.info/?t=142172432500001&r=1&w=2
NeilBrown
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Wednesday, January 14, 2015 3:55 PM
> To: 'Pasi Kärkkäinen'
> Cc: 'neilb@suse.de'; 'linux-raid'
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
> Could you able to find something on this issue.
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Friday, January 2, 2015 12:08 PM
> To: 'Pasi Kärkkäinen'
> Cc: neilb@suse.de; linux-raid
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
>
> I have add the bug in
> https://bugzilla.redhat.com/show_bug.cgi?id=1178080
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Wednesday, December 31, 2014 10:18 PM
> To: Manibalan P
> Cc: neilb@suse.de; linux-raid
> Subject: Re: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> On Tue, Dec 30, 2014 at 11:06:47AM +0000, Manibalan P wrote:
> > Dear Neil,
> >
>
> Hello,
>
> > Few this for you kind attention,
> > 1. I tried the same test with FC11 (2.6.32 kernel before MD code
> > change). And the issue is not there 2. But with Centos 6.4 (2.6.32 kernel after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Also, a bug has been raise with RHEL regarding this issue. Please find the bug link "https://access.redhat.com/support/cases/#/case/01320319"
> >
>
> That support case URL can only be accessed by you and Redhat. Do you happen to have a public bugzilla link?
>
>
> Thanks,
>
> -- Pasi
>
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 24, 2014 12:15 PM
> > To: neilb@suse.de; 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> >
> > Dear Neil,
> >
> > Few this for you kind attention,
> > 1. I tried the same tesst with FC11 (2.6 kernel before MD code change). And the issue is not there 2. But with Centos 6.4 (2.6 after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Thursday, December 18, 2014 11:38 AM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear neil,
> >
> > I also compiled the latest 3.18 kernel on CentOS 6.4 with GIT MD pull patches form 3.19, that also ran in to the same issue after removing a drive during resync.
> >
> > Dec 17 19:07:32 ITX002590129362 kernel: Linux version 3.18.0 (root@mycentos6) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Wed Dec 17 15:59:09 EST 2014 Dec 17 19:07:32 ITX002590129362 kernel: Command line: ro root=/dev/md255 rd_NO_LVM rd_NO_DM rhgb quiet md_mod.start_ro=1 nmi_watchdog=1 md_mod.start_dirty_degraded=1 ??? Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sda6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdb6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdc6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdh6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdi6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdj6> Dec 17 19:10:15 ITX002590129362 kernel: async_tx: api initialized (async) Dec 17 19:10:15 ITX002590129362 kernel: xor: measuring software checksum speed
> > Dec 17 19:10:15 ITX002590129362 kernel: prefetch64-sse: 10048.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: generic_sse: 8824.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: xor: using function: prefetch64-sse (10048.000 MB/sec)
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x1 5921 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x2 6933 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x4 7476 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: using algorithm
> > sse2x4
> > (7476 MB/s) Dec 17 19:10:15 ITX002590129362 kernel: raid6: using
> > ssse3x2 recovery algorithm Dec 17 19:10:15 ITX002590129362 kernel: md:
> > raid6 personality registered for level 6 Dec 17 19:10:15
> > ITX002590129362 kernel: md: raid5 personality registered for level 5
> > Dec 17 19:10:15 ITX002590129362 kernel: md: raid4 personality
> > registered for level 4 Dec 17 19:10:15 ITX002590129362 kernel:
> > md/raid:md0: not clean -- starting background reconstruction Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdj6
> > operational as raid disk 5 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdi6 operational as raid disk 4 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/raid:md0: device sdh6 operational as raid disk 3 Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdc6
> > operational as raid disk 2 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdb6 operational as raid disk 1 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/ra
> > id:md0: device sda6 operational as raid disk 0 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: allocated 0kB Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: raid level 5 active with 6 out of 6 devices, algorithm 2 Dec 17 19:10:15 ITX002590129362 kernel: md0: detected capacity change from 0 to 2361059573760 Dec 17 19:10:15 ITX002590129362 kernel: md0: unknown partition table Dec 17 19:10:35 ITX002590129362 kernel: md: md0 switched to read-write mode.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: resync of RAID array md0 Dec 17 19:10:35 ITX002590129362 kernel: md: minimum _guaranteed_ speed: 10000 KB/sec/disk.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using maximum available idle IO bandwidth (but not more than 30000 KB/sec) for resync.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using 128k window, over a total of 461144448k.
> > ???
> > Started IOs using fio tool.
> >
> > ./fio --name=md0 --filename=/dev/md0 --thread --numjobs=10
> > --direct=1 --group_reporting --unlink=0 --loops=1 --offset=0
> > --randrepeat=1 --norandommap --scramble_buffers=1 --stonewall
> > --ioengine=libaio --rw=randwrite --bs=8704 --iodepth=4000
> > --runtime=3000
> > --blockalign=512
> >
> > ???
> > Removed a drive form the system..
> >
> > Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) ..
> > Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 02 69 03 70 00 00 10 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 40436592 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 51 b3 d0 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 206681040 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 3a f3 40 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 205189952 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK ??? Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb 00 00 00 08 00 Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:25 ITX002590129362 kernel: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb f0 00 00 10 00 Dec 17 19:13:25 ITX002590129362 aghswap: devpath [0:0:7:0] action [remove] devtype [scsi_disk] Dec 17 19:13:25 ITX002590129362 aghswap: MHSA: Sent event 0 0 7 0 remove scsi_disk Dec 17 19:13:25 ITX002590129362 kernel: mpt2sas0: removing handle(0x0011), sas_addr(0x500605ba0101e305) Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Disk failure on sdh6, disabling device.
> > Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Operation continuing on 5 devices.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: md0: resync interrupted.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: checkpointing resync of md0.
> > ..
> > Log messages after enabling debufgs on raid5.c, it is getting repeated continuously.
> >
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0 handling stripe 273480328, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x18 read (null) write ffff8808029b6b00 written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=273480328 for sector 273480328, rmw=2 rcw=1 handling stripe 65238568, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081a956b00 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=65238568 for sector 65238568, rmw=2 rcw=1 handling stripe 713868672, state=0x2041 cnt=1, pd_idx=4, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x10 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081f020100 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=713868672 for sector 713868672, rmw=2 rcw=1 handling stripe 729622496, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622496 for sector 729622496, rmw=2 rcw=1 handling stripe 729622504, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622504 for sector 729622504, rmw=2 rcw=1 handling stripe 245773680, state=0x2041 cnt=1, pd_idx=0, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081cab7a00 written (null)
> > check 0: state 0x10 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=245773680 for sector 245773680, rmw=2 rcw=1 handling stripe 867965560, state=0x2041 cnt=1, pd_idx=1, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff880802b2bf00 written (null)
> > check 1: state 0x10 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=867965560 for sector 867965560, rmw=2 rcw=1 handling stripe 550162280, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x18 read (null) write ffff880802b08800 written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1
> > force RCW max_degraded=1, recovery_cp=7036944 sh->sector=550162280
> > for sector 550162280, rmw=2 rcw=1
> >
> >
> > Thanks,
> > Manibalan
> >
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:11 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > The same Issue is reproducible in the latest upstream kernel also.
> >
> > Tested in "3.17.6" latest stable upstream kernel and find the same issue.
> >
> > [root@root ~]# modinfo raid456
> > filename: /lib/modules/3.17.6/kernel/drivers/md/raid456.ko
> > alias: raid6
> > alias: raid5
> > alias: md-level-6
> > alias: md-raid6
> > alias: md-personality-8
> > alias: md-level-4
> > alias: md-level-5
> > alias: md-raid4
> > alias: md-raid5
> > alias: md-personality-4
> > description: RAID4/5/6 (striping with parity) personality for MD
> > license: GPL
> > srcversion: 0EEF680023FDC7410F7989A
> > depends: async_raid6_recov,async_pq,async_tx,async_memcpy,async_xor
> > intree: Y
> > vermagic: 3.17.6 SMP mod_unload modversions
> > parm: devices_handle_discard_safely:Set to Y if all devices in each array reliably return zeroes on reads from discarded regions (bool)
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:01 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > We are facing IO struck issue with raid5 in the following scenario.
> > (please see the attachment for the complete information) In RAID5
> > array, if a drive is removed while initialization and the same time
> > if IO is happening to that md. Then IO is getting struck, and
> > md_raid5 thread is using 100 % of CPU. Also the md state showing as
> > resync=PENDING
> >
> > Kernel : Issue found in the following kernels RHEL 6.5
> > (2.6.32-431.el6.x86_64) CentOS 7 (kernel-3.10.0-123.13.1.el7.x86_64)
> >
> > Steps to Reproduce the issue:
> >
> > 1. Created a raid 5 md with 4 drives using the below mdadm command.
> > mdadm -C /dev/md0 -c 64 -l 5 -f -n 4 -e 1.2 /dev/sdb6 /dev/sdc6
> > /dev/sdd6 /dev/sde6
> >
> > 2. Make the md writable
> > mdadm ???readwrite /dev/md0
> >
> > 3. Now md will start initialization
> >
> > 4. Run FIO Tool, the the below said configuration /usr/bin/fio
> > --name=md0 --filename=/dev/md0 --thread --numjobs=10 --direct=1
> > --group_reporting --unlink=0 --loops=1 --offset=0 --randrepeat=1
> > --norandommap --scramble_buffers=1 --stonewall --ioengine=libaio
> > --rw=randwrite --bs=8704 --iodepth=4000 --runtime=3000
> > --blockalign=512
> >
> > 4. During MD initialzing, remove a drive(either using MDADM set
> > faulty/remove or remove manually)
> >
> > 5. Now the IO will struck, and cat /proc/mdstat shows states with
> > resync=PENDING
> > --------------------------------------------------------------------
> > --
> > ----------------------- top - output show, md_raid5 using 100% cpu
> >
> > top - 17:55:06 up 1:09, 3 users, load average: 11.98, 8.53, 3.99
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 2690 root 20 0 0 0 0 R 100.0 0.0 6:44.41 md0_raid5
> > --------------------------------------------------------------------
> > --
> > -----------------------
> > dmesg - show the stack trace
> >
> > INFO: task fio:2715 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000a 0 2715 2654 0x00000080
> > ffff88043b623598 0000000000000082 0000000000000000 ffffffff81058d53
> > ffff88043b623548 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b40b098 ffff88043b623fd8 000000000000fbc8 ffff88043b40b098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff8140fa39>] ?
> > md_wakeup_thread+0x39/0x70 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffffa0308f66>] ?
> > make_request+0x306/0xc6c [raid456] [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81122283>] ?
> > mempool_alloc+0x63/0x140 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c767a>] do_direct_IO+0x7ca/0xfa0 [<ffffffff811c8196>]
> > __blockdev_direct_IO_newtrunc+0x346/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2717 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000004 0 2717 2654 0x00000080
> > ffff880439e97698 0000000000000082 ffff880439e97628 ffffffff81058d53
> > ffff880439e97648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0adab8 ffff880439e97fd8 000000000000fbc8 ffff88043b0adab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2718 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000005 0 2718 2654 0x00000080
> > ffff88043bc13698 0000000000000082 ffff88043bc13628 ffffffff81058d53
> > ffff88043bc13648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ad058 ffff88043bc13fd8 000000000000fbc8 ffff88043b0ad058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2719 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000001 0 2719 2654 0x00000080
> > ffff880439ebb698 0000000000000082 ffff880439ebb628 ffffffff81058d53
> > ffff880439ebb648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ac5f8 ffff880439ebbfd8 000000000000fbc8 ffff88043b0ac5f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2720 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000008 0 2720 2654 0x00000080
> > ffff88043b8cf698 0000000000000082 ffff88043b8cf628 ffffffff81058d53
> > ffff88043b8cf648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89af8 ffff88043b8cffd8 000000000000fbc8 ffff880439e89af8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2721 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2721 2654 0x00000080
> > ffff88043b047698 0000000000000082 ffff88043b047628 ffffffff81058d53
> > ffff88043b047648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89098 ffff88043b047fd8 000000000000fbc8 ffff880439e89098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2722 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2722 2654 0x00000080
> > ffff880439ea3698 0000000000000082 ffff880439ea3628 ffffffff81058d53
> > ffff880439ea3648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e88638 ffff880439ea3fd8 000000000000fbc8 ffff880439e88638 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2723 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000006 0 2723 2654 0x00000080
> > ffff88043bf5f698 0000000000000082 ffff88043bf5f628 ffffffff81058d53
> > ffff88043bf5f648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183ab8 ffff88043bf5ffd8 000000000000fbc8 ffff88043a183ab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2724 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000b 0 2724 2654 0x00000080
> > ffff88043be05698 0000000000000082 ffff88043be05628 ffffffff81058d53
> > ffff88043be05648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183058 ffff88043be05fd8 000000000000fbc8 ffff88043a183058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2725 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000003 0 2725 2654 0x00000080
> > ffff88043be07698 0000000000000082 ffff88043be07628 ffffffff81058d53
> > ffff88043be07648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a1825f8 ffff88043be07fd8 000000000000fbc8 ffff88043a1825f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> >
> > [root@root ~]# cat /proc/2690/stack
> > [<ffffffff810686da>] __cond_resched+0x2a/0x40 [<ffffffffa030361c>]
> > ops_run_io+0x2c/0x920 [raid456] [<ffffffffa03052cc>]
> > handle_stripe+0x9cc/0x2980 [raid456] [<ffffffffa03078a4>]
> > raid5d+0x624/0x850 [raid456] [<ffffffff81416f05>]
> > md_thread+0x115/0x150 [<ffffffff8109aef6>] kthread+0x96/0xa0
> > [<ffffffff8100c20a>] child_rip+0xa/0x20 [<ffffffffffffffff>]
> > 0xffffffffffffffff
> >
> > [root@root ~]# cat /proc/2690/stat
> > 2690 (md0_raid5) R 2 0 0 0 -1 2149613632 0 0 0 0 0 68495 0 0 20 0 1
> > 0
> > 350990 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483391 256 0 0 0
> > 17
> > 2 0 0 6855 0 0 [root@root ~]# cat /proc/2690/statm
> > 0 0 0 0 0 0 0
> > [root@root ~]# cat /proc/2690/stat
> > stat statm status
> > [root@root ~]# cat /proc/2690/status
> > Name: md0_raid5
> > State: R (running)
> > Tgid: 2690
> > Pid: 2690
> > PPid: 2
> > TracerPid: 0
> > Uid: 0 0 0 0
> > Gid: 0 0 0 0
> > Utrace: 0
> > FDSize: 64
> > Groups:
> > Threads: 1
> > SigQ: 2/128402
> > SigPnd: 0000000000000000
> > ShdPnd: 0000000000000000
> > SigBlk: 0000000000000000
> > SigIgn: fffffffffffffeff
> > SigCgt: 0000000000000100
> > CapInh: 0000000000000000
> > CapPrm: ffffffffffffffff
> > CapEff: fffffffffffffeff
> > CapBnd: ffffffffffffffff
> > Cpus_allowed: ffffff
> > Cpus_allowed_list: 0-23
> > Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
> > Mems_allowed_list: 0-1
> > voluntary_ctxt_switches: 5411612
> > nonvoluntary_ctxt_switches: 257032
> >
> >
> > Thanks,
> > Manibalan.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Off-Topic Write cache disabling?
From: Weedy @ 2015-02-04 5:00 UTC (permalink / raw)
To: Roger Heflin; +Cc: Linux RAID
In-Reply-To: <CAAMCDedvF+Nv9LYqM7Ddgg2xMQKtgQDntOsG0krxWCV4H+zQmA@mail.gmail.com>
On Tue, Feb 3, 2015 at 1:49 PM, Roger Heflin <rogerheflin@gmail.com> wrote:
> In general if you completely disable write caching things will likely
> be unusable. When I have benchmarked things attempting to turn off
> write cache performance has been mostly unusable.
This has little to do with the problem I wanted help with.
^ permalink raw reply
* Re: Off-Topic Write cache disabling?
From: Roger Heflin @ 2015-02-04 0:38 UTC (permalink / raw)
To: Alireza Haghdoost; +Cc: Weedy, Linux RAID
In-Reply-To: <CAB-428kgGgMTiyUgq=0geAMsLGju2QeJvL=GfcChX0W5cBfPBg@mail.gmail.com>
When dirty_cache hits 50mb it writes enough out to get the dirty_cache
down to 25MB (since 25000000 is the lower setting)
On Tue, Feb 3, 2015 at 1:16 PM, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>> and it syncs down to 25mb when it syncs.
>
> What do you mean by "syncs down to 25mb" ?
^ permalink raw reply
* [md PATCH 26/26] md: wakeup thread upon rdev_dec_pending()
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
From: Hannes Reinecke <hare@suse.de>
After each call to rdev_dec_pending() we should wakeup the
md thread if the device is found to be faulty.
Otherwise we'll incur heavy delays on failing devices.
Signed-off-by: Neil Brown <nfbrown@suse.de>
Signed-off-by: Hannes Reinecke <hare@suse.de>
---
drivers/md/md.h | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 14367d919351..318ca8fd430f 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -474,13 +474,6 @@ static inline int mddev_trylock(struct mddev *mddev)
}
extern void mddev_unlock(struct mddev *mddev);
-static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev)
-{
- int faulty = test_bit(Faulty, &rdev->flags);
- if (atomic_dec_and_test(&rdev->nr_pending) && faulty)
- set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
-}
-
static inline void md_sync_acct(struct block_device *bdev, unsigned long nr_sectors)
{
atomic_add(nr_sectors, &bdev->bd_contains->bd_disk->sync_io);
@@ -666,4 +659,14 @@ static inline int mddev_check_plugged(struct mddev *mddev)
return !!blk_check_plugged(md_unplug, mddev,
sizeof(struct blk_plug_cb));
}
+
+static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev)
+{
+ int faulty = test_bit(Faulty, &rdev->flags);
+ if (atomic_dec_and_test(&rdev->nr_pending) && faulty) {
+ set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
+ md_wakeup_thread(mddev->thread);
+ }
+}
+
#endif /* _MD_MD_H */
^ permalink raw reply related
* [md PATCH 25/26] md: make reconfig_mutex optional for writes to md sysfs files.
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
Rather than using mddev_lock() to take the reconfig_mutex
when writing to any md sysfs file, we only take mddev_lock()
in the particular _store() functions that require it.
Admittedly this is most, but it isn't all.
This also allows us to remove special-case handling for new_dev_store
(in md_attr_store).
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 346 ++++++++++++++++++++++++++++++++++++----------------
drivers/md/raid5.c | 134 +++++++++++---------
2 files changed, 316 insertions(+), 164 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 6517f4ece44e..4c3ab0639c58 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3256,26 +3256,32 @@ static ssize_t
level_store(struct mddev *mddev, const char *buf, size_t len)
{
char clevel[16];
- ssize_t rv = len;
+ ssize_t rv;
+ size_t slen = len;
struct md_personality *pers, *oldpers;
long level;
void *priv, *oldpriv;
struct md_rdev *rdev;
+ if (slen == 0 || slen >= sizeof(clevel))
+ return -EINVAL;
+
+ rv = mddev_lock(mddev);
+ if (rv)
+ return rv;
+
if (mddev->pers == NULL) {
- if (len == 0)
- return 0;
- if (len >= sizeof(mddev->clevel))
- return -ENOSPC;
- strncpy(mddev->clevel, buf, len);
- if (mddev->clevel[len-1] == '\n')
- len--;
- mddev->clevel[len] = 0;
+ strncpy(mddev->clevel, buf, slen);
+ if (mddev->clevel[slen-1] == '\n')
+ slen--;
+ mddev->clevel[slen] = 0;
mddev->level = LEVEL_NONE;
- return rv;
+ rv = len;
+ goto out_unlock;
}
+ rv = -EROFS;
if (mddev->ro)
- return -EROFS;
+ goto out_unlock;
/* request to change the personality. Need to ensure:
* - array is not engaged in resync/recovery/reshape
@@ -3283,25 +3289,25 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
* - new personality will access other array.
*/
+ rv = -EBUSY;
if (mddev->sync_thread ||
test_bit(MD_RECOVERY_RUNNING, &mddev->recovery) ||
mddev->reshape_position != MaxSector ||
mddev->sysfs_active)
- return -EBUSY;
+ goto out_unlock;
+ rv = -EINVAL;
if (!mddev->pers->quiesce) {
printk(KERN_WARNING "md: %s: %s does not support online personality change\n",
mdname(mddev), mddev->pers->name);
- return -EINVAL;
+ goto out_unlock;
}
/* Now find the new personality */
- if (len == 0 || len >= sizeof(clevel))
- return -EINVAL;
- strncpy(clevel, buf, len);
- if (clevel[len-1] == '\n')
- len--;
- clevel[len] = 0;
+ strncpy(clevel, buf, slen);
+ if (clevel[slen-1] == '\n')
+ slen--;
+ clevel[slen] = 0;
if (kstrtol(clevel, 10, &level))
level = LEVEL_NONE;
@@ -3312,20 +3318,23 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
if (!pers || !try_module_get(pers->owner)) {
spin_unlock(&pers_lock);
printk(KERN_WARNING "md: personality %s not loaded\n", clevel);
- return -EINVAL;
+ rv = -EINVAL;
+ goto out_unlock;
}
spin_unlock(&pers_lock);
if (pers == mddev->pers) {
/* Nothing to do! */
module_put(pers->owner);
- return rv;
+ rv = len;
+ goto out_unlock;
}
if (!pers->takeover) {
module_put(pers->owner);
printk(KERN_WARNING "md: %s: %s does not support personality takeover\n",
mdname(mddev), clevel);
- return -EINVAL;
+ rv = -EINVAL;
+ goto out_unlock;
}
rdev_for_each(rdev, mddev)
@@ -3345,7 +3354,8 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
module_put(pers->owner);
printk(KERN_WARNING "md: %s: %s would not accept array\n",
mdname(mddev), clevel);
- return PTR_ERR(priv);
+ rv = PTR_ERR(priv);
+ goto out_unlock;
}
/* Looks like we have a winner */
@@ -3438,6 +3448,9 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
md_update_sb(mddev, 1);
sysfs_notify(&mddev->kobj, NULL, "level");
md_new_event(mddev);
+ rv = len;
+out_unlock:
+ mddev_unlock(mddev);
return rv;
}
@@ -3460,28 +3473,32 @@ layout_store(struct mddev *mddev, const char *buf, size_t len)
{
char *e;
unsigned long n = simple_strtoul(buf, &e, 10);
+ int err;
if (!*buf || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->pers) {
- int err;
if (mddev->pers->check_reshape == NULL)
- return -EBUSY;
- if (mddev->ro)
- return -EROFS;
- mddev->new_layout = n;
- err = mddev->pers->check_reshape(mddev);
- if (err) {
- mddev->new_layout = mddev->layout;
- return err;
+ err = -EBUSY;
+ else if (mddev->ro)
+ err = -EROFS;
+ else {
+ mddev->new_layout = n;
+ err = mddev->pers->check_reshape(mddev);
+ if (err)
+ mddev->new_layout = mddev->layout;
}
} else {
mddev->new_layout = n;
if (mddev->reshape_position == MaxSector)
mddev->layout = n;
}
- return len;
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_layout =
__ATTR(layout, S_IRUGO|S_IWUSR, layout_show, layout_store);
@@ -3504,32 +3521,39 @@ static ssize_t
raid_disks_store(struct mddev *mddev, const char *buf, size_t len)
{
char *e;
- int rv = 0;
+ int err;
unsigned long n = simple_strtoul(buf, &e, 10);
if (!*buf || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->pers)
- rv = update_raid_disks(mddev, n);
+ err = update_raid_disks(mddev, n);
else if (mddev->reshape_position != MaxSector) {
struct md_rdev *rdev;
int olddisks = mddev->raid_disks - mddev->delta_disks;
+ err = -EINVAL;
rdev_for_each(rdev, mddev) {
if (olddisks < n &&
rdev->data_offset < rdev->new_data_offset)
- return -EINVAL;
+ goto out_unlock;
if (olddisks > n &&
rdev->data_offset > rdev->new_data_offset)
- return -EINVAL;
+ goto out_unlock;
}
+ err = 0;
mddev->delta_disks = n - olddisks;
mddev->raid_disks = n;
mddev->reshape_backwards = (mddev->delta_disks < 0);
} else
mddev->raid_disks = n;
- return rv ? rv : len;
+out_unlock:
+ mddev_unlock(mddev);
+ return err ? err : len;
}
static struct md_sysfs_entry md_raid_disks =
__ATTR(raid_disks, S_IRUGO|S_IWUSR, raid_disks_show, raid_disks_store);
@@ -3548,30 +3572,34 @@ chunk_size_show(struct mddev *mddev, char *page)
static ssize_t
chunk_size_store(struct mddev *mddev, const char *buf, size_t len)
{
+ int err;
char *e;
unsigned long n = simple_strtoul(buf, &e, 10);
if (!*buf || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->pers) {
- int err;
if (mddev->pers->check_reshape == NULL)
- return -EBUSY;
- if (mddev->ro)
- return -EROFS;
- mddev->new_chunk_sectors = n >> 9;
- err = mddev->pers->check_reshape(mddev);
- if (err) {
- mddev->new_chunk_sectors = mddev->chunk_sectors;
- return err;
+ err = -EBUSY;
+ else if (mddev->ro)
+ err = -EROFS;
+ else {
+ mddev->new_chunk_sectors = n >> 9;
+ err = mddev->pers->check_reshape(mddev);
+ if (err)
+ mddev->new_chunk_sectors = mddev->chunk_sectors;
}
} else {
mddev->new_chunk_sectors = n >> 9;
if (mddev->reshape_position == MaxSector)
mddev->chunk_sectors = n >> 9;
}
- return len;
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_chunk_size =
__ATTR(chunk_size, S_IRUGO|S_IWUSR, chunk_size_show, chunk_size_store);
@@ -3587,20 +3615,27 @@ resync_start_show(struct mddev *mddev, char *page)
static ssize_t
resync_start_store(struct mddev *mddev, const char *buf, size_t len)
{
+ int err;
char *e;
unsigned long long n = simple_strtoull(buf, &e, 10);
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->pers && !test_bit(MD_RECOVERY_FROZEN, &mddev->recovery))
- return -EBUSY;
- if (cmd_match(buf, "none"))
+ err = -EBUSY;
+ else if (cmd_match(buf, "none"))
n = MaxSector;
else if (!*buf || (*e && *e != '\n'))
- return -EINVAL;
+ err = -EINVAL;
- mddev->recovery_cp = n;
- if (mddev->pers)
- set_bit(MD_CHANGE_CLEAN, &mddev->flags);
- return len;
+ if (!err) {
+ mddev->recovery_cp = n;
+ if (mddev->pers)
+ set_bit(MD_CHANGE_CLEAN, &mddev->flags);
+ }
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_resync_start =
__ATTR(resync_start, S_IRUGO|S_IWUSR, resync_start_show, resync_start_store);
@@ -3698,8 +3733,39 @@ static int restart_array(struct mddev *mddev);
static ssize_t
array_state_store(struct mddev *mddev, const char *buf, size_t len)
{
- int err = -EINVAL;
+ int err;
enum array_state st = match_word(buf, array_states);
+
+ if (mddev->pers && (st == active || st == clean) && mddev->ro != 1) {
+ /* don't take reconfig_mutex when toggling between
+ * clean and active
+ */
+ spin_lock(&mddev->lock);
+ if (st == active) {
+ restart_array(mddev);
+ clear_bit(MD_CHANGE_PENDING, &mddev->flags);
+ wake_up(&mddev->sb_wait);
+ err = 0;
+ } else /* st == clean */ {
+ restart_array(mddev);
+ if (atomic_read(&mddev->writes_pending) == 0) {
+ if (mddev->in_sync == 0) {
+ mddev->in_sync = 1;
+ if (mddev->safemode == 1)
+ mddev->safemode = 0;
+ set_bit(MD_CHANGE_CLEAN, &mddev->flags);
+ }
+ err = 0;
+ } else
+ err = -EBUSY;
+ }
+ spin_unlock(&mddev->lock);
+ return err;
+ }
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ err = -EINVAL;
switch(st) {
case bad_word:
break;
@@ -3775,14 +3841,14 @@ array_state_store(struct mddev *mddev, const char *buf, size_t len)
/* these cannot be set */
break;
}
- if (err)
- return err;
- else {
+
+ if (!err) {
if (mddev->hold_active == UNTIL_IOCTL)
mddev->hold_active = 0;
sysfs_notify_dirent_safe(mddev->sysfs_state);
- return len;
}
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_array_state =
__ATTR(array_state, S_IRUGO|S_IWUSR, array_state_show, array_state_store);
@@ -3843,6 +3909,11 @@ new_dev_store(struct mddev *mddev, const char *buf, size_t len)
minor != MINOR(dev))
return -EOVERFLOW;
+ flush_workqueue(md_misc_wq);
+
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->persistent) {
rdev = md_import_device(dev, mddev->major_version,
mddev->minor_version);
@@ -3866,6 +3937,7 @@ new_dev_store(struct mddev *mddev, const char *buf, size_t len)
out:
if (err)
export_rdev(rdev);
+ mddev_unlock(mddev);
return err ? err : len;
}
@@ -3877,7 +3949,11 @@ bitmap_store(struct mddev *mddev, const char *buf, size_t len)
{
char *end;
unsigned long chunk, end_chunk;
+ int err;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (!mddev->bitmap)
goto out;
/* buf should be <chunk> <chunk> ... or <chunk>-<chunk> ... (range) */
@@ -3895,6 +3971,7 @@ bitmap_store(struct mddev *mddev, const char *buf, size_t len)
}
bitmap_unplug(mddev->bitmap); /* flush the bits to disk */
out:
+ mddev_unlock(mddev);
return len;
}
@@ -3922,6 +3999,9 @@ size_store(struct mddev *mddev, const char *buf, size_t len)
if (err < 0)
return err;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (mddev->pers) {
err = update_size(mddev, sectors);
md_update_sb(mddev, 1);
@@ -3932,6 +4012,7 @@ size_store(struct mddev *mddev, const char *buf, size_t len)
else
err = -ENOSPC;
}
+ mddev_unlock(mddev);
return err ? err : len;
}
@@ -3961,21 +4042,28 @@ metadata_store(struct mddev *mddev, const char *buf, size_t len)
{
int major, minor;
char *e;
+ int err;
/* Changing the details of 'external' metadata is
* always permitted. Otherwise there must be
* no devices attached to the array.
*/
+
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ err = -EBUSY;
if (mddev->external && strncmp(buf, "external:", 9) == 0)
;
else if (!list_empty(&mddev->disks))
- return -EBUSY;
+ goto out_unlock;
+ err = 0;
if (cmd_match(buf, "none")) {
mddev->persistent = 0;
mddev->external = 0;
mddev->major_version = 0;
mddev->minor_version = 90;
- return len;
+ goto out_unlock;
}
if (strncmp(buf, "external:", 9) == 0) {
size_t namelen = len-9;
@@ -3989,22 +4077,27 @@ metadata_store(struct mddev *mddev, const char *buf, size_t len)
mddev->external = 1;
mddev->major_version = 0;
mddev->minor_version = 90;
- return len;
+ goto out_unlock;
}
major = simple_strtoul(buf, &e, 10);
+ err = -EINVAL;
if (e==buf || *e != '.')
- return -EINVAL;
+ goto out_unlock;
buf = e+1;
minor = simple_strtoul(buf, &e, 10);
if (e==buf || (*e && *e != '\n') )
- return -EINVAL;
+ goto out_unlock;
+ err = -ENOENT;
if (major >= ARRAY_SIZE(super_types) || super_types[major].name == NULL)
- return -ENOENT;
+ goto out_unlock;
mddev->major_version = major;
mddev->minor_version = minor;
mddev->persistent = 1;
mddev->external = 0;
- return len;
+ err = 0;
+out_unlock:
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_metadata =
@@ -4049,7 +4142,10 @@ action_store(struct mddev *mddev, const char *page, size_t len)
flush_workqueue(md_misc_wq);
if (mddev->sync_thread) {
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
- md_reap_sync_thread(mddev);
+ if (mddev_lock(mddev) == 0) {
+ md_reap_sync_thread(mddev);
+ mddev_unlock(mddev);
+ }
}
} else if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery) ||
test_bit(MD_RECOVERY_NEEDED, &mddev->recovery))
@@ -4063,7 +4159,11 @@ action_store(struct mddev *mddev, const char *page, size_t len)
int err;
if (mddev->pers->start_reshape == NULL)
return -EINVAL;
- err = mddev->pers->start_reshape(mddev);
+ err = mddev_lock(mddev);
+ if (!err) {
+ err = mddev->pers->start_reshape(mddev);
+ mddev_unlock(mddev);
+ }
if (err)
return err;
sysfs_notify(&mddev->kobj, NULL, "degraded");
@@ -4346,14 +4446,20 @@ suspend_lo_store(struct mddev *mddev, const char *buf, size_t len)
{
char *e;
unsigned long long new = simple_strtoull(buf, &e, 10);
- unsigned long long old = mddev->suspend_lo;
+ unsigned long long old;
+ int err;
- if (mddev->pers == NULL ||
- mddev->pers->quiesce == NULL)
- return -EINVAL;
if (buf == e || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ err = -EINVAL;
+ if (mddev->pers == NULL ||
+ mddev->pers->quiesce == NULL)
+ goto unlock;
+ old = mddev->suspend_lo;
mddev->suspend_lo = new;
if (new >= old)
/* Shrinking suspended region */
@@ -4363,7 +4469,10 @@ suspend_lo_store(struct mddev *mddev, const char *buf, size_t len)
mddev->pers->quiesce(mddev, 1);
mddev->pers->quiesce(mddev, 0);
}
- return len;
+ err = 0;
+unlock:
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_suspend_lo =
__ATTR(suspend_lo, S_IRUGO|S_IWUSR, suspend_lo_show, suspend_lo_store);
@@ -4379,14 +4488,20 @@ suspend_hi_store(struct mddev *mddev, const char *buf, size_t len)
{
char *e;
unsigned long long new = simple_strtoull(buf, &e, 10);
- unsigned long long old = mddev->suspend_hi;
+ unsigned long long old;
+ int err;
- if (mddev->pers == NULL ||
- mddev->pers->quiesce == NULL)
- return -EINVAL;
if (buf == e || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ err = -EINVAL;
+ if (mddev->pers == NULL ||
+ mddev->pers->quiesce == NULL)
+ goto unlock;
+ old = mddev->suspend_hi;
mddev->suspend_hi = new;
if (new <= old)
/* Shrinking suspended region */
@@ -4396,7 +4511,10 @@ suspend_hi_store(struct mddev *mddev, const char *buf, size_t len)
mddev->pers->quiesce(mddev, 1);
mddev->pers->quiesce(mddev, 0);
}
- return len;
+ err = 0;
+unlock:
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_suspend_hi =
__ATTR(suspend_hi, S_IRUGO|S_IWUSR, suspend_hi_show, suspend_hi_store);
@@ -4416,11 +4534,17 @@ reshape_position_store(struct mddev *mddev, const char *buf, size_t len)
{
struct md_rdev *rdev;
char *e;
+ int err;
unsigned long long new = simple_strtoull(buf, &e, 10);
- if (mddev->pers)
- return -EBUSY;
+
if (buf == e || (*e && *e != '\n'))
return -EINVAL;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ err = -EBUSY;
+ if (mddev->pers)
+ goto unlock;
mddev->reshape_position = new;
mddev->delta_disks = 0;
mddev->reshape_backwards = 0;
@@ -4429,7 +4553,10 @@ reshape_position_store(struct mddev *mddev, const char *buf, size_t len)
mddev->new_chunk_sectors = mddev->chunk_sectors;
rdev_for_each(rdev, mddev)
rdev->new_data_offset = rdev->data_offset;
- return len;
+ err = 0;
+unlock:
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_reshape_position =
@@ -4447,6 +4574,8 @@ static ssize_t
reshape_direction_store(struct mddev *mddev, const char *buf, size_t len)
{
int backwards = 0;
+ int err;
+
if (cmd_match(buf, "forwards"))
backwards = 0;
else if (cmd_match(buf, "backwards"))
@@ -4456,16 +4585,19 @@ reshape_direction_store(struct mddev *mddev, const char *buf, size_t len)
if (mddev->reshape_backwards == backwards)
return len;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
/* check if we are allowed to change */
if (mddev->delta_disks)
- return -EBUSY;
-
- if (mddev->persistent &&
+ err = -EBUSY;
+ else if (mddev->persistent &&
mddev->major_version == 0)
- return -EINVAL;
-
- mddev->reshape_backwards = backwards;
- return len;
+ err = -EINVAL;
+ else
+ mddev->reshape_backwards = backwards;
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_reshape_direction =
@@ -4486,6 +4618,11 @@ static ssize_t
array_size_store(struct mddev *mddev, const char *buf, size_t len)
{
sector_t sectors;
+ int err;
+
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
if (strncmp(buf, "default", 7) == 0) {
if (mddev->pers)
@@ -4496,19 +4633,22 @@ array_size_store(struct mddev *mddev, const char *buf, size_t len)
mddev->external_size = 0;
} else {
if (strict_blocks_to_sectors(buf, §ors) < 0)
- return -EINVAL;
- if (mddev->pers && mddev->pers->size(mddev, 0, 0) < sectors)
- return -E2BIG;
-
- mddev->external_size = 1;
+ err = -EINVAL;
+ else if (mddev->pers && mddev->pers->size(mddev, 0, 0) < sectors)
+ err = -E2BIG;
+ else
+ mddev->external_size = 1;
}
- mddev->array_sectors = sectors;
- if (mddev->pers) {
- set_capacity(mddev->gendisk, mddev->array_sectors);
- revalidate_disk(mddev->gendisk);
+ if (!err) {
+ mddev->array_sectors = sectors;
+ if (mddev->pers) {
+ set_capacity(mddev->gendisk, mddev->array_sectors);
+ revalidate_disk(mddev->gendisk);
+ }
}
- return len;
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry md_array_size =
@@ -4596,13 +4736,7 @@ md_attr_store(struct kobject *kobj, struct attribute *attr,
}
mddev_get(mddev);
spin_unlock(&all_mddevs_lock);
- if (entry->store == new_dev_store)
- flush_workqueue(md_misc_wq);
- rv = mddev_lock(mddev);
- if (!rv) {
- rv = entry->store(mddev, page, length);
- mddev_unlock(mddev);
- }
+ rv = entry->store(mddev, page, length);
mddev_put(mddev);
return rv;
}
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index d5b80174b3b3..aa76865b804b 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -5400,21 +5400,25 @@ EXPORT_SYMBOL(raid5_set_cache_size);
static ssize_t
raid5_store_stripe_cache_size(struct mddev *mddev, const char *page, size_t len)
{
- struct r5conf *conf = mddev->private;
+ struct r5conf *conf;
unsigned long new;
int err;
if (len >= PAGE_SIZE)
return -EINVAL;
- if (!conf)
- return -ENODEV;
-
if (kstrtoul(page, 10, &new))
return -EINVAL;
- err = raid5_set_cache_size(mddev, new);
+ err = mddev_lock(mddev);
if (err)
return err;
- return len;
+ conf = mddev->private;
+ if (!conf)
+ err = -ENODEV;
+ else
+ err = raid5_set_cache_size(mddev, new);
+ mddev_unlock(mddev);
+
+ return err ?: len;
}
static struct md_sysfs_entry
@@ -5438,19 +5442,27 @@ raid5_show_preread_threshold(struct mddev *mddev, char *page)
static ssize_t
raid5_store_preread_threshold(struct mddev *mddev, const char *page, size_t len)
{
- struct r5conf *conf = mddev->private;
+ struct r5conf *conf;
unsigned long new;
+ int err;
+
if (len >= PAGE_SIZE)
return -EINVAL;
- if (!conf)
- return -ENODEV;
-
if (kstrtoul(page, 10, &new))
return -EINVAL;
- if (new > conf->max_nr_stripes)
- return -EINVAL;
- conf->bypass_threshold = new;
- return len;
+
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ conf = mddev->private;
+ if (!conf)
+ err = -ENODEV;
+ else if (new > conf->max_nr_stripes)
+ err = -EINVAL;
+ else
+ conf->bypass_threshold = new;
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry
@@ -5475,29 +5487,35 @@ raid5_show_skip_copy(struct mddev *mddev, char *page)
static ssize_t
raid5_store_skip_copy(struct mddev *mddev, const char *page, size_t len)
{
- struct r5conf *conf = mddev->private;
+ struct r5conf *conf;
unsigned long new;
+ int err;
+
if (len >= PAGE_SIZE)
return -EINVAL;
- if (!conf)
- return -ENODEV;
-
if (kstrtoul(page, 10, &new))
return -EINVAL;
new = !!new;
- if (new == conf->skip_copy)
- return len;
- mddev_suspend(mddev);
- conf->skip_copy = new;
- if (new)
- mddev->queue->backing_dev_info.capabilities |=
- BDI_CAP_STABLE_WRITES;
- else
- mddev->queue->backing_dev_info.capabilities &=
- ~BDI_CAP_STABLE_WRITES;
- mddev_resume(mddev);
- return len;
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ conf = mddev->private;
+ if (!conf)
+ err = -ENODEV;
+ else if (new != conf->skip_copy) {
+ mddev_suspend(mddev);
+ conf->skip_copy = new;
+ if (new)
+ mddev->queue->backing_dev_info.capabilities |=
+ BDI_CAP_STABLE_WRITES;
+ else
+ mddev->queue->backing_dev_info.capabilities &=
+ ~BDI_CAP_STABLE_WRITES;
+ mddev_resume(mddev);
+ }
+ mddev_unlock(mddev);
+ return err ?: len;
}
static struct md_sysfs_entry
@@ -5538,7 +5556,7 @@ static int alloc_thread_groups(struct r5conf *conf, int cnt,
static ssize_t
raid5_store_group_thread_cnt(struct mddev *mddev, const char *page, size_t len)
{
- struct r5conf *conf = mddev->private;
+ struct r5conf *conf;
unsigned long new;
int err;
struct r5worker_group *new_groups, *old_groups;
@@ -5546,41 +5564,41 @@ raid5_store_group_thread_cnt(struct mddev *mddev, const char *page, size_t len)
if (len >= PAGE_SIZE)
return -EINVAL;
- if (!conf)
- return -ENODEV;
-
if (kstrtoul(page, 10, &new))
return -EINVAL;
- if (new == conf->worker_cnt_per_group)
- return len;
-
- mddev_suspend(mddev);
+ err = mddev_lock(mddev);
+ if (err)
+ return err;
+ conf = mddev->private;
+ if (!conf)
+ err = -ENODEV;
+ else if (new != conf->worker_cnt_per_group) {
+ mddev_suspend(mddev);
- old_groups = conf->worker_groups;
- if (old_groups)
- flush_workqueue(raid5_wq);
+ old_groups = conf->worker_groups;
+ if (old_groups)
+ flush_workqueue(raid5_wq);
- err = alloc_thread_groups(conf, new,
- &group_cnt, &worker_cnt_per_group,
- &new_groups);
- if (!err) {
- spin_lock_irq(&conf->device_lock);
- conf->group_cnt = group_cnt;
- conf->worker_cnt_per_group = worker_cnt_per_group;
- conf->worker_groups = new_groups;
- spin_unlock_irq(&conf->device_lock);
+ err = alloc_thread_groups(conf, new,
+ &group_cnt, &worker_cnt_per_group,
+ &new_groups);
+ if (!err) {
+ spin_lock_irq(&conf->device_lock);
+ conf->group_cnt = group_cnt;
+ conf->worker_cnt_per_group = worker_cnt_per_group;
+ conf->worker_groups = new_groups;
+ spin_unlock_irq(&conf->device_lock);
- if (old_groups)
- kfree(old_groups[0].workers);
- kfree(old_groups);
+ if (old_groups)
+ kfree(old_groups[0].workers);
+ kfree(old_groups);
+ }
+ mddev_resume(mddev);
}
+ mddev_unlock(mddev);
- mddev_resume(mddev);
-
- if (err)
- return err;
- return len;
+ return err ?: len;
}
static struct md_sysfs_entry
^ permalink raw reply related
* [md PATCH 24/26] md: move mddev_lock and related to md.h
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
The one which is not inline (mddev_unlock) gets EXPORTed.
This makes the locking available to personality modules so that it
doesn't have to be imposed upon them.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 26 ++------------------------
drivers/md/md.h | 24 ++++++++++++++++++++++++
2 files changed, 26 insertions(+), 24 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index c17de6d0fb42..6517f4ece44e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -590,32 +590,9 @@ static struct mddev *mddev_find(dev_t unit)
goto retry;
}
-static inline int __must_check mddev_lock(struct mddev *mddev)
-{
- return mutex_lock_interruptible(&mddev->reconfig_mutex);
-}
-
-/* Sometimes we need to take the lock in a situation where
- * failure due to interrupts is not acceptable.
- */
-static inline void mddev_lock_nointr(struct mddev *mddev)
-{
- mutex_lock(&mddev->reconfig_mutex);
-}
-
-static inline int mddev_is_locked(struct mddev *mddev)
-{
- return mutex_is_locked(&mddev->reconfig_mutex);
-}
-
-static inline int mddev_trylock(struct mddev *mddev)
-{
- return mutex_trylock(&mddev->reconfig_mutex);
-}
-
static struct attribute_group md_redundancy_group;
-static void mddev_unlock(struct mddev *mddev)
+void mddev_unlock(struct mddev *mddev)
{
if (mddev->to_remove) {
/* These cannot be removed under reconfig_mutex as
@@ -657,6 +634,7 @@ static void mddev_unlock(struct mddev *mddev)
md_wakeup_thread(mddev->thread);
spin_unlock(&pers_lock);
}
+EXPORT_SYMBOL_GPL(mddev_unlock);
static struct md_rdev *find_rdev_nr_rcu(struct mddev *mddev, int nr)
{
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 6bf3faa951ec..14367d919351 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -450,6 +450,30 @@ struct mddev {
void (*sync_super)(struct mddev *mddev, struct md_rdev *rdev);
};
+static inline int __must_check mddev_lock(struct mddev *mddev)
+{
+ return mutex_lock_interruptible(&mddev->reconfig_mutex);
+}
+
+/* Sometimes we need to take the lock in a situation where
+ * failure due to interrupts is not acceptable.
+ */
+static inline void mddev_lock_nointr(struct mddev *mddev)
+{
+ mutex_lock(&mddev->reconfig_mutex);
+}
+
+static inline int mddev_is_locked(struct mddev *mddev)
+{
+ return mutex_is_locked(&mddev->reconfig_mutex);
+}
+
+static inline int mddev_trylock(struct mddev *mddev)
+{
+ return mutex_trylock(&mddev->reconfig_mutex);
+}
+extern void mddev_unlock(struct mddev *mddev);
+
static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev)
{
int faulty = test_bit(Faulty, &rdev->flags);
^ permalink raw reply related
* [md PATCH 22/26] md: minor cleanup in safe_delay_store.
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
There isn't really much room for races with ->safemode_delay.
But as I am trying to clean up any racy code and will soon
be removing reconfig_mutex protection from most _store()
functions:
- only set mddev->safemode_delay once, to ensure no code
can see an intermediate value
- use safemode_timer to call md_safemode_timeout() rather than
calling it directly, to ensure it never races with itself.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index f11aff6bd5e2..e68824d2d21e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3242,11 +3242,13 @@ safe_delay_store(struct mddev *mddev, const char *cbuf, size_t len)
mddev->safemode_delay = 0;
else {
unsigned long old_delay = mddev->safemode_delay;
- mddev->safemode_delay = (msec*HZ)/1000;
- if (mddev->safemode_delay == 0)
- mddev->safemode_delay = 1;
- if (mddev->safemode_delay < old_delay || old_delay == 0)
- md_safemode_timeout((unsigned long)mddev);
+ unsigned long new_delay = (msec*HZ)/1000;
+
+ if (new_delay == 0)
+ new_delay = 1;
+ mddev->safemode_delay = new_delay;
+ if (new_delay < old_delay || old_delay == 0)
+ mod_timer(&mddev->safemode_timer, jiffies+1);
}
return len;
}
^ permalink raw reply related
* [md PATCH 23/26] md: use mddev->lock to protect updates to resync_{min, max}.
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
There are interdependencies between these two sysfs attributes
and whether a resync is currently running.
Rather than depending on reconfig_mutex to ensure no races when
testing these interdependencies are met, use the spinlock.
This will allow the mutex to be remove from protecting this
code in a subsequent patch.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 58 +++++++++++++++++++++++++++++++++++++++++++------------
drivers/md/md.h | 2 ++
2 files changed, 47 insertions(+), 13 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index e68824d2d21e..c17de6d0fb42 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -4269,22 +4269,36 @@ static ssize_t
min_sync_store(struct mddev *mddev, const char *buf, size_t len)
{
unsigned long long min;
+ int err;
+ int chunk;
+
if (kstrtoull(buf, 10, &min))
return -EINVAL;
+
+ spin_lock(&mddev->lock);
+ err = -EINVAL;
if (min > mddev->resync_max)
- return -EINVAL;
+ goto out_unlock;
+
+ err = -EBUSY;
if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
- return -EBUSY;
+ goto out_unlock;
/* Must be a multiple of chunk_size */
- if (mddev->chunk_sectors) {
+ chunk = mddev->chunk_sectors;
+ if (chunk) {
sector_t temp = min;
- if (sector_div(temp, mddev->chunk_sectors))
- return -EINVAL;
+
+ err = -EINVAL;
+ if (sector_div(temp, chunk))
+ goto out_unlock;
}
mddev->resync_min = min;
+ err = 0;
- return len;
+out_unlock:
+ spin_unlock(&mddev->lock);
+ return err ?: len;
}
static struct md_sysfs_entry md_min_sync =
@@ -4302,29 +4316,42 @@ max_sync_show(struct mddev *mddev, char *page)
static ssize_t
max_sync_store(struct mddev *mddev, const char *buf, size_t len)
{
+ int err;
+ spin_lock(&mddev->lock);
if (strncmp(buf, "max", 3) == 0)
mddev->resync_max = MaxSector;
else {
unsigned long long max;
+ int chunk;
+
+ err = -EINVAL;
if (kstrtoull(buf, 10, &max))
- return -EINVAL;
+ goto out_unlock;
if (max < mddev->resync_min)
- return -EINVAL;
+ goto out_unlock;
+
+ err = -EBUSY;
if (max < mddev->resync_max &&
mddev->ro == 0 &&
test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
- return -EBUSY;
+ goto out_unlock;
/* Must be a multiple of chunk_size */
- if (mddev->chunk_sectors) {
+ chunk = mddev->chunk_sectors;
+ if (chunk) {
sector_t temp = max;
- if (sector_div(temp, mddev->chunk_sectors))
- return -EINVAL;
+
+ err = -EINVAL;
+ if (sector_div(temp, chunk))
+ goto out_unlock;
}
mddev->resync_max = max;
}
wake_up(&mddev->recovery_wait);
- return len;
+ err = 0;
+out_unlock:
+ spin_unlock(&mddev->lock);
+ return err ?: len;
}
static struct md_sysfs_entry md_max_sync =
@@ -7585,6 +7612,7 @@ void md_do_sync(struct md_thread *thread)
skip:
set_bit(MD_CHANGE_DEVS, &mddev->flags);
+ spin_lock(&mddev->lock);
if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
/* We completed so min/max setting can be forgotten if used. */
if (test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery))
@@ -7593,6 +7621,8 @@ void md_do_sync(struct md_thread *thread)
} else if (test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery))
mddev->resync_min = mddev->curr_resync_completed;
mddev->curr_resync = 0;
+ spin_unlock(&mddev->lock);
+
wake_up(&resync_wait);
set_bit(MD_RECOVERY_DONE, &mddev->recovery);
md_wakeup_thread(mddev->thread);
@@ -7793,7 +7823,9 @@ void md_check_recovery(struct mddev *mddev)
* any transients in the value of "sync_action".
*/
mddev->curr_resync_completed = 0;
+ spin_lock(&mddev->lock);
set_bit(MD_RECOVERY_RUNNING, &mddev->recovery);
+ spin_unlock(&mddev->lock);
/* Clear some bits that don't mean anything, but
* might be left set
*/
diff --git a/drivers/md/md.h b/drivers/md/md.h
index b4fbd6a63fcf..6bf3faa951ec 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -394,6 +394,8 @@ struct mddev {
* pers (also protected by reconfig_mutex and pending IO).
* clearing ->bitmap
* clearing ->bitmap_info.file
+ * changing ->resync_{min,max}
+ * setting MD_RECOVERY_RUNNING (which interacts with resync_{min,max})
*/
spinlock_t lock;
wait_queue_head_t sb_wait; /* for waiting on superblock updates */
^ permalink raw reply related
* [md PATCH 21/26] md: move GET_BITMAP_FILE ioctl out from mddev_lock.
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
It makes more sense to report bitmap_info->file, rather than
bitmap->file (the later is only available once the array is
active).
With that change, use mddev->lock to protect bitmap_info being
set to NULL, and we can call get_bitmap_file() without taking
the mutex.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 58 ++++++++++++++++++++++++++++++-------------------------
drivers/md/md.h | 1 +
2 files changed, 33 insertions(+), 26 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 1e32f2e93d1d..f11aff6bd5e2 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5292,8 +5292,11 @@ static int do_md_stop(struct mddev *mddev, int mode,
bitmap_destroy(mddev);
if (mddev->bitmap_info.file) {
- fput(mddev->bitmap_info.file);
+ struct file *f = mddev->bitmap_info.file;
+ spin_lock(&mddev->lock);
mddev->bitmap_info.file = NULL;
+ spin_unlock(&mddev->lock);
+ fput(f);
}
mddev->bitmap_info.offset = 0;
@@ -5503,32 +5506,30 @@ static int get_bitmap_file(struct mddev *mddev, void __user * arg)
{
mdu_bitmap_file_t *file = NULL; /* too big for stack allocation */
char *ptr;
- int err = -ENOMEM;
+ int err;
file = kmalloc(sizeof(*file), GFP_NOIO);
-
if (!file)
- goto out;
+ return -ENOMEM;
+ err = 0;
+ spin_lock(&mddev->lock);
/* bitmap disabled, zero the first byte and copy out */
- if (!mddev->bitmap || !mddev->bitmap->storage.file) {
+ if (!mddev->bitmap_info.file)
file->pathname[0] = '\0';
- goto copy_out;
- }
-
- ptr = d_path(&mddev->bitmap->storage.file->f_path,
- file->pathname, sizeof(file->pathname));
- if (IS_ERR(ptr))
- goto out;
-
- memmove(file->pathname, ptr,
- sizeof(file->pathname)-(ptr-file->pathname));
+ else if ((ptr = d_path(&mddev->bitmap_info.file->f_path,
+ file->pathname, sizeof(file->pathname))),
+ IS_ERR(ptr))
+ err = PTR_ERR(ptr);
+ else
+ memmove(file->pathname, ptr,
+ sizeof(file->pathname)-(ptr-file->pathname));
+ spin_unlock(&mddev->lock);
-copy_out:
- err = 0;
- if (copy_to_user(arg, file, sizeof(*file)))
+ if (err == 0 &&
+ copy_to_user(arg, file, sizeof(*file)))
err = -EFAULT;
-out:
+
kfree(file);
return err;
}
@@ -5900,9 +5901,13 @@ static int set_bitmap_file(struct mddev *mddev, int fd)
mddev->pers->quiesce(mddev, 0);
}
if (fd < 0) {
- if (mddev->bitmap_info.file)
- fput(mddev->bitmap_info.file);
- mddev->bitmap_info.file = NULL;
+ struct file *f = mddev->bitmap_info.file;
+ if (f) {
+ spin_lock(&mddev->lock);
+ mddev->bitmap_info.file = NULL;
+ spin_unlock(&mddev->lock);
+ fput(f);
+ }
}
return err;
@@ -6315,6 +6320,11 @@ static int md_ioctl(struct block_device *bdev, fmode_t mode,
case SET_DISK_FAULTY:
err = set_disk_faulty(mddev, new_decode_dev(arg));
goto out;
+
+ case GET_BITMAP_FILE:
+ err = get_bitmap_file(mddev, argp);
+ goto out;
+
}
if (cmd == ADD_NEW_DISK)
@@ -6406,10 +6416,6 @@ static int md_ioctl(struct block_device *bdev, fmode_t mode,
* Commands even a read-only array can execute:
*/
switch (cmd) {
- case GET_BITMAP_FILE:
- err = get_bitmap_file(mddev, argp);
- goto unlock;
-
case RESTART_ARRAY_RW:
err = restart_array(mddev);
goto unlock;
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 8770308a8052..b4fbd6a63fcf 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -393,6 +393,7 @@ struct mddev {
* in_sync - and related safemode and MD_CHANGE changes
* pers (also protected by reconfig_mutex and pending IO).
* clearing ->bitmap
+ * clearing ->bitmap_info.file
*/
spinlock_t lock;
wait_queue_head_t sb_wait; /* for waiting on superblock updates */
^ permalink raw reply related
* [md PATCH 20/26] md: tidy up set_bitmap_file
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
1/ delay setting mddev->bitmap_info.file until 'f' looks
usable, so we don't have to unset it.
2/ Don't allow bitmap file to be set if bitmap_info.file
is already set.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 01793f24f8de..1e32f2e93d1d 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5851,22 +5851,24 @@ static int set_bitmap_file(struct mddev *mddev, int fd)
if (fd >= 0) {
struct inode *inode;
- if (mddev->bitmap)
+ struct file *f;
+
+ if (mddev->bitmap || mddev->bitmap_info.file)
return -EEXIST; /* cannot add when bitmap is present */
- mddev->bitmap_info.file = fget(fd);
+ f = fget(fd);
- if (mddev->bitmap_info.file == NULL) {
+ if (f == NULL) {
printk(KERN_ERR "%s: error: failed to get bitmap file\n",
mdname(mddev));
return -EBADF;
}
- inode = mddev->bitmap_info.file->f_mapping->host;
+ inode = f->f_mapping->host;
if (!S_ISREG(inode->i_mode)) {
printk(KERN_ERR "%s: error: bitmap file must be a regular file\n",
mdname(mddev));
err = -EBADF;
- } else if (!(mddev->bitmap_info.file->f_mode & FMODE_WRITE)) {
+ } else if (!(f->f_mode & FMODE_WRITE)) {
printk(KERN_ERR "%s: error: bitmap file must open for write\n",
mdname(mddev));
err = -EBADF;
@@ -5876,10 +5878,10 @@ static int set_bitmap_file(struct mddev *mddev, int fd)
err = -EBUSY;
}
if (err) {
- fput(mddev->bitmap_info.file);
- mddev->bitmap_info.file = NULL;
+ fput(f);
return err;
}
+ mddev->bitmap_info.file = f;
mddev->bitmap_info.offset = 0; /* file overrides offset */
} else if (mddev->bitmap == NULL)
return -ENOENT; /* cannot remove what isn't there */
^ permalink raw reply related
* [md PATCH 19/26] md: remove unnecessary 'buf' from get_bitmap_file.
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
'buf' is only used because d_path fills from the end of the
buffer instead of from the start.
We don't need a separate buf to handle that, we just need to use
memmove() to move the string to the start.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index cbbac7b7238a..01793f24f8de 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5502,7 +5502,7 @@ static int get_array_info(struct mddev *mddev, void __user *arg)
static int get_bitmap_file(struct mddev *mddev, void __user * arg)
{
mdu_bitmap_file_t *file = NULL; /* too big for stack allocation */
- char *ptr, *buf = NULL;
+ char *ptr;
int err = -ENOMEM;
file = kmalloc(sizeof(*file), GFP_NOIO);
@@ -5516,23 +5516,19 @@ static int get_bitmap_file(struct mddev *mddev, void __user * arg)
goto copy_out;
}
- buf = kmalloc(sizeof(file->pathname), GFP_KERNEL);
- if (!buf)
- goto out;
-
ptr = d_path(&mddev->bitmap->storage.file->f_path,
- buf, sizeof(file->pathname));
+ file->pathname, sizeof(file->pathname));
if (IS_ERR(ptr))
goto out;
- strcpy(file->pathname, ptr);
+ memmove(file->pathname, ptr,
+ sizeof(file->pathname)-(ptr-file->pathname));
copy_out:
err = 0;
if (copy_to_user(arg, file, sizeof(*file)))
err = -EFAULT;
out:
- kfree(buf);
kfree(file);
return err;
}
^ permalink raw reply related
* [md PATCH 18/26] md: remove mddev_lock from rdev_attr_show()
From: NeilBrown @ 2015-02-03 21:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150203213948.3448.80258.stgit@notabene.brown>
No rdev attributes need locking for 'show', though
state_show() might benefit from ensuring it sees a
consistent set of flags.
None even use rdev->mddev, so testing for it isn't really
needed and it certainly doesn't need to be held constant.
So improve state_show() and remove the locking.
Signed-off-by: NeilBrown <neilb@suse.de>
---
drivers/md/md.c | 36 ++++++++++++++----------------------
1 file changed, 14 insertions(+), 22 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index b7f48238a92d..cbbac7b7238a 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -2419,40 +2419,41 @@ state_show(struct md_rdev *rdev, char *page)
{
char *sep = "";
size_t len = 0;
+ unsigned long flags = ACCESS_ONCE(rdev->flags);
- if (test_bit(Faulty, &rdev->flags) ||
+ if (test_bit(Faulty, &flags) ||
rdev->badblocks.unacked_exist) {
len+= sprintf(page+len, "%sfaulty",sep);
sep = ",";
}
- if (test_bit(In_sync, &rdev->flags)) {
+ if (test_bit(In_sync, &flags)) {
len += sprintf(page+len, "%sin_sync",sep);
sep = ",";
}
- if (test_bit(WriteMostly, &rdev->flags)) {
+ if (test_bit(WriteMostly, &flags)) {
len += sprintf(page+len, "%swrite_mostly",sep);
sep = ",";
}
- if (test_bit(Blocked, &rdev->flags) ||
+ if (test_bit(Blocked, &flags) ||
(rdev->badblocks.unacked_exist
- && !test_bit(Faulty, &rdev->flags))) {
+ && !test_bit(Faulty, &flags))) {
len += sprintf(page+len, "%sblocked", sep);
sep = ",";
}
- if (!test_bit(Faulty, &rdev->flags) &&
- !test_bit(In_sync, &rdev->flags)) {
+ if (!test_bit(Faulty, &flags) &&
+ !test_bit(In_sync, &flags)) {
len += sprintf(page+len, "%sspare", sep);
sep = ",";
}
- if (test_bit(WriteErrorSeen, &rdev->flags)) {
+ if (test_bit(WriteErrorSeen, &flags)) {
len += sprintf(page+len, "%swrite_error", sep);
sep = ",";
}
- if (test_bit(WantReplacement, &rdev->flags)) {
+ if (test_bit(WantReplacement, &flags)) {
len += sprintf(page+len, "%swant_replacement", sep);
sep = ",";
}
- if (test_bit(Replacement, &rdev->flags)) {
+ if (test_bit(Replacement, &flags)) {
len += sprintf(page+len, "%sreplacement", sep);
sep = ",";
}
@@ -2965,21 +2966,12 @@ rdev_attr_show(struct kobject *kobj, struct attribute *attr, char *page)
{
struct rdev_sysfs_entry *entry = container_of(attr, struct rdev_sysfs_entry, attr);
struct md_rdev *rdev = container_of(kobj, struct md_rdev, kobj);
- struct mddev *mddev = rdev->mddev;
- ssize_t rv;
if (!entry->show)
return -EIO;
-
- rv = mddev ? mddev_lock(mddev) : -EBUSY;
- if (!rv) {
- if (rdev->mddev == NULL)
- rv = -EBUSY;
- else
- rv = entry->show(rdev, page);
- mddev_unlock(mddev);
- }
- return rv;
+ if (!rdev->mddev)
+ return -EBUSY;
+ return entry->show(rdev, page);
}
static ssize_t
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox