* Re: mdadm failed to remove internal bitmap
From: gary @ 2015-02-16 6:42 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20150216143419.553cf93e@notabene.brown>
Hi Neil,
Please check the followings.
> What kernel are you running?
linux48:~ # uname -r
3.12.32-33-default
And 3.12.28-4-default kernel is ok.
> Please use "strace" on mdadm in a case where it fails, and post the result.
linux48:~ # strace mdadm --grow --bitmap=none /dev/md127
execve("/sbin/mdadm", ["mdadm", "--grow", "--bitmap=none",
"/dev/md127"], [/* 58 vars */]) = 0
brk(0) = 0xfe8000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fcab07ad000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=93919, ...}) = 0
mmap(NULL, 93919, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fcab0796000
close(3) = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\34\2\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1978611, ...}) = 0
mmap(NULL, 3832352, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7fcab01e6000
mprotect(0x7fcab0384000, 2097152, PROT_NONE) = 0
mmap(0x7fcab0584000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19e000) = 0x7fcab0584000
mmap(0x7fcab058a000, 14880, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fcab058a000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fcab0795000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fcab0794000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fcab0793000
arch_prctl(ARCH_SET_FS, 0x7fcab0794700) = 0
mprotect(0x7fcab0584000, 16384, PROT_READ) = 0
mprotect(0x677000, 4096, PROT_READ) = 0
mprotect(0x7fcab07ae000, 4096, PROT_READ) = 0
munmap(0x7fcab0796000, 93919) = 0
getpid() = 6000
brk(0) = 0xfe8000
brk(0x1009000) = 0x1009000
open("/dev/md127", O_RDWR) = 3
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(9, 127), ...}) = 0
ioctl(3, RAID_VERSION, 0x7fff2e10d160) = 0
open("/etc/mdadm.conf", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/etc/mdadm/mdadm.conf", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/etc/mdadm.conf.d", O_RDONLY) = -1 ENOENT (No such file or
directory)
uname({sys="Linux", node="linux48", ...}) = 0
geteuid() = 0
fstat(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(9, 127), ...}) = 0
ioctl(3, RAID_VERSION, 0x7fff2e10c060) = 0
ioctl(3, GET_BITMAP_FILE, 0x7fff2e10c1f0) = 0
ioctl(3, GET_ARRAY_INFO, 0x7fff2e10c1a0) = 0
ioctl(3, SET_ARRAY_INFO, 0x7fff2e10c1a0) = -1 EINVAL (Invalid argument)
write(2, "mdadm: failed to remove internal"..., 41mdadm: failed to
remove internal bitmap.
) = 41
exit_group(1) = ?
+++ exited with 1 +++
Thanks,
gary
^ permalink raw reply
* Re: [md PATCH] md/raid1: round up to bdev_logical_block_size in narrow_write_error
From: NeilBrown @ 2015-02-16 3:54 UTC (permalink / raw)
To: Nate Dailey; +Cc: linux-raid
In-Reply-To: <54DCDC91.6010808@stratus.com>
[-- Attachment #1: Type: text/plain, Size: 1668 bytes --]
On Thu, 12 Feb 2015 12:02:09 -0500 Nate Dailey <nate.dailey@stratus.com>
wrote:
> This modifies raid1's narrow_write_error to round up block_sectors to the
> device's logical block size.
>
> This prevents sd complaining about "Bad block number requested" for non-512-byte
> sector disks.
>
> Signed-off-by: Nate Dailey <nate.dailey@stratus.com>
> ---
>
> diff -Nupr a/drivers/md/raid1.c b/drivers/md/raid1.c
> --- a/drivers/md/raid1.c 2015-02-10 15:29:02.000000000 -0500
> +++ b/drivers/md/raid1.c 2015-02-10 15:29:45.000000000 -0500
> @@ -2206,7 +2206,8 @@ static int narrow_write_error(struct r1b
> if (rdev->badblocks.shift < 0)
> return 0;
>
> - block_sectors = 1 << rdev->badblocks.shift;
> + block_sectors = roundup(1 << rdev->badblocks.shift,
> + bdev_logical_block_size(rdev->bdev) >> 9);
> sector = r1_bio->sector;
> sectors = ((sector + block_sectors)
> & ~(sector_t)(block_sectors - 1))
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Thanks. I've applied this patch and a similar one for RAID10.
This patch had spaces where it should have had tabs (and had no space at all
on one line which should have had a space).
I've fixed all that up, but if you find yourself submitting more patches in
future it would be worth working out how to convince your mailer to send the
patches cleanly with no TAB->space conversions.
Thanks,
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: mdadm failed to remove internal bitmap
From: NeilBrown @ 2015-02-16 3:34 UTC (permalink / raw)
To: gary; +Cc: linux-raid
In-Reply-To: <54DDC806.5010803@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]
On Fri, 13 Feb 2015 17:46:46 +0800 gary <gary.mdjiang@gmail.com> wrote:
> Hi,
>
> I used v3.3.1 mdadm to do some test for bitmap, but when switch bitmap from
> internal to none, the output shows fail info about remove internal
> bitmap, is it
> just a warning? Since the bitmap seems to be cleared, and it doesn't
> show with
> v3.2.6 mdadm with the same steps.
>
> linux:~ # mdadm --create md0 --raid-devices=2 --level=mirror
> --assume-clean /dev/vdb /dev/vdc
> mdadm: Note: this array has metadata at the start and
> may not be suitable as a boot device. If you plan to
> store '/boot' on this device please ensure that
> your boot-loader understands md/v1.x metadata, or use
> --metadata=0.90
> Continue creating array? y
> mdadm: Defaulting to version 1.2 metadata
> mdadm: array /dev/md/md0 started.
> linux:~ # cat /proc/mdstat
> Personalities : [raid1]
> md127 : active raid1 vdc[1] vdb[0]
> 523712 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
> linux:~ # mdadm --grow --bitmap=internal /dev/md127
> linux:~ # cat /proc/mdstat
> Personalities : [raid1]
> md127 : active raid1 vdc[1] vdb[0]
> 523712 blocks super 1.2 [2/2] [UU]
> bitmap: 1/1 pages [4KB], 65536KB chunk
>
> unused devices: <none>
> linux:~ # mdadm --grow --bitmap=none /dev/md127
> mdadm: failed to remove internal bitmap.
> linux:~ # cat /proc/mdstat
> Personalities : [raid1]
> md127 : active raid1 vdc[1] vdb[0]
> 523712 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
>
I cannot reproduce this.
What kernel are you running?
Please use "strace" on mdadm in a case where it fails, and post the result.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: re-add POLICY
From: NeilBrown @ 2015-02-16 3:28 UTC (permalink / raw)
To: Chris; +Cc: linux-raid
In-Reply-To: <loom.20150214T222329-324@post.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]
On Sat, 14 Feb 2015 21:59:34 +0000 (UTC) Chris <email.bug@arcor.de> wrote:
>
> Hi all,
>
> I'd like mdadm to automatically attempt to re-sync raid members after they
> where temporarily removed from the system.
>
> I would have thought "POLICY domain=default action=re-add" should allow this,
> and found a prior post that also seemed to want/test that behaviour.
> But as I understand the answer given there
> http://permalink.gmane.org/gmane.linux.raid/47516
> mdadm is expected to exit with an error (not re-add) upon plugging the
> device back in?
>
> with:
> mdadm: can only add /dev/loop2 to /dev/md0 as a spare, and force-spare is
> not set.
> mdadm: failed to add /dev/loop2 to existing array /dev/md0: Invalid argument.
>
> For one, I don't understand what the error messages is trying to tell me, about
> an invalid argument that was never supplied to --incremental?
>
> But more importantly, how can priorly diconnected devices (marked failed
> with non-future event count) get re-synced automatically when they are
> plugged in again?
> (avoiding manual mdadm /dev/mdX --add /dev/sdYZ hassle)
>
Does your array have a write-intent bitmap configured?
If it does, then "POLICY action=re-add" really should work.
If it doesn't, then maybe you need "POLICY action=spare".
This isn't the default, because depending on exactly how/why the device
failed, it may not be safe to treat it as a spare.
If the above does not help, please report:
- kernel version
- mdadm version
- "mdadm --examine" output of at least one good drive and one failed drive.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: re-add POLICY: conflict detection?
From: Chris @ 2015-02-15 19:03 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <loom.20150214T222329-324@post.gmane.org>
thinking about the "invalid argument" message...
with "action=re-add":
# mdadm --incremental /dev/loop2
mdadm: can only add /dev/loop2 to /dev/md0 as a spare, and force-spare is
not set.
mdadm: failed to add /dev/loop2 to existing array /dev/md0: Invalid argument.
My guess is that mdadm may not be adding back the failed disk, because it is
unsure wether it may have run separately, and may have newer data on it?
I thought it may be possible to clearly distinguish between clean re-adds
and conflicts, by doing something like this:
* If a member fails (or is missing when starting degraded) write this info
into some failed_at_event_count field belonging to the failed member in the
superblock of every remaining raid member device in the array.
Now, if an array part that got unplugged reappears and still has the event
count that matches the failed_at_event_count that was recorded in the
superblocks of the still running disks, and the reappearing part's
superblock has no failed_at_event_count values for any member of the running
array, the reappearing part is ok to be automatically re-synced.
But if the reappearing disk claims a member of the already running array has
failed, or it reappeared with a different event count than its
faile_at_event_count field in the superblocks of the running array says, a
conflict has arisen and a sync may only be done with manual --force.
Cheers,
Chris
^ permalink raw reply
* Re: update mdadm version to 3.3.2 in Centos 6.4
From: alpha lin @ 2015-02-15 18:48 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid
In-Reply-To: <wrfjtwz3t9na.fsf@redhat.com>
Hi Jes,
I have my Oracle database running on Centos 6.4. There will be
great if I can stay in Centos 6.4 rather than upgrade to Centos 6.6.
Alpha.
On Tue, Feb 3, 2015 at 9:19 PM, Jes Sorensen <Jes.Sorensen@redhat.com> wrote:
> alpha lin <lin.alpha@gmail.com> writes:
>> Hi All,
>>
>> I would like to know the possibility of update the mdadm package
>> from 3.2.5 to 3.3.2 in Redhat Centos 6.4 x64 system.
>> The problem I have is I found that I can create a raid volume
>> without any problem. But when reboot the system. The md volume become
>> inactive.
>> Below is my step:
>> 1. Check out mdadm package from the github.
>> 2. unzip the mdadm package and execute "make" and "make install"
>> 3. reboot.
>> 4. after system reboot, run mdadm to create a imsm RAID Volume.
>> [root@localhost rules.d]# mdadm --version
>> mdadm - v3.3.2 - 21st August 2014
>
> Any reason why you haven't upgraded to centos 6.6? 6.4 is ancient, and
> 6.6 will at least come with mdadm-3.3.
>
> Jes
>
>
^ permalink raw reply
* re-add POLICY
From: Chris @ 2015-02-14 21:59 UTC (permalink / raw)
To: linux-raid
Hi all,
I'd like mdadm to automatically attempt to re-sync raid members after they
where temporarily removed from the system.
I would have thought "POLICY domain=default action=re-add" should allow this,
and found a prior post that also seemed to want/test that behaviour.
But as I understand the answer given there
http://permalink.gmane.org/gmane.linux.raid/47516
mdadm is expected to exit with an error (not re-add) upon plugging the
device back in?
with:
mdadm: can only add /dev/loop2 to /dev/md0 as a spare, and force-spare is
not set.
mdadm: failed to add /dev/loop2 to existing array /dev/md0: Invalid argument.
For one, I don't understand what the error messages is trying to tell me, about
an invalid argument that was never supplied to --incremental?
But more importantly, how can priorly diconnected devices (marked failed
with non-future event count) get re-synced automatically when they are
plugged in again?
(avoiding manual mdadm /dev/mdX --add /dev/sdYZ hassle)
Cheers,
Chris
^ permalink raw reply
* [PATCH 1/1] Use dev_t for devnm2devid and devid2devnm
From: Mike Lovell @ 2015-02-14 1:08 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, mlovell
In-Reply-To: <1423876088-7169-1-git-send-email-mlovell@bluehost.com>
Commit 4dd2df0966ec added a trip through makedev(), major(), and minor() for
device major and minor numbers. This would cause mdadm to fail in operating
on a device with a minor number bigger than (2^19)-1 due to it changing
from dev_t to a signed int and back.
Where this was found as a problem was when a array was created with a device
specified as a name like /dev/md/raidname and there were already 128 arrays
on the system. In this case, mdadm would chose 1048575 ((2^20)-1) for the
array and minor number. This would cause the major and minor number to become
negative when generated from devnm2devid() and passed to major() and minor()
in open_dev_excl(). open_dev_excl() would then call dev_open() which would
detect the negative minor number and call open() on the *char containing the
major:minor pair which isn't a valid file.
Signed-off-by: Mike Lovell <mlovell@bluehost.com>
---
Detail.c | 4 ++--
Grow.c | 2 +-
lib.c | 2 +-
mapfile.c | 2 +-
mdadm.h | 4 ++--
mdopen.c | 4 ++--
tests/00largemdnumber | 7 +++++++
util.c | 6 +++---
8 files changed, 19 insertions(+), 12 deletions(-)
create mode 100644 tests/00largemdnumber
diff --git a/Detail.c b/Detail.c
index dd72ede..18fa5df 100644
--- a/Detail.c
+++ b/Detail.c
@@ -130,7 +130,7 @@ int Detail(char *dev, struct context *c)
/* This is a subarray of some container.
* We want the name of the container, and the member
*/
- int devid = devnm2devid(st->container_devnm);
+ dev_t devid = devnm2devid(st->container_devnm);
int cfd, err;
member = subarray;
@@ -573,7 +573,7 @@ This is pretty boring
char path[200];
char vbuf[1024];
int nlen = strlen(sra->sys_name);
- int devid;
+ dev_t devid;
if (de->d_name[0] == '.')
continue;
sprintf(path, "/sys/block/%s/md/metadata_version",
diff --git a/Grow.c b/Grow.c
index b78d063..e8f6a2a 100644
--- a/Grow.c
+++ b/Grow.c
@@ -3467,7 +3467,7 @@ int reshape_container(char *container, char *devname,
int fd;
struct mdstat_ent *mdstat;
char *adev;
- int devid;
+ dev_t devid;
sysfs_free(cc);
diff --git a/lib.c b/lib.c
index 6808f62..e9f7018 100644
--- a/lib.c
+++ b/lib.c
@@ -84,7 +84,7 @@ char *devid2kname(int devid)
return NULL;
}
-char *devid2devnm(int devid)
+char *devid2devnm(dev_t devid)
{
char path[30];
char link[200];
diff --git a/mapfile.c b/mapfile.c
index 41599df..9135450 100644
--- a/mapfile.c
+++ b/mapfile.c
@@ -374,7 +374,7 @@ void RebuildMap(void)
char dn[30];
int dfd;
int ok;
- int devid;
+ dev_t devid;
struct supertype *st;
char *subarray = NULL;
char *path;
diff --git a/mdadm.h b/mdadm.h
index 141f963..13279a5 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1372,8 +1372,8 @@ extern char *find_free_devnm(int use_partitions);
extern void put_md_name(char *name);
extern char *devid2kname(int devid);
-extern char *devid2devnm(int devid);
-extern int devnm2devid(char *devnm);
+extern char *devid2devnm(dev_t devid);
+extern dev_t devnm2devid(char *devnm);
extern char *get_md_name(char *devnm);
extern char DefaultConfFile[];
diff --git a/mdopen.c b/mdopen.c
index 28410f4..e71d758 100644
--- a/mdopen.c
+++ b/mdopen.c
@@ -348,7 +348,7 @@ int create_mddev(char *dev, char *name, int autof, int trustworthy,
if (lstat(devname, &stb) == 0) {
/* Must be the correct device, else error */
if ((stb.st_mode&S_IFMT) != S_IFBLK ||
- stb.st_rdev != (dev_t)devnm2devid(devnm)) {
+ stb.st_rdev != devnm2devid(devnm)) {
pr_err("%s exists but looks wrong, please fix\n",
devname);
return -1;
@@ -452,7 +452,7 @@ char *find_free_devnm(int use_partitions)
if (!use_udev()) {
/* make sure it is new to /dev too, at least as a
* non-standard */
- int devid = devnm2devid(devnm);
+ dev_t devid = devnm2devid(devnm);
if (devid) {
char *dn = map_dev(major(devid),
minor(devid), 0);
diff --git a/tests/00largemdnumber b/tests/00largemdnumber
new file mode 100644
index 0000000..432e5f4
--- /dev/null
+++ b/tests/00largemdnumber
@@ -0,0 +1,7 @@
+
+# create a simple linear with a large device number
+
+mdadm -CR /dev/md1048575 -l linear -n3 $dev0 $dev1 $dev2
+check linear
+testdev /dev/md1048575 3 $mdsize2_l 1
+mdadm -S /dev/md1048575
diff --git a/util.c b/util.c
index 6f1e2b1..a11f995 100644
--- a/util.c
+++ b/util.c
@@ -779,7 +779,7 @@ int get_data_disks(int level, int layout, int raid_disks)
return data_disks;
}
-int devnm2devid(char *devnm)
+dev_t devnm2devid(char *devnm)
{
/* First look in /sys/block/$DEVNM/dev for %d:%d
* If that fails, try parsing out a number
@@ -916,7 +916,7 @@ int dev_open(char *dev, int flags)
int open_dev_flags(char *devnm, int flags)
{
- int devid;
+ dev_t devid;
char buf[20];
devid = devnm2devid(devnm);
@@ -934,7 +934,7 @@ int open_dev_excl(char *devnm)
char buf[20];
int i;
int flags = O_RDWR;
- int devid = devnm2devid(devnm);
+ dev_t devid = devnm2devid(devnm);
long delay = 1000;
sprintf(buf, "%d:%d", major(devid), minor(devid));
--
1.9.1
^ permalink raw reply related
* [PATCH 0/1] Bug fix for mdadm 3.3 with large md device numbers
From: Mike Lovell @ 2015-02-14 1:08 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, mlovell
A co-worker discovered a situation were new arrays were not being created on
a system. Invoking mdadm similar to 'mdadm --create /dev/md/array-name ...'
would fail where 'mdadm --create /dev/md999 ...' would not. This system had
previously created 128 arrays successfully. He was able to work around the
problem by reverting from mdadm 3.3 to 3.2. The error reported from mdadm
was "mdadm: unexpected failure opening /dev/md1048575."
Some digging with strace showed that mdadm would try to call
open("/sys/block/md1048575/dev") which would fail and then would fail on a
call to open("-4087:-1"). I tested 'mdadm --create /dev/md1048575 ...' on an
empty test VM which would fail as well.
The problem was traced to the md device number being used to create a
major:minor pair which would be passed to makedev(). The result, which is a
dev_t or u32, was then being used as a signed int before being passed to
major() and minor() and then into a string as signed ints. This meant that
the major:minor string had negative numbers in it causing dev_open() to not
recognize it as a valid pair and just calling open() on the string.
The large number was generated because the problem system already had 128
arrays on it. This caused find_free_devnm() to loop from 0 to (1<<20)-1, or
1048575. Triggering the bug is done by specifying a md device number larger
than (1<<19)-1 or by creating a md array by name on a system with 128 already
configured arrays.
Originally, I was going to modify find_free_devnm to loop to (1<<19)-1 but,
since 3.2 works with the larger numbers, I decided to change the signed int
use around devnm2devid and devid2devnm. This patch has been tested against
a number of the tests that weren't already failing on my test system and
didn't cause any more tests to fail. I didn't test all but got the basics.
I am new to the mdadm source and not normally a C developer so this may not
be the best way to fix this but it seems to be working.
^ permalink raw reply
* [PATCH 3/3] md bitmap: export bitmap_destroy() to support dm-raid down takover to raid0
From: heinzm @ 2015-02-13 18:48 UTC (permalink / raw)
To: linux-raid; +Cc: Heinz Mauelshagen
From: Heinz Mauelshagen <heinzm@redhat.com>
This patch exports symbol bitmap_destroy to allow dm-raid to remove
bitmaps when performing a down takeover to md raid0.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Tested-by: Heinz Mauelshagen <heinzm@redhat.com>
---
drivers/md/bitmap.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 3a57679..b484d15 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -1631,6 +1631,7 @@ void bitmap_destroy(struct mddev *mddev)
bitmap_free(bitmap);
}
+EXPORT_SYMBOL_GPL(bitmap_destroy);
/*
* initialize the bitmap structure
--
2.1.0
^ permalink raw reply related
* [PATCH 2/3] md raid0: access mddev->queue (request queue member) conditionally because it is not set when accessed from dm-raid
From: heinzm @ 2015-02-13 18:48 UTC (permalink / raw)
To: linux-raid; +Cc: Heinz Mauelshagen
From: Heinz Mauelshagen <heinzm@redhat.com>
The patch makes 3 references to mddev->queue in the raid0 personality
conditional in order to allow for it to be accessed from dm-raid.
Mandatory, because md instances underneath dm-raid don't manage
a request queue of their own which'd lead to oopses without the patch.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Tested-by: Heinz Mauelshagen <heinzm@redhat.com>
---
drivers/md/raid0.c | 48 +++++++++++++++++++++++++++---------------------
1 file changed, 27 insertions(+), 21 deletions(-)
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index a13f738..d2e037d 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -271,14 +271,16 @@ static int create_strip_zones(struct mddev *mddev, struct r0conf **private_conf)
goto abort;
}
- blk_queue_io_min(mddev->queue, mddev->chunk_sectors << 9);
- blk_queue_io_opt(mddev->queue,
- (mddev->chunk_sectors << 9) * mddev->raid_disks);
-
- if (!discard_supported)
- queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
- else
- queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
+ if (mddev->queue) {
+ blk_queue_io_min(mddev->queue, mddev->chunk_sectors << 9);
+ blk_queue_io_opt(mddev->queue,
+ (mddev->chunk_sectors << 9) * mddev->raid_disks);
+
+ if (!discard_supported)
+ queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
+ else
+ queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
+ }
pr_debug("md/raid0:%s: done.\n", mdname(mddev));
*private_conf = conf;
@@ -429,9 +431,12 @@ static int raid0_run(struct mddev *mddev)
}
if (md_check_no_bitmap(mddev))
return -EINVAL;
- blk_queue_max_hw_sectors(mddev->queue, mddev->chunk_sectors);
- blk_queue_max_write_same_sectors(mddev->queue, mddev->chunk_sectors);
- blk_queue_max_discard_sectors(mddev->queue, mddev->chunk_sectors);
+
+ if (mddev->queue) {
+ blk_queue_max_hw_sectors(mddev->queue, mddev->chunk_sectors);
+ blk_queue_max_write_same_sectors(mddev->queue, mddev->chunk_sectors);
+ blk_queue_max_discard_sectors(mddev->queue, mddev->chunk_sectors);
+ }
/* if private is not null, we are here after takeover */
if (mddev->private == NULL) {
@@ -448,16 +453,17 @@ static int raid0_run(struct mddev *mddev)
printk(KERN_INFO "md/raid0:%s: md_size is %llu sectors.\n",
mdname(mddev),
(unsigned long long)mddev->array_sectors);
- /* calculate the max read-ahead size.
- * For read-ahead of large files to be effective, we need to
- * readahead at least twice a whole stripe. i.e. number of devices
- * multiplied by chunk size times 2.
- * If an individual device has an ra_pages greater than the
- * chunk size, then we will not drive that device as hard as it
- * wants. We consider this a configuration error: a larger
- * chunksize should be used in that case.
- */
- {
+
+ if (mddev->queue) {
+ /* calculate the max read-ahead size.
+ * For read-ahead of large files to be effective, we need to
+ * readahead at least twice a whole stripe. i.e. number of devices
+ * multiplied by chunk size times 2.
+ * If an individual device has an ra_pages greater than the
+ * chunk size, then we will not drive that device as hard as it
+ * wants. We consider this a configuration error: a larger
+ * chunksize should be used in that case.
+ */
int stripe = mddev->raid_disks *
(mddev->chunk_sectors << 9) / PAGE_SIZE;
if (mddev->queue->backing_dev_info.ra_pages < 2* stripe)
--
2.1.0
^ permalink raw reply related
* [PATCH 1/3] md core: add 2 API functions for takeover and resize to support dm-raid
From: heinzm @ 2015-02-13 18:48 UTC (permalink / raw)
To: linux-raid; +Cc: Heinz Mauelshagen
From: Heinz Mauelshagen <heinzm@redhat.com>
These 2 added external functions allow the device mapper raid target (dm-raid)
to access the md raid takeover and resize funtionality;
reshape API extensions are not needed in lieu of the existing md personality ones.
The patch makes a reference to mddev->queue conditional as well, because
md instances underneath dm-raid don't manage a request queue of their own.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Tested-by: Heinz Mauelshagen <heinzm@redhat.com>
---
drivers/md/md.c | 39 ++++++++++++++++++++++++++++++---------
drivers/md/md.h | 3 +++
2 files changed, 33 insertions(+), 9 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index c8d2bac..fb9907c 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -3440,7 +3440,8 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
mddev->in_sync = 1;
del_timer_sync(&mddev->safemode_timer);
}
- blk_set_stacking_limits(&mddev->queue->limits);
+ if (mddev->queue)
+ blk_set_stacking_limits(&mddev->queue->limits);
pers->run(mddev);
set_bit(MD_CHANGE_DEVS, &mddev->flags);
mddev_resume(mddev);
@@ -3454,6 +3455,15 @@ out_unlock:
return rv;
}
+/* API to expose level_store() to dm-raid target */
+int md_takeover(struct mddev *mddev, const char *buf)
+{
+ ssize_t r = level_store(mddev, buf, strlen(buf));
+
+ return r < 0 ? (int) r : 0;
+}
+EXPORT_SYMBOL_GPL(md_takeover);
+
static struct md_sysfs_entry md_level =
__ATTR(level, S_IRUGO|S_IWUSR, level_show, level_store);
@@ -3987,18 +3997,15 @@ size_show(struct mddev *mddev, char *page)
static int update_size(struct mddev *mddev, sector_t num_sectors);
-static ssize_t
-size_store(struct mddev *mddev, const char *buf, size_t len)
+/* API to expose size_store() to dm-raid target */
+int md_resize(struct mddev *mddev, sector_t sectors)
{
+ int err;
+
/* If array is inactive, we can reduce the component size, but
* not increase it (except from 0).
* If array is active, we can try an on-line resize
*/
- sector_t sectors;
- int err = strict_blocks_to_sectors(buf, §ors);
-
- if (err < 0)
- return err;
err = mddev_lock(mddev);
if (err)
return err;
@@ -4013,7 +4020,21 @@ size_store(struct mddev *mddev, const char *buf, size_t len)
err = -ENOSPC;
}
mddev_unlock(mddev);
- return err ? err : len;
+ return err;
+}
+EXPORT_SYMBOL_GPL(md_resize);
+
+/* Compatibility wrapper around md_resize() to keep md internal inbterface */
+static ssize_t
+size_store(struct mddev *mddev, const char *buf, size_t len)
+{
+ sector_t dev_sectors;
+ int err = strict_blocks_to_sectors(buf, &dev_sectors);
+
+ if (!err)
+ err = md_resize(mddev, dev_sectors);
+
+ return err ? (ssize_t) err : len;
}
static struct md_sysfs_entry md_size =
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 318ca8f..892a28a 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -646,6 +646,9 @@ extern void md_stop_writes(struct mddev *mddev);
extern int md_rdev_init(struct md_rdev *rdev);
extern void md_rdev_clear(struct md_rdev *rdev);
+extern int md_resize(struct mddev *mddev, sector_t dev_sectors);
+extern int md_takeover(struct mddev *mddev, const char *buf);
+
extern void mddev_suspend(struct mddev *mddev);
extern void mddev_resume(struct mddev *mddev);
extern struct bio *bio_clone_mddev(struct bio *bio, gfp_t gfp_mask,
--
2.1.0
^ permalink raw reply related
* [PATCH 0/3] md raid: enhancements to support the device mapper dm-raid target
From: heinzm @ 2015-02-13 18:47 UTC (permalink / raw)
To: linux-raid; +Cc: Heinz Mauelshagen
From: Heinz Mauelshagen <heinzm@redhat.com>
I'm enhancing the device mapper raid target (dm-raid) to take
advantage of so far unused md raid kernel funtionality:
takeover, reshape, resize, addition and removal of devices to/from raid sets.
This series of patches remove constraints doing so.
Patch #1:
add 2 API functions to allow dm-raid to access the raid takeover
and resize functionality (namely md_takeover() and md_resize());
reshape APIs are not needed in lieu of the existing personalilty ones
Patch #2:
because device mapper core manages a request queue per mapped device
utilizing the md make_request API to pass on bios via the dm-raid target,
no md instance underneath it needs to manage a request queue of its own.
Thus dm-raid can't use the md raid0 personality as is, because the latter
accesses the request queue unconditionally in 3 places via mddev->queue
which this patch addresses.
Patch #3:
when dm-raid processes a down takeover to raid0, it needs to destroy
any existing bitmap, because raid0 does not require one. The patch
exports the bitmap_destroy() API to allow dm-raid to remove bitmaps.
Heinz Mauelshagen (3):
md core: add 2 API functions for takeover and resize to support dm-raid
md raid0: access mddev->queue (request queue member) conditionally
because it is not set when accessed from dm-raid
md bitmap: export bitmap_destroy() to support dm-raid down takover to raid0
drivers/md/bitmap.c | 1 +
drivers/md/md.c | 39 ++++++++++++++++++++++++++++++---------
drivers/md/md.h | 3 +++
drivers/md/raid0.c | 48 +++++++++++++++++++++++++++---------------------
4 files changed, 61 insertions(+), 30 deletions(-)
--
2.1.0
^ permalink raw reply
* Re: [PATCH RESEND] Monitor: fix for regression with container devices
From: Artur Paszkiewicz @ 2015-02-13 15:29 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, pawel.baldysiak
In-Reply-To: <20150211153814.333cb17a@notabene.brown>
On 02/11/2015 05:38 AM, NeilBrown wrote:
> On Mon, 9 Feb 2015 11:13:50 +0100 Artur Paszkiewicz
> <artur.paszkiewicz@intel.com> wrote:
>
> > This patch fixes 2 problems introduced by commit 9a518d8: not closing a
> > file descriptor and ignoring container devices. Array state is always
> > "inactive" for containers, so we make sure that the device is not a
> > container by reading also the "level" sysfs entry.
> >
> > Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
> > Reviewed-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
> > ---
> > Monitor.c | 14 ++++++++++----
> > 1 file changed, 10 insertions(+), 4 deletions(-)
> >
> > diff --git a/Monitor.c b/Monitor.c
> > index 971d2ec..66d67ba 100644
> > --- a/Monitor.c
> > +++ b/Monitor.c
> > @@ -483,11 +483,17 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
> > strncmp(buf,"inact",5) == 0) {
> > if (fd >= 0)
> > close(fd);
> > - if (!st->err)
> > - alert("DeviceDisappeared", dev, NULL, ainfo);
> > - st->err++;
> > - return 0;
> > + fd = sysfs_open(st->devnm, NULL, "level");
> > + if (fd < 0 || read(fd, buf, 10) != 0) {
> > + if (fd >= 0)
> > + close(fd);
> > + if (!st->err)
> > + alert("DeviceDisappeared", dev, NULL, ainfo);
> > + st->err++;
> > + return 0;
> > + }
> > }
> > + close(fd);
> > }
> > fd = open(dev, O_RDONLY);
> > if (fd < 0) {
>
> Thanks for the patch.
>
> I don't think I agree with the logic of using 'level' though.
> For the sort of arrays that I need to ignore here, 'level' will be empty.
>
> It would make sense to test 'metadata' though. If that starts 'external:',
> then we don't want to ignore the array.
>
> Could you confirm that this works please?
>
Hi Neil,
I tested your patch. I assume you wanted to use 'metadata_version',
because there is no 'metadata' attribute, right? I had also thought
about that, but simply looking for 'external:' is not enough to
determine that the array is a container - for volumes inside the
container it looks like this: 'external:/md127/0'. But I think that the
arrays you want to ignore will just have 'none' there, so maybe it can
be done like this?
diff --git a/Monitor.c b/Monitor.c
index 971d2ec..83daf3b 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -483,11 +483,18 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
strncmp(buf,"inact",5) == 0) {
if (fd >= 0)
close(fd);
- if (!st->err)
- alert("DeviceDisappeared", dev, NULL, ainfo);
- st->err++;
- return 0;
+ fd = sysfs_open(st->devnm, NULL, "metadata_version");
+ if (fd < 0 || read(fd, buf, 4) < 0 ||
+ strncmp(buf, "none", 4) == 0) {
+ if (fd >= 0)
+ close(fd);
+ if (!st->err)
+ alert("DeviceDisappeared", dev, NULL, ainfo);
+ st->err++;
+ return 0;
+ }
}
+ close(fd);
}
fd = open(dev, O_RDONLY);
if (fd < 0) {
Thanks,
Artur
> Thanks,
> NeilBrown
>
> diff --git a/Monitor.c b/Monitor.c
> index 971d2ecbea72..6e085cb24993 100644
> --- a/Monitor.c
> +++ b/Monitor.c
> @@ -483,11 +483,18 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
> strncmp(buf,"inact",5) == 0) {
> if (fd >= 0)
> close(fd);
> - if (!st->err)
> - alert("DeviceDisappeared", dev, NULL, ainfo);
> - st->err++;
> - return 0;
> + fd = sysfs_open(st->devnm, NULL, "metadata");
> + if (fd < 0 || read(fd, buf, 9) != 9 ||
> + strncmp(buf, "external:", 9) != 0) {
> + if (fd >= 0)
> + close(fd);
> + if (!st->err)
> + alert("DeviceDisappeared", dev, NULL, ainfo);
> + st->err++;
> + return 0;
> + }
> }
> + close(fd);
> }
> fd = open(dev, O_RDONLY);
> if (fd < 0) {
>
^ permalink raw reply related
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 14:48 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213102746.GO2896@worktop.programming.kicks-ass.net>
On Fri, Feb 13, 2015 at 11:27:46AM +0100, Peter Zijlstra wrote:
> > I've moved blk_flush_plug to the beginning of the function.
>
> > I wondered if it really make sense to call blk_flush_plug with nr_iowait
> > elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
> > for non-"io" reasons and maybe that could upset stuff???
>
> Yeah, good question that. Lemme ponder that a bit.
Yes, I thikn your version makes most sense as, you say, even regular
schedule() call nested in my version would go towards blk delayacct --
and I doubt that was the intent; even though the current kernel works
that way.
I'll move the now rudimentary io_schedule() into sched.h as an inline.
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 10:27 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213194953.0368355d@notabene.brown>
On Fri, Feb 13, 2015 at 07:49:53PM +1100, NeilBrown wrote:
> > Like said, that will still recursive call delayacct_blkio_*() and would
> > increase nr_iowait for a second time; while arguably its still the same
> > one io-wait instance.
>
> No it doesn't. There is no "blk_flush_plug" call between the
> delayacct_blkio_*() calls.
Duh, clearly I needed to still wake up :/
> I've moved blk_flush_plug to the beginning of the function.
> I wondered if it really make sense to call blk_flush_plug with nr_iowait
> elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
> for non-"io" reasons and maybe that could upset stuff???
Yeah, good question that. Lemme ponder that a bit.
> I don't really know. I'm happy with your version. I don't suppose anyone
> else is paying attention and could give a third opinion....
:-)
^ permalink raw reply
* mdadm failed to remove internal bitmap
From: gary @ 2015-02-13 9:46 UTC (permalink / raw)
To: linux-raid
Hi,
I used v3.3.1 mdadm to do some test for bitmap, but when switch bitmap from
internal to none, the output shows fail info about remove internal
bitmap, is it
just a warning? Since the bitmap seems to be cleared, and it doesn't
show with
v3.2.6 mdadm with the same steps.
linux:~ # mdadm --create md0 --raid-devices=2 --level=mirror
--assume-clean /dev/vdb /dev/vdc
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md0 started.
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
unused devices: <none>
linux:~ # mdadm --grow --bitmap=internal /dev/md127
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
bitmap: 1/1 pages [4KB], 65536KB chunk
unused devices: <none>
linux:~ # mdadm --grow --bitmap=none /dev/md127
mdadm: failed to remove internal bitmap.
linux:~ # cat /proc/mdstat
Personalities : [raid1]
md127 : active raid1 vdc[1] vdb[0]
523712 blocks super 1.2 [2/2] [UU]
unused devices: <none>
Thanks,
Gary
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: NeilBrown @ 2015-02-13 8:49 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213083250.GN2896@worktop.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 3876 bytes --]
On Fri, 13 Feb 2015 09:32:50 +0100 Peter Zijlstra <peterz@infradead.org>
wrote:
> On Fri, Feb 13, 2015 at 04:26:00PM +1100, NeilBrown wrote:
> > I choose ... Buzz Lightyear !!!
>
> Great choice!
>
> > From: NeilBrown <neilb@suse.de>
> > Date: Fri, 13 Feb 2015 15:49:17 +1100
> > Subject: [PATCH] sched: prevent recursion in io_schedule()
> >
> > io_schedule() calls blk_flush_plug() which, depending on the
> > contents of current->plug, can initiate arbitrary blk-io requests.
> >
> > Note that this contrasts with blk_schedule_flush_plug() which requires
> > all non-trivial work to be handed off to a separate thread.
> >
> > This makes it possible for io_schedule() to recurse, and initiating
> > block requests could possibly call mempool_alloc() which, in times of
> > memory pressure, uses io_schedule().
> >
> > Apart from any stack usage issues, io_schedule() will not behave
> > correctly when called recursively as delayacct_blkio_start() does
> > not allow for repeated calls.
>
> Which seems to still be an issue with this patch.
>
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 1f37fe7f77a4..90f3de8bc7ca 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
> > */
> > void __sched io_schedule(void)
> > {
> > + io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
> > }
> > EXPORT_SYMBOL(io_schedule);
>
> Might as well move it to sched.h as an inline or so..
>
> > long __sched io_schedule_timeout(long timeout)
> > {
> > + struct rq *rq;
> > long ret;
> > + int old_iowait = current->in_iowait;
> > +
> > + current->in_iowait = 1;
> > + if (old_iowait)
> > + blk_schedule_flush_plug(current);
> > + else
> > + blk_flush_plug(current);
> >
> > delayacct_blkio_start();
> > + rq = raw_rq();
> > atomic_inc(&rq->nr_iowait);
> > ret = schedule_timeout(timeout);
> > + current->in_iowait = old_iowait;
> > atomic_dec(&rq->nr_iowait);
> > delayacct_blkio_end();
> > return ret;
>
> Like said, that will still recursive call delayacct_blkio_*() and would
> increase nr_iowait for a second time; while arguably its still the same
> one io-wait instance.
No it doesn't. There is no "blk_flush_plug" call between the
delayacct_blkio_*() calls.
I've moved blk_flush_plug to the beginning of the function.
>
> So would a little something like:
>
> long __sched io_schedule_timeout(long timeout)
> {
> struct rq *rq;
> long ret;
>
> /*
> * Recursive io_schedule() call; make sure to not recurse
> * on the blk_flush_plug() stuff again.
> */
> if (unlikely(current->in_iowait)) {
> /*
> * Our parent io_schedule() call will already have done
> * all the required io-wait accounting.
> */
> blk_schedule_flush_plug(current);
> return schedule_timeout(timeout);
> }
>
> current->in_iowait = 1;
> delayacct_blkio_start();
> rq = raw_rq();
> atomic_inc(&rq->nr_iowait);
> blk_flush_plug(current);
> ret = schedule_timeout(timeout);
> atomic_dec(&rq->nr_iowait);
> delayacct_blkio_end();
> current->in_iowait = 0;
>
> return ret;
> }
>
> not make more sense?
That does make a similar amount of sense at least....
I wondered if it really make sense to call blk_flush_plug with nr_iowait
elevated and delayacct_blkio active. blk_flush_plug() could call schedule()
for non-"io" reasons and maybe that could upset stuff???
I don't really know. I'm happy with your version. I don't suppose anyone
else is paying attention and could give a third opinion....
Thanks,
NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-13 8:32 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150213162600.059fffb2@notabene.brown>
On Fri, Feb 13, 2015 at 04:26:00PM +1100, NeilBrown wrote:
> I choose ... Buzz Lightyear !!!
Great choice!
> From: NeilBrown <neilb@suse.de>
> Date: Fri, 13 Feb 2015 15:49:17 +1100
> Subject: [PATCH] sched: prevent recursion in io_schedule()
>
> io_schedule() calls blk_flush_plug() which, depending on the
> contents of current->plug, can initiate arbitrary blk-io requests.
>
> Note that this contrasts with blk_schedule_flush_plug() which requires
> all non-trivial work to be handed off to a separate thread.
>
> This makes it possible for io_schedule() to recurse, and initiating
> block requests could possibly call mempool_alloc() which, in times of
> memory pressure, uses io_schedule().
>
> Apart from any stack usage issues, io_schedule() will not behave
> correctly when called recursively as delayacct_blkio_start() does
> not allow for repeated calls.
Which seems to still be an issue with this patch.
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 1f37fe7f77a4..90f3de8bc7ca 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
> */
> void __sched io_schedule(void)
> {
> + io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
> }
> EXPORT_SYMBOL(io_schedule);
Might as well move it to sched.h as an inline or so..
> long __sched io_schedule_timeout(long timeout)
> {
> + struct rq *rq;
> long ret;
> + int old_iowait = current->in_iowait;
> +
> + current->in_iowait = 1;
> + if (old_iowait)
> + blk_schedule_flush_plug(current);
> + else
> + blk_flush_plug(current);
>
> delayacct_blkio_start();
> + rq = raw_rq();
> atomic_inc(&rq->nr_iowait);
> ret = schedule_timeout(timeout);
> + current->in_iowait = old_iowait;
> atomic_dec(&rq->nr_iowait);
> delayacct_blkio_end();
> return ret;
Like said, that will still recursive call delayacct_blkio_*() and would
increase nr_iowait for a second time; while arguably its still the same
one io-wait instance.
So would a little something like:
long __sched io_schedule_timeout(long timeout)
{
struct rq *rq;
long ret;
/*
* Recursive io_schedule() call; make sure to not recurse
* on the blk_flush_plug() stuff again.
*/
if (unlikely(current->in_iowait)) {
/*
* Our parent io_schedule() call will already have done
* all the required io-wait accounting.
*/
blk_schedule_flush_plug(current);
return schedule_timeout(timeout);
}
current->in_iowait = 1;
delayacct_blkio_start();
rq = raw_rq();
atomic_inc(&rq->nr_iowait);
blk_flush_plug(current);
ret = schedule_timeout(timeout);
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
current->in_iowait = 0;
return ret;
}
not make more sense?
^ permalink raw reply
* Re: raid1 narrow_write_error with 4K disks, sd "bad block number requested" messages
From: NeilBrown @ 2015-02-13 6:01 UTC (permalink / raw)
To: Nate Dailey; +Cc: linux-raid, linux-scsi
In-Reply-To: <54DCD8DD.7080103@stratus.com>
[-- Attachment #1: Type: text/plain, Size: 3813 bytes --]
On Thu, 12 Feb 2015 11:46:21 -0500 Nate Dailey <nate.dailey@stratus.com>
wrote:
> On 02/04/2015 11:59 PM, NeilBrown wrote:
> > On Wed, 28 Jan 2015 10:29:46 -0500 Nate Dailey <nate.dailey@stratus.com>
> > wrote:
> >
> >> I'm writing about something that appears to be an issue with raid1's
> >> narrow_write_error, particular to non-512-byte-sector disks. Here's what
> >> I'm doing:
> >>
> >> - 2 disk raid1, 4K disks, each connected to a different SAS HBA
> >> - mount a filesystem on the raid1, run a test that writes to it
> >> - remove one of the SAS HBAs (echo 1 >
> >> /sys/bus/pci/devices/0000\:45\:00.0/remove)
> >>
> >> At this point, writes fail and narrow_write_error breaks them up and
> >> retries, one sector at a time. But these are 512-byte sectors, and sd
> >> doesn't like it:
> >>
> >> [ 2645.310517] sd 3:0:1:0: [sde] Bad block number requested
> >> [ 2645.310610] sd 3:0:1:0: [sde] Bad block number requested
> >> [ 2645.310690] sd 3:0:1:0: [sde] Bad block number requested
> >> ...
> >>
> >> There appears to be no real harm done, but there can be a huge number of
> >> these messages in the log.
> >>
> >> I can avoid this by disabling bad block tracking, but it looks like
> >> maybe the superblock's bblog_shift is intended to address this exact
> >> issue. However, I don't see a way to change it. Presumably this is
> >> something mdadm should be setting up? I don't see bblog_shift ever set
> >> to anything other than 0.
> >>
> >> This is on a RHEL 7.1 kernel, version 3.10.0-221.el7. I took a look at
> >> upstream sd and md changes and nothing jumps out at me that would have
> >> affected this (but I have not tested to see if the bad block messages do
> >> or do not happen on an upstream kernel).
> >>
> >> I'd appreciate any advice re: how to handle this. Thanks!
> >
> > Thanks for the report.
> >
> > narrow_write_error() should use bdev_logical_block_size() and round up to
> > that.
> > Possibly mdadm should get the same information and set bblog_shift
> > accordingly when creating a bad block log.
> >
> > I've made a note to fix that, but I'm happy to review patches too :-)
> >
> > thanks,
> > NeilBrown
> >
>
> I will post a narrow_write_error patch shortly.
>
> I did some experimentation with setting the bblog_shift in mdadm, but it
> didn't work out the way I expected. It turns out that the value is only
> loaded from the superblock if:
>
> 1453 if ((le32_to_cpu(sb->feature_map) & MD_FEATURE_BAD_BLOCKS) &&
> 1454 rdev->badblocks.count == 0) {
> ...
> 1473 rdev->badblocks.shift = sb->bblog_shift;
>
> And this feature bit is only set if any bad blocks have actually been
> recorded.
>
> It also appears to me that the shift is used when loading the bad blocks
> from the superblock, but not when storing the bad block list in the
> superblock.
>
> Seems like these are bugs, but I'm not certain how the code is supposed
> to work (and am getting in a bit over my head with this).
Yes, that's probably a bug.
The
} else if (sb->bblog_offset != 0)
rdev->badblocks.shift = 0;
should be
} else if (sb->bblog_offset != 0)
rdev->badblocks.shift = sb->bblog_shift;
>
> In any case, it doesn't appear to me that there's any harm in having the
> bblog_shift not match the disk's block size (right?).
Having the bblog_shift larger than the disk's block size certainly should not
be a problem. Having it small only causes the problem that you have already
discovered.
NeilBrown
>
> Nate Dailey
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: NeilBrown @ 2015-02-13 5:26 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150210092936.GW21418@twins.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 4838 bytes --]
On Tue, 10 Feb 2015 10:29:36 +0100 Peter Zijlstra <peterz@infradead.org>
wrote:
> On Tue, Feb 10, 2015 at 01:50:17PM +1100, NeilBrown wrote:
> > On Mon, 9 Feb 2015 10:10:00 +0100 Peter Zijlstra <peterz@infradead.org> wrote:
> > > > However, when io_schedule() explicitly calls blk_flush_plug(), then
> > > > @from_schedule=false variant is used, and the unplug functions are allowed to
> > > > allocate memory and block and maybe even call mempool_alloc() which might
> > > > call io_schedule().
> > > >
> > > > This shouldn't be a problem as blk_flush_plug() spliced out the plug list, so
> > > > any recursive call will find an empty list and do nothing.
> > >
> > > Unless, something along the way stuck something back on, right? So
> > > should we stick an:
> > >
> > > WARN_ON(current->in_iowait);
> > >
> > > somewhere near where things are added to this plug list? (and move the
> > > blk_flush_plug() call inside of where that's actually true of course).
> >
> > No, I don't think so.
> >
> > It is certainly possible that some request on plug->cb_list could add
> > something to plug->list - which is processed after ->cb_list.
> >
> > I think the best way to think about this is that the *problem* was that a
> > wait_event loop could spin without making any progress. So any time that
> > clear forward progress is made it is safe sleep without necessitating the
> > warning. Hence sched_annotate_sleep() is reasonable.
> > blk_flush_plug() with definitely have dispatched some requests if it
> > might_sleep(), so the sleep is OK.
>
> Well, yes, but you forget that this gets us back into recursion land.
> io_schedule() calling io_schedule() calling io_schedule() and *boom*
> stack overflow -> dead machine.
>
> We must either guarantee io_schedule() will never call io_schedule() or
> that io_schedule() itself will not add new work to the current plug such
> that calling io_schedule() itself will not recurse on the blk stuff.
>
> Pick either option, but pick one.
I choose ... Buzz Lightyear !!!
Sorry, go carried away there. Uhhmm. I think I pick a/ (But I expect I'll
find a goat... ho hum).
Does this look credible?
Thanks,
NeilBrown
From: NeilBrown <neilb@suse.de>
Date: Fri, 13 Feb 2015 15:49:17 +1100
Subject: [PATCH] sched: prevent recursion in io_schedule()
io_schedule() calls blk_flush_plug() which, depending on the
contents of current->plug, can initiate arbitrary blk-io requests.
Note that this contrasts with blk_schedule_flush_plug() which requires
all non-trivial work to be handed off to a separate thread.
This makes it possible for io_schedule() to recurse, and initiating
block requests could possibly call mempool_alloc() which, in times of
memory pressure, uses io_schedule().
Apart from any stack usage issues, io_schedule() will not behave
correctly when called recursively as delayacct_blkio_start() does
not allow for repeated calls.
So:
- use in_iowait to detect recursion. Set it earlier, and restore
it to the old value.
- move the call to "raw_rq" after the call to blk_flush_plug().
As this is some sort of per-cpu thing, we want some chance that
we are on the right CPU
- When io_schedule() is called recurively, use blk_schedule_flush_plug()
which cannot further recurse.
- as this makes io_schedule() a lot more complex and as io_schedule()
must match io_schedule_timeout(), but all the changes in io_schedule_timeout()
and make io_schedule a simple wrapper for that.
Signed-off-by: NeilBrown <neilb@suse.de>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Peter Zijlstra <peterz@infradead.org>
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1f37fe7f77a4..90f3de8bc7ca 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4420,30 +4420,27 @@ EXPORT_SYMBOL_GPL(yield_to);
*/
void __sched io_schedule(void)
{
- struct rq *rq = raw_rq();
-
- delayacct_blkio_start();
- atomic_inc(&rq->nr_iowait);
- blk_flush_plug(current);
- current->in_iowait = 1;
- schedule();
- current->in_iowait = 0;
- atomic_dec(&rq->nr_iowait);
- delayacct_blkio_end();
+ io_schedule_timeout(MAX_SCHEDULE_TIMEOUT);
}
EXPORT_SYMBOL(io_schedule);
long __sched io_schedule_timeout(long timeout)
{
- struct rq *rq = raw_rq();
+ struct rq *rq;
long ret;
+ int old_iowait = current->in_iowait;
+
+ current->in_iowait = 1;
+ if (old_iowait)
+ blk_schedule_flush_plug(current);
+ else
+ blk_flush_plug(current);
delayacct_blkio_start();
+ rq = raw_rq();
atomic_inc(&rq->nr_iowait);
- blk_flush_plug(current);
- current->in_iowait = 1;
ret = schedule_timeout(timeout);
- current->in_iowait = 0;
+ current->in_iowait = old_iowait;
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
return ret;
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* [md PATCH] md/raid1: round up to bdev_logical_block_size in narrow_write_error
From: Nate Dailey @ 2015-02-12 17:02 UTC (permalink / raw)
To: linux-raid
This modifies raid1's narrow_write_error to round up block_sectors to the
device's logical block size.
This prevents sd complaining about "Bad block number requested" for non-512-byte
sector disks.
Signed-off-by: Nate Dailey <nate.dailey@stratus.com>
---
diff -Nupr a/drivers/md/raid1.c b/drivers/md/raid1.c
--- a/drivers/md/raid1.c 2015-02-10 15:29:02.000000000 -0500
+++ b/drivers/md/raid1.c 2015-02-10 15:29:45.000000000 -0500
@@ -2206,7 +2206,8 @@ static int narrow_write_error(struct r1b
if (rdev->badblocks.shift < 0)
return 0;
- block_sectors = 1 << rdev->badblocks.shift;
+ block_sectors = roundup(1 << rdev->badblocks.shift,
+ bdev_logical_block_size(rdev->bdev) >> 9);
sector = r1_bio->sector;
sectors = ((sector + block_sectors)
& ~(sector_t)(block_sectors - 1))
^ permalink raw reply
* Re: raid1 narrow_write_error with 4K disks, sd "bad block number requested" messages
From: Nate Dailey @ 2015-02-12 16:46 UTC (permalink / raw)
To: linux-raid; +Cc: linux-scsi
In-Reply-To: <20150205155953.64e9b1e4@notabene.brown>
On 02/04/2015 11:59 PM, NeilBrown wrote:
> On Wed, 28 Jan 2015 10:29:46 -0500 Nate Dailey <nate.dailey@stratus.com>
> wrote:
>
>> I'm writing about something that appears to be an issue with raid1's
>> narrow_write_error, particular to non-512-byte-sector disks. Here's what
>> I'm doing:
>>
>> - 2 disk raid1, 4K disks, each connected to a different SAS HBA
>> - mount a filesystem on the raid1, run a test that writes to it
>> - remove one of the SAS HBAs (echo 1 >
>> /sys/bus/pci/devices/0000\:45\:00.0/remove)
>>
>> At this point, writes fail and narrow_write_error breaks them up and
>> retries, one sector at a time. But these are 512-byte sectors, and sd
>> doesn't like it:
>>
>> [ 2645.310517] sd 3:0:1:0: [sde] Bad block number requested
>> [ 2645.310610] sd 3:0:1:0: [sde] Bad block number requested
>> [ 2645.310690] sd 3:0:1:0: [sde] Bad block number requested
>> ...
>>
>> There appears to be no real harm done, but there can be a huge number of
>> these messages in the log.
>>
>> I can avoid this by disabling bad block tracking, but it looks like
>> maybe the superblock's bblog_shift is intended to address this exact
>> issue. However, I don't see a way to change it. Presumably this is
>> something mdadm should be setting up? I don't see bblog_shift ever set
>> to anything other than 0.
>>
>> This is on a RHEL 7.1 kernel, version 3.10.0-221.el7. I took a look at
>> upstream sd and md changes and nothing jumps out at me that would have
>> affected this (but I have not tested to see if the bad block messages do
>> or do not happen on an upstream kernel).
>>
>> I'd appreciate any advice re: how to handle this. Thanks!
>
> Thanks for the report.
>
> narrow_write_error() should use bdev_logical_block_size() and round up to
> that.
> Possibly mdadm should get the same information and set bblog_shift
> accordingly when creating a bad block log.
>
> I've made a note to fix that, but I'm happy to review patches too :-)
>
> thanks,
> NeilBrown
>
I will post a narrow_write_error patch shortly.
I did some experimentation with setting the bblog_shift in mdadm, but it
didn't work out the way I expected. It turns out that the value is only
loaded from the superblock if:
1453 if ((le32_to_cpu(sb->feature_map) & MD_FEATURE_BAD_BLOCKS) &&
1454 rdev->badblocks.count == 0) {
...
1473 rdev->badblocks.shift = sb->bblog_shift;
And this feature bit is only set if any bad blocks have actually been
recorded.
It also appears to me that the shift is used when loading the bad blocks
from the superblock, but not when storing the bad block list in the
superblock.
Seems like these are bugs, but I'm not certain how the code is supposed
to work (and am getting in a bit over my head with this).
In any case, it doesn't appear to me that there's any harm in having the
bblog_shift not match the disk's block size (right?).
Nate Dailey
^ permalink raw reply
* RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
From: Manibalan P @ 2015-02-12 13:56 UTC (permalink / raw)
To: NeilBrown, linux-raid; +Cc: Pasi Kärkkäinen
In-Reply-To: <20150203093040.569aa5e1@notabene.brown>
Dear All,
Gentle reminder !.. Any update on this issue..
I have tested with all the live lock related patch.. and I hope this is some specific scenario..
Sync thread become busy, and this happens only when RAID array is initializing and heavy IO is happening in parallel
Thanks,
Manibalan.
-----Original Message-----
From: Manibalan P
Sent: Wednesday, February 4, 2015 11:27 AM
To: 'NeilBrown'; linux-raid
Cc: Pasi Kärkkäinen
Subject: RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
>> Dear All,
>> Any updates on this issue.
>Probably the same as:
> http://marc.info/?l=linux-raid&m=142283560704091&w=2
Dear Neil
This patch is not fixing this issue.
This issue happens only if a drive removed from a RAID5 array, which is "initializing" and "heavy IO" is performed on the array.
In such case, as soon as the drive removed, the array state changed to resync=PENDING and md0_raid5 thread using 100% of CPU.
Thanks,
Manibalan.
>which follows on from
> http://marc.info/?t=142221642300001&r=1&w=2
>and
> http://marc.info/?t=142172432500001&r=1&w=2
>NeilBrown
-----Original Message-----
From: NeilBrown [mailto:neilb@suse.de]
Sent: Tuesday, February 3, 2015 4:01 AM
To: Manibalan P
Cc: Pasi Kärkkäinen; linux-raid
Subject: Re: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
On Mon, 2 Feb 2015 07:10:14 +0000 Manibalan P <pmanibalan@amiindia.co.in>
wrote:
> Dear All,
> Any updates on this issue.
Probably the same as:
http://marc.info/?l=linux-raid&m=142283560704091&w=2
which follows on from
http://marc.info/?t=142221642300001&r=1&w=2
and
http://marc.info/?t=142172432500001&r=1&w=2
NeilBrown
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Wednesday, January 14, 2015 3:55 PM
> To: 'Pasi Kärkkäinen'
> Cc: 'neilb@suse.de'; 'linux-raid'
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
> Could you able to find something on this issue.
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Manibalan P
> Sent: Friday, January 2, 2015 12:08 PM
> To: 'Pasi Kärkkäinen'
> Cc: neilb@suse.de; linux-raid
> Subject: RE: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> Dear Pasi,
>
> I have add the bug in
> https://bugzilla.redhat.com/show_bug.cgi?id=1178080
>
> Thanks,
> Manibalan.
>
> -----Original Message-----
> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Wednesday, December 31, 2014 10:18 PM
> To: Manibalan P
> Cc: neilb@suse.de; linux-raid
> Subject: Re: md_raid5 using 100% CPU and hang with status
> resync=PENDING, if a drive is removed during initialization
>
> On Tue, Dec 30, 2014 at 11:06:47AM +0000, Manibalan P wrote:
> > Dear Neil,
> >
>
> Hello,
>
> > Few this for you kind attention,
> > 1. I tried the same test with FC11 (2.6.32 kernel before MD code
> > change). And the issue is not there 2. But with Centos 6.4 (2.6.32 kernel after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Also, a bug has been raise with RHEL regarding this issue. Please find the bug link "https://access.redhat.com/support/cases/#/case/01320319"
> >
>
> That support case URL can only be accessed by you and Redhat. Do you happen to have a public bugzilla link?
>
>
> Thanks,
>
> -- Pasi
>
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 24, 2014 12:15 PM
> > To: neilb@suse.de; 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> >
> > Dear Neil,
> >
> > Few this for you kind attention,
> > 1. I tried the same tesst with FC11 (2.6 kernel before MD code change). And the issue is not there 2. But with Centos 6.4 (2.6 after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Thursday, December 18, 2014 11:38 AM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear neil,
> >
> > I also compiled the latest 3.18 kernel on CentOS 6.4 with GIT MD pull patches form 3.19, that also ran in to the same issue after removing a drive during resync.
> >
> > Dec 17 19:07:32 ITX002590129362 kernel: Linux version 3.18.0 (root@mycentos6) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Wed Dec 17 15:59:09 EST 2014 Dec 17 19:07:32 ITX002590129362 kernel: Command line: ro root=/dev/md255 rd_NO_LVM rd_NO_DM rhgb quiet md_mod.start_ro=1 nmi_watchdog=1 md_mod.start_dirty_degraded=1 ??? Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sda6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdb6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdc6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdh6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdi6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdj6> Dec 17 19:10:15 ITX002590129362 kernel: async_tx: api initialized (async) Dec 17 19:10:15 ITX002590129362 kernel: xor: measuring software checksum speed
> > Dec 17 19:10:15 ITX002590129362 kernel: prefetch64-sse: 10048.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: generic_sse: 8824.000 MB/sec
> > Dec 17 19:10:15 ITX002590129362 kernel: xor: using function: prefetch64-sse (10048.000 MB/sec)
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x1 5921 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x2 6933 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x4 7476 MB/s
> > Dec 17 19:10:15 ITX002590129362 kernel: raid6: using algorithm
> > sse2x4
> > (7476 MB/s) Dec 17 19:10:15 ITX002590129362 kernel: raid6: using
> > ssse3x2 recovery algorithm Dec 17 19:10:15 ITX002590129362 kernel: md:
> > raid6 personality registered for level 6 Dec 17 19:10:15
> > ITX002590129362 kernel: md: raid5 personality registered for level 5
> > Dec 17 19:10:15 ITX002590129362 kernel: md: raid4 personality
> > registered for level 4 Dec 17 19:10:15 ITX002590129362 kernel:
> > md/raid:md0: not clean -- starting background reconstruction Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdj6
> > operational as raid disk 5 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdi6 operational as raid disk 4 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/raid:md0: device sdh6 operational as raid disk 3 Dec 17
> > 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdc6
> > operational as raid disk 2 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0:
> > device sdb6 operational as raid disk 1 Dec 17 19:10:15
> > ITX002590129362
> > kernel: md/ra
> > id:md0: device sda6 operational as raid disk 0 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: allocated 0kB Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: raid level 5 active with 6 out of 6 devices, algorithm 2 Dec 17 19:10:15 ITX002590129362 kernel: md0: detected capacity change from 0 to 2361059573760 Dec 17 19:10:15 ITX002590129362 kernel: md0: unknown partition table Dec 17 19:10:35 ITX002590129362 kernel: md: md0 switched to read-write mode.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: resync of RAID array md0 Dec 17 19:10:35 ITX002590129362 kernel: md: minimum _guaranteed_ speed: 10000 KB/sec/disk.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using maximum available idle IO bandwidth (but not more than 30000 KB/sec) for resync.
> > Dec 17 19:10:35 ITX002590129362 kernel: md: using 128k window, over a total of 461144448k.
> > ???
> > Started IOs using fio tool.
> >
> > ./fio --name=md0 --filename=/dev/md0 --thread --numjobs=10
> > --direct=1 --group_reporting --unlink=0 --loops=1 --offset=0
> > --randrepeat=1 --norandommap --scramble_buffers=1 --stonewall
> > --ioengine=libaio --rw=randwrite --bs=8704 --iodepth=4000
> > --runtime=3000
> > --blockalign=512
> >
> > ???
> > Removed a drive form the system..
> >
> > Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) ..
> > Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 02 69 03 70 00 00 10 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 40436592 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 51 b3 d0 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 206681040 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 3a f3 40 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 205189952 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK ??? Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb 00 00 00 08 00 Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:25 ITX002590129362 kernel: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
> > Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb f0 00 00 10 00 Dec 17 19:13:25 ITX002590129362 aghswap: devpath [0:0:7:0] action [remove] devtype [scsi_disk] Dec 17 19:13:25 ITX002590129362 aghswap: MHSA: Sent event 0 0 7 0 remove scsi_disk Dec 17 19:13:25 ITX002590129362 kernel: mpt2sas0: removing handle(0x0011), sas_addr(0x500605ba0101e305) Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Disk failure on sdh6, disabling device.
> > Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Operation continuing on 5 devices.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: md0: resync interrupted.
> > Dec 17 19:13:25 ITX002590129362 kernel: md: checkpointing resync of md0.
> > ..
> > Log messages after enabling debufgs on raid5.c, it is getting repeated continuously.
> >
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0
> > bypass_count: 0
> > __get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0 handling stripe 273480328, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x18 read (null) write ffff8808029b6b00 written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=273480328 for sector 273480328, rmw=2 rcw=1 handling stripe 65238568, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x10 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081a956b00 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=65238568 for sector 65238568, rmw=2 rcw=1 handling stripe 713868672, state=0x2041 cnt=1, pd_idx=4, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x10 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff88081f020100 written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=713868672 for sector 713868672, rmw=2 rcw=1 handling stripe 729622496, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622496 for sector 729622496, rmw=2 rcw=1 handling stripe 729622504, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622504 for sector 729622504, rmw=2 rcw=1 handling stripe 245773680, state=0x2041 cnt=1, pd_idx=0, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x11 read (null) write (null) written (null)
> > check 1: state 0x18 read (null) write ffff88081cab7a00 written (null)
> > check 0: state 0x10 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=245773680 for sector 245773680, rmw=2 rcw=1 handling stripe 867965560, state=0x2041 cnt=1, pd_idx=1, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x11 read (null) write (null) written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x18 read (null) write ffff880802b2bf00 written (null)
> > check 1: state 0x10 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=867965560 for sector 867965560, rmw=2 rcw=1 handling stripe 550162280, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
> > check 5: state 0x11 read (null) write (null) written (null)
> > check 4: state 0x18 read (null) write ffff880802b08800 written (null)
> > check 3: state 0x0 read (null) write (null) written (null)
> > check 2: state 0x10 read (null) write (null) written (null)
> > check 1: state 0x11 read (null) write (null) written (null)
> > check 0: state 0x11 read (null) write (null) written (null)
> > locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1
> > force RCW max_degraded=1, recovery_cp=7036944 sh->sector=550162280
> > for sector 550162280, rmw=2 rcw=1
> >
> >
> > Thanks,
> > Manibalan
> >
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:11 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > The same Issue is reproducible in the latest upstream kernel also.
> >
> > Tested in "3.17.6" latest stable upstream kernel and find the same issue.
> >
> > [root@root ~]# modinfo raid456
> > filename: /lib/modules/3.17.6/kernel/drivers/md/raid456.ko
> > alias: raid6
> > alias: raid5
> > alias: md-level-6
> > alias: md-raid6
> > alias: md-personality-8
> > alias: md-level-4
> > alias: md-level-5
> > alias: md-raid4
> > alias: md-raid5
> > alias: md-personality-4
> > description: RAID4/5/6 (striping with parity) personality for MD
> > license: GPL
> > srcversion: 0EEF680023FDC7410F7989A
> > depends: async_raid6_recov,async_pq,async_tx,async_memcpy,async_xor
> > intree: Y
> > vermagic: 3.17.6 SMP mod_unload modversions
> > parm: devices_handle_discard_safely:Set to Y if all devices in each array reliably return zeroes on reads from discarded regions (bool)
> >
> > Thanks,
> > Manibalan.
> >
> > -----Original Message-----
> > From: Manibalan P
> > Sent: Wednesday, December 17, 2014 12:01 PM
> > To: 'linux-raid'
> > Cc: 'NeilBrown'
> > Subject: RE: md_raid5 using 100% CPU and hang with status
> > resync=PENDING, if a drive is removed during initialization
> >
> > Dear Neil,
> >
> > We are facing IO struck issue with raid5 in the following scenario.
> > (please see the attachment for the complete information) In RAID5
> > array, if a drive is removed while initialization and the same time
> > if IO is happening to that md. Then IO is getting struck, and
> > md_raid5 thread is using 100 % of CPU. Also the md state showing as
> > resync=PENDING
> >
> > Kernel : Issue found in the following kernels RHEL 6.5
> > (2.6.32-431.el6.x86_64) CentOS 7 (kernel-3.10.0-123.13.1.el7.x86_64)
> >
> > Steps to Reproduce the issue:
> >
> > 1. Created a raid 5 md with 4 drives using the below mdadm command.
> > mdadm -C /dev/md0 -c 64 -l 5 -f -n 4 -e 1.2 /dev/sdb6 /dev/sdc6
> > /dev/sdd6 /dev/sde6
> >
> > 2. Make the md writable
> > mdadm ???readwrite /dev/md0
> >
> > 3. Now md will start initialization
> >
> > 4. Run FIO Tool, the the below said configuration /usr/bin/fio
> > --name=md0 --filename=/dev/md0 --thread --numjobs=10 --direct=1
> > --group_reporting --unlink=0 --loops=1 --offset=0 --randrepeat=1
> > --norandommap --scramble_buffers=1 --stonewall --ioengine=libaio
> > --rw=randwrite --bs=8704 --iodepth=4000 --runtime=3000
> > --blockalign=512
> >
> > 4. During MD initialzing, remove a drive(either using MDADM set
> > faulty/remove or remove manually)
> >
> > 5. Now the IO will struck, and cat /proc/mdstat shows states with
> > resync=PENDING
> > --------------------------------------------------------------------
> > --
> > ----------------------- top - output show, md_raid5 using 100% cpu
> >
> > top - 17:55:06 up 1:09, 3 users, load average: 11.98, 8.53, 3.99
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 2690 root 20 0 0 0 0 R 100.0 0.0 6:44.41 md0_raid5
> > --------------------------------------------------------------------
> > --
> > -----------------------
> > dmesg - show the stack trace
> >
> > INFO: task fio:2715 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000a 0 2715 2654 0x00000080
> > ffff88043b623598 0000000000000082 0000000000000000 ffffffff81058d53
> > ffff88043b623548 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b40b098 ffff88043b623fd8 000000000000fbc8 ffff88043b40b098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff8140fa39>] ?
> > md_wakeup_thread+0x39/0x70 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffffa0308f66>] ?
> > make_request+0x306/0xc6c [raid456] [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81122283>] ?
> > mempool_alloc+0x63/0x140 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c767a>] do_direct_IO+0x7ca/0xfa0 [<ffffffff811c8196>]
> > __blockdev_direct_IO_newtrunc+0x346/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2717 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000004 0 2717 2654 0x00000080
> > ffff880439e97698 0000000000000082 ffff880439e97628 ffffffff81058d53
> > ffff880439e97648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0adab8 ffff880439e97fd8 000000000000fbc8 ffff88043b0adab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2718 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000005 0 2718 2654 0x00000080
> > ffff88043bc13698 0000000000000082 ffff88043bc13628 ffffffff81058d53
> > ffff88043bc13648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ad058 ffff88043bc13fd8 000000000000fbc8 ffff88043b0ad058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2719 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000001 0 2719 2654 0x00000080
> > ffff880439ebb698 0000000000000082 ffff880439ebb628 ffffffff81058d53
> > ffff880439ebb648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043b0ac5f8 ffff880439ebbfd8 000000000000fbc8 ffff88043b0ac5f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2720 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000008 0 2720 2654 0x00000080
> > ffff88043b8cf698 0000000000000082 ffff88043b8cf628 ffffffff81058d53
> > ffff88043b8cf648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89af8 ffff88043b8cffd8 000000000000fbc8 ffff880439e89af8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2721 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2721 2654 0x00000080
> > ffff88043b047698 0000000000000082 ffff88043b047628 ffffffff81058d53
> > ffff88043b047648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e89098 ffff88043b047fd8 000000000000fbc8 ffff880439e89098 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2722 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000000 0 2722 2654 0x00000080
> > ffff880439ea3698 0000000000000082 ffff880439ea3628 ffffffff81058d53
> > ffff880439ea3648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff880439e88638 ffff880439ea3fd8 000000000000fbc8 ffff880439e88638 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2723 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000006 0 2723 2654 0x00000080
> > ffff88043bf5f698 0000000000000082 ffff88043bf5f628 ffffffff81058d53
> > ffff88043bf5f648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183ab8 ffff88043bf5ffd8 000000000000fbc8 ffff88043a183ab8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2724 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 000000000000000b 0 2724 2654 0x00000080
> > ffff88043be05698 0000000000000082 ffff88043be05628 ffffffff81058d53
> > ffff88043be05648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a183058 ffff88043be05fd8 000000000000fbc8 ffff88043a183058 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> > INFO: task fio:2725 blocked for more than 120 seconds.
> > Not tainted 2.6.32-431.el6.x86_64 #1 "echo 0 >
> > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > fio D 0000000000000003 0 2725 2654 0x00000080
> > ffff88043be07698 0000000000000082 ffff88043be07628 ffffffff81058d53
> > ffff88043be07648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
> > ffff88043a1825f8 ffff88043be07fd8 000000000000fbc8 ffff88043a1825f8 Call Trace:
> > [<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ?
> > md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>]
> > get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ?
> > default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ?
> > prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>]
> > make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ?
> > autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ?
> > mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>]
> > md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ?
> > bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ?
> > __bio_add_page+0x110/0x230 [<ffffffff81266c50>]
> > generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ?
> > do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120
> > [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
> > [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20
> > [<ffffffff811c9137>]
> > __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>]
> > blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ?
> > blkdev_get_block+0x0/0x20 [<ffffffff81120552>]
> > generic_file_direct_write+0xc2/0x190
> > [<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
> > [<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>]
> > blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ?
> > blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>]
> > aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>]
> > aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>]
> > do_io_submit+0x291/0x920 [<ffffffff811d83f0>]
> > sys_io_submit+0x10/0x20 [<ffffffff8100b072>]
> > system_call_fastpath+0x16/0x1b
> >
> > [root@root ~]# cat /proc/2690/stack
> > [<ffffffff810686da>] __cond_resched+0x2a/0x40 [<ffffffffa030361c>]
> > ops_run_io+0x2c/0x920 [raid456] [<ffffffffa03052cc>]
> > handle_stripe+0x9cc/0x2980 [raid456] [<ffffffffa03078a4>]
> > raid5d+0x624/0x850 [raid456] [<ffffffff81416f05>]
> > md_thread+0x115/0x150 [<ffffffff8109aef6>] kthread+0x96/0xa0
> > [<ffffffff8100c20a>] child_rip+0xa/0x20 [<ffffffffffffffff>]
> > 0xffffffffffffffff
> >
> > [root@root ~]# cat /proc/2690/stat
> > 2690 (md0_raid5) R 2 0 0 0 -1 2149613632 0 0 0 0 0 68495 0 0 20 0 1
> > 0
> > 350990 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483391 256 0 0 0
> > 17
> > 2 0 0 6855 0 0 [root@root ~]# cat /proc/2690/statm
> > 0 0 0 0 0 0 0
> > [root@root ~]# cat /proc/2690/stat
> > stat statm status
> > [root@root ~]# cat /proc/2690/status
> > Name: md0_raid5
> > State: R (running)
> > Tgid: 2690
> > Pid: 2690
> > PPid: 2
> > TracerPid: 0
> > Uid: 0 0 0 0
> > Gid: 0 0 0 0
> > Utrace: 0
> > FDSize: 64
> > Groups:
> > Threads: 1
> > SigQ: 2/128402
> > SigPnd: 0000000000000000
> > ShdPnd: 0000000000000000
> > SigBlk: 0000000000000000
> > SigIgn: fffffffffffffeff
> > SigCgt: 0000000000000100
> > CapInh: 0000000000000000
> > CapPrm: ffffffffffffffff
> > CapEff: fffffffffffffeff
> > CapBnd: ffffffffffffffff
> > Cpus_allowed: ffffff
> > Cpus_allowed_list: 0-23
> > Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
> > Mems_allowed_list: 0-1
> > voluntary_ctxt_switches: 5411612
> > nonvoluntary_ctxt_switches: 257032
> >
> >
> > Thanks,
> > Manibalan.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: RAID 1 metadata - keep separate from mirror disks ?
From: Suresh Babu Kandukuru @ 2015-02-12 11:42 UTC (permalink / raw)
To: Phil Turmel, linux-raid
In-Reply-To: <54DB6BF0.3040309@turmel.org>
Thanks Phil . This helps.
/Suresh
-----Original Message-----
From: Phil Turmel [mailto:philip@turmel.org]
Sent: Wednesday, February 11, 2015 8:19 PM
To: Suresh Babu Kandukuru; linux-raid@vger.kernel.org
Subject: Re: RAID 1 metadata - keep separate from mirror disks ?
Good morning Suresh,
On 02/11/2015 07:14 AM, Suresh Babu Kandukuru wrote:
> Hi There,
>
> On the RAID 1 metadata: is there any way to keep the metadata
> separate from the mirror disks? Could you guide us on this ?,
> please. In general, we need to keep all metadata off the device
> itself, leaving all the device available for user data. This is
> particularly important in the migration case, where we want to take an
> existing LUN and add a second leg to it to create the mirror device
> without changing any of the data or metadata on the LUN.
If you look at "man 4 md" you'll see some options. If a legacy array type meets your needs, you can operate without metadata at all. Use "mdadm --build" to assemble your raid at each boot.
Or, if your storage server can insert a leg ahead of you current LUN, you can then create the array with an explicit data offset matching the size of the inserted leg. Create it degraded with the existing LUN, then add (a) LUN(s) to start mirroring. This process will leave you the option to resize with more legs later.
Or you can add a leg to the end and create your array with version 1.0 metadata, which is placed at the end of the device.
Finally, you could write your own metadata container service for use with mdmon. (That's a bit beyond my ability, sorry.)
Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox