* [SOLVED] Re: On RAID5 read error during syncing - array .A.A
From: Emery Guevremont @ 2014-12-29 19:40 UTC (permalink / raw)
To: Emery Guevremont, linux-raid
Hello,
I just wanted to document in this thread the commands and steps I
followed with Robin's help after having a second drive fail during the
synching of a RAID5 array.
- First step I did was take a picture of my error on screen. And save
a copy of the output of
mdadm -E /dev/sd[abcd]3
and
mdadm --detail --scan /dev/md0.
and
smartctl -a /dev/sda #for each drives since it was nice to have the
serial numbers match up with each UUID device mdadm -E gave.
- Second step I did was to boot with a liveusb and use ddrescue to
clone 3 of my 4 drives. The first drive that failed was totally
malfunctioning, but I was able to ddrescue my 3 other drives. I booted
with ubuntu's rescue remix live usb distro and ddrescued each drive
one at a time with a new drives I purchased.
ddrescue -f -n /dev/sda /dev/sdb /mnt/sdc2/ddrescue.sda.log
ddrescue -f -r3 /dev/sda /dev/sdb /mnt/sdc2/ddrescue.sda.log
- After the cloning process was done, I booted again with only the new
cloned drives. I ran
mdadm -E /dev/sd[abc]3 | grep "UUID Dev"
and compared that output to the one I took original when everything
first failed. This was important since the ordering of my drives did
not match anymore. After figuring out the correct order I ran thing
command that Robin helped me sort out.
mdadm -v --create --assume-clean --level=5 --chunk=512
--size=1952795136 --raid-devices=4 /dev/md0 /dev/sdd3 /dev/sdb3
/dev/sdc3 missing
- At this point I was able to run
fsck -n /dev/md0
which told me my ext4 partition was clean.
From this point on you have 2 options. mount in read-only and backup,
or insert a new drive to add to your array. I went the safe way and
mounted in read-only:
mount -o ro,noload /dev/md0 /mnt
rsync -axvSH /mnt/ /mnt2/
- The final step for me was to add a new fourth drive to my degraded
raid5 array.
mdadm --add /dev/md0 /dev/sda3
A cat /proc/mdstat confirmed the resynching process.
^ permalink raw reply
* Re: Big trouble during reassemble a Raid5
From: John Stoffel @ 2014-12-29 18:32 UTC (permalink / raw)
To: sylvain.depuille; +Cc: linux-raid
In-Reply-To: <938521225.11264152.1419768981146.JavaMail.zimbra@laposte.net>
Sylvain,
I would recommend that you buy a replacement disk for the one
throwing errors and then run dd_rescue to copy as much data from the
dying disk to the replacement. Then, and only then, do you try to
reassemble the array with the --force option.
That disk is dying, and dying quickly.
Can you also post the output of
mdadm -E /dev/sd[bcde]1 for each disk, even the dying one, so we can
look at the counts and give you some more advice. Also, the output of
the mdadm --assemble --force /dev/md2 /dev/sd[bcde]1 would also be
good. The more info the better.
Good luck!
John
sylvain> i'm sorry to ask this questions but the raid 5 with 4 disk is
sylvain> in big trouble during re-assemble. 2 disks are out of order.
sylvain> I have change a disk of the raid 5 (sde) to growing the raid.
sylvain> But a second disk (sdc) have too many bad sector during the
sylvain> re-assemble, and shutdown the re-assemble. "mdadm --assemble
sylvain> --force /dev/md2 /dev/sd[bcde]1" I have try to correct bad
sylvain> sectors with badblocks, but it's finished by no more spare
sylvain> sectors and the disk still have some bad sector. badblocks
sylvain> -b 512 -o badblocks-sdc.txt -v -n /dev/sdc 1140170000
sylvain> 1140169336 1140169400 1140169401 1140169402 1140169403
sylvain> 1140169404 1140169405 1140169406 1140169407 1140169416
sylvain> 1140169417 1140169418 1140169419 1140169420 1140169421
sylvain> 1140169422 1140169423
sylvain> For information the mdadm examine return : cat mdadm-exam.txt
sylvain> /dev/sdb: MBR Magic : aa55 Partition[0] : 1953523120 sectors
sylvain> at 2048 (type fd) /dev/sdc: MBR Magic : aa55 Partition[0] :
sylvain> 1953520002 sectors at 63 (type fd) /dev/sdd: MBR Magic : aa55
sylvain> Partition[0] : 1953520002 sectors at 63 (type fd) /dev/sde:
sylvain> MBR Magic : aa55 Partition[0] : 4294965247 sectors at 2048
sylvain> (type fd)
sylvain> I have 2 way to solve the issue. The first, is to have
sylvain> special command to pass bad sector during re-assemble as
sylvain> "mdadm --assemble --force /dev/md2 /dev/sd[bcde]1" The second
sylvain> is change the disk sde with the old good one, but some datas
sylvain> have been changed on the raid since i have remove it. But
sylvain> these datas are not important. It's only logs and history
sylvain> activity.
sylvain> What can i do to recover a maximum datas without too many
sylvain> risk?
sylvain> Thank's in advance Best Regards
sylvain> ---------------------------------- Sylvain Depuille (in
sylvain> trouble)
sylvain> sylvain.depuille@laposte.net -- To unsubscribe from this
sylvain> list: send the line "unsubscribe linux-raid" in the body of a
sylvain> message to majordomo@vger.kernel.org More majordomo info at
sylvain> http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Big trouble during reassemble a Raid5
From: sylvain.depuille @ 2014-12-28 12:16 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <2105542796.11263344.1419768937013.JavaMail.zimbra@laposte.net>
Hello,
i'm sorry to ask this questions but the raid 5 with 4 disk is in big trouble during re-assemble. 2 disks are out of order.
I have change a disk of the raid 5 (sde) to growing the raid.
But a second disk (sdc) have too many bad sector during the re-assemble, and shutdown the re-assemble.
"mdadm --assemble --force /dev/md2 /dev/sd[bcde]1"
I have try to correct bad sectors with badblocks, but it's finished by no more spare sectors and the disk still have some bad sector.
badblocks -b 512 -o badblocks-sdc.txt -v -n /dev/sdc 1140170000 1140169336
1140169400
1140169401
1140169402
1140169403
1140169404
1140169405
1140169406
1140169407
1140169416
1140169417
1140169418
1140169419
1140169420
1140169421
1140169422
1140169423
For information the mdadm examine return :
cat mdadm-exam.txt
/dev/sdb:
MBR Magic : aa55
Partition[0] : 1953523120 sectors at 2048 (type fd)
/dev/sdc:
MBR Magic : aa55
Partition[0] : 1953520002 sectors at 63 (type fd)
/dev/sdd:
MBR Magic : aa55
Partition[0] : 1953520002 sectors at 63 (type fd)
/dev/sde:
MBR Magic : aa55
Partition[0] : 4294965247 sectors at 2048 (type fd)
I have 2 way to solve the issue.
The first, is to have special command to pass bad sector during re-assemble as "mdadm --assemble --force /dev/md2 /dev/sd[bcde]1"
The second is change the disk sde with the old good one, but some datas have been changed on the raid since i have remove it. But these datas are not important. It's only logs and history activity.
What can i do to recover a maximum datas without too many risk?
Thank's in advance
Best Regards
----------------------------------
Sylvain Depuille (in trouble)
sylvain.depuille@laposte.net
^ permalink raw reply
* A recommendation from Friend, screw college, I'm dropping out
From: fernalynbuendia @ 2014-12-28 11:29 UTC (permalink / raw)
To: rwpremiumpainters, good_mom2003, linux-raid, bennukoop, mari0z
Hello,
Hey!
I told you I wouldn't let you down!
First, thanks for putting up that cash so we could keep production on track. I really appreciate it.
Just give me another week and I'll have that $20k back in your pocket.
I've found something that can fund this whole project here:
==> ( http://tinyurl.com/buendia915 )
I'll pay you back and put in for the rest of the film myself.
We'll be showing “Mass Plan” at Cannes next year - guaranteed!
Best,
Fernalyn Buendia
Click below to view the webpage that your friend has recommended:
Friend, screw college, I'm dropping out has used the Tell-a-Friend form to send you this link.
We look forward to your visit!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] drivers:md: Remove unneeded fix me comments in dm-bio-prison.h
From: Nicholas Krause @ 2014-12-28 4:12 UTC (permalink / raw)
To: agk; +Cc: linux-raid, dm-devel, linux-kernel, snitzer
Removes two unneeded fix me comments for removing the header files,
persistent-data/dm-block-manager.h and dm-thin-metadata.h. These
headers are needed in order to build this file successfully and
for it to run,thus the comments should be removed as they are no
longer needed.
Signed-off-by: Nicholas Krause <xerofoify@gmail.com>
---
drivers/md/dm-bio-prison.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/dm-bio-prison.h b/drivers/md/dm-bio-prison.h
index 74cf011..15c31cf 100644
--- a/drivers/md/dm-bio-prison.h
+++ b/drivers/md/dm-bio-prison.h
@@ -7,8 +7,8 @@
#ifndef DM_BIO_PRISON_H
#define DM_BIO_PRISON_H
-#include "persistent-data/dm-block-manager.h" /* FIXME: for dm_block_t */
-#include "dm-thin-metadata.h" /* FIXME: for dm_thin_id */
+#include "persistent-data/dm-block-manager.h"
+#include "dm-thin-metadata.h"
#include <linux/bio.h>
#include <linux/rbtree.h>
--
2.1.0
^ permalink raw reply related
* Re: md_raid5 recovering failed need help
From: Phil Turmel @ 2014-12-27 15:24 UTC (permalink / raw)
To: Stephan Hafiz, linux-raid
In-Reply-To: <!&!AAAAAAAAAAAYAAAAAAAAAOgLllvGWotKlJcBYHqu4VUCgQAAEAAAAJjrgwWxkKFFjrGxfwSsmpoBAAAAAA==@hafis.de>
Good morning David,
{ or Stephan ? }
On 12/25/2014 09:24 AM, Stephan Hafiz wrote:
> Hi! I’m from germany and my raid and me needs help.
> My english isn’t very good, but i think it’s sufficient. And i think, this mailinglist is my last hope ☺
This is the right place for problems with linux raid arrays.
> So on, …. Here ist my problem.
> The raid5 has lost 2 of 5 disks. First one disk and then the second one.
Ok. Not uncommon.
[trim /]
> !SMART Status
> for i in a b c d e f; do echo Device sd$i; smartctl -H /dev/sd$i | egrep overall; echo; done;
> Device sda
> SMART overall-health self-assessment test result: PASSED
>
> Device sdb
> SMART overall-health self-assessment test result: PASSED
>
> Device sdc
> SMART overall-health self-assessment test result: PASSED
>
> Device sdd
> SMART overall-health self-assessment test result: PASSED
>
> Device sde
> SMART overall-health self-assessment test result: PASSED
>
> Device sdf
> SMART overall-health self-assessment test result: PASSED
It is extremely common to have an overall result of "PASSED" when you
aren't safe at all. Please redo this without trimming, like so:
for x in /dev/sd[b-f] ; do echo $x ; smartctl -x $x ; done
Paste the result at the end of you next mail--no need to attach nor need
for pastebin services.
Also, if you still have any syslogs from the time of the failure, it
would be good to see the kernel messages that triggered the drive
ejections from the raid.
> !mdadm version
> mdadm - v3.2.5 - 18th May 2012
> I have read about recent versions 3.3.x @ raid.wiki.kernel.org, i haven’t tested this version.
It may be necessary. You haven't reported your distro nor your kernel
version.
> !superblock informations
> Only the Events from sdb1 are off
[trim /]
Very good report! You've saved all the superblocks and you haven't
tried to do any --create operations.
[trim /]
> !reassemble force
> mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 --force
> mdadm: ignoring /dev/sdd1 as it reports /dev/sdc1 as failed
> mdadm: ignoring /dev/sde1 as it reports /dev/sdc1 as failed
> mdadm: ignoring /dev/sdf1 as it reports /dev/sdc1 as failed
> mdadm: /dev/md0 assembled from 1 drive - not enough to start the array.
This should have worked. Hmmm.
> i hope i don’t get the award „paint onself in to the corner“ ……
Probably not. :-)
The simplest way forward would probably be to boot a rescue CD (I
generally use the one from sysrescuecd.org) that has a recent kernel and
mdadm combination. Such CDs will probably attempt to assemble your
array during boot to /dev/md127 instead of /dev/md0, but it will fail.
So, within the rescue environment, do:
mdadm --stop /dev/md127 {or whatever shows in /proc/mdstat}
mdadm --assemble --force --verbose /dev/md0 /dev/sd[b-f]1
If that doesn't work, show us the verbose output, along with the
matching part of the dmesg.
If it does work, just do a clean shutdown and reboot back into your
regular OS.
> merry christmas … David
And Merry Christmas to you!
When you are done celebrating the revival of your array, you will need
to find out why it broke in the first place. The most common cause seen
on this list is the use of consumer-grade drives without dealing with
the timeout mismatch problem. You might want to review this old thread:
http://marc.info/?l=linux-raid&m=135811522817345&w=1
Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH][MD] raid : fix raid recoverying all data corruption, when j=0(sectors) interrupted by "echo idle > /sys/block/md0/md/sync_action"
From: 董杨 @ 2014-12-27 8:20 UTC (permalink / raw)
To: neilb; +Cc: linux-raid
Hi, Neil
I find a problem unintentionally: Raid5 is recovering, interrupted by
"echo idle > /sys/block/md0/md/sync_action", at this point,
j=0(sector), after a while, recovery thread is wake up, now j = 3
because when j=0, mddev->curr_resync = 3 by kernel code, so recovery
thread will start on the basic of j=3, which will lead all data
corruption. To avoid the problem,can we amend variable j by 8 sectors
alignment? as shown below:
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 709755f..fe4a5f9 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7400,7 +7400,7 @@ void md_do_sync(struct md_thread *thread)
if (test_bit(MD_RECOVERY_INTR, &mddev->recovery))
break;
- sectors = mddev->pers->sync_request(mddev, j, &skipped,
+ sectors = mddev->pers->sync_request(mddev, (j &
~((sector_t)STRIPE_SECTORS-1)), &skipped,
currspeed < speed_min(mddev));
if (sectors == 0) {
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
^ permalink raw reply related
* Re: [PATCH 00/24] Clustered MD RAID1
From: Marcus @ 2014-12-27 0:47 UTC (permalink / raw)
To: Peter Kieser; +Cc: Goldwyn Rodrigues, Neil Brown, lzhong, john, linux-raid
In-Reply-To: <549DE299.3010205@kieser.ca>
I don't think it's any different than exposing the same block device
to multiple servers, like an iSCSI or FC volume. It just allows you to
also treat a RAID device like you would any other SAN-type block
device by keeping the MD metadata consistent. You still have to
either put something on top of it that will be aware of the fact that
other hosts can see the block device and maintains its own consistency
(clustered FS), or be sure to only use it on one host at a time, again
like you would treat any other SAN volume.
On Fri, Dec 26, 2014 at 2:35 PM, Peter Kieser <peter@kieser.ca> wrote:
>
> On 2014-12-23 9:24 AM, Goldwyn Rodrigues wrote:
>>
>> This md device can be used as a regular shared device. There are no
>> restrictions on the type of filesystem or LVM you can use, as long as
>> you observe clustering rules of using a shared device.
>
>
> This is very vague. I feel like this is a huge layering violation. LVM or
> filesystem on top of mdadm should not have to be aware of what is going on
> below. Otherwise, I feel like other clustered filesystems that include RAID
> are better suited. As it stands, what are the benefits of using this over
> $CLUSTER_FS?
>
> -Peter
>
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: Goldwyn Rodrigues @ 2014-12-26 22:49 UTC (permalink / raw)
To: Peter Kieser, neilb; +Cc: lzhong, john, linux-raid
In-Reply-To: <549DE299.3010205@kieser.ca>
Hi Peter,
On 12/26/2014 04:35 PM, Peter Kieser wrote:
>
> On 2014-12-23 9:24 AM, Goldwyn Rodrigues wrote:
>> This md device can be used as a regular shared device. There are no
>> restrictions on the type of filesystem or LVM you can use, as long as
>> you observe clustering rules of using a shared device.
>
> This is very vague. I feel like this is a huge layering violation. LVM
> or filesystem on top of mdadm should not have to be aware of what is
> going on below.
How is this a layering violation? I said there are _no_ restrictions as
long you follow the clustering rules of using a shared device. This
device can be used as a regular device with the filesystem being unaware
of how the data is handled underneath. The only thing the md device
ensures is that the data (between mirrors) is consistent for the cluster.
By clustering rules I mean rules such as:
- Use a clustered filesystem
- Use cLVM (as opposed to usual LVM) on top of MD-RAID.
- Use local filesystems in active/passive mode
> Otherwise, I feel like other clustered filesystems that
> include RAID are better suited.
such as?
> As it stands, what are the benefits of
> using this over $CLUSTER_FS?
This is not a cluster filesystem. This is a RAID which is made cluster
aware for your shared storage. This provides redundancy at the device
level for a clustered filesystem. Your cluster shared storage is a
single point of failure (in case hardware-based raid is not used) and
this software-based solution avoids making it a single point of failure.
An example usage scenario (not limited to) could be using it across two
iSCSI devices to ensure higher availability.
--
Goldwyn
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: Peter Kieser @ 2014-12-26 22:35 UTC (permalink / raw)
To: Goldwyn Rodrigues, neilb; +Cc: lzhong, john, linux-raid
In-Reply-To: <5499A55E.3010705@suse.de>
[-- Attachment #1: Type: text/plain, Size: 588 bytes --]
On 2014-12-23 9:24 AM, Goldwyn Rodrigues wrote:
> This md device can be used as a regular shared device. There are no
> restrictions on the type of filesystem or LVM you can use, as long as
> you observe clustering rules of using a shared device.
This is very vague. I feel like this is a huge layering violation. LVM
or filesystem on top of mdadm should not have to be aware of what is
going on below. Otherwise, I feel like other clustered filesystems that
include RAID are better suited. As it stands, what are the benefits of
using this over $CLUSTER_FS?
-Peter
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4291 bytes --]
^ permalink raw reply
* md_raid5 recovering failed need help
From: Stephan Hafiz @ 2014-12-25 14:24 UTC (permalink / raw)
To: linux-raid; +Cc: forum
Hi! I’m from germany and my raid and me needs help.
My english isn’t very good, but i think it’s sufficient. And i think, this mailinglist is my last hope ☺
So on, …. Here ist my problem.
The raid5 has lost 2 of 5 disks. First one disk and then the second one.
I have been trying for several weeks to solve the problem alone … unsucessful ☹
And here some informations about this disaster.
!SMART Status
for i in a b c d e f; do echo Device sd$i; smartctl -H /dev/sd$i | egrep overall; echo; done;
Device sda
SMART overall-health self-assessment test result: PASSED
Device sdb
SMART overall-health self-assessment test result: PASSED
Device sdc
SMART overall-health self-assessment test result: PASSED
Device sdd
SMART overall-health self-assessment test result: PASSED
Device sde
SMART overall-health self-assessment test result: PASSED
Device sdf
SMART overall-health self-assessment test result: PASSED
!mdadm version
mdadm - v3.2.5 - 18th May 2012
I have read about recent versions 3.3.x @ raid.wiki.kernel.org, i haven’t tested this version.
!superblock informations
Only the Events from sdb1 are off
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x2
Array UUID : 19fc2188:8d10ae41:fee05ce5:06321b30
Name : pluto:0 (local to host pluto)
Creation Time : Tue Sep 24 22:45:51 2013
Raid Level : raid5
Raid Devices : 5
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Recovery Offset : 99655656 sectors
State : clean
Device UUID : 4aab22d5:21663d86:98f49007:0b776d30
Update Time : Thu Dec 18 20:12:19 2014
Checksum : f7a68c46 - correct
Events : 195
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAA ('A' == active, '.' == missing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 19fc2188:8d10ae41:fee05ce5:06321b30
Name : pluto:0 (local to host pluto)
Creation Time : Tue Sep 24 22:45:51 2013
Raid Level : raid5
Raid Devices : 5
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 61b9afe4:ba695181:7d1a1431:3e52621b
Update Time : Sat Dec 20 20:49:35 2014
Checksum : 10d2aaf0 - correct
Events : 217
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : .AAAA ('A' == active, '.' == missing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 19fc2188:8d10ae41:fee05ce5:06321b30
Name : pluto:0 (local to host pluto)
Creation Time : Tue Sep 24 22:45:51 2013
Raid Level : raid5
Raid Devices : 5
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : bf5ed2c0:95e837fa:e8b8eabf:b5cc8dba
Update Time : Sun Dec 21 12:22:33 2014
Checksum : 93dbc2b8 - correct
Events : 217
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : ..AAA ('A' == active, '.' == missing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 19fc2188:8d10ae41:fee05ce5:06321b30
Name : pluto:0 (local to host pluto)
Creation Time : Tue Sep 24 22:45:51 2013
Raid Level : raid5
Raid Devices : 5
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 52f8c258:96d54789:0cbdad3d:b17e85ae
Update Time : Sun Dec 21 12:22:33 2014
Checksum : 2c96ff6c - correct
Events : 217
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : ..AAA ('A' == active, '.' == missing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 19fc2188:8d10ae41:fee05ce5:06321b30
Name : pluto:0 (local to host pluto)
Creation Time : Tue Sep 24 22:45:51 2013
Raid Level : raid5
Raid Devices : 5
Avail Dev Size : 5860268943 (2794.39 GiB 3000.46 GB)
Array Size : 11720536064 (11177.57 GiB 12001.83 GB)
Used Dev Size : 5860268032 (2794.39 GiB 3000.46 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 418df97d:1658bced:ca887a61:2d7f3c05
Update Time : Sun Dec 21 12:22:33 2014
Checksum : 30c5e317 - correct
Events : 217
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : ..AAA ('A' == active, '.' == missing)
!Process Status md
cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : inactive sdf1[5](S) sdc1[1](S) sdd1[2](S) sde1[3](S) sdb1[6](S)
14650672357 blocks super 1.2
unused devices: <none>
!reassemble force
mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 --force
mdadm: ignoring /dev/sdd1 as it reports /dev/sdc1 as failed
mdadm: ignoring /dev/sde1 as it reports /dev/sdc1 as failed
mdadm: ignoring /dev/sdf1 as it reports /dev/sdc1 as failed
mdadm: /dev/md0 assembled from 1 drive - not enough to start the array.
i hope i don’t get the award „paint onself in to the corner“ ……
merry christmas … David
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
From: Manibalan P @ 2014-12-24 6:45 UTC (permalink / raw)
To: linux-raid; +Cc: NeilBrown
Dear Neil,
Few this for you kind attention,
1. I tried the same tesst with FC11 (2.6 kernel before MD code change). And the issue is not there
2. But with Centos 6.4 (2.6 after MD code change). I am getting this issue.. and also even with the latest kernel, able to reproduce the issue.
Thanks,
Manibalan.
-----Original Message-----
From: Manibalan P
Sent: Thursday, December 18, 2014 11:38 AM
To: 'linux-raid'
Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
Subject: RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
Dear neil,
I also compiled the latest 3.18 kernel on CentOS 6.4 with GIT MD pull patches form 3.19, that also ran in to the same issue after removing a drive during resync.
Dec 17 19:07:32 ITX002590129362 kernel: Linux version 3.18.0 (root@mycentos6) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Wed Dec 17 15:59:09 EST 2014 Dec 17 19:07:32 ITX002590129362 kernel: Command line: ro root=/dev/md255 rd_NO_LVM rd_NO_DM rhgb quiet md_mod.start_ro=1 nmi_watchdog=1 md_mod.start_dirty_degraded=1 … Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sda6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdb6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdc6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdh6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdi6> Dec 17 19:10:15 ITX002590129362 kernel: md: bind<sdj6> Dec 17 19:10:15 ITX002590129362 kernel: async_tx: api initialized (async) Dec 17 19:10:15 ITX002590129362 kernel: xor: measuring software checksum speed
Dec 17 19:10:15 ITX002590129362 kernel: prefetch64-sse: 10048.000 MB/sec
Dec 17 19:10:15 ITX002590129362 kernel: generic_sse: 8824.000 MB/sec
Dec 17 19:10:15 ITX002590129362 kernel: xor: using function: prefetch64-sse (10048.000 MB/sec)
Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x1 5921 MB/s
Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x2 6933 MB/s
Dec 17 19:10:15 ITX002590129362 kernel: raid6: sse2x4 7476 MB/s
Dec 17 19:10:15 ITX002590129362 kernel: raid6: using algorithm sse2x4 (7476 MB/s) Dec 17 19:10:15 ITX002590129362 kernel: raid6: using ssse3x2 recovery algorithm Dec 17 19:10:15 ITX002590129362 kernel: md: raid6 personality registered for level 6 Dec 17 19:10:15 ITX002590129362 kernel: md: raid5 personality registered for level 5 Dec 17 19:10:15 ITX002590129362 kernel: md: raid4 personality registered for level 4 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: not clean -- starting background reconstruction Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdj6 operational as raid disk 5 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdi6 operational as raid disk 4 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdh6 operational as raid disk 3 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdc6 operational as raid disk 2 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sdb6 operational as raid disk 1 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: device sda6 operational as raid disk 0 Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: allocated 0kB Dec 17 19:10:15 ITX002590129362 kernel: md/raid:md0: raid level 5 active with 6 out of 6 devices, algorithm 2 Dec 17 19:10:15 ITX002590129362 kernel: md0: detected capacity change from 0 to 2361059573760 Dec 17 19:10:15 ITX002590129362 kernel: md0: unknown partition table Dec 17 19:10:35 ITX002590129362 kernel: md: md0 switched to read-write mode.
Dec 17 19:10:35 ITX002590129362 kernel: md: resync of RAID array md0 Dec 17 19:10:35 ITX002590129362 kernel: md: minimum _guaranteed_ speed: 10000 KB/sec/disk.
Dec 17 19:10:35 ITX002590129362 kernel: md: using maximum available idle IO bandwidth (but not more than 30000 KB/sec) for resync.
Dec 17 19:10:35 ITX002590129362 kernel: md: using 128k window, over a total of 461144448k.
…
Started IOs using fio tool.
./fio --name=md0 --filename=/dev/md0 --thread --numjobs=10 --direct=1 --group_reporting --unlink=0 --loops=1 --offset=0 --randrepeat=1 --norandommap --scramble_buffers=1 --stonewall --ioengine=libaio --rw=randwrite --bs=8704 --iodepth=4000 --runtime=3000 --blockalign=512
…
Removed a drive form the system..
Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Dec 17 19:13:23 ITX002590129362 kernel: mpt2sas0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) ..
Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 02 69 03 70 00 00 10 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 40436592 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 51 b3 d0 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 206681040 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
Dec 17 19:13:23 ITX002590129362 kernel: Read(10): 28 00 0c 3a f3 40 00 00 18 00 Dec 17 19:13:23 ITX002590129362 kernel: blk_update_request: I/O error, dev sdh, sector 205189952 Dec 17 19:13:23 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:23 ITX002590129362 kernel: Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK … Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb 00 00 00 08 00 Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] Dec 17 19:13:25 ITX002590129362 kernel: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Dec 17 19:13:25 ITX002590129362 kernel: sd 0:0:7:0: [sdh] CDB:
Dec 17 19:13:25 ITX002590129362 kernel: Read(10): 28 00 26 8d eb f0 00 00 10 00 Dec 17 19:13:25 ITX002590129362 aghswap: devpath [0:0:7:0] action [remove] devtype [scsi_disk] Dec 17 19:13:25 ITX002590129362 aghswap: MHSA: Sent event 0 0 7 0 remove scsi_disk Dec 17 19:13:25 ITX002590129362 kernel: mpt2sas0: removing handle(0x0011), sas_addr(0x500605ba0101e305) Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Disk failure on sdh6, disabling device.
Dec 17 19:13:25 ITX002590129362 kernel: md/raid:md0: Operation continuing on 5 devices.
Dec 17 19:13:25 ITX002590129362 kernel: md: md0: resync interrupted.
Dec 17 19:13:25 ITX002590129362 kernel: md: checkpointing resync of md0.
..
Log messages after enabling debufgs on raid5.c, it is getting repeated continuously.
__get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0
__get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0
__get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0
__get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0
__get_priority_stripe: handle: busy hold: empty full_writes: 0 bypass_count: 0 handling stripe 273480328, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x10 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x11 read (null) write (null) written (null)
check 1: state 0x11 read (null) write (null) written (null)
check 0: state 0x18 read (null) write ffff8808029b6b00 written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=273480328 for sector 273480328, rmw=2 rcw=1 handling stripe 65238568, state=0x2041 cnt=1, pd_idx=5, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x10 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x18 read (null) write ffff88081a956b00 written (null)
check 1: state 0x11 read (null) write (null) written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=65238568 for sector 65238568, rmw=2 rcw=1 handling stripe 713868672, state=0x2041 cnt=1, pd_idx=4, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x10 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x18 read (null) write ffff88081f020100 written (null)
check 1: state 0x11 read (null) write (null) written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=713868672 for sector 713868672, rmw=2 rcw=1 handling stripe 729622496, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x10 read (null) write (null) written (null)
check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622496 for sector 729622496, rmw=2 rcw=1 handling stripe 729622504, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x10 read (null) write (null) written (null)
check 1: state 0x18 read (null) write ffff88081b9bae00 written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=729622504 for sector 729622504, rmw=2 rcw=1 handling stripe 245773680, state=0x2041 cnt=1, pd_idx=0, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x11 read (null) write (null) written (null)
check 1: state 0x18 read (null) write ffff88081cab7a00 written (null)
check 0: state 0x10 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=245773680 for sector 245773680, rmw=2 rcw=1 handling stripe 867965560, state=0x2041 cnt=1, pd_idx=1, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x11 read (null) write (null) written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x18 read (null) write ffff880802b2bf00 written (null)
check 1: state 0x10 read (null) write (null) written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=867965560 for sector 867965560, rmw=2 rcw=1 handling stripe 550162280, state=0x2041 cnt=1, pd_idx=2, qd_idx=-1 , check:0, reconstruct:0
check 5: state 0x11 read (null) write (null) written (null)
check 4: state 0x18 read (null) write ffff880802b08800 written (null)
check 3: state 0x0 read (null) write (null) written (null)
check 2: state 0x10 read (null) write (null) written (null)
check 1: state 0x11 read (null) write (null) written (null)
check 0: state 0x11 read (null) write (null) written (null)
locked=0 uptodate=3 to_read=0 to_write=1 failed=1 failed_num=3,-1 force RCW max_degraded=1, recovery_cp=7036944 sh->sector=550162280 for sector 550162280, rmw=2 rcw=1
Thanks,
Manibalan
-----Original Message-----
From: Manibalan P
Sent: Wednesday, December 17, 2014 12:11 PM
To: 'linux-raid'
Cc: 'NeilBrown'; Vijayarankan Muthirisavengopal; Dinakaran N
Subject: RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
Dear Neil,
The same Issue is reproducible in the latest upstream kernel also.
Tested in "3.17.6" latest stable upstream kernel and find the same issue.
[root@root ~]# modinfo raid456
filename: /lib/modules/3.17.6/kernel/drivers/md/raid456.ko
alias: raid6
alias: raid5
alias: md-level-6
alias: md-raid6
alias: md-personality-8
alias: md-level-4
alias: md-level-5
alias: md-raid4
alias: md-raid5
alias: md-personality-4
description: RAID4/5/6 (striping with parity) personality for MD
license: GPL
srcversion: 0EEF680023FDC7410F7989A
depends: async_raid6_recov,async_pq,async_tx,async_memcpy,async_xor
intree: Y
vermagic: 3.17.6 SMP mod_unload modversions
parm: devices_handle_discard_safely:Set to Y if all devices in each array reliably return zeroes on reads from discarded regions (bool)
Thanks,
Manibalan.
-----Original Message-----
From: Manibalan P
Sent: Wednesday, December 17, 2014 12:01 PM
To: 'linux-raid'
Cc: 'NeilBrown'
Subject: RE: md_raid5 using 100% CPU and hang with status resync=PENDING, if a drive is removed during initialization
Dear Neil,
We are facing IO struck issue with raid5 in the following scenario. (please see the attachment for the complete information) In RAID5 array, if a drive is removed while initialization and the same time if IO is happening to that md. Then IO is getting struck, and md_raid5 thread is using 100 % of CPU. Also the md state showing as resync=PENDING
Kernel : Issue found in the following kernels RHEL 6.5 (2.6.32-431.el6.x86_64) CentOS 7 (kernel-3.10.0-123.13.1.el7.x86_64)
Steps to Reproduce the issue:
1. Created a raid 5 md with 4 drives using the below mdadm command.
mdadm -C /dev/md0 -c 64 -l 5 -f -n 4 -e 1.2 /dev/sdb6 /dev/sdc6 /dev/sdd6 /dev/sde6
2. Make the md writable
mdadm –readwrite /dev/md0
3. Now md will start initialization
4. Run FIO Tool, the the below said configuration /usr/bin/fio --name=md0 --filename=/dev/md0 --thread --numjobs=10 --direct=1 --group_reporting --unlink=0 --loops=1 --offset=0 --randrepeat=1 --norandommap --scramble_buffers=1 --stonewall --ioengine=libaio --rw=randwrite --bs=8704 --iodepth=4000 --runtime=3000 --blockalign=512
4. During MD initialzing, remove a drive(either using MDADM set faulty/remove or remove manually)
5. Now the IO will struck, and cat /proc/mdstat shows states with resync=PENDING
---------------------------------------------------------------------------------------------
top - output show, md_raid5 using 100% cpu
top - 17:55:06 up 1:09, 3 users, load average: 11.98, 8.53, 3.99
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2690 root 20 0 0 0 0 R 100.0 0.0 6:44.41 md0_raid5
---------------------------------------------------------------------------------------------
dmesg - show the stack trace
INFO: task fio:2715 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 000000000000000a 0 2715 2654 0x00000080
ffff88043b623598 0000000000000082 0000000000000000 ffffffff81058d53
ffff88043b623548 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043b40b098 ffff88043b623fd8 000000000000fbc8 ffff88043b40b098 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff8140fa39>] ? md_wakeup_thread+0x39/0x70 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffffa0308f66>] ? make_request+0x306/0xc6c [raid456] [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81122283>] ? mempool_alloc+0x63/0x140 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c767a>] do_direct_IO+0x7ca/0xfa0 [<ffffffff811c8196>] __blockdev_direct_IO_newtrunc+0x346/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2717 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000004 0 2717 2654 0x00000080
ffff880439e97698 0000000000000082 ffff880439e97628 ffffffff81058d53
ffff880439e97648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043b0adab8 ffff880439e97fd8 000000000000fbc8 ffff88043b0adab8 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2718 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000005 0 2718 2654 0x00000080
ffff88043bc13698 0000000000000082 ffff88043bc13628 ffffffff81058d53
ffff88043bc13648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043b0ad058 ffff88043bc13fd8 000000000000fbc8 ffff88043b0ad058 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8e50>] __blockdev_direct_IO_newtrunc+0x1000/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2719 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000001 0 2719 2654 0x00000080
ffff880439ebb698 0000000000000082 ffff880439ebb628 ffffffff81058d53
ffff880439ebb648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043b0ac5f8 ffff880439ebbfd8 000000000000fbc8 ffff88043b0ac5f8 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2720 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000008 0 2720 2654 0x00000080
ffff88043b8cf698 0000000000000082 ffff88043b8cf628 ffffffff81058d53
ffff88043b8cf648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff880439e89af8 ffff88043b8cffd8 000000000000fbc8 ffff880439e89af8 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2721 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000000 0 2721 2654 0x00000080
ffff88043b047698 0000000000000082 ffff88043b047628 ffffffff81058d53
ffff88043b047648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff880439e89098 ffff88043b047fd8 000000000000fbc8 ffff880439e89098 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2722 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000000 0 2722 2654 0x00000080
ffff880439ea3698 0000000000000082 ffff880439ea3628 ffffffff81058d53
ffff880439ea3648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff880439e88638 ffff880439ea3fd8 000000000000fbc8 ffff880439e88638 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2723 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000006 0 2723 2654 0x00000080
ffff88043bf5f698 0000000000000082 ffff88043bf5f628 ffffffff81058d53
ffff88043bf5f648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043a183ab8 ffff88043bf5ffd8 000000000000fbc8 ffff88043a183ab8 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2724 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 000000000000000b 0 2724 2654 0x00000080
ffff88043be05698 0000000000000082 ffff88043be05628 ffffffff81058d53
ffff88043be05648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043a183058 ffff88043be05fd8 000000000000fbc8 ffff88043a183058 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task fio:2725 blocked for more than 120 seconds.
Not tainted 2.6.32-431.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fio D 0000000000000003 0 2725 2654 0x00000080
ffff88043be07698 0000000000000082 ffff88043be07628 ffffffff81058d53
ffff88043be07648 ffff880230e49cc0 ffff8802389aa228 ffff88043b2ad1b8
ffff88043a1825f8 ffff88043be07fd8 000000000000fbc8 ffff88043a1825f8 Call Trace:
[<ffffffff81058d53>] ? __wake_up+0x53/0x70 [<ffffffffa030334b>] ? md_raid5_unplug_device+0x7b/0x100 [raid456] [<ffffffffa0304146>] get_active_stripe+0x236/0x830 [raid456] [<ffffffff81065df0>] ? default_wake_function+0x0/0x20 [<ffffffff8109b5ce>] ? prepare_to_wait+0x4e/0x80 [<ffffffffa0308e15>] make_request+0x1b5/0xc6c [raid456] [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff811220e5>] ? mempool_alloc_slab+0x15/0x20 [<ffffffff81415b41>] md_make_request+0xe1/0x230 [<ffffffff811c3fd2>] ? bvec_alloc_bs+0x62/0x110 [<ffffffff811c32f0>] ? __bio_add_page+0x110/0x230 [<ffffffff81266c50>] generic_make_request+0x240/0x5a0 [<ffffffff811c742c>] ? do_direct_IO+0x57c/0xfa0 [<ffffffff81267020>] submit_bio+0x70/0x120 [<ffffffff811c8acd>] __blockdev_direct_IO_newtrunc+0xc7d/0x1270
[<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c9137>] __blockdev_direct_IO+0x77/0xe0 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff811c53b7>] blkdev_direct_IO+0x57/0x60 [<ffffffff811c4330>] ? blkdev_get_block+0x0/0x20 [<ffffffff81120552>] generic_file_direct_write+0xc2/0x190
[<ffffffff81121e71>] __generic_file_aio_write+0x3a1/0x490
[<ffffffff811d64c0>] ? aio_read_evt+0xa0/0x170 [<ffffffff811c490c>] blkdev_aio_write+0x3c/0xa0 [<ffffffff811c48d0>] ? blkdev_aio_write+0x0/0xa0 [<ffffffff811d4f64>] aio_rw_vect_retry+0x84/0x200 [<ffffffff811d6924>] aio_run_iocb+0x64/0x170 [<ffffffff811d7d51>] do_io_submit+0x291/0x920 [<ffffffff811d83f0>] sys_io_submit+0x10/0x20 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[root@root ~]# cat /proc/2690/stack
[<ffffffff810686da>] __cond_resched+0x2a/0x40 [<ffffffffa030361c>] ops_run_io+0x2c/0x920 [raid456] [<ffffffffa03052cc>] handle_stripe+0x9cc/0x2980 [raid456] [<ffffffffa03078a4>] raid5d+0x624/0x850 [raid456] [<ffffffff81416f05>] md_thread+0x115/0x150 [<ffffffff8109aef6>] kthread+0x96/0xa0 [<ffffffff8100c20a>] child_rip+0xa/0x20 [<ffffffffffffffff>] 0xffffffffffffffff
[root@root ~]# cat /proc/2690/stat
2690 (md0_raid5) R 2 0 0 0 -1 2149613632 0 0 0 0 0 68495 0 0 20 0 1 0 350990 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483391 256 0 0 0 17 2 0 0 6855 0 0 [root@root ~]# cat /proc/2690/statm
0 0 0 0 0 0 0
[root@root ~]# cat /proc/2690/stat
stat statm status
[root@root ~]# cat /proc/2690/status
Name: md0_raid5
State: R (running)
Tgid: 2690
Pid: 2690
PPid: 2
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
Utrace: 0
FDSize: 64
Groups:
Threads: 1
SigQ: 2/128402
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: fffffffffffffeff
SigCgt: 0000000000000100
CapInh: 0000000000000000
CapPrm: ffffffffffffffff
CapEff: fffffffffffffeff
CapBnd: ffffffffffffffff
Cpus_allowed: ffffff
Cpus_allowed_list: 0-23
Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003
Mems_allowed_list: 0-1
voluntary_ctxt_switches: 5411612
nonvoluntary_ctxt_switches: 257032
Thanks,
Manibalan.
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: John Stoffel @ 2014-12-23 21:47 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: neilb, lzhong, john, linux-raid
In-Reply-To: <5499A55E.3010705@suse.de>
This is much improved, thanks for takign the time to listen to my
comments. But I do have some questions and clarifications I think you
neeed to make.
Goldwyn> A howto to use cluster-md:
Goldwyn> 1. With your corosync/pacemaker based cluster with DLM
Goldwyn> running execute: # mdadm --create md0 --bitmap=clustered
Goldwyn> --raid-devices=2 --level=mirror --assume-clean <device1>
Goldwyn> <device2>
What are <device1> and <device2>? in terms of block devices? Are
they local? Shared across the cluster? Are the iSCSI or FibreChannel
block devices accessible from all nodes at the same time? It's not
clear, and this is a *key* issue to address and make sure end users
understand.
Goldwyn> With the option of --bitmap=clustered, it automatically
Goldwyn> creates multiple bitmaps (one for each node). The default
Goldwyn> currently is set to 4 nodes. However, you can set it by
Goldwyn> --nodes=<number> option. It also detects the cluster name
Goldwyn> which is required for creating a clustered md. In order to
Goldwyn> specify that, use --cluster-name=<name>
Goldwyn> 2. On other nodes, issue: # mdadm --assemble md0 <device1>
Goldwyn> <device2>
Same comment here, what are the limits/restrictions/expectations of
devices?
Goldwyn> This md device can be used as a regular shared device. There
Goldwyn> are no restrictions on the type of filesystem or LVM you can
Goldwyn> use, as long as you observe clustering rules of using a
Goldwyn> shared device.
Another place where you need to be more explicit. For example, if I'm
running ext3, I assume my cluster needs to be running in an
Active/Passive mode, so that only one node is accessing the filesystem
at a time, correct?
But if I'm running glusterfs, I could be using the block device and
the filesystem in an Active/Active mode?
Goldwyn> There is only one special case as opposed to a regular
Goldwyn> non-clustered md, which is to add a device. This is because
Goldwyn> all nodes should be able to "see" the device before adding
Goldwyn> it.
So this little snipped implies that my question above about device
restrictions is that you MUST be able to see each device from all
nodes, correct?
Goldwyn> You can (hot) add a spare device by issuing the regular --add
Goldwyn> command.
Goldwyn> # mdadm --manage /dev/md0 --add <device3>
Again, this device needs to be visible to all nodes.
Goldwyn> The other nodes must acknowledge that they see the device by
Goldwyn> issuing:
Goldwyn> # mdadm --manage /dev/md0 --cluster-confirm 2:<device3>
Goldwyn> where 2 is the raid slot number. This step can be automated
Goldwyn> using a udev script because the module sends a udev event
Goldwyn> when another node issues an --add. The uevent is with the
Goldwyn> usual device name parameters and:
Goldwyn> EVENT=ADD_DEVICE DEVICE_UUID=<uuid of the device>
Goldwyn> RAID_DISK=<slot number>
Goldwyn> Usually, you would use blkid to find the devices uuid and
Goldwyn> issue the --cluster-confirm command.
Goldwyn> If the node does not "see" the device, it must issue (or
Goldwyn> timeout):
Goldwyn> # mdadm --manage /dev/md0 --cluster-confirm 2:missing
This seems wrong, if the other node doesn't see the new device, how
will the other nodes know that it's there or not? What is the
communication channel between nodes? Do they use the DLM?
And what are the performance impacts and benefits of using this setup?
Why not use a cluster aware filesystem which already does RAID in some
manner? I'm honestly clueless about glusterFS or other Linux based
cluster filesystems, haven't had any need to run them, nor the time to
play with them.
I think you've got a good start now on the documentation, but you need
to expand more on the why and what is expected as an infrastructure.
Just a note that all devices must be visible on all nodes would be a
key thing to include.
Thanks,
John
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: Goldwyn Rodrigues @ 2014-12-23 17:24 UTC (permalink / raw)
To: neilb; +Cc: lzhong, john, linux-raid
In-Reply-To: <20141218161456.GA29504@shrek.lan>
Since, I missed on important setup details, I am sending an addendum to
explain in more detail how to setup a cluster-md.
Requirements
============
You would require a multi-node cluster setup using corosync and
pacemaker. You can read more about how to setup a cluster from the
one of the guides available [3]. Make sure that you are using corosync
version greater than 2.3.1
You need to have the Distributed Lock Manager (DLM) service running on
all the nodes of the cluster. A simple CRM configuration on my virtual
cluster is:
node 1084752262: node3
node 1084752351: node1
node 1084752358: node2
primitive dlm ocf:pacemaker:controld \
op start interval=0 timeout=90 \
op stop interval=0 timeout=100 \
op monitor interval=60 timeout=60
primitive stone stonith:external/libvirt \
params hostlist="node1,node2,node3"
hypervisor_uri="qemu+tcp://vmhnost/system" \
op start timeout=120s interval=0 \
op stop timeout=120s interval=0 \
op monitor interval=40s timeout=120s \
meta target-role=Started
group base-group dlm
clone base-clone base-group \
meta interleave=true target-role=Started
Note: The configuration may be different for your scenario.
This work requires some patches for the mdadm tool [1]. The changes in
mdadm are basic enough to get the clustered-md up and running. There are
a couple of options checks which are missing. Use with care. You would
need the corosync libraries to compile cluster related stuff in mdadm.
Download and install the patched mdadm.
A howto to use cluster-md:
1. With your corosync/pacemaker based cluster with DLM running execute:
# mdadm --create md0 --bitmap=clustered --raid-devices=2 --level=mirror
--assume-clean <device1> <device2>
With the option of --bitmap=clustered, it automatically creates multiple
bitmaps (one for each node). The default currently is set to 4 nodes.
However, you can set it by --nodes=<number> option.
It also detects the cluster name which is required for creating a
clustered md. In order to specify that, use --cluster-name=<name>
2. On other nodes, issue:
# mdadm --assemble md0 <device1> <device2>
This md device can be used as a regular shared device. There are no
restrictions on the type of filesystem or LVM you can use, as long as
you observe clustering rules of using a shared device.
There is only one special case as opposed to a regular non-clustered md,
which is to add a device. This is because all nodes should be able to
"see" the device before adding it.
You can (hot) add a spare device by issuing the regular --add command.
# mdadm --manage /dev/md0 --add <device3>
The other nodes must acknowledge that they see the device by issuing:
# mdadm --manage /dev/md0 --cluster-confirm 2:<device3>
where 2 is the raid slot number. This step can be automated
using a udev script because the module sends a udev event when another
node issues an --add. The uevent is with the usual device name
parameters and:
EVENT=ADD_DEVICE
DEVICE_UUID=<uuid of the device>
RAID_DISK=<slot number>
Usually, you would use blkid to find the devices uuid and issue the
--cluster-confirm command.
If the node does not "see" the device, it must issue (or timeout):
# mdadm --manage /dev/md0 --cluster-confirm 2:missing
References:
[1] mdadm tool changes: https://github.com/goldwynr/mdadm branch:cluster-md
[2] Patches against stable 3.14: https://github.com/goldwynr/linux
branch: cluster-md-devel
[3] A guide to setup a cluster using corosync/pacemaker
https://www.suse.com/documentation/sle-ha-12/singlehtml/book_sleha/book_sleha.html
(Note: The basic concepts of this guide should work with any distro)
--
--
Goldwyn
^ permalink raw reply
* [PATCH] Monitor: fix for regression with container devices
From: Artur Paszkiewicz @ 2014-12-23 13:55 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, pawel.baldysiak, Artur Paszkiewicz
This patch fixes 2 problems introduced by commit 9a518d8: not closing a
file descriptor and ignoring container devices. Array state is always
'inactive' for containers, so we make sure that the device is not a
container by reading the "level" sysfs entry.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
---
Monitor.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/Monitor.c b/Monitor.c
index 971d2ec..66d67ba 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -483,11 +483,17 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
strncmp(buf,"inact",5) == 0) {
if (fd >= 0)
close(fd);
- if (!st->err)
- alert("DeviceDisappeared", dev, NULL, ainfo);
- st->err++;
- return 0;
+ fd = sysfs_open(st->devnm, NULL, "level");
+ if (fd < 0 || read(fd, buf, 10) != 0) {
+ if (fd >= 0)
+ close(fd);
+ if (!st->err)
+ alert("DeviceDisappeared", dev, NULL, ainfo);
+ st->err++;
+ return 0;
+ }
}
+ close(fd);
}
fd = open(dev, O_RDONLY);
if (fd < 0) {
--
2.1.2
^ permalink raw reply related
* Re: [RFC PATCH 12/17] md/raid10: make sync_request_write() call bio_copy_data()
From: Dongsu Park @ 2014-12-23 12:31 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-kernel, Jens Axboe, Kent Overstreet, Ming Lin, Neil Brown,
linux-raid
In-Reply-To: <20141223104830.GD27600@infradead.org>
On 23.12.2014 02:48, Christoph Hellwig wrote:
> On Mon, Dec 22, 2014 at 12:48:39PM +0100, Dongsu Park wrote:
> > From: Kent Overstreet <kmo@daterainc.com>
> >
> > Refactor sync_request_write() of md/raid10 to use bio_copy_data()
> > instead of open coding bio_vec iterations.
>
> Seems like another one for the prep sweries?
Right. I'll also move this to the prep series with iov_iter conversions etc.
Thanks,
Dongsu
^ permalink raw reply
* Re: [RFC PATCH 12/17] md/raid10: make sync_request_write() call bio_copy_data()
From: Christoph Hellwig @ 2014-12-23 10:48 UTC (permalink / raw)
To: Dongsu Park
Cc: linux-kernel, Jens Axboe, Kent Overstreet, Ming Lin, Neil Brown,
linux-raid
In-Reply-To: <b7767767957ca758ac492e94dfb320bea4f52dd7.1419241597.git.dongsu.park@profitbricks.com>
On Mon, Dec 22, 2014 at 12:48:39PM +0100, Dongsu Park wrote:
> From: Kent Overstreet <kmo@daterainc.com>
>
> Refactor sync_request_write() of md/raid10 to use bio_copy_data()
> instead of open coding bio_vec iterations.
Seems like another one for the prep sweries?
^ permalink raw reply
* A recommendation from Fernalyn Buendia
From: fernalynbuendia @ 2014-12-23 1:31 UTC (permalink / raw)
To: rwpremiumpainters, good_mom2003, linux-raid, bennukoop, mari0z
Hello,
How Does $1,853.72/day sound? (It's $45K/mo)
-------------
MCB gives YOU a rare opportunity to
multiply YOUR GAINS by 10X with its
completely automated app.
YOU can now CASH IN BIG TIME without
having to move a finger....
...As MCB will not just show how to
multiply your income 10x but also it
will do it for you.
So from now on, you can relax and forget
about how to increase your income,
MCB will do it for you...
Now that’s something I say...
But you don’t have to trust my words...
You can actually VIEW what 223 ordinary
people are obtaining each from this
amazing app...
==> http://tinyurl.com/Fernalyn143
You’ve seen the diary of our current members...
You’ve seen Live videos...
You’ve also seen the Summary...
Now that’s what I call 100% TRANSPARENCY!
Only a HANDFUL of apps have dared to
be 100% transparent and MCB is one of them!
==> http://tinyurl.com/Fernalyn143
Plus, YOU get ongoing training videos
expert profit tactics and Premium Customer
Care services... AT NO COST!
Yes, you DON’T pay anything at all...
But YOU must ACT NOW...
==> http://tinyurl.com/Fernalyn143
The free offer is VALID only for a
LIMITED TIME...
After that, you might need to spend $297
per month... But why would you WAIT another
minute?
Those who roped in their licenses early
on are earning profits at This very moment
with MCB...HANDSFREE and STRESS FREE...
If you ACT NOW, you can enjoy 85% profits
within the next 5 minutes, Starting with
$10, $20 or even $5...It’s totally up to you...
YOU decide your financial goals and MCB
will achieve it for you...
So, don’t wait another minute...
==> http://tinyurl.com/Fernalyn143
All the best,
Fernalyn Buendia
Note: If the link is broken please copy paste this:
http://ohhyeah.mcbot.cpa.clicksure.com
Click below to view the webpage that your friend has recommended:
Fernalyn Buendia has used the Tell-a-Friend form to send you this link.
We look forward to your visit!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 01/24] md-cluster: Design Documentation
From: John Stoffel @ 2014-12-22 16:24 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: John Stoffel, neilb, lzhong, linux-raid
In-Reply-To: <5494A8D6.3090702@suse.de>
Goldwyn> Thanks for the review.
You're welcome. I'm not qualified to comment on the actual code
really, but I felt that you needed to be alot more up-front and
detailed in your docs, esp since you were writing a readme on this.
You should also talk more about the splitbrain possibilities, esp
with non-cluster aware filesystems like ext3/4 which might be setup
and used on there.
The detailed design docs are great too, but maybe they should really
be in the md-cluster-design.txt, while the md-cluster.txt file talks
about how to use it and what to expect.
Goldwyn> On 12/19/2014 09:38 AM, John Stoffel wrote:
>>>>>>> "Goldwyn" == Goldwyn Rodrigues <rgoldwyn@suse.de> writes:
>>
>> This is an interesting concept, but I think you're glossing over the
>> details here way too much. You're so close to the trees, that you're
>> missing the forest. You need to spell out the requirements in terms
>> of software, configuration, etc ahead of time.
>>
>> Showing how people can configure this for testing would be good as
>> well. Right now though, I wouldn't touch this with a ten foot pole.
Goldwyn> I mentioned a quick howto in patch zero. However, putting it in the
Goldwyn> design document will not hurt. Currently, it is known to work with
Goldwyn> corosync 2.3.x and pacemaker 1.1 on Kernels 3.14.x
>>
Goldwyn> Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Goldwyn> ---
Goldwyn> Documentation/md-cluster.txt | 178 +++++++++++++++++++++++++++++++++++++++++++
Goldwyn> 1 file changed, 178 insertions(+)
Goldwyn> create mode 100644 Documentation/md-cluster.txt
>>
Goldwyn> diff --git a/Documentation/md-cluster.txt b/Documentation/md-cluster.txt
Goldwyn> new file mode 100644
Goldwyn> index 0000000..038d0f0
Goldwyn> --- /dev/null
Goldwyn> +++ b/Documentation/md-cluster.txt
Goldwyn> @@ -0,0 +1,178 @@
Goldwyn> +The cluster MD is a shared-device RAID for a cluster.
>>
>>
>> How is this cluster setup? What are the restrictions? You just
>> straight into the ondisk format, without any introduction to the
>> problem and how you solve it.
Goldwyn> The cluster is a regular corosync/pacemaker cluster with DLM setup. I
Goldwyn> mentioned this in patch zero as well. However, I assumed configuring a
Goldwyn> cluster is not in the scope of the design document. This is the design
Goldwyn> of cluster-md. I agree it could use a foreword though.
>>
Goldwyn> +
Goldwyn> +
Goldwyn> +1. On-disk format
Goldwyn> +
Goldwyn> +Separate write-intent-bitmap are used for each cluster node.
Goldwyn> +The bitmaps record all writes that may have been started on that node,
Goldwyn> +and may not yet have finished. The on-disk layout is:
Goldwyn> +
Goldwyn> +0 4k 8k 12k
Goldwyn> +-------------------------------------------------------------------
Goldwyn> +| idle | md super | bm super [0] + bits |
Goldwyn> +| bm bits[0, contd] | bm super[1] + bits | bm bits[1, contd] |
Goldwyn> +| bm super[2] + bits | bm bits [2, contd] | bm super[3] + bits |
Goldwyn> +| bm bits [3, contd] | | |
Goldwyn> +
Goldwyn> +During "normal" functioning we assume the filesystem ensures that only one
Goldwyn> +node writes to any given block at a time, so a write
Goldwyn> +request will
Goldwyn> + - set the appropriate bit (if not already set)
Goldwyn> + - commit the write to all mirrors
Goldwyn> + - schedule the bit to be cleared after a timeout.
Goldwyn> +
Goldwyn> +Reads are just handled normally. It is up to the filesystem to
Goldwyn> +ensure one node doesn't read from a location where another node (or the same
Goldwyn> +node) is writing.
>>
>>
>> GAH! So what filesystem(s) are supported and known to work? Why this
>> this information not in the introduction? You just toss off this
>> statement without any context.
Goldwyn> The point here is data integrity is the responsibility of the
Goldwyn> filesystem. The cluster-md just ensures that all it has confirmed as
Goldwyn> written is stable and mirrored (RAID1). As for filesystem support, all
Goldwyn> device based filesystems are supported. However, we are targeting
Goldwyn> cluster based filesystems such as ocfs2. Yes, it could be moved in the
Goldwyn> Introduction.
>>
>> And you also seem to imply that I can't just put LVM volumes ontop of
>> this mirror either, which to me is a huge layering violation. If I'm
Goldwyn> No, I am not implying LVM cannot be used. LVM can be used in conjunction
Goldwyn> with cluster-md.
>> using MD to build RAID1 devices, I don't care how MD handles
>> reads/writes being out of sync. My filesystem or volumes on top get
>> consistent storage without having to know anything special.
Goldwyn> If you are reading the design document of cluster-md. I think you should
Goldwyn> be concerned on how out of sync data is handled in order to understand
Goldwyn> the design better. Filesystem just treat this as a normal block device
Goldwyn> and do not need to know anything special.
>>
>>
Goldwyn> +2. DLM Locks for management
Goldwyn> +
Goldwyn> +There are two locks for managing the device:
Goldwyn> +
Goldwyn> +2.1 Bitmap lock resource (bm_lockres)
Goldwyn> +
Goldwyn> + The bm_lockres protects individual node bitmaps. They are named in the
Goldwyn> + form bitmap001 for node 1, bitmap002 for node and so on. When a node
Goldwyn> + joins the cluster, it acquires the lock in PW mode and it stays so
>>
>> PW is what? Make sure you expand all your acronyms the first time you
>> use them so we can confirm we all understand them please.
Goldwyn> PW is Protected Write. I will add that.
>>
Goldwyn> + during the lifetime the node is part of the cluster. The lock resource
Goldwyn> + number is based on the slot number returned by the DLM subsystem. Since
Goldwyn> + DLM starts node count from one and bitmap slots start from zero, one is
Goldwyn> + subtracted from the DLM slot number to arrive at the bitmap slot number.
>>
>> Why do you bother? Why not just make the bitmap slots start at 1 and
>> reserve zero for a special case? Say that the bitmap is setup but not
>> initialized?
Goldwyn> What would that special case be? The bitmap setup is not a two-step
Goldwyn> process. If it is setup, it is also initialized.
>>
Goldwyn> +
Goldwyn> +3. Communication
Goldwyn> +
Goldwyn> +Each node has to communicate with other nodes when starting or ending
Goldwyn> +resync, and metadata superblock updates.
>>
>> HOW!!!! Does this all depend on DRDB being installed? Or some other
>> HA software?
Goldwyn> DLM. Mentioned later in the design. Yes, I will add that as well.
>>
Goldwyn> +
Goldwyn> +3.1 Message Types
Goldwyn> +
Goldwyn> + There are 3 types, of messages which are passed
Goldwyn> +
Goldwyn> + 3.1.1 METADATA_UPDATED: informs other nodes that the metadata has been
Goldwyn> + updated, and the node must re-read the md superblock. This is performed
Goldwyn> + synchronously.
Goldwyn> +
Goldwyn> + 3.1.2 RESYNC: informs other nodes that a resync is initiated or ended
Goldwyn> + so that each node may suspend or resume the region.
Goldwyn> +
Goldwyn> +3.2 Communication mechanism
Goldwyn> +
Goldwyn> + The DLM LVB is used to communicate within nodes of the cluster. There
Goldwyn> + are three resources used for the purpose:
Goldwyn> +
Goldwyn> + 3.2.1 Token: The resource which protects the entire communication
Goldwyn> + system. The node having the token resource is allowed to
Goldwyn> + communicate.
Goldwyn> +
Goldwyn> + 3.2.2 Message: The lock resource which carries the data to
Goldwyn> + communicate.
Goldwyn> +
Goldwyn> + 3.2.3 Ack: The resource, acquiring which means the message has been
Goldwyn> + acknowledged by all nodes in the cluster. The BAST of the resource
Goldwyn> + is used to inform the receive node that a node wants to communicate.
Goldwyn> +
Goldwyn> +The algorithm is:
Goldwyn> +
Goldwyn> + 1. receive status
Goldwyn> +
Goldwyn> + sender receiver receiver
Goldwyn> + ACK:CR ACK:CR ACK:CR
Goldwyn> +
Goldwyn> + 2. sender get EX of TOKEN
Goldwyn> + sender get EX of MESSAGE
Goldwyn> + sender receiver receiver
Goldwyn> + TOKEN:EX ACK:CR ACK:CR
Goldwyn> + MESSAGE:EX
Goldwyn> + ACK:CR
Goldwyn> +
Goldwyn> + Sender checks that it still needs to send a message. Messages received
Goldwyn> + or other events that happened while waiting for the TOKEN may have made
Goldwyn> + this message inappropriate or redundant.
Goldwyn> +
Goldwyn> + 3. sender write LVB.
Goldwyn> + sender down-convert MESSAGE from EX to CR
Goldwyn> + sender try to get EX of ACK
Goldwyn> + [ wait until all receiver has *processed* the MESSAGE ]
Goldwyn> +
Goldwyn> + [ triggered by bast of ACK ]
Goldwyn> + receiver get CR of MESSAGE
Goldwyn> + receiver read LVB
Goldwyn> + receiver processes the message
Goldwyn> + [ wait finish ]
Goldwyn> + receiver release ACK
Goldwyn> +
Goldwyn> + sender receiver receiver
Goldwyn> + TOKEN:EX MESSAGE:CR MESSAGE:CR
Goldwyn> + MESSAGE:CR
Goldwyn> + ACK:EX
Goldwyn> +
Goldwyn> + 4. triggered by grant of EX on ACK (indicating all receivers have processed
Goldwyn> + message)
Goldwyn> + sender down-convert ACK from EX to CR
Goldwyn> + sender release MESSAGE
Goldwyn> + sender release TOKEN
Goldwyn> + receiver upconvert to EX of MESSAGE
Goldwyn> + receiver get CR of ACK
Goldwyn> + receiver release MESSAGE
Goldwyn> +
Goldwyn> + sender receiver receiver
Goldwyn> + ACK:CR ACK:CR ACK:CR
Goldwyn> +
Goldwyn> +
Goldwyn> +4. Handling Failures
Goldwyn> +
Goldwyn> +4.1 Node Failure
Goldwyn> + When a node fails, the DLM informs the cluster with the slot. The node
>>
>> This needs to be re-worded. The cluster is the entire group of
>> machines, I think you mean:
>>
>> The DLM informs the node with the slot.
Goldwyn> Correct.
>>
>> And is a node failure as simple as a reboot? How about if the entire
>> cluster crashes, how to do you know which node is the more upto date
>> and should be the master?
Goldwyn> There is not concept of master here since everything is distributed. We
Goldwyn> do not want a central dependency. A node failure is it's inability to
Goldwyn> respond. It is usually STONITHd (Shoot the Other node in the Head) by
Goldwyn> the cluster resource management.
Goldwyn> The concept of bitmap is that data needs to be synced (that what I had
Goldwyn> been trying to explain in the point where you mentioned about
Goldwyn> filesystem). In case of a cluster failure, The first node to come up
Goldwyn> performs the "bitmap recovery" for all the bitmaps.
>>
Goldwyn> + starts a cluster recovery thread. The cluster recovery thread:
Goldwyn> + - acquires the bitmap<number> lock of the failed node
Goldwyn> + - opens the bitmap
Goldwyn> + - reads the bitmap of the failed node
Goldwyn> + - copies the set bitmap to local node
Goldwyn> + - cleans the bitmap of the failed node
Goldwyn> + - releases bitmap<number> lock of the failed node
Goldwyn> + - initiates resync of the bitmap on the current node
Goldwyn> +
Goldwyn> + The resync process, is the regular md resync. However, in a clustered
Goldwyn> + environment when a resync is performed, it needs to tell other nodes
Goldwyn> + of the areas which are suspended. Before a resync starts, the node
Goldwyn> + send out RESYNC_START with the (lo,hi) range of the area which needs
Goldwyn> + to be suspended. Each node maintains a suspend_list, which contains
Goldwyn> + the list of ranges which are currently suspended. On receiving
Goldwyn> + RESYNC_START, the node adds the range to the suspend_list. Similarly,
Goldwyn> + when the node performing resync finishes, it send RESYNC_FINISHED
Goldwyn> + to other nodes and other nodes remove the corresponding entry from
Goldwyn> + the suspend_list.
Goldwyn> +
Goldwyn> + A helper function, should_suspend() can be used to check if a particular
Goldwyn> + I/O range should be suspended or not.
Goldwyn> +
Goldwyn> +4.2 Device Failure
Goldwyn> + Device failures are handled and communicated with the metadata update
Goldwyn> + routine.
Goldwyn> +
Goldwyn> +5. Adding a new Device
Goldwyn> +For adding a new device, it is necessary that all nodes "see" the new device
Goldwyn> +to be added. For this, the following algorithm is used:
Goldwyn> +
Goldwyn> + 1. Node 1 issues mdadm --manage /dev/mdX --add /dev/sdYY which issues
Goldwyn> + ioctl(ADD_NEW_DISC with disc.state set to MD_DISK_CLUSTER_ADD)
Goldwyn> + 2. Node 1 sends NEWDISK with uuid and slot number
Goldwyn> + 3. Other nodes issue kobject_uevent_env with uuid and slot number
Goldwyn> + (Steps 4,5 could be a udev rule)
Goldwyn> + 4. In userspace, the node searches for the disk, perhaps
Goldwyn> + using blkid -t SUB_UUID=""
Goldwyn> + 5. Other nodes issue either of the following depending on whether the disk
Goldwyn> + was found:
Goldwyn> + ioctl(ADD_NEW_DISK with disc.state set to MD_DISK_CANDIDATE and
Goldwyn> + disc.number set to slot number)
Goldwyn> + ioctl(CLUSTERED_DISK_NACK)
Goldwyn> + 6. Other nodes drop lock on no-new-devs (CR) if device is found
Goldwyn> + 7. Node 1 attempts EX lock on no-new-devs
Goldwyn> + 8. If node 1 gets the lock, it sends METADATA_UPDATED after unmarking the disk
Goldwyn> + as SpareLocal
Goldwyn> + 9. If not (get no-new-dev lock), it fails the operation and sends METADATA_UPDATED
Goldwyn> + 10. Other nodes get the information whether a disk is added or not
Goldwyn> + by the following METADATA_UPDATED.
Goldwyn> +
Goldwyn> +
Goldwyn> --
Goldwyn> 2.1.2
>>
Goldwyn> --
Goldwyn> Goldwyn
^ permalink raw reply
* Re: 4 disks outage in RAID6
From: Phil Turmel @ 2014-12-22 16:23 UTC (permalink / raw)
To: Mark Kolama, linux-raid
In-Reply-To: <CAMvZ8g5dv-eXrgTAr58ib06LM+NhzbY7hkmFxaH7V71iL8Eg+w@mail.gmail.com>
Good morning Mark,
On 12/20/2014 12:03 PM, Mark Kolama wrote:
> Dear List,
>
> due to a controller failure, a raid6 with 16 drives
> lost 4 drives at once. The failure was noticed a few days later.
>
> a examine output of all 16 drive is listed at
> http://pastebin.com/4WH9xp7K
Ok. In the future, paste these in your email. kernel.org has a
generous size limit and this sort of stuff should stay in the archives.
As long as posters trim replies appropriately, it's not a problem.
> As you can see the event count differs on 4 drives with
> about 150 comparing to the other 12 drives.
>
> I have already tried:
> mdadm --assemble --scan:
> assembled from 12 drives - not enough to start the array.
>
> Then i tried:
> mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
> /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1
> /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 --force
This should have worked, unless the array wasn't stopped first. You
didn't show the actual response from mdadm, so we don't know.
There have also been bugs in various assembly features, so a report of
your kernel version and your mdadm version would be appropriate.
> /proc/mdstat after that:
>
> Personalities : [raid6] [raid5] [raid4]
> md0 : inactive sda1[0](S) sdp1[15](S) sdo1[14](S) sdn1[13](S)
> sdm1[12](S) sdl1[11](S) sdk1[10](S) sdj1[9](S) sdi1[8](S) sdh1[7](S)
> sdg1[6](S) sdf1[5](S) sde1[4](S) sdd1[3](S) sdc1[2](S) sdb1[1](S)
> 62353932288 blocks super 1.2
>
> No success either.
Try --force again, like so:
mdadm --stop /dev/md0
mdadm -Avf /dev/md0 /dev/sd[a-p]1
Show all of the output. Also show the tail of dmesg where this
operation happens.
> So the next try would be recreating the array ?
Absolutely not.
Phil
^ permalink raw reply
* [RFC PATCH 14/17] block: kill merge_bvec_fn() completely
From: Dongsu Park @ 2014-12-22 11:48 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, Kent Overstreet, Ming Lin, Dongsu Park, drbd-dev,
Jiri Kosina, Yehuda Sadeh, Sage Weil, Alex Elder, ceph-devel,
Alasdair Kergon, Mike Snitzer, dm-devel, Neil Brown, linux-raid,
Christoph Hellwig, Ming Lei, Martin K. Petersen
In-Reply-To: <cover.1419241597.git.dongsu.park@profitbricks.com>
From: Kent Overstreet <kmo@daterainc.com>
As generic_make_request() is now able to handle arbitrarily sized bios,
it's no longer necessary for each individual block driver to define its
own ->merge_bvec_fn() callback. Remove every invocation completely.
Signed-off-by: Kent Overstreet <kmo@daterainc.com>
[dpark: also remove ->merge_bvec_fn() in dm-thin as well as
dm-era-target, and resolve merge conflicts]
Signed-off-by: Dongsu Park <dongsu.park@profitbricks.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: drbd-dev@lists.linbit.com
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Yehuda Sadeh <yehuda@inktank.com>
Cc: Sage Weil <sage@inktank.com>
Cc: Alex Elder <elder@kernel.org>
Cc: ceph-devel@vger.kernel.org
Cc: Alasdair Kergon <agk@redhat.com>
Cc: Mike Snitzer <snitzer@redhat.com>
Cc: dm-devel@redhat.com
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Ming Lei <ming.lei@canonical.com>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
---
block/blk-merge.c | 17 +-----
block/blk-settings.c | 22 --------
drivers/block/drbd/drbd_int.h | 1 -
drivers/block/drbd/drbd_main.c | 1 -
drivers/block/drbd/drbd_req.c | 35 ------------
drivers/block/pktcdvd.c | 21 -------
drivers/block/rbd.c | 47 ----------------
drivers/md/dm-cache-target.c | 21 -------
drivers/md/dm-crypt.c | 16 ------
drivers/md/dm-era-target.c | 15 -----
drivers/md/dm-flakey.c | 16 ------
drivers/md/dm-linear.c | 16 ------
drivers/md/dm-snap.c | 15 -----
drivers/md/dm-stripe.c | 21 -------
drivers/md/dm-table.c | 8 ---
drivers/md/dm-thin.c | 31 -----------
drivers/md/dm-verity.c | 16 ------
drivers/md/dm.c | 120 +---------------------------------------
drivers/md/dm.h | 2 -
drivers/md/linear.c | 46 ----------------
drivers/md/md.c | 2 -
drivers/md/md.h | 8 ---
drivers/md/multipath.c | 21 -------
drivers/md/raid0.c | 57 -------------------
drivers/md/raid0.h | 2 -
drivers/md/raid1.c | 59 +-------------------
drivers/md/raid10.c | 122 +----------------------------------------
drivers/md/raid5.c | 28 ----------
include/linux/blkdev.h | 10 ----
include/linux/device-mapper.h | 4 --
30 files changed, 9 insertions(+), 791 deletions(-)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 3bc2068..8cd7a83 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -69,24 +69,13 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
struct bio *split;
struct bio_vec bv = { 0 }, bvprv = { 0 };
struct bvec_iter iter;
- unsigned seg_size = 0, nsegs = 0;
+ unsigned seg_size = 0, nsegs = 0, sectors = 0;
int prev = 0;
- struct bvec_merge_data bvm = {
- .bi_bdev = bio->bi_bdev,
- .bi_sector = bio->bi_iter.bi_sector,
- .bi_size = 0,
- .bi_rw = bio->bi_rw,
- };
-
bio_for_each_segment(bv, bio, iter) {
- if (q->merge_bvec_fn &&
- q->merge_bvec_fn(q, &bvm, &bv) < (int) bv.bv_len)
- goto split;
-
- bvm.bi_size += bv.bv_len;
+ sectors += bv.bv_len >> 9;
- if (bvm.bi_size >> 9 > queue_max_sectors(q))
+ if (sectors > queue_max_sectors(q))
goto split;
if (prev && blk_queue_cluster(q)) {
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 6ed2cbe..463a10a 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -53,28 +53,6 @@ void blk_queue_unprep_rq(struct request_queue *q, unprep_rq_fn *ufn)
}
EXPORT_SYMBOL(blk_queue_unprep_rq);
-/**
- * blk_queue_merge_bvec - set a merge_bvec function for queue
- * @q: queue
- * @mbfn: merge_bvec_fn
- *
- * Usually queues have static limitations on the max sectors or segments that
- * we can put in a request. Stacking drivers may have some settings that
- * are dynamic, and thus we have to query the queue whether it is ok to
- * add a new bio_vec to a bio at a given offset or not. If the block device
- * has such limitations, it needs to register a merge_bvec_fn to control
- * the size of bio's sent to it. Note that a block device *must* allow a
- * single page to be added to an empty bio. The block device driver may want
- * to use the bio_split() function to deal with these bio's. By default
- * no merge_bvec_fn is defined for a queue, and only the fixed limits are
- * honored.
- */
-void blk_queue_merge_bvec(struct request_queue *q, merge_bvec_fn *mbfn)
-{
- q->merge_bvec_fn = mbfn;
-}
-EXPORT_SYMBOL(blk_queue_merge_bvec);
-
void blk_queue_softirq_done(struct request_queue *q, softirq_done_fn *fn)
{
q->softirq_done_fn = fn;
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index b905e98..63ce2b0 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1449,7 +1449,6 @@ extern void do_submit(struct work_struct *ws);
extern void __drbd_make_request(struct drbd_device *, struct bio *, unsigned long);
extern void drbd_make_request(struct request_queue *q, struct bio *bio);
extern int drbd_read_remote(struct drbd_device *device, struct drbd_request *req);
-extern int drbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bvm, struct bio_vec *bvec);
extern int is_valid_ar_handle(struct drbd_request *, sector_t);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 1fc8342..f49f53e 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2775,7 +2775,6 @@ enum drbd_ret_code drbd_create_device(struct drbd_config_context *adm_ctx, unsig
This triggers a max_bio_size message upon first attach or connect */
blk_queue_max_hw_sectors(q, DRBD_MAX_BIO_SIZE_SAFE >> 8);
blk_queue_bounce_limit(q, BLK_BOUNCE_ANY);
- blk_queue_merge_bvec(q, drbd_merge_bvec);
q->queue_lock = &resource->req_lock;
device->md_io.page = alloc_page(GFP_KERNEL);
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index dee706d..b57d30b 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -1509,41 +1509,6 @@ void drbd_make_request(struct request_queue *q, struct bio *bio)
__drbd_make_request(device, bio, start_jif);
}
-/* This is called by bio_add_page().
- *
- * q->max_hw_sectors and other global limits are already enforced there.
- *
- * We need to call down to our lower level device,
- * in case it has special restrictions.
- *
- * We also may need to enforce configured max-bio-bvecs limits.
- *
- * As long as the BIO is empty we have to allow at least one bvec,
- * regardless of size and offset, so no need to ask lower levels.
- */
-int drbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bvm, struct bio_vec *bvec)
-{
- struct drbd_device *device = (struct drbd_device *) q->queuedata;
- unsigned int bio_size = bvm->bi_size;
- int limit = DRBD_MAX_BIO_SIZE;
- int backing_limit;
-
- if (bio_size && get_ldev(device)) {
- unsigned int max_hw_sectors = queue_max_hw_sectors(q);
- struct request_queue * const b =
- device->ldev->backing_bdev->bd_disk->queue;
- if (b->merge_bvec_fn) {
- bvm->bi_bdev = device->ldev->backing_bdev;
- backing_limit = b->merge_bvec_fn(b, bvm, bvec);
- limit = min(limit, backing_limit);
- }
- put_ldev(device);
- if ((limit >> 9) > max_hw_sectors)
- limit = max_hw_sectors << 9;
- }
- return limit;
-}
-
void request_timer_fn(unsigned long data)
{
struct drbd_device *device = (struct drbd_device *) data;
diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c
index ea10bd9..85eac23 100644
--- a/drivers/block/pktcdvd.c
+++ b/drivers/block/pktcdvd.c
@@ -2505,26 +2505,6 @@ end_io:
-static int pkt_merge_bvec(struct request_queue *q, struct bvec_merge_data *bmd,
- struct bio_vec *bvec)
-{
- struct pktcdvd_device *pd = q->queuedata;
- sector_t zone = get_zone(bmd->bi_sector, pd);
- int used = ((bmd->bi_sector - zone) << 9) + bmd->bi_size;
- int remaining = (pd->settings.size << 9) - used;
- int remaining2;
-
- /*
- * A bio <= PAGE_SIZE must be allowed. If it crosses a packet
- * boundary, pkt_make_request() will split the bio.
- */
- remaining2 = PAGE_SIZE - bmd->bi_size;
- remaining = max(remaining, remaining2);
-
- BUG_ON(remaining < 0);
- return remaining;
-}
-
static void pkt_init_queue(struct pktcdvd_device *pd)
{
struct request_queue *q = pd->disk->queue;
@@ -2532,7 +2512,6 @@ static void pkt_init_queue(struct pktcdvd_device *pd)
blk_queue_make_request(q, pkt_make_request);
blk_queue_logical_block_size(q, CD_FRAMESIZE);
blk_queue_max_hw_sectors(q, PACKET_MAX_SECTORS);
- blk_queue_merge_bvec(q, pkt_merge_bvec);
q->queuedata = pd;
}
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 3ec85df..0c0e2d0 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -3459,52 +3459,6 @@ static void rbd_request_fn(struct request_queue *q)
queue_work(rbd_wq, &rbd_dev->rq_work);
}
-/*
- * a queue callback. Makes sure that we don't create a bio that spans across
- * multiple osd objects. One exception would be with a single page bios,
- * which we handle later at bio_chain_clone_range()
- */
-static int rbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bmd,
- struct bio_vec *bvec)
-{
- struct rbd_device *rbd_dev = q->queuedata;
- sector_t sector_offset;
- sector_t sectors_per_obj;
- sector_t obj_sector_offset;
- int ret;
-
- /*
- * Find how far into its rbd object the partition-relative
- * bio start sector is to offset relative to the enclosing
- * device.
- */
- sector_offset = get_start_sect(bmd->bi_bdev) + bmd->bi_sector;
- sectors_per_obj = 1 << (rbd_dev->header.obj_order - SECTOR_SHIFT);
- obj_sector_offset = sector_offset & (sectors_per_obj - 1);
-
- /*
- * Compute the number of bytes from that offset to the end
- * of the object. Account for what's already used by the bio.
- */
- ret = (int) (sectors_per_obj - obj_sector_offset) << SECTOR_SHIFT;
- if (ret > bmd->bi_size)
- ret -= bmd->bi_size;
- else
- ret = 0;
-
- /*
- * Don't send back more than was asked for. And if the bio
- * was empty, let the whole thing through because: "Note
- * that a block device *must* allow a single page to be
- * added to an empty bio."
- */
- rbd_assert(bvec->bv_len <= PAGE_SIZE);
- if (ret > (int) bvec->bv_len || !bmd->bi_size)
- ret = (int) bvec->bv_len;
-
- return ret;
-}
-
static void rbd_free_disk(struct rbd_device *rbd_dev)
{
struct gendisk *disk = rbd_dev->disk;
@@ -3771,7 +3725,6 @@ static int rbd_init_disk(struct rbd_device *rbd_dev)
q->limits.max_discard_sectors = segment_size / SECTOR_SIZE;
q->limits.discard_zeroes_data = 1;
- blk_queue_merge_bvec(q, rbd_merge_bvec);
disk->queue = q;
q->queuedata = rbd_dev;
diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c
index 1e96d78..72fd9d3 100644
--- a/drivers/md/dm-cache-target.c
+++ b/drivers/md/dm-cache-target.c
@@ -3277,26 +3277,6 @@ static int cache_iterate_devices(struct dm_target *ti,
return r;
}
-/*
- * We assume I/O is going to the origin (which is the volume
- * more likely to have restrictions e.g. by being striped).
- * (Looking up the exact location of the data would be expensive
- * and could always be out of date by the time the bio is submitted.)
- */
-static int cache_bvec_merge(struct dm_target *ti,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct cache *cache = ti->private;
- struct request_queue *q = bdev_get_queue(cache->origin_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = cache->origin_dev->bdev;
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void set_discard_limits(struct cache *cache, struct queue_limits *limits)
{
/*
@@ -3340,7 +3320,6 @@ static struct target_type cache_target = {
.status = cache_status,
.message = cache_message,
.iterate_devices = cache_iterate_devices,
- .merge = cache_bvec_merge,
.io_hints = cache_io_hints,
};
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 08981be..723c176 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1951,21 +1951,6 @@ error:
return -EINVAL;
}
-static int crypt_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct crypt_config *cc = ti->private;
- struct request_queue *q = bdev_get_queue(cc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = cc->dev->bdev;
- bvm->bi_sector = cc->start + dm_target_offset(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int crypt_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -1986,7 +1971,6 @@ static struct target_type crypt_target = {
.preresume = crypt_preresume,
.resume = crypt_resume,
.message = crypt_message,
- .merge = crypt_merge,
.iterate_devices = crypt_iterate_devices,
};
diff --git a/drivers/md/dm-era-target.c b/drivers/md/dm-era-target.c
index ad913cd..0119ebf 100644
--- a/drivers/md/dm-era-target.c
+++ b/drivers/md/dm-era-target.c
@@ -1673,20 +1673,6 @@ static int era_iterate_devices(struct dm_target *ti,
return fn(ti, era->origin_dev, 0, get_dev_size(era->origin_dev), data);
}
-static int era_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct era *era = ti->private;
- struct request_queue *q = bdev_get_queue(era->origin_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = era->origin_dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void era_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
struct era *era = ti->private;
@@ -1717,7 +1703,6 @@ static struct target_type era_target = {
.status = era_status,
.message = era_message,
.iterate_devices = era_iterate_devices,
- .merge = era_merge,
.io_hints = era_io_hints
};
diff --git a/drivers/md/dm-flakey.c b/drivers/md/dm-flakey.c
index b257e46..d955b3e 100644
--- a/drivers/md/dm-flakey.c
+++ b/drivers/md/dm-flakey.c
@@ -387,21 +387,6 @@ static int flakey_ioctl(struct dm_target *ti, unsigned int cmd, unsigned long ar
return r ? : __blkdev_driver_ioctl(dev->bdev, dev->mode, cmd, arg);
}
-static int flakey_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct flakey_c *fc = ti->private;
- struct request_queue *q = bdev_get_queue(fc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = fc->dev->bdev;
- bvm->bi_sector = flakey_map_sector(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int flakey_iterate_devices(struct dm_target *ti, iterate_devices_callout_fn fn, void *data)
{
struct flakey_c *fc = ti->private;
@@ -419,7 +404,6 @@ static struct target_type flakey_target = {
.end_io = flakey_end_io,
.status = flakey_status,
.ioctl = flakey_ioctl,
- .merge = flakey_merge,
.iterate_devices = flakey_iterate_devices,
};
diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c
index 53e848c..7dd5fc8 100644
--- a/drivers/md/dm-linear.c
+++ b/drivers/md/dm-linear.c
@@ -130,21 +130,6 @@ static int linear_ioctl(struct dm_target *ti, unsigned int cmd,
return r ? : __blkdev_driver_ioctl(dev->bdev, dev->mode, cmd, arg);
}
-static int linear_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct linear_c *lc = ti->private;
- struct request_queue *q = bdev_get_queue(lc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = lc->dev->bdev;
- bvm->bi_sector = linear_map_sector(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int linear_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -162,7 +147,6 @@ static struct target_type linear_target = {
.map = linear_map,
.status = linear_status,
.ioctl = linear_ioctl,
- .merge = linear_merge,
.iterate_devices = linear_iterate_devices,
};
diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c
index 864b03f..2e6bb7e 100644
--- a/drivers/md/dm-snap.c
+++ b/drivers/md/dm-snap.c
@@ -2234,20 +2234,6 @@ static void origin_status(struct dm_target *ti, status_type_t type,
}
}
-static int origin_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct dm_origin *o = ti->private;
- struct request_queue *q = bdev_get_queue(o->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = o->dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int origin_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -2265,7 +2251,6 @@ static struct target_type origin_target = {
.map = origin_map,
.resume = origin_resume,
.status = origin_status,
- .merge = origin_merge,
.iterate_devices = origin_iterate_devices,
};
diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c
index f8b37d4..09bb2fe 100644
--- a/drivers/md/dm-stripe.c
+++ b/drivers/md/dm-stripe.c
@@ -412,26 +412,6 @@ static void stripe_io_hints(struct dm_target *ti,
blk_limits_io_opt(limits, chunk_size * sc->stripes);
}
-static int stripe_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct stripe_c *sc = ti->private;
- sector_t bvm_sector = bvm->bi_sector;
- uint32_t stripe;
- struct request_queue *q;
-
- stripe_map_sector(sc, bvm_sector, &stripe, &bvm_sector);
-
- q = bdev_get_queue(sc->stripe[stripe].dev->bdev);
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = sc->stripe[stripe].dev->bdev;
- bvm->bi_sector = sc->stripe[stripe].physical_start + bvm_sector;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static struct target_type stripe_target = {
.name = "striped",
.version = {1, 5, 1},
@@ -443,7 +423,6 @@ static struct target_type stripe_target = {
.status = stripe_status,
.iterate_devices = stripe_iterate_devices,
.io_hints = stripe_io_hints,
- .merge = stripe_merge,
};
int __init dm_stripe_init(void)
diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 3afae9e..6c14cb4 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -443,14 +443,6 @@ static int dm_set_device_limits(struct dm_target *ti, struct dm_dev *dev,
q->limits.alignment_offset,
(unsigned long long) start << SECTOR_SHIFT);
- /*
- * Check if merge fn is supported.
- * If not we'll force DM to use PAGE_SIZE or
- * smaller I/O, just to be safe.
- */
- if (dm_queue_merge_is_compulsory(q) && !ti->type->merge)
- blk_limits_max_hw_sectors(limits,
- (unsigned int) (PAGE_SIZE >> 9));
return 0;
}
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index 8735543..8e0dd5e 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -3548,20 +3548,6 @@ static int pool_iterate_devices(struct dm_target *ti,
return fn(ti, pt->data_dev, 0, ti->len, data);
}
-static int pool_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct pool_c *pt = ti->private;
- struct request_queue *q = bdev_get_queue(pt->data_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = pt->data_dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void set_discard_limits(struct pool_c *pt, struct queue_limits *limits)
{
struct pool *pool = pt->pool;
@@ -3653,7 +3639,6 @@ static struct target_type pool_target = {
.resume = pool_resume,
.message = pool_message,
.status = pool_status,
- .merge = pool_merge,
.iterate_devices = pool_iterate_devices,
.io_hints = pool_io_hints,
};
@@ -3979,21 +3964,6 @@ err:
DMEMIT("Error");
}
-static int thin_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct thin_c *tc = ti->private;
- struct request_queue *q = bdev_get_queue(tc->pool_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = tc->pool_dev->bdev;
- bvm->bi_sector = dm_target_offset(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int thin_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -4028,7 +3998,6 @@ static struct target_type thin_target = {
.presuspend = thin_presuspend,
.postsuspend = thin_postsuspend,
.status = thin_status,
- .merge = thin_merge,
.iterate_devices = thin_iterate_devices,
};
diff --git a/drivers/md/dm-verity.c b/drivers/md/dm-verity.c
index 7a7bab8..25d76a8 100644
--- a/drivers/md/dm-verity.c
+++ b/drivers/md/dm-verity.c
@@ -564,21 +564,6 @@ static int verity_ioctl(struct dm_target *ti, unsigned cmd,
cmd, arg);
}
-static int verity_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct dm_verity *v = ti->private;
- struct request_queue *q = bdev_get_queue(v->data_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = v->data_dev->bdev;
- bvm->bi_sector = verity_map_sector(v, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int verity_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -872,7 +857,6 @@ static struct target_type verity_target = {
.map = verity_map,
.status = verity_status,
.ioctl = verity_ioctl,
- .merge = verity_merge,
.iterate_devices = verity_iterate_devices,
.io_hints = verity_io_hints,
};
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 5ce28a4..7cf0dde 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -116,9 +116,8 @@ EXPORT_SYMBOL_GPL(dm_get_rq_mapinfo);
#define DMF_FREEING 3
#define DMF_DELETING 4
#define DMF_NOFLUSH_SUSPENDING 5
-#define DMF_MERGE_IS_OPTIONAL 6
-#define DMF_DEFERRED_REMOVE 7
-#define DMF_SUSPENDED_INTERNALLY 8
+#define DMF_DEFERRED_REMOVE 6
+#define DMF_SUSPENDED_INTERNALLY 7
/*
* A dummy definition to make RCU happy.
@@ -1586,60 +1585,6 @@ static void __split_and_process_bio(struct mapped_device *md,
* CRUD END
*---------------------------------------------------------------*/
-static int dm_merge_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mapped_device *md = q->queuedata;
- struct dm_table *map = dm_get_live_table_fast(md);
- struct dm_target *ti;
- sector_t max_sectors;
- int max_size = 0;
-
- if (unlikely(!map))
- goto out;
-
- ti = dm_table_find_target(map, bvm->bi_sector);
- if (!dm_target_is_valid(ti))
- goto out;
-
- /*
- * Find maximum amount of I/O that won't need splitting
- */
- max_sectors = min(max_io_len(bvm->bi_sector, ti),
- (sector_t) queue_max_sectors(q));
- max_size = (max_sectors << SECTOR_SHIFT) - bvm->bi_size;
- if (unlikely(max_size < 0)) /* this shouldn't _ever_ happen */
- max_size = 0;
-
- /*
- * merge_bvec_fn() returns number of bytes
- * it can accept at this offset
- * max is precomputed maximal io size
- */
- if (max_size && ti->type->merge)
- max_size = ti->type->merge(ti, bvm, biovec, max_size);
- /*
- * If the target doesn't support merge method and some of the devices
- * provided their merge_bvec method (we know this by looking for the
- * max_hw_sectors that dm_set_device_limits may set), then we can't
- * allow bios with multiple vector entries. So always set max_size
- * to 0, and the code below allows just one page.
- */
- else if (queue_max_hw_sectors(q) <= PAGE_SIZE >> 9)
- max_size = 0;
-
-out:
- dm_put_live_table_fast(md);
- /*
- * Always allow an entire first page
- */
- if (max_size <= biovec->bv_len && !(bvm->bi_size >> SECTOR_SHIFT))
- max_size = biovec->bv_len;
-
- return max_size;
-}
-
/*
* The request function that just remaps the bio built up by
* dm_merge_bvec.
@@ -2030,7 +1975,6 @@ static void dm_init_md_queue(struct mapped_device *md)
md->queue->backing_dev_info.congested_data = md;
blk_queue_make_request(md->queue, dm_request);
blk_queue_bounce_limit(md->queue, BLK_BOUNCE_ANY);
- blk_queue_merge_bvec(md->queue, dm_merge_bvec);
}
/*
@@ -2241,59 +2185,6 @@ static void __set_size(struct mapped_device *md, sector_t size)
}
/*
- * Return 1 if the queue has a compulsory merge_bvec_fn function.
- *
- * If this function returns 0, then the device is either a non-dm
- * device without a merge_bvec_fn, or it is a dm device that is
- * able to split any bios it receives that are too big.
- */
-int dm_queue_merge_is_compulsory(struct request_queue *q)
-{
- struct mapped_device *dev_md;
-
- if (!q->merge_bvec_fn)
- return 0;
-
- if (q->make_request_fn == dm_request) {
- dev_md = q->queuedata;
- if (test_bit(DMF_MERGE_IS_OPTIONAL, &dev_md->flags))
- return 0;
- }
-
- return 1;
-}
-
-static int dm_device_merge_is_compulsory(struct dm_target *ti,
- struct dm_dev *dev, sector_t start,
- sector_t len, void *data)
-{
- struct block_device *bdev = dev->bdev;
- struct request_queue *q = bdev_get_queue(bdev);
-
- return dm_queue_merge_is_compulsory(q);
-}
-
-/*
- * Return 1 if it is acceptable to ignore merge_bvec_fn based
- * on the properties of the underlying devices.
- */
-static int dm_table_merge_is_optional(struct dm_table *table)
-{
- unsigned i = 0;
- struct dm_target *ti;
-
- while (i < dm_table_get_num_targets(table)) {
- ti = dm_table_get_target(table, i++);
-
- if (ti->type->iterate_devices &&
- ti->type->iterate_devices(ti, dm_device_merge_is_compulsory, NULL))
- return 0;
- }
-
- return 1;
-}
-
-/*
* Returns old map, which caller must destroy.
*/
static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
@@ -2302,7 +2193,6 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
struct dm_table *old_map;
struct request_queue *q = md->queue;
sector_t size;
- int merge_is_optional;
size = dm_table_get_size(t);
@@ -2328,17 +2218,11 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
__bind_mempools(md, t);
- merge_is_optional = dm_table_merge_is_optional(t);
-
old_map = rcu_dereference_protected(md->map, lockdep_is_held(&md->suspend_lock));
rcu_assign_pointer(md->map, t);
md->immutable_target_type = dm_table_get_immutable_target_type(t);
dm_table_set_restrictions(t, q, limits);
- if (merge_is_optional)
- set_bit(DMF_MERGE_IS_OPTIONAL, &md->flags);
- else
- clear_bit(DMF_MERGE_IS_OPTIONAL, &md->flags);
if (old_map)
dm_sync_table(md);
diff --git a/drivers/md/dm.h b/drivers/md/dm.h
index 84b0f9e4..08f47fc 100644
--- a/drivers/md/dm.h
+++ b/drivers/md/dm.h
@@ -76,8 +76,6 @@ bool dm_table_request_based(struct dm_table *t);
void dm_table_free_md_mempools(struct dm_table *t);
struct dm_md_mempools *dm_table_get_md_mempools(struct dm_table *t);
-int dm_queue_merge_is_compulsory(struct request_queue *q);
-
void dm_lock_md_type(struct mapped_device *md);
void dm_unlock_md_type(struct mapped_device *md);
void dm_set_md_type(struct mapped_device *md, unsigned type);
diff --git a/drivers/md/linear.c b/drivers/md/linear.c
index 64713b7..d831a5b 100644
--- a/drivers/md/linear.c
+++ b/drivers/md/linear.c
@@ -52,51 +52,6 @@ static inline struct dev_info *which_dev(struct mddev *mddev, sector_t sector)
return conf->disks + lo;
}
-/**
- * linear_mergeable_bvec -- tell bio layer if two requests can be merged
- * @q: request queue
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can take at this offset
- */
-static int linear_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- struct dev_info *dev0;
- unsigned long maxsectors, bio_sectors = bvm->bi_size >> 9;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int maxbytes = biovec->bv_len;
- struct request_queue *subq;
-
- rcu_read_lock();
- dev0 = which_dev(mddev, sector);
- maxsectors = dev0->end_sector - sector;
- subq = bdev_get_queue(dev0->rdev->bdev);
- if (subq->merge_bvec_fn) {
- bvm->bi_bdev = dev0->rdev->bdev;
- bvm->bi_sector -= dev0->end_sector - dev0->rdev->sectors;
- maxbytes = min(maxbytes, subq->merge_bvec_fn(subq, bvm,
- biovec));
- }
- rcu_read_unlock();
-
- if (maxsectors < bio_sectors)
- maxsectors = 0;
- else
- maxsectors -= bio_sectors;
-
- if (maxsectors <= (PAGE_SIZE >> 9 ) && bio_sectors == 0)
- return maxbytes;
-
- if (maxsectors > (maxbytes >> 9))
- return maxbytes;
- else
- return maxsectors << 9;
-}
-
static int linear_congested(void *data, int bits)
{
struct mddev *mddev = data;
@@ -217,7 +172,6 @@ static int linear_run (struct mddev *mddev)
mddev->private = conf;
md_set_array_sectors(mddev, linear_size(mddev, 0, 0));
- blk_queue_merge_bvec(mddev->queue, linear_mergeable_bvec);
mddev->queue->backing_dev_info.congested_fn = linear_congested;
mddev->queue->backing_dev_info.congested_data = mddev;
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 48234eb..0e34b76 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5033,7 +5033,6 @@ static void md_clean(struct mddev *mddev)
mddev->changed = 0;
mddev->degraded = 0;
mddev->safemode = 0;
- mddev->merge_check_needed = 0;
mddev->bitmap_info.offset = 0;
mddev->bitmap_info.default_offset = 0;
mddev->bitmap_info.default_space = 0;
@@ -5201,7 +5200,6 @@ static int do_md_stop(struct mddev *mddev, int mode,
__md_stop_writes(mddev);
__md_stop(mddev);
- mddev->queue->merge_bvec_fn = NULL;
mddev->queue->backing_dev_info.congested_fn = NULL;
/* tell userspace to handle 'inactive' */
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 03cec5b..4932445 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -132,10 +132,6 @@ enum flag_bits {
Bitmap_sync, /* ..actually, not quite In_sync. Need a
* bitmap-based recovery to get fully in sync
*/
- Unmerged, /* device is being added to array and should
- * be considerred for bvec_merge_fn but not
- * yet for actual IO
- */
WriteMostly, /* Avoid reading if at all possible */
AutoDetected, /* added by auto-detect */
Blocked, /* An error occurred but has not yet
@@ -366,10 +362,6 @@ struct mddev {
int degraded; /* whether md should consider
* adding a spare
*/
- int merge_check_needed; /* at least one
- * member device
- * has a
- * merge_bvec_fn */
atomic_t recovery_active; /* blocks scheduled, but not written */
wait_queue_head_t recovery_wait;
diff --git a/drivers/md/multipath.c b/drivers/md/multipath.c
index 399272f..2f82954 100644
--- a/drivers/md/multipath.c
+++ b/drivers/md/multipath.c
@@ -261,18 +261,6 @@ static int multipath_add_disk(struct mddev *mddev, struct md_rdev *rdev)
disk_stack_limits(mddev->gendisk, rdev->bdev,
rdev->data_offset << 9);
- /* as we don't honour merge_bvec_fn, we must never risk
- * violating it, so limit ->max_segments to one, lying
- * within a single page.
- * (Note: it is very unlikely that a device with
- * merge_bvec_fn will be involved in multipath.)
- */
- if (q->merge_bvec_fn) {
- blk_queue_max_segments(mddev->queue, 1);
- blk_queue_segment_boundary(mddev->queue,
- PAGE_CACHE_SIZE - 1);
- }
-
spin_lock_irq(&conf->device_lock);
mddev->degraded--;
rdev->raid_disk = path;
@@ -436,15 +424,6 @@ static int multipath_run (struct mddev *mddev)
disk_stack_limits(mddev->gendisk, rdev->bdev,
rdev->data_offset << 9);
- /* as we don't honour merge_bvec_fn, we must never risk
- * violating it, not that we ever expect a device with
- * a merge_bvec_fn to be involved in multipath */
- if (rdev->bdev->bd_disk->queue->merge_bvec_fn) {
- blk_queue_max_segments(mddev->queue, 1);
- blk_queue_segment_boundary(mddev->queue,
- PAGE_CACHE_SIZE - 1);
- }
-
if (!test_bit(Faulty, &rdev->flags))
working_disks++;
}
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index ba6b85d..bc4c0b6 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -195,9 +195,6 @@ static int create_strip_zones(struct mddev *mddev, struct r0conf **private_conf)
disk_stack_limits(mddev->gendisk, rdev1->bdev,
rdev1->data_offset << 9);
- if (rdev1->bdev->bd_disk->queue->merge_bvec_fn)
- conf->has_merge_bvec = 1;
-
if (!smallest || (rdev1->sectors < smallest->sectors))
smallest = rdev1;
cnt++;
@@ -354,59 +351,6 @@ static struct md_rdev *map_sector(struct mddev *mddev, struct strip_zone *zone,
+ sector_div(sector, zone->nb_dev)];
}
-/**
- * raid0_mergeable_bvec -- tell bio layer if two requests can be merged
- * @q: request queue
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can accept at this offset
- */
-static int raid0_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- struct r0conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- sector_t sector_offset = sector;
- int max;
- unsigned int chunk_sectors = mddev->chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
- struct strip_zone *zone;
- struct md_rdev *rdev;
- struct request_queue *subq;
-
- if (is_power_of_2(chunk_sectors))
- max = (chunk_sectors - ((sector & (chunk_sectors-1))
- + bio_sectors)) << 9;
- else
- max = (chunk_sectors - (sector_div(sector, chunk_sectors)
- + bio_sectors)) << 9;
- if (max < 0)
- max = 0; /* bio_add cannot handle a negative return */
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- if (max < biovec->bv_len)
- /* too small already, no need to check further */
- return max;
- if (!conf->has_merge_bvec)
- return max;
-
- /* May need to check subordinate device */
- sector = sector_offset;
- zone = find_zone(mddev->private, §or_offset);
- rdev = map_sector(mddev, zone, sector, §or_offset);
- subq = bdev_get_queue(rdev->bdev);
- if (subq->merge_bvec_fn) {
- bvm->bi_bdev = rdev->bdev;
- bvm->bi_sector = sector_offset + zone->dev_start +
- rdev->data_offset;
- return min(max, subq->merge_bvec_fn(subq, bvm, biovec));
- } else
- return max;
-}
-
static sector_t raid0_size(struct mddev *mddev, sector_t sectors, int raid_disks)
{
sector_t array_sectors = 0;
@@ -471,7 +415,6 @@ static int raid0_run(struct mddev *mddev)
mddev->queue->backing_dev_info.ra_pages = 2* stripe;
}
- blk_queue_merge_bvec(mddev->queue, raid0_mergeable_bvec);
dump_zones(mddev);
ret = md_integrity_register(mddev);
diff --git a/drivers/md/raid0.h b/drivers/md/raid0.h
index 05539d9..7127a62 100644
--- a/drivers/md/raid0.h
+++ b/drivers/md/raid0.h
@@ -12,8 +12,6 @@ struct r0conf {
struct md_rdev **devlist; /* lists of rdevs, pointed to
* by strip_zone->dev */
int nr_strip_zones;
- int has_merge_bvec; /* at least one member has
- * a merge_bvec_fn */
};
#endif
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 40b35be..c2f236c 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -551,7 +551,6 @@ static int read_balance(struct r1conf *conf, struct r1bio *r1_bio, int *max_sect
rdev = rcu_dereference(conf->mirrors[disk].rdev);
if (r1_bio->bios[disk] == IO_BLOCKED
|| rdev == NULL
- || test_bit(Unmerged, &rdev->flags)
|| test_bit(Faulty, &rdev->flags))
continue;
if (!test_bit(In_sync, &rdev->flags) &&
@@ -701,39 +700,6 @@ static int read_balance(struct r1conf *conf, struct r1bio *r1_bio, int *max_sect
return best_disk;
}
-static int raid1_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- struct r1conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max = biovec->bv_len;
-
- if (mddev->merge_check_needed) {
- int disk;
- rcu_read_lock();
- for (disk = 0; disk < conf->raid_disks * 2; disk++) {
- struct md_rdev *rdev = rcu_dereference(
- conf->mirrors[disk].rdev);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = sector +
- rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- }
- rcu_read_unlock();
- }
- return max;
-
-}
-
int md_raid1_congested(struct mddev *mddev, int bits)
{
struct r1conf *conf = mddev->private;
@@ -1266,8 +1232,7 @@ read_again:
break;
}
r1_bio->bios[i] = NULL;
- if (!rdev || test_bit(Faulty, &rdev->flags)
- || test_bit(Unmerged, &rdev->flags)) {
+ if (!rdev || test_bit(Faulty, &rdev->flags)) {
if (i < conf->raid_disks)
set_bit(R1BIO_Degraded, &r1_bio->state);
continue;
@@ -1611,7 +1576,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
struct raid1_info *p;
int first = 0;
int last = conf->raid_disks - 1;
- struct request_queue *q = bdev_get_queue(rdev->bdev);
if (mddev->recovery_disabled == conf->recovery_disabled)
return -EBUSY;
@@ -1619,11 +1583,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
if (rdev->raid_disk >= 0)
first = last = rdev->raid_disk;
- if (q->merge_bvec_fn) {
- set_bit(Unmerged, &rdev->flags);
- mddev->merge_check_needed = 1;
- }
-
for (mirror = first; mirror <= last; mirror++) {
p = conf->mirrors+mirror;
if (!p->rdev) {
@@ -1655,19 +1614,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
break;
}
}
- if (err == 0 && test_bit(Unmerged, &rdev->flags)) {
- /* Some requests might not have seen this new
- * merge_bvec_fn. We must wait for them to complete
- * before merging the device fully.
- * First we make sure any code which has tested
- * our function has submitted the request, then
- * we wait for all outstanding requests to complete.
- */
- synchronize_sched();
- freeze_array(conf, 0);
- unfreeze_array(conf);
- clear_bit(Unmerged, &rdev->flags);
- }
md_integrity_add_rdev(rdev, mddev);
if (mddev->queue && blk_queue_discard(bdev_get_queue(rdev->bdev)))
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
@@ -2810,8 +2756,6 @@ static struct r1conf *setup_conf(struct mddev *mddev)
goto abort;
disk->rdev = rdev;
q = bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn)
- mddev->merge_check_needed = 1;
disk->head_position = 0;
disk->seq_start = MaxSector;
@@ -2957,7 +2901,6 @@ static int run(struct mddev *mddev)
if (mddev->queue) {
mddev->queue->backing_dev_info.congested_fn = raid1_congested;
mddev->queue->backing_dev_info.congested_data = mddev;
- blk_queue_merge_bvec(mddev->queue, raid1_mergeable_bvec);
if (discard_supported)
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD,
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 4a40354..08772a9 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -672,94 +672,6 @@ static sector_t raid10_find_virt(struct r10conf *conf, sector_t sector, int dev)
return (vchunk << geo->chunk_shift) + offset;
}
-/**
- * raid10_mergeable_bvec -- tell bio layer if a two requests can be merged
- * @q: request queue
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can accept at this offset
- * This requires checking for end-of-chunk if near_copies != raid_disks,
- * and for subordinate merge_bvec_fns if merge_check_needed.
- */
-static int raid10_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- struct r10conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max;
- unsigned int chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
- struct geom *geo = &conf->geo;
-
- chunk_sectors = (conf->geo.chunk_mask & conf->prev.chunk_mask) + 1;
- if (conf->reshape_progress != MaxSector &&
- ((sector >= conf->reshape_progress) !=
- conf->mddev->reshape_backwards))
- geo = &conf->prev;
-
- if (geo->near_copies < geo->raid_disks) {
- max = (chunk_sectors - ((sector & (chunk_sectors - 1))
- + bio_sectors)) << 9;
- if (max < 0)
- /* bio_add cannot handle a negative return */
- max = 0;
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- } else
- max = biovec->bv_len;
-
- if (mddev->merge_check_needed) {
- struct {
- struct r10bio r10_bio;
- struct r10dev devs[conf->copies];
- } on_stack;
- struct r10bio *r10_bio = &on_stack.r10_bio;
- int s;
- if (conf->reshape_progress != MaxSector) {
- /* Cannot give any guidance during reshape */
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- return 0;
- }
- r10_bio->sector = sector;
- raid10_find_phys(conf, r10_bio);
- rcu_read_lock();
- for (s = 0; s < conf->copies; s++) {
- int disk = r10_bio->devs[s].devnum;
- struct md_rdev *rdev = rcu_dereference(
- conf->mirrors[disk].rdev);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = r10_bio->devs[s].addr
- + rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- rdev = rcu_dereference(conf->mirrors[disk].replacement);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = r10_bio->devs[s].addr
- + rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- }
- rcu_read_unlock();
- }
- return max;
-}
-
/*
* This routine returns the disk from which the requested read should
* be done. There is a per-array 'next expected sequential IO' sector
@@ -822,12 +734,10 @@ retry:
disk = r10_bio->devs[slot].devnum;
rdev = rcu_dereference(conf->mirrors[disk].replacement);
if (rdev == NULL || test_bit(Faulty, &rdev->flags) ||
- test_bit(Unmerged, &rdev->flags) ||
r10_bio->devs[slot].addr + sectors > rdev->recovery_offset)
rdev = rcu_dereference(conf->mirrors[disk].rdev);
if (rdev == NULL ||
- test_bit(Faulty, &rdev->flags) ||
- test_bit(Unmerged, &rdev->flags))
+ test_bit(Faulty, &rdev->flags))
continue;
if (!test_bit(In_sync, &rdev->flags) &&
r10_bio->devs[slot].addr + sectors > rdev->recovery_offset)
@@ -1336,11 +1246,9 @@ retry_write:
blocked_rdev = rrdev;
break;
}
- if (rdev && (test_bit(Faulty, &rdev->flags)
- || test_bit(Unmerged, &rdev->flags)))
+ if (rdev && (test_bit(Faulty, &rdev->flags)))
rdev = NULL;
- if (rrdev && (test_bit(Faulty, &rrdev->flags)
- || test_bit(Unmerged, &rrdev->flags)))
+ if (rrdev && (test_bit(Faulty, &rrdev->flags)))
rrdev = NULL;
r10_bio->devs[i].bio = NULL;
@@ -1787,7 +1695,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
int mirror;
int first = 0;
int last = conf->geo.raid_disks - 1;
- struct request_queue *q = bdev_get_queue(rdev->bdev);
if (mddev->recovery_cp < MaxSector)
/* only hot-add to in-sync arrays, as recovery is
@@ -1800,11 +1707,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
if (rdev->raid_disk >= 0)
first = last = rdev->raid_disk;
- if (q->merge_bvec_fn) {
- set_bit(Unmerged, &rdev->flags);
- mddev->merge_check_needed = 1;
- }
-
if (rdev->saved_raid_disk >= first &&
conf->mirrors[rdev->saved_raid_disk].rdev == NULL)
mirror = rdev->saved_raid_disk;
@@ -1843,19 +1745,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
rcu_assign_pointer(p->rdev, rdev);
break;
}
- if (err == 0 && test_bit(Unmerged, &rdev->flags)) {
- /* Some requests might not have seen this new
- * merge_bvec_fn. We must wait for them to complete
- * before merging the device fully.
- * First we make sure any code which has tested
- * our function has submitted the request, then
- * we wait for all outstanding requests to complete.
- */
- synchronize_sched();
- freeze_array(conf, 0);
- unfreeze_array(conf);
- clear_bit(Unmerged, &rdev->flags);
- }
md_integrity_add_rdev(rdev, mddev);
if (mddev->queue && blk_queue_discard(bdev_get_queue(rdev->bdev)))
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
@@ -2404,7 +2293,6 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10
d = r10_bio->devs[sl].devnum;
rdev = rcu_dereference(conf->mirrors[d].rdev);
if (rdev &&
- !test_bit(Unmerged, &rdev->flags) &&
test_bit(In_sync, &rdev->flags) &&
is_badblock(rdev, r10_bio->devs[sl].addr + sect, s,
&first_bad, &bad_sectors) == 0) {
@@ -2458,7 +2346,6 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10
d = r10_bio->devs[sl].devnum;
rdev = rcu_dereference(conf->mirrors[d].rdev);
if (!rdev ||
- test_bit(Unmerged, &rdev->flags) ||
!test_bit(In_sync, &rdev->flags))
continue;
@@ -3657,8 +3544,6 @@ static int run(struct mddev *mddev)
disk->rdev = rdev;
}
q = bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn)
- mddev->merge_check_needed = 1;
diff = (rdev->new_data_offset - rdev->data_offset);
if (!mddev->reshape_backwards)
diff = -diff;
@@ -3757,7 +3642,6 @@ static int run(struct mddev *mddev)
stripe /= conf->geo.near_copies;
if (mddev->queue->backing_dev_info.ra_pages < 2 * stripe)
mddev->queue->backing_dev_info.ra_pages = 2 * stripe;
- blk_queue_merge_bvec(mddev->queue, raid10_mergeable_bvec);
}
if (md_integrity_register(mddev))
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 40e464c..6008a30 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4103,32 +4103,6 @@ static int raid5_congested(void *data, int bits)
md_raid5_congested(mddev, bits);
}
-/* We want read requests to align with chunks where possible,
- * but write requests don't need to.
- */
-static int raid5_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max;
- unsigned int chunk_sectors = mddev->chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
-
- if ((bvm->bi_rw & 1) == WRITE)
- return biovec->bv_len; /* always allow writes to be mergeable */
-
- if (mddev->new_chunk_sectors < mddev->chunk_sectors)
- chunk_sectors = mddev->new_chunk_sectors;
- max = (chunk_sectors - ((sector & (chunk_sectors - 1)) + bio_sectors)) << 9;
- if (max < 0) max = 0;
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- else
- return max;
-}
-
static int in_chunk_boundary(struct mddev *mddev, struct bio *bio)
{
sector_t sector = bio->bi_iter.bi_sector + get_start_sect(bio->bi_bdev);
@@ -6152,8 +6126,6 @@ static int run(struct mddev *mddev)
if (mddev->queue->backing_dev_info.ra_pages < 2 * stripe)
mddev->queue->backing_dev_info.ra_pages = 2 * stripe;
- blk_queue_merge_bvec(mddev->queue, raid5_mergeable_bvec);
-
mddev->queue->backing_dev_info.congested_data = mddev;
mddev->queue->backing_dev_info.congested_fn = raid5_congested;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index c03e37a..7a8c95c 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -239,14 +239,6 @@ typedef int (prep_rq_fn) (struct request_queue *, struct request *);
typedef void (unprep_rq_fn) (struct request_queue *, struct request *);
struct bio_vec;
-struct bvec_merge_data {
- struct block_device *bi_bdev;
- sector_t bi_sector;
- unsigned bi_size;
- unsigned long bi_rw;
-};
-typedef int (merge_bvec_fn) (struct request_queue *, struct bvec_merge_data *,
- struct bio_vec *);
typedef void (softirq_done_fn)(struct request *);
typedef int (dma_drain_needed_fn)(struct request *);
typedef int (lld_busy_fn) (struct request_queue *q);
@@ -327,7 +319,6 @@ struct request_queue {
make_request_fn *make_request_fn;
prep_rq_fn *prep_rq_fn;
unprep_rq_fn *unprep_rq_fn;
- merge_bvec_fn *merge_bvec_fn;
softirq_done_fn *softirq_done_fn;
rq_timed_out_fn *rq_timed_out_fn;
dma_drain_needed_fn *dma_drain_needed;
@@ -1036,7 +1027,6 @@ extern void blk_queue_lld_busy(struct request_queue *q, lld_busy_fn *fn);
extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
extern void blk_queue_unprep_rq(struct request_queue *, unprep_rq_fn *ufn);
-extern void blk_queue_merge_bvec(struct request_queue *, merge_bvec_fn *);
extern void blk_queue_dma_alignment(struct request_queue *, int);
extern void blk_queue_update_dma_alignment(struct request_queue *, int);
extern void blk_queue_softirq_done(struct request_queue *, softirq_done_fn *);
diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h
index ca6d2acc..2f7f2df 100644
--- a/include/linux/device-mapper.h
+++ b/include/linux/device-mapper.h
@@ -77,9 +77,6 @@ typedef int (*dm_message_fn) (struct dm_target *ti, unsigned argc, char **argv);
typedef int (*dm_ioctl_fn) (struct dm_target *ti, unsigned int cmd,
unsigned long arg);
-typedef int (*dm_merge_fn) (struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size);
-
/*
* These iteration functions are typically used to check (and combine)
* properties of underlying devices.
@@ -153,7 +150,6 @@ struct target_type {
dm_status_fn status;
dm_message_fn message;
dm_ioctl_fn ioctl;
- dm_merge_fn merge;
dm_busy_fn busy;
dm_iterate_devices_fn iterate_devices;
dm_io_hints_fn io_hints;
--
2.1.0
^ permalink raw reply related
* [RFC PATCH 13/17] md/raid5: get rid of bio_fits_rdev()
From: Dongsu Park @ 2014-12-22 11:48 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, Kent Overstreet, Ming Lin, Dongsu Park, Neil Brown,
linux-raid
In-Reply-To: <cover.1419241597.git.dongsu.park@profitbricks.com>
From: Kent Overstreet <kmo@daterainc.com>
Remove bio_fits_rdev() completely, because ->merge_bvec_fn() has now
gone. There's no point in calling bio_fits_rdev() only for ensuring
aligned read from rdev.
Signed-off-by: Kent Overstreet <kmo@daterainc.com>
[dpark: add more description in commit message]
Signed-off-by: Dongsu Park <dongsu.park@profitbricks.com>
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
---
drivers/md/raid5.c | 23 +----------------------
1 file changed, 1 insertion(+), 22 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index c1b0d52..40e464c 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4218,25 +4218,6 @@ static void raid5_align_endio(struct bio *bi, int error)
add_bio_to_retry(raid_bi, conf);
}
-static int bio_fits_rdev(struct bio *bi)
-{
- struct request_queue *q = bdev_get_queue(bi->bi_bdev);
-
- if (bio_sectors(bi) > queue_max_sectors(q))
- return 0;
- blk_recount_segments(q, bi);
- if (bi->bi_phys_segments > queue_max_segments(q))
- return 0;
-
- if (q->merge_bvec_fn)
- /* it's too hard to apply the merge_bvec_fn at this stage,
- * just just give up
- */
- return 0;
-
- return 1;
-}
-
static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
{
struct r5conf *conf = mddev->private;
@@ -4290,11 +4271,9 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
align_bi->bi_bdev = rdev->bdev;
__clear_bit(BIO_SEG_VALID, &align_bi->bi_flags);
- if (!bio_fits_rdev(align_bi) ||
- is_badblock(rdev, align_bi->bi_iter.bi_sector,
+ if (is_badblock(rdev, align_bi->bi_iter.bi_sector,
bio_sectors(align_bi),
&first_bad, &bad_sectors)) {
- /* too big in some way, or has a known bad block */
bio_put(align_bi);
rdev_dec_pending(rdev, mddev);
return 0;
--
2.1.0
^ permalink raw reply related
* [RFC PATCH 12/17] md/raid10: make sync_request_write() call bio_copy_data()
From: Dongsu Park @ 2014-12-22 11:48 UTC (permalink / raw)
To: linux-kernel
Cc: Jens Axboe, Kent Overstreet, Ming Lin, Dongsu Park, Neil Brown,
linux-raid
In-Reply-To: <cover.1419241597.git.dongsu.park@profitbricks.com>
From: Kent Overstreet <kmo@daterainc.com>
Refactor sync_request_write() of md/raid10 to use bio_copy_data()
instead of open coding bio_vec iterations.
Signed-off-by: Kent Overstreet <kmo@daterainc.com>
[dpark: add more description in commit message]
Signed-off-by: Dongsu Park <dongsu.park@profitbricks.com>
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
---
drivers/md/raid10.c | 20 +++++---------------
1 file changed, 5 insertions(+), 15 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 32e282f..4a40354 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -2107,18 +2107,11 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
tbio->bi_vcnt = vcnt;
tbio->bi_iter.bi_size = r10_bio->sectors << 9;
tbio->bi_rw = WRITE;
- tbio->bi_private = r10_bio;
tbio->bi_iter.bi_sector = r10_bio->devs[i].addr;
-
- for (j=0; j < vcnt ; j++) {
- tbio->bi_io_vec[j].bv_offset = 0;
- tbio->bi_io_vec[j].bv_len = PAGE_SIZE;
-
- memcpy(page_address(tbio->bi_io_vec[j].bv_page),
- page_address(fbio->bi_io_vec[j].bv_page),
- PAGE_SIZE);
- }
tbio->bi_end_io = end_sync_write;
+ tbio->bi_private = r10_bio;
+
+ bio_copy_data(tbio, fbio);
d = r10_bio->devs[i].devnum;
atomic_inc(&conf->mirrors[d].rdev->nr_pending);
@@ -2134,17 +2127,14 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
* that are active
*/
for (i = 0; i < conf->copies; i++) {
- int j, d;
+ int d;
tbio = r10_bio->devs[i].repl_bio;
if (!tbio || !tbio->bi_end_io)
continue;
if (r10_bio->devs[i].bio->bi_end_io != end_sync_write
&& r10_bio->devs[i].bio != fbio)
- for (j = 0; j < vcnt; j++)
- memcpy(page_address(tbio->bi_io_vec[j].bv_page),
- page_address(fbio->bi_io_vec[j].bv_page),
- PAGE_SIZE);
+ bio_copy_data(tbio, fbio);
d = r10_bio->devs[i].devnum;
atomic_inc(&r10_bio->remaining);
md_sync_acct(conf->mirrors[d].replacement->bdev,
--
2.1.0
^ permalink raw reply related
* Re: Expand RAID5 array or switch to RAID10
From: Robin Hill @ 2014-12-22 9:15 UTC (permalink / raw)
To: George Duffield; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <CAG__1a7YqAWMccgDeYksY_t10Zyy8Rc-veZM_P7aaXs-DtFdfQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2396 bytes --]
On Wed Dec 17, 2014 at 12:11:27pm +0200, George Duffield wrote:
> Hi
>
> I have a RAID5 array comprising four 3TB drives and I've basically got
> 100GB remaining free so it's time to expand the array. With this in
> mind, and considering my requirement is predominantly to increase
> capacity, I'm wondering whether it'd be best to add another 3TB drive
> and expand the array whilst retaining RAID5 or to switch out to a
> RAID10 array. My thinking is 1) adding a 5th 3TB drive to the
> existing array may result in drive failure during the rebuild; and 2)
> RAID10 is very expensive to implement. Hence, I'm tempted to consider
> other options (recognising that whilst it'd be convenient I don't
> actually need everything stored on a single array). One such option
> would be running two raid5 arrays comprised of 3x 3TB each yielding
> 12TB of storage across the two arrays.
>
>
> Questions:
> + Is my assumption re RAID5 drive failure correct/ likely?
> + Is there a non-destructive way to migrate from RAID5 to RAID10?
> + 2 x RAID5 arrays seems pretty appealing - from a reliability and
> cost effectiveness standpoint - yes/no?
RAID5 with 3TB drives is definitely a dodgy area. The rebuild times are
sufficiently long, and the amount of data that needs to be read
sufficiently high, that an unreadable block or complete drive failure
during rebuild are too probable to be worth the risk (in my opinion
anyway).
You can non-destructively convert from a RAID5 to a RAID0, then from the
RAID0 to a RAID10 (near-2). This does mean going via RAID0 though, so
there's a data-loss risk there.
I wouldn't run RAID5 on these size disks at all - anything over 2T is
too risky for my liking, no matter how many disks there are in the
array.
If you just want to reduce the risk of a secondary failure during
rebuild, RAID6 would be the safer option. RAID10 is definitely more
performant but there's a far higher capacity overhead, so it very much
depends on your use case as to which would be better.
My advice would be to add two additional drives, converting to RAID6 and
expanding the array.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: Expand RAID5 array or switch to RAID10
From: forumscollective @ 2014-12-22 5:57 UTC (permalink / raw)
To: linux-raid@vger.kernel.org
In-Reply-To: <CAG__1a7YqAWMccgDeYksY_t10Zyy8Rc-veZM_P7aaXs-DtFdfQ@mail.gmail.com>
anyone?
Sent from my iPhone
> On 17 Dec 2014, at 12:11, George Duffield <forumscollective@gmail.com> wrote:
>
> Hi
>
> I have a RAID5 array comprising four 3TB drives and I've basically got
> 100GB remaining free so it's time to expand the array. With this in
> mind, and considering my requirement is predominantly to increase
> capacity, I'm wondering whether it'd be best to add another 3TB drive
> and expand the array whilst retaining RAID5 or to switch out to a
> RAID10 array. My thinking is 1) adding a 5th 3TB drive to the
> existing array may result in drive failure during the rebuild; and 2)
> RAID10 is very expensive to implement. Hence, I'm tempted to consider
> other options (recognising that whilst it'd be convenient I don't
> actually need everything stored on a single array). One such option
> would be running two raid5 arrays comprised of 3x 3TB each yielding
> 12TB of storage across the two arrays.
>
>
> Questions:
> + Is my assumption re RAID5 drive failure correct/ likely?
> + Is there a non-destructive way to migrate from RAID5 to RAID10?
> + 2 x RAID5 arrays seems pretty appealing - from a reliability and
> cost effectiveness standpoint - yes/no?
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox