* Boot option raid=parttitionable and autodetection
From: Bostjan Skufca @ 2014-09-10 23:31 UTC (permalink / raw)
To: linux-raid
Hi all,
does kernel boot time parameter raid=partitionable ofer any additional
feature over not using it at all?
Besides changing autodetected device names from /dev/mdX to /dev/md_dX
I currently can not see any other difference, save lilo being unable
to work with them.
Normal raid devices can be partitioned without hassle, unfortunately
in-kernel autodetection of nested raid arrays does not work (talking
about raid0 over raid1 here, not raid10), but that is more or less
understandable.
Another quick question:
What is the status of in-kernel autodetection vs initramfs based one?
As I could gather from various sites it seems initramfs is preferable,
but what does that mean for in-kernel detection? Is it slowly being
phased out?
Tnx for info,
b.
^ permalink raw reply
* Re: Boot option raid=parttitionable and autodetection
From: NeilBrown @ 2014-09-11 0:02 UTC (permalink / raw)
To: Bostjan Skufca; +Cc: linux-raid
In-Reply-To: <CAEp_DRBuccHOzkrJe8UKP1bKLdk_GM1TB8rqB_7_9w-fgdUZ0g@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]
On Thu, 11 Sep 2014 01:31:13 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> Hi all,
>
> does kernel boot time parameter raid=partitionable ofer any additional
> feature over not using it at all?
>
> Besides changing autodetected device names from /dev/mdX to /dev/md_dX
> I currently can not see any other difference, save lilo being unable
> to work with them.
You have identified the one singular thing which that parameter does. It
does nothing else.
>
> Normal raid devices can be partitioned without hassle,
This was not true when that parameter was added. It is true now.
So best to ignore that parameter.
> unfortunately
> in-kernel autodetection of nested raid arrays does not work (talking
> about raid0 over raid1 here, not raid10), but that is more or less
> understandable.
I discourage use of in-kernel autodetection. Use mdadm from an initrd
instead.
>
>
> Another quick question:
> What is the status of in-kernel autodetection vs initramfs based one?
initramfs based is preferred at least by me.
> As I could gather from various sites it seems initramfs is preferable,
> but what does that mean for in-kernel detection? Is it slowly being
> phased out?
Not it isn't being phase out, but it is not being enhanced at all. Some new
features only work with v1.x metadata and that cannot be used with in-kernel
autodetection.
NeilBrown
>
> Tnx for info,
> b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Add disks and convert level 0 to level 5
From: NeilBrown @ 2014-09-11 0:27 UTC (permalink / raw)
To: Michael Muratet; +Cc: linux-raid
In-Reply-To: <8CF94689-0AD6-4B87-BC0C-815F794549E7@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]
On Wed, 10 Sep 2014 11:15:24 -0500 Michael Muratet <muratetm@gmail.com> wrote:
> I have a two-disk RAID0 system that is working splendidly, thanks to the list for the help.
>
> I managed to get my hands on more identical disks and since I have the disks and because I'm adding precious data, I'd like to add two more disks and grow to RAID5.
>
> I have partitioned the two new drives to type 'fd', /dev/sde and /dev/sdf
>
> I believe the command to accomplish the change is this:
>
> mdadm /dev/md0 --grow --level=5 --add /dev/sde /dev/sdf
>
> Following the old adage "measure twice, cut once", is this syntax correct? Is there any danger of data loss in such a conversion?
I recommend creating a few loop-back devices and experimenting.
i.e.:
create some 100M files.
use "losetup" to turn them into block devices.
create an 2-device raid0
try converting it as you suggest.
You find it doesn't do quite what you expected, but should be easy to fix.
Providing your new devices are reliable (as least read/write the entire drive
once if you feel at all cautious) there is no particular danger of data loss.
NeilBrown
>
> Thanks
>
> Mike--
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: NeilBrown @ 2014-09-11 0:31 UTC (permalink / raw)
To: Bostjan Skufca; +Cc: linux-raid
In-Reply-To: <CAEp_DRAVPBvA34kgdjWqO6f6489SbmHQf-dFXC_SwSQd8e0C2w@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]
On Wed, 10 Sep 2014 23:24:11 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> Hi,
>
> I have a simple question:
> - Where is the code that is used for actual RAID 10 creation? In
> kernel or in mdadm?
Depends on exactly what you mean ... probably in mdadm.
>
>
> Explanation:
>
> I was dissatisfied with single-threaded RAID 1 sequential read
> performance (basically boils down to the speed of one disk). I figured
> that instead of using level 1 I could create RAID level 10 and use two
> equally-sized partitions on each drive (instead of one).
>
> It turns out that if array is created properly, it is capable of
> sequential reads at almost 2x single device speed, as expected (on
> SSD!) and what would anyone expect from ordinary RAID 1.
>
> What does "properly" actually mean?
> I was doing some benchmarks with various raid configurations and
> figured out that the order of devices submitted to creation command is
> significant. It also makes raid10 created in such mode reliable or
> unreliable to a device failure (not partition failure, device failure,
> which means that two raid underlying devices fail at once).
I don't think you've really explained what "properly" means. How exactly do
you get better throughput?
If you want double-speed single-thread throughput on 2 devices, then create a
2-device RAID10 with "--layout=f2".
>
> Sum:
> - if such array is created properly, it has redundancy in place and
> performs as expected
> - if not, it performs as raid1 and fails with one physical disk failure
>
> I am trying to find the code responsible for creation of RAID 10 in
> order to try and make it more inteligent about where to place RAID 10
> parts if it gets a list of devices to use, and some of those devices
> are on the same physical disks.
mdadm uses the devices in the order that you list them.
>
> Thanks for hints,
> b.
>
NeilBrown
>
>
> PS: More details about testing is available here, but be warned, it is
> still a bit hectic to read:
> http://blog.a2o.si/2014/09/07/linux-software-raid-why-you-should-always-use-raid-10-instead-of-raid-1/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Howto start reshape from 100% when change readonly raid to readwrite
From: Apple Yin @ 2014-09-11 3:06 UTC (permalink / raw)
To: linux-raid
I met a prolem which confused me long time.
Background:
I have two machines which connected to the same HDDs, both of them are
linux software raid. Machine 1 called Master, Machine 2 called Slave.
Master can readwrite the HDD, Slave can read the HDD only.
/dev/md1 was created on Master, and level is raid5 with 3 active
disks + 1 spare disk. On slave I assemble the md1 as readonly with the
same disks.Here is my steps:
Create md1(raid5) on Master:
#mdadm --create /dev/md1 -l 5 -n 3 -x 1 /dev/sd[bcde]
Assemble md1(raid5) as readonly on Slave:
#mdadm -A /dev/md1 /dev/sd[bcde] -o
When Master doing the reshape(#mdadm --grow --raid-devices 4 /dev/md1
), Slave need to do the same reshape( #mdadm --grow --raid-devices 4
/dev/md1).But md on Slave is readonly, so md_do_sync() not start.
After Master finish the reshape, all data has moved from 3disks to
4disks. How can I ask Slave's md1 to start the reshape from 100% after
set it to readwrite ? Here is my steps:
1.On Slave, I modify the reshape_position_store() of md.c in kernel,
to let user space to set reshape_position to “none” (100%):
static ssize_t
reshape_position_store(struct mddev *mddev, const char *buf, size_t len)
{
…
if (cmd_match(buf, “none”))
;
else if (buf == e || (*e && *e != ‘\n’))
return -EINVAL;
if (cmd_match(buf, “none”))
mddev->reshape_position = MaxSector;
else
mddev->reshape_position = new;
…
}
#echo none > /sys/block/md1/md/reshape_position
2. change readonly raid to readwrite on Slave
#mdadm –readwrite /dev/md1
Result: the reshape didn’t start . Slave's md1 size still the old size
( with 3 disk size)
BRs,
AppleYin
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/5] Fixes for RAID1 resync
From: Brassow Jonathan @ 2014-09-11 3:45 UTC (permalink / raw)
To: NeilBrown; +Cc: Eivind Sarto, linux-raid, majianpeng
In-Reply-To: <20140910062039.26400.36745.stgit@notabene.brown>
On Sep 10, 2014, at 1:20 AM, NeilBrown wrote:
>
> Jon: could you test with these patches on top of what you
> have just in case something happens to fix the problem without
> me realising it?
I'm on it. The test is running. I'll know later tomorrow.
brassow
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: Bostjan Skufca @ 2014-09-11 4:48 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140911103110.42449c9e@notabene.brown>
On 11 September 2014 02:31, NeilBrown <neilb@suse.de> wrote:
> On Wed, 10 Sep 2014 23:24:11 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
>> What does "properly" actually mean?
>> I was doing some benchmarks with various raid configurations and
>> figured out that the order of devices submitted to creation command is
>> significant. It also makes raid10 created in such mode reliable or
>> unreliable to a device failure (not partition failure, device failure,
>> which means that two raid underlying devices fail at once).
>
> I don't think you've really explained what "properly" means. How exactly do
> you get better throughput?
>
> If you want double-speed single-thread throughput on 2 devices, then create a
> 2-device RAID10 with "--layout=f2".
I went and retested a few things and I see I must have done something
wrong before:
- regardless whether I use --layout flag or not, and
- regardless of device cli arg order at array creation time,
= I always get double-speed single-thread throughput. Yaay!
Anyway, the thing is that regardless of -using -layout=f2 or not,
redundancy STILL depends on the order of command line arguments passed
to mdadm --create.
If I do:
- "sda1 sdb1 sda2 sdb2" - redundandcy is ok
- "sda1 sda2 sdb1 sdb2" - redundancy fails
Is there a flag that ensures redundancy in this particular case?
If not, don't you think the naive user (me, for example) would assume
that code is smart enough to ensure basic redundancy, if there are at
least two devices available?
Because, if someone wants only performance and no redundancy, they
will look no further than raid 0. But raid10 strongly hints at
redundancy being incorporated in it. (I admit this is anecdotal, based
on my own experience and thought flow.)
b.
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: NeilBrown @ 2014-09-11 4:59 UTC (permalink / raw)
To: Bostjan Skufca; +Cc: linux-raid
In-Reply-To: <CAEp_DRANDtdBCmRCqfiMeyfDg2+-q_EGsrs97QcE744Otqg0Og@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2834 bytes --]
On Thu, 11 Sep 2014 06:48:31 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> On 11 September 2014 02:31, NeilBrown <neilb@suse.de> wrote:
> > On Wed, 10 Sep 2014 23:24:11 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> >> What does "properly" actually mean?
> >> I was doing some benchmarks with various raid configurations and
> >> figured out that the order of devices submitted to creation command is
> >> significant. It also makes raid10 created in such mode reliable or
> >> unreliable to a device failure (not partition failure, device failure,
> >> which means that two raid underlying devices fail at once).
> >
> > I don't think you've really explained what "properly" means. How exactly do
> > you get better throughput?
> >
> > If you want double-speed single-thread throughput on 2 devices, then create a
> > 2-device RAID10 with "--layout=f2".
>
> I went and retested a few things and I see I must have done something
> wrong before:
> - regardless whether I use --layout flag or not, and
> - regardless of device cli arg order at array creation time,
> = I always get double-speed single-thread throughput. Yaay!
>
> Anyway, the thing is that regardless of -using -layout=f2 or not,
> redundancy STILL depends on the order of command line arguments passed
> to mdadm --create.
> If I do:
> - "sda1 sdb1 sda2 sdb2" - redundandcy is ok
> - "sda1 sda2 sdb1 sdb2" - redundancy fails
>
> Is there a flag that ensures redundancy in this particular case?
> If not, don't you think the naive user (me, for example) would assume
> that code is smart enough to ensure basic redundancy, if there are at
> least two devices available?
I cannot guess what other people will assume. I certainly cannot guard
against all possible incorrect assumptions.
If you create an array which doesn't have true redundancy you will get a
message from the kernel saying:
%s: WARNING: %s appears to be on the same physical disk as %s.
True protection against single-disk failure might be compromised.
Maybe mdadm could produce a similar message...
>
> Because, if someone wants only performance and no redundancy, they
> will look no further than raid 0. But raid10 strongly hints at
> redundancy being incorporated in it. (I admit this is anecdotal, based
> on my own experience and thought flow.)
I really don't think there is any value is splitting a device into multiple
partitions and putting more than one partition per device into an array.
Have you tried using just one partition per device, making a RAID10 with
--layout=f2 ??
NeilBrown
>
>
> b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: Bostjan Skufca @ 2014-09-11 5:20 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140911145911.47c0d857@notabene.brown>
On 11 September 2014 06:59, NeilBrown <neilb@suse.de> wrote:
> On Thu, 11 Sep 2014 06:48:31 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
>
>> On 11 September 2014 02:31, NeilBrown <neilb@suse.de> wrote:
>> > On Wed, 10 Sep 2014 23:24:11 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
>> >> What does "properly" actually mean?
>> >> I was doing some benchmarks with various raid configurations and
>> >> figured out that the order of devices submitted to creation command is
>> >> significant. It also makes raid10 created in such mode reliable or
>> >> unreliable to a device failure (not partition failure, device failure,
>> >> which means that two raid underlying devices fail at once).
>> >
>> > I don't think you've really explained what "properly" means. How exactly do
>> > you get better throughput?
>> >
>> > If you want double-speed single-thread throughput on 2 devices, then create a
>> > 2-device RAID10 with "--layout=f2".
>>
>> I went and retested a few things and I see I must have done something
>> wrong before:
>> - regardless whether I use --layout flag or not, and
>> - regardless of device cli arg order at array creation time,
>> = I always get double-speed single-thread throughput. Yaay!
>>
>> Anyway, the thing is that regardless of -using -layout=f2 or not,
>> redundancy STILL depends on the order of command line arguments passed
>> to mdadm --create.
>> If I do:
>> - "sda1 sdb1 sda2 sdb2" - redundandcy is ok
>> - "sda1 sda2 sdb1 sdb2" - redundancy fails
>>
>> Is there a flag that ensures redundancy in this particular case?
>> If not, don't you think the naive user (me, for example) would assume
>> that code is smart enough to ensure basic redundancy, if there are at
>> least two devices available?
>
> I cannot guess what other people will assume. I certainly cannot guard
> against all possible incorrect assumptions.
>
> If you create an array which doesn't have true redundancy you will get a
> message from the kernel saying:
>
> %s: WARNING: %s appears to be on the same physical disk as %s.
> True protection against single-disk failure might be compromised.
>
> Maybe mdadm could produce a similar message...
I've seen it. Kernel produces this message in both cases.
>> Because, if someone wants only performance and no redundancy, they
>> will look no further than raid 0. But raid10 strongly hints at
>> redundancy being incorporated in it. (I admit this is anecdotal, based
>> on my own experience and thought flow.)
>
> I really don't think there is any value is splitting a device into multiple
> partitions and putting more than one partition per device into an array.
> Have you tried using just one partition per device, making a RAID10 with
> --layout=f2 ??
Yep, I tried raid10 on 4 devices with layout=f2, it works as expected.
No problem there.
And I know it is better if you have 4 devices for raid10, you are
right there. That is the expected use case.
But if you only have 2, you are limited to the options with those two.
Now, if I create raid1 on those two, I get bad single-threaded read
performance. This usually does not happen with hardware RAIDs.
This is the reason I started looking into posibility of using multiple
partitions per disk, to get something which reads off both disks even
for single "client". Raid10 seemed an option, and it works, albeit a
bit hackish ATM.
This is also the reason I asked for code locations, to look at it and
maybe send in patches for review which make a bit more inteligent
data-placement guesses in the case mentioned above. Would this be an
option of interest to actually pull it it?
b.
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: NeilBrown @ 2014-09-11 5:46 UTC (permalink / raw)
To: Bostjan Skufca; +Cc: linux-raid
In-Reply-To: <CAEp_DRBOeg8r8qUnMKM7tR9YbcP6Yb2HupUzSb1zzFPv7Q3ePA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4303 bytes --]
On Thu, 11 Sep 2014 07:20:48 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> On 11 September 2014 06:59, NeilBrown <neilb@suse.de> wrote:
> > On Thu, 11 Sep 2014 06:48:31 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> >
> >> On 11 September 2014 02:31, NeilBrown <neilb@suse.de> wrote:
> >> > On Wed, 10 Sep 2014 23:24:11 +0200 Bostjan Skufca <bostjan@a2o.si> wrote:
> >> >> What does "properly" actually mean?
> >> >> I was doing some benchmarks with various raid configurations and
> >> >> figured out that the order of devices submitted to creation command is
> >> >> significant. It also makes raid10 created in such mode reliable or
> >> >> unreliable to a device failure (not partition failure, device failure,
> >> >> which means that two raid underlying devices fail at once).
> >> >
> >> > I don't think you've really explained what "properly" means. How exactly do
> >> > you get better throughput?
> >> >
> >> > If you want double-speed single-thread throughput on 2 devices, then create a
> >> > 2-device RAID10 with "--layout=f2".
> >>
> >> I went and retested a few things and I see I must have done something
> >> wrong before:
> >> - regardless whether I use --layout flag or not, and
> >> - regardless of device cli arg order at array creation time,
> >> = I always get double-speed single-thread throughput. Yaay!
> >>
> >> Anyway, the thing is that regardless of -using -layout=f2 or not,
> >> redundancy STILL depends on the order of command line arguments passed
> >> to mdadm --create.
> >> If I do:
> >> - "sda1 sdb1 sda2 sdb2" - redundandcy is ok
> >> - "sda1 sda2 sdb1 sdb2" - redundancy fails
> >>
> >> Is there a flag that ensures redundancy in this particular case?
> >> If not, don't you think the naive user (me, for example) would assume
> >> that code is smart enough to ensure basic redundancy, if there are at
> >> least two devices available?
> >
> > I cannot guess what other people will assume. I certainly cannot guard
> > against all possible incorrect assumptions.
> >
> > If you create an array which doesn't have true redundancy you will get a
> > message from the kernel saying:
> >
> > %s: WARNING: %s appears to be on the same physical disk as %s.
> > True protection against single-disk failure might be compromised.
> >
> > Maybe mdadm could produce a similar message...
>
> I've seen it. Kernel produces this message in both cases.
>
>
> >> Because, if someone wants only performance and no redundancy, they
> >> will look no further than raid 0. But raid10 strongly hints at
> >> redundancy being incorporated in it. (I admit this is anecdotal, based
> >> on my own experience and thought flow.)
> >
> > I really don't think there is any value is splitting a device into multiple
> > partitions and putting more than one partition per device into an array.
> > Have you tried using just one partition per device, making a RAID10 with
> > --layout=f2 ??
>
> Yep, I tried raid10 on 4 devices with layout=f2, it works as expected.
> No problem there.
But did you try RAID10 with just 2 devices?
> And I know it is better if you have 4 devices for raid10, you are
> right there. That is the expected use case.
>
> But if you only have 2, you are limited to the options with those two.
You can still use RAID10 on 2 devices - that is not a limit (just like you can
use RAID5 on 2 devices).
NeilBrown
> Now, if I create raid1 on those two, I get bad single-threaded read
> performance. This usually does not happen with hardware RAIDs.
>
> This is the reason I started looking into posibility of using multiple
> partitions per disk, to get something which reads off both disks even
> for single "client". Raid10 seemed an option, and it works, albeit a
> bit hackish ATM.
>
> This is also the reason I asked for code locations, to look at it and
> maybe send in patches for review which make a bit more inteligent
> data-placement guesses in the case mentioned above. Would this be an
> option of interest to actually pull it it?
>
> b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: [patch v2 0/6] raid5: automatically batch adjacent full stripe write
From: NeilBrown @ 2014-09-11 7:15 UTC (permalink / raw)
To: shli; +Cc: linux-raid
In-Reply-To: <20140910124009.810771309@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 601 bytes --]
On Wed, 10 Sep 2014 20:40:09 +0800 shli@kernel.org wrote:
> This is the 2nd attempt to make adjacent full stripe write together. The main
> change against v1 is how to detect if stripes can be batched. Also some bugs
> are fixed.
>
Thanks a lot for this - it looks quite good. I've read through most of it
and don't see anything worth mentioning.... except maybe to wonder why
to_addr_page() has an unused 'sh' argument. Maybe I'll just remove that(?).
I've pushed it you the 'devel' branch of my md git tree. I'll try to do some
testing of my own next week.
Thanks,
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Add disks and convert level 0 to level 5
From: Robin Hill @ 2014-09-11 7:27 UTC (permalink / raw)
To: Michael Muratet; +Cc: linux-raid
In-Reply-To: <8CF94689-0AD6-4B87-BC0C-815F794549E7@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 945 bytes --]
On Wed Sep 10, 2014 at 11:15:24AM -0500, Michael Muratet wrote:
> I have a two-disk RAID0 system that is working splendidly, thanks to
> the list for the help.
>
> I managed to get my hands on more identical disks and since I have the
> disks and because I'm adding precious data, I'd like to add two more
> disks and grow to RAID5.
>
> I have partitioned the two new drives to type 'fd', /dev/sde and /dev/sdf
>
> I believe the command to accomplish the change is this:
>
> mdadm /dev/md0 --grow --level=5 --add /dev/sde /dev/sdf
>
If you've partitioned the drives then presumably you'd want to use the
partitions here (/dev/sde1 and /dev/sdf1?) rather than the full drives.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: Question about RAID1 plug/unplug code
From: Alexander Lyakas @ 2014-09-11 8:22 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, Yair Hershko
In-Reply-To: <20140910193631.0c8cd1f4@notabene.brown>
Hi Neil,
On Wed, Sep 10, 2014 at 12:36 PM, NeilBrown <neilb@suse.de> wrote:
> On Wed, 10 Sep 2014 11:01:30 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
> wrote:
>
>> Hello Neil,
>>
>> On Tue, Sep 9, 2014 at 12:45 PM, NeilBrown <neilb@suse.de> wrote:
>> > On Tue, 9 Sep 2014 11:33:13 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
>> > wrote:
>> >
>> >> Hi Neil,
>> >>
>> >>
>> >> On Tue, Sep 9, 2014 at 4:45 AM, NeilBrown <neilb@suse.de> wrote:
>> >> > On Mon, 8 Sep 2014 16:55:52 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> Hi Neil,
>> >> >> We have been seeing high latency on the md/raid1 block device, due to
>> >> >> the fact that all WRITEs are handed off to raid1d thread. This thread
>> >> >> also calls bitmap_unplug(), which writes the bitmap synchronously.
>> >> >> While it waits for the bitmap, it cannot trigger other WRITEs waiting
>> >> >> in its pending_bio_list. This is especially seen with SSDs: MD's
>> >> >> latency is much higher that SSD latency (I have been stoned by Peter
>> >> >> Grandi when I brought up this issue previously for raid5).
>> >> >>
>> >> >> Then I have noticed the commit:
>> >> >>
>> >> >> commit f54a9d0e59c4bea3db733921ca9147612a6f292c
>> >> >> Author: NeilBrown <neilb@suse.de>
>> >> >> Date: Thu Aug 2 08:33:20 2012 +1000
>> >> >>
>> >> >> md/raid1: submit IO from originating thread instead of md thread.
>> >> >>
>> >> >> Looking at the code, I learned that to avoid switching into raid1d,
>> >> >> the caller has to use blk_start_plug/blk_finish_plug. So I added these
>> >> >> calls in our kernel module, which submits bios to MD. Results were
>> >> >> awesome, MD latency got down significantly.
>> >> >
>> >> > That's good to hear.
>> >> >
>> >> >>
>> >> >> So I have several questions about this plug/unplug thing.
>> >> >>
>> >> >> 1/ Originally this infrastructure was supposed to help IO schedulers
>> >> >> in merging requests. It is useful when one has a bunch of requests to
>> >> >> submit in one shot.
>> >> >
>> >> > That is exactly the whole point of plugging: allow the device to handle a
>> >> > batch of requests together instead of one at a time.
>> >> >
>> >> >> But in MD case, thus infrastructure is used for a different purpose:
>> >> >> not to merge requests (which may help bandwidth, but probably not
>> >> >> latency), but to avoid making raid1d a bottleneck, to be able to
>> >> >> submit requests from multiple threads in parallel, which brings down
>> >> >> latency significantly in our case. Indeed "struct blk_plug" has a
>> >> >> special "cb_list", which is used only by MD.
>> >> >
>> >> > I don't think the way md uses plugging is conceptually different from any
>> >> > other use: it is always about gathering a batch together.
>> >> > "cb_list" is handled by blk_check_plugged() which is also used by
>> >> > block/umem.c and btrfs.
>> >> >
>> >> > The base plugging code assumes that it is only gathering a batch of requests
>> >> > for a single device - if the target device changes then the batch is flushed.
>> >> > It also assumed that it was "struct request" that was batched.
>> >> > Devices like md that want to queue 'struct bio', something else was needed.
>> >> > Also with layered devices it can be useful to gather multiple batches for
>> >> > multiple layers.
>> >> > So I created "cb_list" etc and a more generic interface.
>> >> >
>> >> >> In my case I have only individual bios (not a bunch of bios), and I
>> >> >> after wrap them with plug/unplug, MD latency gets better. So we are
>> >> >> using the plug infrastructure for a different purpose.
>> >> >> Is my understanding correct? Was this your intention?
>> >> >
>> >> > I don't really understand what you are doing. There is no point in using
>> >> > plugging for individual bios. The main point for raid1 writes is to gather
>> >> > a lot of writes together so that all multiple bitmap bits can be set all at
>> >> > once.
>> >> > It should be possible to submit individual bios directly from make_request
>> >> > without passing them to raid1d and without using plugging.
>> >> Can you pls explain how it is possible?
>> >> You have this code for WRITEs:
>> >> cb = blk_check_plugged(raid1_unplug, mddev, sizeof(*plug));
>> >> if (cb)
>> >> plug = container_of(cb, struct raid1_plug_cb, cb);
>> >> else
>> >> plug = NULL;
>> >> spin_lock_irqsave(&conf->device_lock, flags);
>> >> if (plug) {
>> >> bio_list_add(&plug->pending, mbio);
>> >> plug->pending_cnt++;
>> >> } else {
>> >> bio_list_add(&conf->pending_bio_list, mbio);
>> >> conf->pending_count++;
>> >> }
>> >> spin_unlock_irqrestore(&conf->device_lock, flags);
>> >>
>> >> If the thread blk_check_plugged returns NULL, then you always hand the
>> >> WRITE to raid1d. So the only option to avoid handoff to raid1d is for
>> >> the caller to plug. Otherwise, all WRITEs are handed off to raid1d and
>> >> latency becomes terrible.
>> >> So in my case, I use plug/unplug for individual bios only to avoid the
>> >> handoff to raid1d.
>> >> What am I missing in this analysis?
>> >
>> > if blk_check_plugged succeeds then it has arranged for raid1_unplug to be
>> > called a little later by that same process.
>> > So there is nothing to stop you calling raid1_unplug immediately.
>> >
>> > raid1_unplug essentially does:
>> > bitmap_unplug()
>> > generic_make_request()
>> >
>> > so you can very nearly just do that, without any plugging.
>> I am sorry, but I did not understand your reply. Maybe I did not
>> explain myself, I will try again.
>>
>> I am not changing raid1.c code. I just want to avoid the handoff to
>> raid1d on WRITEs. According to your code, there are only two possible
>> flows:
>>
>> Flow 1 - with plugging
>> # caller calls blk_start_plug
>> # caller calls submit_bio
>> # blk_check_plugged succeeds, and bio is put onto plug->pending list
>> # caller calls blk_finish_plug
>> # raid1_unplug is called in the same caller's thread, so it does
>> bitmap_unplug and generic_make_request
>>
>> Flow 2 - without plugging
>> # caller calls submit_bio
>> # blk_check_plugged fails, and bio is put onto conf->pending_bio_list,
>> which means it will be submitted by raid1d
>>
>> My conclusion from that: to avoid the handoff to raid1, caller always
>> need to plug, even if it has a single bio to submit. But you said "it
>> should be possible to submit individual bios directly from
>> make_request without passing them to raid1d and without using
>> plugging". So can you explain how it is possible? I prefer not to
>> change raid1.c code.
>>
>> >
>> > There is a bit of extra subtlety but I can't really know how relevant that
>> > might be to you without actually seeing you code.
>> My code (in a different kernel module, not in raid1.c) is simply doing
>> submit_bio. I want to wrap this with plug/unplug to avoid the handoff
>> to raid1d and improve raid1 latency.
>>
>
> I think I need to see the code you are working with to be able to suggest
> anything used.
I am working with kernel 3.8.13. But your master branch has the same
code with respect to plug/unplug logic.
> But if it works with plugging, then just do it that way(?).
It works perfectly, and latency is much better. The only doubt is with
bitmap_unplug being called from multiple threads now. However, it can
happen for anybody that uses plug/unplug on top of MD raid1 (like ext4
for example). So question is whether it is safe for MD users to
plug/unplug when submitting bios to MD. If not, would you be fixing
this?
Alex.
^ permalink raw reply
* Re: [patch v2 0/6] raid5: automatically batch adjacent full stripe write
From: Shaohua Li @ 2014-09-11 14:38 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140911171525.4ec2a88d@notabene.brown>
On Thu, Sep 11, 2014 at 05:15:25PM +1000, NeilBrown wrote:
> On Wed, 10 Sep 2014 20:40:09 +0800 shli@kernel.org wrote:
>
> > This is the 2nd attempt to make adjacent full stripe write together. The main
> > change against v1 is how to detect if stripes can be batched. Also some bugs
> > are fixed.
> >
>
> Thanks a lot for this - it looks quite good. I've read through most of it
> and don't see anything worth mentioning.... except maybe to wonder why
> to_addr_page() has an unused 'sh' argument. Maybe I'll just remove that(?).
Can be removed. I just want the parameters of to_addr_conv/page are the same.
> I've pushed it you the 'devel' branch of my md git tree. I'll try to do some
> testing of my own next week.
Thanks!
Thanks,
Shaohua
^ permalink raw reply
* access to old raid1 partitions after reinstall on two new disks
From: system.admin @ 2014-09-11 16:02 UTC (permalink / raw)
To: linux-raid
Hello there,
we have a machine running on four raid-1 partitions (2 disks). The
system had to be reinstalled on two new disks since one of the old disks
failed and we afterwards saw defective sectors on the remaining disk. So
the system was restored from backup on two complete new disks. We now
have the disk with the defective sectors on which the old system was
installed as third disk installed in the machine. Since we could not
recover everything from backup, we would like to start the old raid
devices from the old disk again and try to access the old filesystems
again.
The fresh installed system has of course the same md0, md1, md2, md3
devices as the old one had. Therefore I wonder how to access the old
raid devices, which had the same names.
Is there a way to get the old partitions running as additional raid
devices like md4, md5, md6 and md7 (or under different names) and should
it be possible to mount the filesystems of those new / old raid devices
in the fresh installed system?
Thanks for your help!
Regards
Hans
^ permalink raw reply
* Re: [PATCH 0/5] Fixes for RAID1 resync
From: Brassow Jonathan @ 2014-09-11 17:12 UTC (permalink / raw)
To: Brassow Jonathan; +Cc: NeilBrown, Eivind Sarto, linux-raid, majianpeng
In-Reply-To: <D96E0116-919C-4F4B-B4B6-2564B99449FB@redhat.com>
On Sep 10, 2014, at 10:45 PM, Brassow Jonathan wrote:
>
> On Sep 10, 2014, at 1:20 AM, NeilBrown wrote:
>
>>
>> Jon: could you test with these patches on top of what you
>> have just in case something happens to fix the problem without
>> me realising it?
>
> I'm on it. The test is running. I'll know later tomorrow.
>
> brassow
The test is still failing from here. I grabbed 3.17.0-rc4, added the 5 patches, and got the attached backtraces when testing. As I said, the hangs are not exactly the same. This set shows the mdX_raid1 thread in the middle of handling a read failure.
brassow
Sep 11 07:48:02 bp-01 kernel: INFO: task dmeventd:27071 blocked for more than 12
0 seconds.
Sep 11 07:48:02 bp-01 kernel: Tainted: G E 3.17.0-rc4 #1
Sep 11 07:48:02 bp-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
Sep 11 07:48:02 bp-01 kernel: dmeventd D 0000000000000003 0 27071 1 0x00000080
Sep 11 07:48:02 bp-01 kernel: ffff8804038efae8 0000000000000082 ffff8800dbccf460 ffff88021721c0d0
Sep 11 07:48:02 bp-01 kernel: ffff8804038ec010 0000000000012bc0 0000000000012bc0 ffff88041432b180
Sep 11 07:48:02 bp-01 kernel: ffff8804038efb28 ffff88021fa72bc0 ffff88041432b180 ffff88041432b180
Sep 11 07:48:02 bp-01 kernel: Call Trace:
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81580999>] schedule+0x29/0x70
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81580a6c>] io_schedule+0x8c/0xd0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811d0044>] dio_await_completion+0x54/0xd0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811d264a>] do_blockdev_direct_IO+0x7fa/0xbd0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81143655>] ? pagevec_lookup_tag+0x25/0x40
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81141327>] ? write_cache_pages+0x147/0x510
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd850>] ? I_BDEV+0x10/0x10
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811d2a6c>] __blockdev_direct_IO+0x4c/0x50
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd850>] ? I_BDEV+0x10/0x10
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811ce7ee>] blkdev_direct_IO+0x4e/0x50
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd850>] ? I_BDEV+0x10/0x10
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811376e3>] generic_file_read_iter+0x143/0x150
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cdab7>] blkdev_read_iter+0x37/0x40
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811977af>] new_sync_read+0x8f/0xc0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81197cb3>] vfs_read+0xa3/0x110
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811b47a3>] ? __fdget+0x13/0x20
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81198266>] SyS_read+0x56/0xd0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff810ec4e6>] ? __audit_syscall_exit+0x216/0x2c0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81584612>] system_call_fastpath+0x16/0x1b
Sep 11 07:48:02 bp-01 kernel: INFO: task kworker/u129:4:24399 blocked for more than 120 seconds.
Sep 11 07:48:02 bp-01 kernel: Tainted: G E 3.17.0-rc4 #1
Sep 11 07:48:02 bp-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 11 07:48:02 bp-01 kernel: kworker/u129:4 D 0000000000000000 0 24399 2 0x00000080
Sep 11 07:48:02 bp-01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-253:16)
Sep 11 07:48:02 bp-01 kernel: ffff8801fc8bb468 0000000000000046 0000000100000000 ffffffff81a19480
Sep 11 07:48:02 bp-01 kernel: ffff8801fc8b8010 0000000000012bc0 0000000000012bc0 ffff88021721ce40
Sep 11 07:48:02 bp-01 kernel: ffff8801fc8bb558 ffff880414f9e940 ffff880414f9e9b8 ffff8801fc8bb498
Sep 11 07:48:02 bp-01 kernel: Call Trace:
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81580999>] schedule+0x29/0x70
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0408e9d>] wait_barrier+0xbd/0x230 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa040baaa>] make_request+0x9a/0xc00 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81137ff0>] ? mempool_alloc+0x60/0x170
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81137e95>] ? mempool_alloc_slab+0x15/0x20
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81137ff0>] ? mempool_alloc+0x60/0x170
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0423018>] raid_map+0x18/0x20 [dm_raid]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa000336a>] __map_bio+0x4a/0x120 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0003723>] __clone_and_map_data_bio+0x113/0x130 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa00037ac>] __split_and_process_non_flush+0x6c/0xb0 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0003991>] __split_and_process_bio+0x1a1/0x200 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0003b12>] _dm_request+0x122/0x190 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0003ba8>] dm_request+0x28/0x40 [dm_mod]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81269620>] generic_make_request+0xc0/0x100
Sep 11 07:48:02 bp-01 kernel: [<ffffffff812696d1>] submit_bio+0x71/0x140
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c9686>] _submit_bh+0x146/0x220
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c9770>] submit_bh+0x10/0x20
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811ccf93>] __block_write_full_page.clone.0+0x1a3/0x340
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd850>] ? I_BDEV+0x10/0x10
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd850>] ? I_BDEV+0x10/0x10
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811cd306>] block_write_full_page+0xc6/0x100
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811ce8f8>] blkdev_writepage+0x18/0x20
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81140067>] __writepage+0x17/0x50
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81141424>] write_cache_pages+0x244/0x510
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81140050>] ? set_page_dirty+0x60/0x60
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81141741>] generic_writepages+0x51/0x80
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81141790>] do_writepages+0x20/0x40
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811bfee9>] __writeback_single_inode+0x49/0x230
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c3329>] writeback_sb_inodes+0x249/0x360
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c34de>] __writeback_inodes_wb+0x9e/0xd0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c370b>] wb_writeback+0x1fb/0x2c0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c3976>] wb_do_writeback+0x1a6/0x1f0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff811c3a30>] bdi_writeback_workfn+0x70/0x210
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8106b762>] process_one_work+0x182/0x450
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8106bb4f>] worker_thread+0x11f/0x3c0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8106ba30>] ? process_one_work+0x450/0x450
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8107083e>] kthread+0xce/0xf0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8158456c>] ret_from_fork+0x7c/0xb0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:02 bp-01 kernel: INFO: task mdX_raid1:27151 blocked for more than 120 seconds.
Sep 11 07:48:02 bp-01 kernel: Tainted: G E 3.17.0-rc4 #1
Sep 11 07:48:02 bp-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 11 07:48:02 bp-01 kernel: mdX_raid1 D 0000000000000002 0 27151 2 0x00000080
Sep 11 07:48:02 bp-01 kernel: ffff880415bebc88 0000000000000046 0000000000000296 ffff880217260f00
Sep 11 07:48:02 bp-01 kernel: ffff880415be8010 0000000000012bc0 0000000000012bc0 ffff8803fc6982d0
Sep 11 07:48:02 bp-01 kernel: ffff8800de22bd20 ffff880414f9e940 ffff880414f9e9b8 ffff880415bebca8
Sep 11 07:48:02 bp-01 kernel: Call Trace:
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81580999>] schedule+0x29/0x70
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa0408d34>] freeze_array+0x74/0xc0 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8126a7a3>] ? blk_queue_bio+0x143/0x320
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa040a14d>] handle_read_error+0x3d/0x300 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81269620>] ? generic_make_request+0xc0/0x100
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa040841b>] ? sync_request_write+0xab/0x1a0 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffffa040a515>] raid1d+0x105/0x170 [raid1]
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81460e76>] md_thread+0x116/0x150
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81460d60>] ? md_rdev_init+0x110/0x110
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8107083e>] kthread+0xce/0xf0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:02 bp-01 kernel: [<ffffffff8158456c>] ret_from_fork+0x7c/0xb0
Sep 11 07:48:02 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:03 bp-01 kernel: INFO: task mdX_resync:27154 blocked for more than 120 seconds.
Sep 11 07:48:03 bp-01 kernel: Tainted: G E 3.17.0-rc4 #1
Sep 11 07:48:03 bp-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 11 07:48:03 bp-01 kernel: mdX_resync D 0000000000000004 0 27154 2 0x00000080
Sep 11 07:48:03 bp-01 kernel: ffff880405777c58 0000000000000046 ffff880405777bf8 ffff88021726cf40
Sep 11 07:48:03 bp-01 kernel: ffff880405774010 0000000000012bc0 0000000000012bc0 ffff88040284f180
Sep 11 07:48:03 bp-01 kernel: ffff88041400323c ffff880405777db8 ffff88041400323c ffff880414003010
Sep 11 07:48:03 bp-01 kernel: Call Trace:
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81580999>] schedule+0x29/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffff814608e7>] md_do_sync+0xac7/0xd40
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81460e76>] md_thread+0x116/0x150
Sep 11 07:48:03 bp-01 kernel: [<ffffffff815804be>] ? __schedule+0x34e/0x6e0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81460d60>] ? md_rdev_init+0x110/0x110
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8107083e>] kthread+0xce/0xf0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81580999>] ? schedule+0x29/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8158456c>] ret_from_fork+0x7c/0xb0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:03 bp-01 kernel: INFO: task kjournald:27205 blocked for more than 120 seconds.
Sep 11 07:48:03 bp-01 kernel: Tainted: G E 3.17.0-rc4 #1
Sep 11 07:48:03 bp-01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 11 07:48:03 bp-01 kernel: kjournald D 0000000000000000 0 27205 2 0x00000080
Sep 11 07:48:03 bp-01 kernel: ffff8803f5c07868 0000000000000046 ffff8803f5c07988 ffffffff81a19480
Sep 11 07:48:03 bp-01 kernel: ffff8803f5c04010 0000000000012bc0 0000000000012bc0 ffff8803f5774ec0
Sep 11 07:48:03 bp-01 kernel: ffff880216e06000 ffff880414f9e940 ffff880414f9e9b8 ffff8803f5c07898
Sep 11 07:48:03 bp-01 kernel: Call Trace:
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81580999>] schedule+0x29/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0408e9d>] wait_barrier+0xbd/0x230 [raid1]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa040baaa>] make_request+0x9a/0xc00 [raid1]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81137ff0>] ? mempool_alloc+0x60/0x170
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0423018>] raid_map+0x18/0x20 [dm_raid]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa000336a>] __map_bio+0x4a/0x120 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0003723>] __clone_and_map_data_bio+0x113/0x130 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa00037ac>] __split_and_process_non_flush+0x6c/0xb0 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0003991>] __split_and_process_bio+0x1a1/0x200 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0003b12>] _dm_request+0x122/0x190 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0003ba8>] dm_request+0x28/0x40 [dm_mod]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81269620>] generic_make_request+0xc0/0x100
Sep 11 07:48:03 bp-01 kernel: [<ffffffff812696d1>] submit_bio+0x71/0x140
Sep 11 07:48:03 bp-01 kernel: [<ffffffff811c9686>] _submit_bh+0x146/0x220
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa042e553>] journal_do_submit_data+0x43/0x60 [jbd]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa042ea12>] journal_submit_data_buffers+0x202/0x2f0 [jbd]
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa042eda6>] journal_commit_transaction+0x2a6/0xf80 [jbd]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8108847f>] ? put_prev_entity+0x2f/0x400
Sep 11 07:48:03 bp-01 kernel: [<ffffffff810b21bb>] ? try_to_del_timer_sync+0x5b/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa0432ae1>] kjournald+0xf1/0x270 [jbd]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8108eb00>] ? bit_waitqueue+0xe0/0xe0
Sep 11 07:48:03 bp-01 kernel: [<ffffffffa04329f0>] ? commit_timeout+0x10/0x10 [jbd]
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8107083e>] kthread+0xce/0xf0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 07:48:03 bp-01 kernel: [<ffffffff8158456c>] ret_from_fork+0x7c/0xb0
Sep 11 07:48:03 bp-01 kernel: [<ffffffff81070770>] ? kthread_freezable_should_stop+0x70/0x70
Sep 11 08:40:01 bp-01 auditd[1981]: Audit daemon rotating log files
^ permalink raw reply
* Re: access to old raid1 partitions after reinstall on two new disks
From: Robin Hill @ 2014-09-11 17:16 UTC (permalink / raw)
To: system.admin; +Cc: linux-raid
In-Reply-To: <5411C7B1.1010205@yum.de>
[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]
On Thu Sep 11, 2014 at 06:02:57pm +0200, system.admin@yum.de wrote:
> Hello there,
>
> we have a machine running on four raid-1 partitions (2 disks). The
> system had to be reinstalled on two new disks since one of the old disks
> failed and we afterwards saw defective sectors on the remaining disk. So
> the system was restored from backup on two complete new disks. We now
> have the disk with the defective sectors on which the old system was
> installed as third disk installed in the machine. Since we could not
> recover everything from backup, we would like to start the old raid
> devices from the old disk again and try to access the old filesystems
> again.
>
> The fresh installed system has of course the same md0, md1, md2, md3
> devices as the old one had. Therefore I wonder how to access the old
> raid devices, which had the same names.
>
> Is there a way to get the old partitions running as additional raid
> devices like md4, md5, md6 and md7 (or under different names) and should
> it be possible to mount the filesystems of those new / old raid devices
> in the fresh installed system?
>
Yes, just specify the md device name as part of the assemble.
mdadm -A /dev/md4 /dev/sdf1
If they're using RAID superblock versions 0.9 or 1.0 then you don't even
have to assemble the RAID - you can just mount any of the members
directly.
And yes, you should be able to mount the filesystems inside your new
system the same as with any other block device.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* frequent disk activity with mdadm-3.3
From: Marco Schindler @ 2014-09-11 22:05 UTC (permalink / raw)
To: linux-raid; +Cc: neilb
Hello,
Im seeing frequent disk activity on all raid drives from mdadm since upgrading to 3.3.1/3.3.2 from 3.2.
It keeps drives from sleeping (disk access every ~15 minutes). Is it intentional?
I reported a similar issue for udev a few weeks ago: https://bugs.gentoo.org/show_bug.cgi?id=518748
Marco
^ permalink raw reply
* Re: frequent disk activity with mdadm-3.3
From: NeilBrown @ 2014-09-11 22:24 UTC (permalink / raw)
To: Marco Schindler; +Cc: linux-raid
In-Reply-To: <AB305DF4-9DCC-47E1-9B29-130E211D4B4E@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 749 bytes --]
On Fri, 12 Sep 2014 00:05:08 +0200 Marco Schindler
<marco.schindler@gmail.com> wrote:
> Hello,
>
> Im seeing frequent disk activity on all raid drives from mdadm since upgrading to 3.3.1/3.3.2 from 3.2.
> It keeps drives from sleeping (disk access every ~15 minutes). Is it intentional?
No.
>
> I reported a similar issue for udev a few weeks ago: https://bugs.gentoo.org/show_bug.cgi?id=518748
In that bug report you mention upgraded udev. Here you mention upgrading
mdadm.... a bit confusing.
Can you use "blktrace" to gather details on exactly what is being read and
when, and hopefully which process is doing it?
Is "mdadm --monitor" (or "-F") running? If you kill it does the disk
activity go away?
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: frequent disk activity with mdadm-3.3
From: Marco Schindler @ 2014-09-11 22:45 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20140912082403.549fc298@notabene.brown>
On 12.09.2014, at 00:24, NeilBrown <neilb@suse.de> wrote:
> On Fri, 12 Sep 2014 00:05:08 +0200 Marco Schindler
> <marco.schindler@gmail.com> wrote:
>
> In that bug report you mention upgraded udev. Here you mention upgrading
> mdadm.... a bit confusing.
totally unsure if this is related, it may be coincidence.
just found it’s odd to have a similar issue with two different processes in such a short timeframe.
I have blocked >sys-fs/udev-212 and >sys-fs/mdadm-3.2 for now to give drives some rest.
> Can you use "blktrace" to gather details on exactly what is being read and
> when, and hopefully which process is doing it?
>
> Is "mdadm --monitor" (or "-F") running? If you kill it does the disk
> activity go away?
>
> NeilBrown
yes I have (and have been) running "mdadm --monitor --scan —daemonise"
I have /proc/sys/vm/block_dump enabled and it shows process mdadm acessing drives every 15 minutes.
I wouldn’t be aware of mdadm being invoked elsewhere, so I presume it won’t happen when daemon is not running.
Will see if I can get some more info with blktrace and report back..
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Status of discard support in MD RAID
From: Brassow Jonathan @ 2014-09-11 23:38 UTC (permalink / raw)
To: linux-raid@vger.kernel.org Raid; +Cc: NeilBrown
Neil (or anyone else),
I know that trim/discard support was added back in 2012 (commit 9db90880). However, I thought there were still issues regarding what happens when various sync operations occur. I'd like to turn on discard support in dm-raid.c (a oneline patch) if things are in order. I can enable any, all or none depending on your recommendation. (I assume RAID1/10 is easier than the parity RAIDs.)
Thanks for any information,
brassow
^ permalink raw reply
* Re: Status of discard support in MD RAID
From: Chris Murphy @ 2014-09-12 0:46 UTC (permalink / raw)
To: Brassow Jonathan; +Cc: linux-raid@vger.kernel.org Raid, NeilBrown
In-Reply-To: <1ED0286A-56DA-491D-853A-1C1045449201@redhat.com>
On Sep 11, 2014, at 5:38 PM, Brassow Jonathan <jbrassow@redhat.com> wrote:
> Neil (or anyone else),
>
> I know that trim/discard support was added back in 2012 (commit 9db90880). However, I thought there were still issues regarding what happens when various sync operations occur. I'd like to turn on discard support in dm-raid.c (a oneline patch) if things are in order. I can enable any, all or none depending on your recommendation. (I assume RAID1/10 is easier than the parity RAIDs.)
If all the controller and drive support it then it should pass through, but there's the problem whether the SSD supports deterministic trim. If it doesn't, a check check > md/sync_action will report mismatches in md/mismatch_cnt; and a repair will probably corrupt the volume. So you can still use trim with a drive that returns non-deterministic results with raid0/1/10, but you can't rely on the result of md/mismatch_cnt and you can't do repair type scrubs.
For raid5/6, it's a problem to use trim if the drive returns non-deterministically for trimmed blocks. I'd think that in addition to DRAT being supported, it'd need to support DZAT.
smartctl --identify=wb /dev/diskX | grep -i trim
Chris Murphy
^ permalink raw reply
* Re: Howto start reshape from 100% when change readonly raid to readwrite
From: NeilBrown @ 2014-09-12 4:05 UTC (permalink / raw)
To: Apple Yin; +Cc: linux-raid
In-Reply-To: <CAFvwrujsWB_zdvW0xfoGtOOrH5TH89m9CLEj-fBqn17N88LHpg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]
On Thu, 11 Sep 2014 11:06:59 +0800 Apple Yin <bravery.apple@gmail.com> wrote:
> I met a prolem which confused me long time.
> Background:
> I have two machines which connected to the same HDDs, both of them are
> linux software raid. Machine 1 called Master, Machine 2 called Slave.
> Master can readwrite the HDD, Slave can read the HDD only.
>
> /dev/md1 was created on Master, and level is raid5 with 3 active
> disks + 1 spare disk. On slave I assemble the md1 as readonly with the
> same disks.Here is my steps:
> Create md1(raid5) on Master:
> #mdadm --create /dev/md1 -l 5 -n 3 -x 1 /dev/sd[bcde]
> Assemble md1(raid5) as readonly on Slave:
> #mdadm -A /dev/md1 /dev/sd[bcde] -o
>
>
> When Master doing the reshape(#mdadm --grow --raid-devices 4 /dev/md1
> ), Slave need to do the same reshape( #mdadm --grow --raid-devices 4
> /dev/md1).But md on Slave is readonly, so md_do_sync() not start.
> After Master finish the reshape, all data has moved from 3disks to
> 4disks. How can I ask Slave's md1 to start the reshape from 100% after
> set it to readwrite ? Here is my steps:
There is no way that you can make this reliable.
Even without a --grow happening, the read-only mount of changeable data could
end up very confused. You definitely cannot get md/raid5 to track the
changes made by another instance of md when reshaping.
If you unmount and --stop the array before starting the reshape, then
re-assemble and re-mount it, the more obvious probably should go away.
But it is likely there will still be less-obvious problems.
md might support clusters one day (though raid1 is much more likely than
raid5) but it certainly doesn't today.
I suggest you try to find a different solution to your underlying problem.
NeilBrown
>
> 1.On Slave, I modify the reshape_position_store() of md.c in kernel,
> to let user space to set reshape_position to “none” (100%):
> static ssize_t
> reshape_position_store(struct mddev *mddev, const char *buf, size_t len)
> {
> …
> if (cmd_match(buf, “none”))
> ;
> else if (buf == e || (*e && *e != ‘\n’))
> return -EINVAL;
>
> if (cmd_match(buf, “none”))
> mddev->reshape_position = MaxSector;
> else
> mddev->reshape_position = new;
> …
> }
> #echo none > /sys/block/md1/md/reshape_position
>
> 2. change readonly raid to readwrite on Slave
> #mdadm –readwrite /dev/md1
>
>
> Result: the reshape didn’t start . Slave's md1 size still the old size
> ( with 3 disk size)
>
>
>
> BRs,
> AppleYin
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Question about RAID1 plug/unplug code
From: NeilBrown @ 2014-09-12 6:16 UTC (permalink / raw)
To: Alexander Lyakas; +Cc: linux-raid, Yair Hershko
In-Reply-To: <CAGRgLy6kQmmhgUkMHEhOwjhyLU-0Pn1Gb4j+9VnasaHTTSjgnw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 8831 bytes --]
On Thu, 11 Sep 2014 11:22:40 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
wrote:
> Hi Neil,
>
> On Wed, Sep 10, 2014 at 12:36 PM, NeilBrown <neilb@suse.de> wrote:
> > On Wed, 10 Sep 2014 11:01:30 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
> > wrote:
> >
> >> Hello Neil,
> >>
> >> On Tue, Sep 9, 2014 at 12:45 PM, NeilBrown <neilb@suse.de> wrote:
> >> > On Tue, 9 Sep 2014 11:33:13 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Neil,
> >> >>
> >> >>
> >> >> On Tue, Sep 9, 2014 at 4:45 AM, NeilBrown <neilb@suse.de> wrote:
> >> >> > On Mon, 8 Sep 2014 16:55:52 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
> >> >> > wrote:
> >> >> >
> >> >> >> Hi Neil,
> >> >> >> We have been seeing high latency on the md/raid1 block device, due to
> >> >> >> the fact that all WRITEs are handed off to raid1d thread. This thread
> >> >> >> also calls bitmap_unplug(), which writes the bitmap synchronously.
> >> >> >> While it waits for the bitmap, it cannot trigger other WRITEs waiting
> >> >> >> in its pending_bio_list. This is especially seen with SSDs: MD's
> >> >> >> latency is much higher that SSD latency (I have been stoned by Peter
> >> >> >> Grandi when I brought up this issue previously for raid5).
> >> >> >>
> >> >> >> Then I have noticed the commit:
> >> >> >>
> >> >> >> commit f54a9d0e59c4bea3db733921ca9147612a6f292c
> >> >> >> Author: NeilBrown <neilb@suse.de>
> >> >> >> Date: Thu Aug 2 08:33:20 2012 +1000
> >> >> >>
> >> >> >> md/raid1: submit IO from originating thread instead of md thread.
> >> >> >>
> >> >> >> Looking at the code, I learned that to avoid switching into raid1d,
> >> >> >> the caller has to use blk_start_plug/blk_finish_plug. So I added these
> >> >> >> calls in our kernel module, which submits bios to MD. Results were
> >> >> >> awesome, MD latency got down significantly.
> >> >> >
> >> >> > That's good to hear.
> >> >> >
> >> >> >>
> >> >> >> So I have several questions about this plug/unplug thing.
> >> >> >>
> >> >> >> 1/ Originally this infrastructure was supposed to help IO schedulers
> >> >> >> in merging requests. It is useful when one has a bunch of requests to
> >> >> >> submit in one shot.
> >> >> >
> >> >> > That is exactly the whole point of plugging: allow the device to handle a
> >> >> > batch of requests together instead of one at a time.
> >> >> >
> >> >> >> But in MD case, thus infrastructure is used for a different purpose:
> >> >> >> not to merge requests (which may help bandwidth, but probably not
> >> >> >> latency), but to avoid making raid1d a bottleneck, to be able to
> >> >> >> submit requests from multiple threads in parallel, which brings down
> >> >> >> latency significantly in our case. Indeed "struct blk_plug" has a
> >> >> >> special "cb_list", which is used only by MD.
> >> >> >
> >> >> > I don't think the way md uses plugging is conceptually different from any
> >> >> > other use: it is always about gathering a batch together.
> >> >> > "cb_list" is handled by blk_check_plugged() which is also used by
> >> >> > block/umem.c and btrfs.
> >> >> >
> >> >> > The base plugging code assumes that it is only gathering a batch of requests
> >> >> > for a single device - if the target device changes then the batch is flushed.
> >> >> > It also assumed that it was "struct request" that was batched.
> >> >> > Devices like md that want to queue 'struct bio', something else was needed.
> >> >> > Also with layered devices it can be useful to gather multiple batches for
> >> >> > multiple layers.
> >> >> > So I created "cb_list" etc and a more generic interface.
> >> >> >
> >> >> >> In my case I have only individual bios (not a bunch of bios), and I
> >> >> >> after wrap them with plug/unplug, MD latency gets better. So we are
> >> >> >> using the plug infrastructure for a different purpose.
> >> >> >> Is my understanding correct? Was this your intention?
> >> >> >
> >> >> > I don't really understand what you are doing. There is no point in using
> >> >> > plugging for individual bios. The main point for raid1 writes is to gather
> >> >> > a lot of writes together so that all multiple bitmap bits can be set all at
> >> >> > once.
> >> >> > It should be possible to submit individual bios directly from make_request
> >> >> > without passing them to raid1d and without using plugging.
> >> >> Can you pls explain how it is possible?
> >> >> You have this code for WRITEs:
> >> >> cb = blk_check_plugged(raid1_unplug, mddev, sizeof(*plug));
> >> >> if (cb)
> >> >> plug = container_of(cb, struct raid1_plug_cb, cb);
> >> >> else
> >> >> plug = NULL;
> >> >> spin_lock_irqsave(&conf->device_lock, flags);
> >> >> if (plug) {
> >> >> bio_list_add(&plug->pending, mbio);
> >> >> plug->pending_cnt++;
> >> >> } else {
> >> >> bio_list_add(&conf->pending_bio_list, mbio);
> >> >> conf->pending_count++;
> >> >> }
> >> >> spin_unlock_irqrestore(&conf->device_lock, flags);
> >> >>
> >> >> If the thread blk_check_plugged returns NULL, then you always hand the
> >> >> WRITE to raid1d. So the only option to avoid handoff to raid1d is for
> >> >> the caller to plug. Otherwise, all WRITEs are handed off to raid1d and
> >> >> latency becomes terrible.
> >> >> So in my case, I use plug/unplug for individual bios only to avoid the
> >> >> handoff to raid1d.
> >> >> What am I missing in this analysis?
> >> >
> >> > if blk_check_plugged succeeds then it has arranged for raid1_unplug to be
> >> > called a little later by that same process.
> >> > So there is nothing to stop you calling raid1_unplug immediately.
> >> >
> >> > raid1_unplug essentially does:
> >> > bitmap_unplug()
> >> > generic_make_request()
> >> >
> >> > so you can very nearly just do that, without any plugging.
> >> I am sorry, but I did not understand your reply. Maybe I did not
> >> explain myself, I will try again.
> >>
> >> I am not changing raid1.c code. I just want to avoid the handoff to
> >> raid1d on WRITEs. According to your code, there are only two possible
> >> flows:
> >>
> >> Flow 1 - with plugging
> >> # caller calls blk_start_plug
> >> # caller calls submit_bio
> >> # blk_check_plugged succeeds, and bio is put onto plug->pending list
> >> # caller calls blk_finish_plug
> >> # raid1_unplug is called in the same caller's thread, so it does
> >> bitmap_unplug and generic_make_request
> >>
> >> Flow 2 - without plugging
> >> # caller calls submit_bio
> >> # blk_check_plugged fails, and bio is put onto conf->pending_bio_list,
> >> which means it will be submitted by raid1d
> >>
> >> My conclusion from that: to avoid the handoff to raid1, caller always
> >> need to plug, even if it has a single bio to submit. But you said "it
> >> should be possible to submit individual bios directly from
> >> make_request without passing them to raid1d and without using
> >> plugging". So can you explain how it is possible? I prefer not to
> >> change raid1.c code.
> >>
> >> >
> >> > There is a bit of extra subtlety but I can't really know how relevant that
> >> > might be to you without actually seeing you code.
> >> My code (in a different kernel module, not in raid1.c) is simply doing
> >> submit_bio. I want to wrap this with plug/unplug to avoid the handoff
> >> to raid1d and improve raid1 latency.
> >>
> >
> > I think I need to see the code you are working with to be able to suggest
> > anything used.
> I am working with kernel 3.8.13. But your master branch has the same
> code with respect to plug/unplug logic.
>
> > But if it works with plugging, then just do it that way(?).
> It works perfectly, and latency is much better. The only doubt is with
> bitmap_unplug being called from multiple threads now. However, it can
> happen for anybody that uses plug/unplug on top of MD raid1 (like ext4
> for example). So question is whether it is safe for MD users to
> plug/unplug when submitting bios to MD. If not, would you be fixing
> this?
Didn't I already answer that question?
Date: Tue, 9 Sep 2014 11:45:38 +1000
Hmmm... there could be an issue there. It is possible that some callers of
bitmap_unplug won't block when they should. bitmap_unplug should probably
wait unconditionally.
See
http://git.neil.brown.name/?p=md.git;a=commitdiff;h=339f60d943f848eb516aa4b82b5e187dbbe088dc
(not tested yet).
NeilBrown
>
> Alex.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Raid 1 vs Raid 10 single thread performance
From: David Brown @ 2014-09-12 8:49 UTC (permalink / raw)
To: Bostjan Skufca, linux-raid
In-Reply-To: <CAEp_DRAVPBvA34kgdjWqO6f6489SbmHQf-dFXC_SwSQd8e0C2w@mail.gmail.com>
On 10/09/14 23:24, Bostjan Skufca wrote:
> Hi,
>
> I have a simple question:
> - Where is the code that is used for actual RAID 10 creation? In
> kernel or in mdadm?
>
>
> Explanation:
>
> I was dissatisfied with single-threaded RAID 1 sequential read
> performance (basically boils down to the speed of one disk). I figured
> that instead of using level 1 I could create RAID level 10 and use two
> equally-sized partitions on each drive (instead of one).
>
> It turns out that if array is created properly, it is capable of
> sequential reads at almost 2x single device speed, as expected (on
> SSD!) and what would anyone expect from ordinary RAID 1.
>
> What does "properly" actually mean?
> I was doing some benchmarks with various raid configurations and
> figured out that the order of devices submitted to creation command is
> significant. It also makes raid10 created in such mode reliable or
> unreliable to a device failure (not partition failure, device failure,
> which means that two raid underlying devices fail at once).
>
> Sum:
> - if such array is created properly, it has redundancy in place and
> performs as expected
> - if not, it performs as raid1 and fails with one physical disk failure
>
> I am trying to find the code responsible for creation of RAID 10 in
> order to try and make it more inteligent about where to place RAID 10
> parts if it gets a list of devices to use, and some of those devices
> are on the same physical disks.
>
> Thanks for hints,
> b.
>
>
>
> PS: More details about testing is available here, but be warned, it is
> still a bit hectic to read:
> http://blog.a2o.si/2014/09/07/linux-software-raid-why-you-should-always-use-raid-10-instead-of-raid-1/
Hi,
First let me applaud your enthusiasm for trying to inform people about
raid in your blog, your interest in investigating different ideas in the
hope of making md raid faster and/or easier and/or safer.
Then let me tell you your entire blog post is wasted, because md already
has a solution that is faster, easier and safer than anything you have
come up with so far.
You are absolutely correct about the single-threaded read performance of
raid1 pairs - for a number of reasons, a single thread read will get
reads from only one disk. This is not a problem in many cases, because
you often have multiple simultaneous reads on "typical" systems with
raid1. But for some cases, such as a high performance desktop, it can
be a limitation.
You are also correct that the solution is basically to split the drives
into two parts, pair up halves from each disk as raid1 mirrors, and
stripe the two mirrors as raid0.
And you are correct that you have to get the sets right, or you will may
lose redundancy and/or speed.
Fortunately, Neil and the other md raid developers are way ahead of you.
Neil gave you the pointers in one of his replies, but I suspect you did
not understand that Linux raid10 is not limited to the arrangement of
traditional raid10, and thus did not see his point.
md raid and mdadmin already support a very flexible form of raid10.
Unlike traditional raid10 that requires a multiple of 4 disks, Linux
raid10 can work with /any/ number of disks greater than 1. There are
various layouts that can be used for this - the Wikipedia entry gives
some useful diagrams:
<http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10>
You can also read about it in the mdadm manual page, and various
documents and resources around the web.
In your particular case, what you want is to use "--layout raid10,f2" on
your two disks. This asks md to split each disk (or the partitions you
use) into two parts, without creating any new partitions. The first
half of disk 1 is mirrored with the second half of disk 2, and vice
versa, then these mirrors are striped. This is very similar to the
layout you are trying to achieve, except for four points:
The mirrors are crossed-over, so that a first half is mirrored with a
second half. This makes no difference on an SSD, but makes a huge
difference on a hard disk.
mdadm and md raid get the ordering right every time - there is no need
to worry about the ordering of the two disks.
You don't have to have extra partitions, automatic detection works, and
the layout has one less layer, meaning less complexity and lower latency
and overheads.
md raid knows more about the layout, and can use it to optimise the speed.
In particular, md will (almost) always read from the outer halves of the
disks. On a hard disk, this can be twice the speed of the inner layers.
Obviously you pay a penalty in writing when you have such an arrangement
- writes need to go to both disks, and involve significant head
movement. There are other raid10 layouts that have lower streamed read
speeds but also lower write latencies (choose the balance you want).
With this in mind, I hope you can try out raid10,f2 layout on your
system and then change your blog to show how easy this all is with md
raid, how practical it is for a fast workstation or desktop, and how
much faster such a setup is than anything that can be achieved with
hardware raid cards or anything other than md raid.
mvh.,
David
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox