* Re: [PATCH 0/1] RFC: Use /dev/md/X as default name
From: Jes Sorensen @ 2015-02-18 22:11 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20150219083708.2ab46006@notabene.brown>
NeilBrown <neilb@suse.de> writes:
> On Wed, 18 Feb 2015 13:12:09 -0500 Jes Sorensen <Jes.Sorensen@redhat.com>
> wrote:
>
>> Jes.Sorensen@redhat.com writes:
>> > From: Jes Sorensen <Jes.Sorensen@redhat.com>
>> >
>> > I have received some issues for when creating an array using a
>> > /dev/mdX name, the matching symlink in /dev/md/X isn't
>> > created. Whereas if you create /dev/md/X, /dev/mdX is created
>> > automatically.
>> >
>> > I was trying to see if there was a better way of dealing with this,
>> > but I couldn't find one. If you have suggestions for a better solution
>> > I am all ears.
>> >
>> > Thoughts?
>>
>> Hi Neil,
>>
>> Any thoughts on this one?
>
> Thanks for the reminder....
>
> I'm not sure that I really see the problem.
>
> "I ask it to create /dev/mdX and it doesn't create /dev/md/X".
>
> Well ... no. You didn't ask it to. If you want it to create /dev/md/X,
> then ... ask it to.
>
> /dev/mdX is the canonical name. It always gets created.
> /dev/md/X is a convenient alias. It gets created if requested.
>
> Is there really a problem here worth solving?
>
> Maybe I missed something.
I have had complaints in Fedora from the installer people that they rely
on the /dev/md/ name being created when they create a new device. It is
also inconsistent because /dev/md/<X> will be created if you run
'mdadm -As' later on.
I don't see it as a major issue, but I can see why it is frustrating for
some and I think there is something to be said for being consistent in
behavior.
Cheers,
Jes
^ permalink raw reply
* Re: [PATCH 0/1] RFC: Use /dev/md/X as default name
From: NeilBrown @ 2015-02-18 22:32 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid
In-Reply-To: <wrfj61ay6fai.fsf@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2738 bytes --]
On Wed, 18 Feb 2015 17:11:33 -0500 Jes Sorensen <Jes.Sorensen@redhat.com>
wrote:
> NeilBrown <neilb@suse.de> writes:
> > On Wed, 18 Feb 2015 13:12:09 -0500 Jes Sorensen <Jes.Sorensen@redhat.com>
> > wrote:
> >
> >> Jes.Sorensen@redhat.com writes:
> >> > From: Jes Sorensen <Jes.Sorensen@redhat.com>
> >> >
> >> > I have received some issues for when creating an array using a
> >> > /dev/mdX name, the matching symlink in /dev/md/X isn't
> >> > created. Whereas if you create /dev/md/X, /dev/mdX is created
> >> > automatically.
> >> >
> >> > I was trying to see if there was a better way of dealing with this,
> >> > but I couldn't find one. If you have suggestions for a better solution
> >> > I am all ears.
> >> >
> >> > Thoughts?
> >>
> >> Hi Neil,
> >>
> >> Any thoughts on this one?
> >
> > Thanks for the reminder....
> >
> > I'm not sure that I really see the problem.
> >
> > "I ask it to create /dev/mdX and it doesn't create /dev/md/X".
> >
> > Well ... no. You didn't ask it to. If you want it to create /dev/md/X,
> > then ... ask it to.
> >
> > /dev/mdX is the canonical name. It always gets created.
> > /dev/md/X is a convenient alias. It gets created if requested.
> >
> > Is there really a problem here worth solving?
> >
> > Maybe I missed something.
>
> I have had complaints in Fedora from the installer people that they rely
> on the /dev/md/ name being created when they create a new device. It is
> also inconsistent because /dev/md/<X> will be created if you run
> 'mdadm -As' later on.
If they rely on the /dev/md/ name being created, then surely they should ask
for it to be created.
Is it really harder to run "mdadm -C /dev/md/0" than "mdadm -C /dev/md0" ??
If you create an array as "/dev/md0", then after subsequent "mdadm
-As" /dev/md0 will exist.
If you create an array as "/dev/md/0", then after subsequent mdadm
-As", /dev/md/0 will exist.
The fact that something unasked for also exists is a bonus.
>
> I don't see it as a major issue, but I can see why it is frustrating for
> some and I think there is something to be said for being consistent in
> behavior.
I think the behaviour is perfectly consistent. It just follows rules that
are slightly less trivial that some people appear to want.
However....
If you changed
mp = map_by_uuid(&map, info->uuid);
if (mp && mp->path &&
strncmp(mp->path, "/dev/md/", 8) == 0) {
printf("MD_DEVNAME=");
print_escape(mp->path+8);
putchar('\n');
}
in Detail.c so that when mp->path were "/dev/md0", MD_DEVNAME became "0",
then you should get the result that you are after, and I probably wouldn't
object to the patch.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: NeilBrown @ 2015-02-19 6:04 UTC (permalink / raw)
To: linux RAID
[-- Attachment #1: Type: text/plain, Size: 3303 bytes --]
Hi all,
as you probably know, when md is doing resync and notices other IO it
throttles the resync to a configured "minimum", which defaults to
1MB/sec/device.
On a lot of modern devices, that is extremely slow.
I don't want to change the default (not all drives are the same) so I
wanted to come up with something that it a little bit dynamic.
After a bit of pondering and a bit of trial and error, I have the following.
It sometimes does what I want. I don't think it is ever really bad.
I'd appreciate it if people could test it on different hardware, different
configs, different loads.
What I have been doing is running
while :; do cat /sys/block/md0/md/sync_speed; sleep 5;
done > /root/some-file
while a resync is happening and a load is being imposed.
I do this with the old kernel and with this patch applied, then use
gnuplot to look at the sync_speed graphs.
I'd like to see that the new code is never slower than the old, and rarely more
than 20% of the available throughput when there is significant load.
Any test results or other observations most welcome,
Thanks,
NeilBrown
When md notices non-sync IO happening while it is trying
to resync (or reshape or recover) it slows down to the
set minimum.
The default minimum might have made sense many years ago
but the drives have become faster. Changing the default
to match the times isn't really a long term solution.
This patch changes the code so that instead of waiting until the speed
has dropped to the target, it just waits until pending requests
have completed, and then waits about as long again.
This means that the delay inserted is a function of the speed
of the devices.
Test show that:
- for some loads, the resync speed is unchanged. For those loads
increasing the minimum doesn't change the speed either.
So this is a good result. To increase resync speed under such
loads we would probably need to increase the resync window
size.
- for other loads, resync speed does increase to a reasonable
fraction (e.g. 20%) of maximum possible, and throughput of
the load only drops a little bit (e.g. 10%)
- for other loads, throughput of the non-sync load drops quite a bit
more. These seem to be latency-sensitive loads.
So it isn't a perfect solution, but it is mostly an improvement.
Signed-off-by: NeilBrown <neilb@suse.de>
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 94741ee6ae69..ce6624b3cc1b 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7669,11 +7669,20 @@ void md_do_sync(struct md_thread *thread)
/((jiffies-mddev->resync_mark)/HZ +1) +1;
if (currspeed > speed_min(mddev)) {
- if ((currspeed > speed_max(mddev)) ||
- !is_mddev_idle(mddev, 0)) {
+ if (currspeed > speed_max(mddev)) {
msleep(500);
goto repeat;
}
+ if (!is_mddev_idle(mddev, 0)) {
+ /*
+ * Give other IO more of a chance.
+ * The faster the devices, the less we wait.
+ */
+ unsigned long start = jiffies;
+ wait_event(mddev->recovery_wait,
+ !atomic_read(&mddev->recovery_active));
+ schedule_timeout_uninterruptible(jiffies-start);
+ }
}
}
printk(KERN_INFO "md: %s: %s %s.\n",mdname(mddev), desc,
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* Re: What are mdadm maintainers to do? (error recovery redundancy/data loss)
From: Chris Murphy @ 2015-02-19 6:12 UTC (permalink / raw)
Cc: linux-raid
In-Reply-To: <loom.20150218T102011-486@post.gmane.org>
On Wed, Feb 18, 2015 at 4:04 AM, Chris <email.bug@arcor.de> wrote:
>>
>
> Hello all,
>
> the discussion about SCTERC boils down to letting the drive attempt ERC a
> little more or less. For any given disk experience seems to tell the slight
> difference is, that if ERC is allowed longer you may see the first
> unrecoverable erros (UREs) just a little (maybe only a month) later.
>
> UREs are inevitable. Thus, if I run a filesystem on just a single drive it
> will get corrupted at some point, nothing to do about it.
On a single randomly selective drive, I disagree. In aggregate, that's
true, eventually it will happen, you just won't know which drive or
when it'll happen. I have a number of 5+ year old drives that have
never reported a URE. Meanwhile another drive has so many bad sectors
I only keep it around for abusive purposes.
>
> Wait, except..., use a redundant raid! And here it makes a lot of a
> difference that the drive's ERC actually terminates before the controller
> timeout, to not loose all your redundacy again and be in hight risk of UREs
> showing up during the re-sync.
>
> So for a proper comparison we need to look at the difference it makes in the
> usage scenarios (error delay vs. loosing redundant error resilence + URE
> triggering), not at the single recoverable/unrecoverable error incidence. It
> looks to me, that it makes a lot of a differnce to redundant raids and no
> qualitative difference to single disk filesystems.
>
> And we need to keep in mind that single disk filesystems do also depend on
> the disk to stop grinding away with ERC attempts before the controller
> timout. Otherwise disk reset may make the system clear buffers and loose
> open files? Without prolonging the linux default controller timout, SCTERC
> can prevent that where supported.
To get to one size fits all, where SCT ERC is disabled (consumer
drive), and the kernel command timer is increased accordingly, we
still need the delay reportable to user space. You can't have a by
default 2-3 minute showstopper without an explanation so that the user
can tune this back to 30 seconds or get rid of the drive or some other
mitigation. Otherwise this is a 2-3 minute silent failure. I know a
huge number of users who would assume this is a crash and force power
off the system.
The option where SCT ERC is configurable, you could also do this one
size fits all by setting this to say 50-70 deciseconds, and for read
failures to cause recovery if raid1+ is used, or cause a read retry
if it's single, raid0, or linear. In other words, control the retries
in software for these drives.
>> I don't know if a udev rule can say "If the drive exclusively uses md,
>> lvm, btrfs, zfs raid1, 4+ or nested of those, and if the drive does
>> not support configurable SCT ERC, then change the kernel command timer
>> for those devices to ~120 seconds" then that might be a plausible
>> solution to use consumer drives the manufacturer rather explicitly
>> proscribes from use in raid...
>
> The script called by the udev rule could do that, but can be kept as simple
> as proposed, and can set SCTERC regardles, because setting SCTERC below the
> controller timout makes a qualitative difference in running the redundant
> arrays and a marginal difference in running non-redundant filesystems. (And
> nevertheless, set long controller timout for devices that don's support SCTERC.)
I can't agree at all, lacking facts, that this change is marginal for
non-redundant configurations. I've seen no data how common long
recovery incidents are, or how much more common data loss would be if
long recovery were prevented.
The mere fact they exist suggests they're necessary. It may very well
be that the ECC code or hardware used is so slow that it really does
take so unbelievably long (really 30 seconds is an eternity, and a
minute seems outrageous, and 2-3 minutes seems wholly ridiculous as in
worthy of brutal unrelenting ridicule); but that doesn't even matter
even if it is true, that's the behavior of the ECC whether we like it
or not, we can't just willy nilly turn these things off without
understanding the consequences. Just saying it's marginal doesn't make
it true.
So if SCT ERC is short, now you have to have a mitigation for the
possibly higher number of URE's this will result in, in the form of
kernel instigated read retries on read fail. And in fact, this may be
false. The retries the drive does internally might be completely
different than the kernel doing another read. The way data is encoded
on the drive these days bears no resemblance to discreet 1's and 0's.
And you also need a reliable opt out for SSD's. Their failures seem
rather different.
--
Chris Murphy
^ permalink raw reply
* mdadm raid 5 one disk overwritten file system failed
From: John Andre Taule @ 2015-02-19 7:38 UTC (permalink / raw)
To: linux-raid
Hi!
Case: mdadm Raid 5 4 2TB disks. ext4 formatted spanning the raid.
Attack: dd if=/dev/zero of=/dev/sdb bs=1M
Expected result would be a raid that could be recovered without data loss.
Result was that the file system failed and not possible to recover.
As I understand it if this was a "hardware type fake" raid controller, the
outcome would be uncertain. However I'm a bit confused as to why the raid
(or more specifically the file system) would fail so horrible when losing
one disk. Is there perhaps critical information written "outside" the raid
on the physical disk, and this where overwritten in the attack?
It would be nice to have an exact idea as to why it failed so hard, and how
obvious it should be that this attack would have more consequence then a
degraded raid.
//Regards
^ permalink raw reply
* Re: mdadm raid 5 one disk overwritten file system failed
From: Mikael Abrahamsson @ 2015-02-19 11:20 UTC (permalink / raw)
To: John Andre Taule; +Cc: linux-raid
In-Reply-To: <020601d04c17$08c10290$1a4307b0$@johnandre.net>
On Thu, 19 Feb 2015, John Andre Taule wrote:
> Hi!
>
> Case: mdadm Raid 5 4 2TB disks. ext4 formatted spanning the raid.
> Attack: dd if=/dev/zero of=/dev/sdb bs=1M
>
> Expected result would be a raid that could be recovered without data loss.
>
> Result was that the file system failed and not possible to recover.
>
> As I understand it if this was a "hardware type fake" raid controller, the
> outcome would be uncertain. However I'm a bit confused as to why the raid
> (or more specifically the file system) would fail so horrible when losing
> one disk. Is there perhaps critical information written "outside" the raid
> on the physical disk, and this where overwritten in the attack?
Did you stop the array before you did the dd command, or you just did it?
If you just did it, most likely you overwrote the superblock on the drive
(located near the beginning of the drive by recent default), plus part of
the file system.
> It would be nice to have an exact idea as to why it failed so hard, and
> how obvious it should be that this attack would have more consequence
> then a degraded raid.
Because if the drive was active then the operating system most likely
didn't notice that you overwrote part of the data on the disk and the
drive wasn't failed.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* SV: mdadm raid 5 one disk overwritten file system failed
From: John Andre Taule @ 2015-02-19 14:00 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <alpine.DEB.2.02.1502191218140.4007@uplift.swm.pp.se>
The array was not stopped before dd was running. The "hacker" logged on,
left the command running and logged of. It was discovered the next morning
about 5 hours later, and there was very high load on the server, I think
this is why the command where discovered at all. This is how that raid have
performed earlier when a drive have failed.
I'm a bit surprised that overwriting anything on the physical disk should
corrupt the file system on the raid. I would think that would be similar to
a disk crashing or failing in other ways.
What you say that Linux might not have seen the disk as failing is
interesting. This could explain why the file system got corrupted.
-----Opprinnelig melding-----
Fra: Mikael Abrahamsson [mailto:swmike@swm.pp.se]
Sendt: 19. februar 2015 12:20
Til: John Andre Taule
Kopi: linux-raid@vger.kernel.org
Emne: Re: mdadm raid 5 one disk overwritten file system failed
On Thu, 19 Feb 2015, John Andre Taule wrote:
> Hi!
>
> Case: mdadm Raid 5 4 2TB disks. ext4 formatted spanning the raid.
> Attack: dd if=/dev/zero of=/dev/sdb bs=1M
>
> Expected result would be a raid that could be recovered without data loss.
>
> Result was that the file system failed and not possible to recover.
>
> As I understand it if this was a "hardware type fake" raid controller,
> the outcome would be uncertain. However I'm a bit confused as to why
> the raid (or more specifically the file system) would fail so horrible
> when losing one disk. Is there perhaps critical information written
> "outside" the raid on the physical disk, and this where overwritten in the
attack?
Did you stop the array before you did the dd command, or you just did it?
If you just did it, most likely you overwrote the superblock on the drive
(located near the beginning of the drive by recent default), plus part of
the file system.
> It would be nice to have an exact idea as to why it failed so hard,
> and how obvious it should be that this attack would have more
> consequence then a degraded raid.
Because if the drive was active then the operating system most likely didn't
notice that you overwrote part of the data on the disk and the drive wasn't
failed.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: mdadm raid 5 one disk overwritten file system failed
From: Mikael Abrahamsson @ 2015-02-19 14:23 UTC (permalink / raw)
To: John Andre Taule; +Cc: linux-raid
In-Reply-To: <021f01d04c4c$5def1740$19cd45c0$@johnandre.net>
On Thu, 19 Feb 2015, John Andre Taule wrote:
> I'm a bit surprised that overwriting anything on the physical disk
> should corrupt the file system on the raid. I would think that would be
> similar to a disk crashing or failing in other ways.
Errr, in raid5 you have data blocks and parity blocks. WHen you overwrite
one of the component drives with zeroes, you're effectively doing the same
as writing 0:es to a non-raid drive every 3 $stripesize. You're zero:ing a
lot of the filesystem information.
> What you say that Linux might not have seen the disk as failing is
> interesting. This could explain why the file system got corrupted.
Correct. There is no mechanism that periodically checks the contents of
the superblock and fails the drive if it's not there anymore. So the drive
is never failed.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply
* Re: mdadm raid 5 one disk overwritten file system failed
From: Adam Goryachev @ 2015-02-19 14:39 UTC (permalink / raw)
To: John Andre Taule; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1502191521450.4007@uplift.swm.pp.se>
On 20/02/2015 01:23, Mikael Abrahamsson wrote:
> On Thu, 19 Feb 2015, John Andre Taule wrote:
>
>> I'm a bit surprised that overwriting anything on the physical disk
>> should corrupt the file system on the raid. I would think that would
>> be similar to a disk crashing or failing in other ways.
>
> Errr, in raid5 you have data blocks and parity blocks. WHen you
> overwrite one of the component drives with zeroes, you're effectively
> doing the same as writing 0:es to a non-raid drive every 3
> $stripesize. You're zero:ing a lot of the filesystem information.
>
>> What you say that Linux might not have seen the disk as failing is
>> interesting. This could explain why the file system got corrupted.
>
> Correct. There is no mechanism that periodically checks the contents
> of the superblock and fails the drive if it's not there anymore. So
> the drive is never failed.
>
In addition, there is no checking of the data when read to confirm that
the data on the first 4 disks = the checksum on the 5th disk (assuming a
5 disk raid5). This applies equally to all raid levels as currently
working from linux md raid. While there are some use cases where it
would be nice to confirm that the data read is correct, this has not yet
been implemented (for live operation, you can schedule a check at
periodic intervals).
Even if MD noticed that the value of the first 4 disks did not equal the
checksum on the 5th disk, it has no method to determine which disk
contained the wrong value (could be any of the data stripes, or the
parity stripe). raid6 begins to allow for this type of check, and I
remember a lot of work being done on this, however, I think that was
still an offline tool more useful for data recovery from partially
failed multiple drives.
From memory, there are filesystems which will do what you are asking
(check that the data received from disk is correct, use multiple 'disks'
and ensure protection from x failed drives, etc. I am certain zfs and
btrfs both support this. (I've never used either due to stability
concerns, but I read about them every now and then....)
Regards,
Adam
^ permalink raw reply
* SV: mdadm raid 5 one disk overwritten file system failed
From: John Andre Taule @ 2015-02-19 16:21 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54E5F5A7.3090609@websitemanagers.com.au>
How common would this knowledge be?
Personally I would never do something like this on a live system just
because there is too many unknown variables in play. I know what the
different raids do. I am not working on this full time, my day to day work
is toward the user end of the application stack. Usually we use Areca
hardware raids, but this particular raid used mdadm, well because that's
what was available at the time. Its been stable enough, I think its survived
2 or 3 failed drives, of course not at the same time.
I would like to thank the list for confirming my suspicion that there was
something else at play here that made the /dev/zero do more damage then the
"hacker" believed it would do.
//Regards
-----Opprinnelig melding-----
Fra: Adam Goryachev [mailto:mailinglists@websitemanagers.com.au]
Sendt: 19. februar 2015 15:40
Til: John Andre Taule
Kopi: linux-raid@vger.kernel.org
Emne: Re: mdadm raid 5 one disk overwritten file system failed
On 20/02/2015 01:23, Mikael Abrahamsson wrote:
> On Thu, 19 Feb 2015, John Andre Taule wrote:
>
>> I'm a bit surprised that overwriting anything on the physical disk
>> should corrupt the file system on the raid. I would think that would
>> be similar to a disk crashing or failing in other ways.
>
> Errr, in raid5 you have data blocks and parity blocks. WHen you
> overwrite one of the component drives with zeroes, you're effectively
> doing the same as writing 0:es to a non-raid drive every 3
> $stripesize. You're zero:ing a lot of the filesystem information.
>
>> What you say that Linux might not have seen the disk as failing is
>> interesting. This could explain why the file system got corrupted.
>
> Correct. There is no mechanism that periodically checks the contents
> of the superblock and fails the drive if it's not there anymore. So
> the drive is never failed.
>
In addition, there is no checking of the data when read to confirm that the
data on the first 4 disks = the checksum on the 5th disk (assuming a
5 disk raid5). This applies equally to all raid levels as currently working
from linux md raid. While there are some use cases where it would be nice to
confirm that the data read is correct, this has not yet been implemented
(for live operation, you can schedule a check at periodic intervals).
Even if MD noticed that the value of the first 4 disks did not equal the
checksum on the 5th disk, it has no method to determine which disk contained
the wrong value (could be any of the data stripes, or the parity stripe).
raid6 begins to allow for this type of check, and I remember a lot of work
being done on this, however, I think that was still an offline tool more
useful for data recovery from partially failed multiple drives.
From memory, there are filesystems which will do what you are asking (check
that the data received from disk is correct, use multiple 'disks'
and ensure protection from x failed drives, etc. I am certain zfs and btrfs
both support this. (I've never used either due to stability concerns, but I
read about them every now and then....)
Regards,
Adam
^ permalink raw reply
* Re: mdadm raid 5 one disk overwritten file system failed
From: Piergiorgio Sartor @ 2015-02-19 17:15 UTC (permalink / raw)
To: John Andre Taule; +Cc: linux-raid
In-Reply-To: <020601d04c17$08c10290$1a4307b0$@johnandre.net>
On Thu, Feb 19, 2015 at 08:38:19AM +0100, John Andre Taule wrote:
> Hi!
>
> Case: mdadm Raid 5 4 2TB disks. ext4 formatted spanning the raid.
> Attack: dd if=/dev/zero of=/dev/sdb bs=1M
>
> Expected result would be a raid that could be recovered without data loss.
>
> Result was that the file system failed and not possible to recover.
>
> As I understand it if this was a "hardware type fake" raid controller, the
> outcome would be uncertain. However I'm a bit confused as to why the raid
> (or more specifically the file system) would fail so horrible when losing
> one disk. Is there perhaps critical information written "outside" the raid
> on the physical disk, and this where overwritten in the attack?
>
> It would be nice to have an exact idea as to why it failed so hard, and how
> obvious it should be that this attack would have more consequence then a
> degraded raid.
In this situation, there is no HDD failure.
The kernel, the md driver, the sata driver and so on,
cannot detect any failure, because there is none.
The HDD is alive and kicking and well writing.
Just to be clear and avoid confusion, the (redundant)
RAID does *not* check, at each read operation, that
the data is consistent. It does only use redundancy
in order to re-generate missing data *after* a failure
is detected.
So, writing to a RAID component does not trigger any
error, hence no failure, hence no reconstruction, but
a corrupted filesystem.
bye,
pg
>
> //Regards
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
piergiorgio
^ permalink raw reply
* Re: [PATCH] [md] raid5: check faulty flag for array status during recovery.
From: NeilBrown @ 2015-02-19 21:51 UTC (permalink / raw)
To: Eric Mei; +Cc: linux-raid, eric.mei
In-Reply-To: <94BC57C5-6223-435B-96FD-7DA2F6B4E561@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3665 bytes --]
On Tue, 6 Jan 2015 15:24:24 -0700 Eric Mei <meijia@gmail.com> wrote:
> Hi Neil,
>
> In a MDRAID derived work we found and fixed a data corruption bug. We think this also affect vanilla MDRAID, but we didn’t directly prove that by constructing a test to show the corruption. Following is the theoretical analysis, please kindly review and see if I missed something.
>
> To rebuild a stripe, MD checks whether array will be optimal after rebuild complete, if that’s true, we’ll mark the WIB bit to be cleared, the purpose is to enable “incremental rebuild”. The code section is like this:
>
> /* Need to check if array will still be degraded after recovery/resync
> * We don't need to check the 'failed' flag as when that gets set,
> * recovery aborts.
> */
> for (i = 0; i < conf->raid_disks; i++)
> if (conf->disks[i].rdev == NULL)
> still_degraded = 1;
>
> The problem is that only checking rdev == NULL might not be enough. Suppose both 2 drives D0 and D1 failed and marked as Faulty; We immediately removed D0 from array, but because some lingering IO on D1, it remains in array with Faulty flags on. A new drive pulled in, rebuild against D0 starts. Now because no rdev is NULL, MD thinks array will be optimal. If some writes happened before rebuild reaches the region, their dirty bits in WIB will be cleared. When later add D1 back into array, we’ll skip rebuilding those stripes, thus data corruption.
>
> The attached patch (against 3.18.0-rc6) is supposed to fix this issue.
>
> Thanks
> Eric
>
Hi Eric,
sorry for the delay, and thanks for the reminder...
The issue you described could only affect RAID6 as it requires the array to
continue with two failed drives.
However in the RAID6 case I think you are correct - there is a chance of
corruption if there is a double failure and a delay in removing one device.
Your patch isn't quite safe as conf->disks[i].rdev can become NULL at any
moment, so it could become NULL between testing and de-referencing.
So I've modified it as follows.
Thanks,
NeilBrown
Author: Eric Mei <eric.mei@seagate.com>
Date: Tue Jan 6 09:35:02 2015 -0800
raid5: check faulty flag for array status during recovery.
When we have more than 1 drive failure, it's possible we start
rebuild one drive while leaving another faulty drive in array.
To determine whether array will be optimal after building, current
code only check whether a drive is missing, which could potentially
lead to data corruption. This patch is to add checking Faulty flag.
Signed-off-by: NeilBrown <neilb@suse.de>
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index bc6d7595ad76..022a0d99e110 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -5120,12 +5120,17 @@ static inline sector_t sync_request(struct mddev *mddev, sector_t sector_nr, int
schedule_timeout_uninterruptible(1);
}
/* Need to check if array will still be degraded after recovery/resync
- * We don't need to check the 'failed' flag as when that gets set,
- * recovery aborts.
+ * Note in case of > 1 drive failures it's possible we're rebuilding
+ * one drive while leaving another faulty drive in array.
*/
- for (i = 0; i < conf->raid_disks; i++)
- if (conf->disks[i].rdev == NULL)
+ rcu_read_lock();
+ for (i = 0; i < conf->raid_disks; i++) {
+ struct md_rdev *rdev = ACCESS_ONCE(conf->disks[i].rdev);
+
+ if (rdev == NULL || test_bit(Faulty, &rdev->flags))
still_degraded = 1;
+ }
+ rcu_read_unlock();
bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* Re: mdadm raid 5 one disk overwritten file system failed
From: Wols Lists @ 2015-02-19 22:15 UTC (permalink / raw)
To: Adam Goryachev, John Andre Taule; +Cc: linux-raid
In-Reply-To: <54E5F5A7.3090609@websitemanagers.com.au>
On 19/02/15 14:39, Adam Goryachev wrote:
> From memory, there are filesystems which will do what you are asking
> (check that the data received from disk is correct, use multiple 'disks'
> and ensure protection from x failed drives, etc. I am certain zfs and
> btrfs both support this. (I've never used either due to stability
> concerns, but I read about them every now and then....)
When I used Pr1mes, I don't remember whether it was hardware or
software, but I believe their drives implemented some form of parity
check and recovery.
Basically, every eight-bit byte you wrote went to disk as sixteen bits -
a data byte and a parity byte. I don't know how it worked but (1) you
could reconstruct either byte from the other, and (2) for any 1-bit
error you could tell which of the data or parity bytes was corrupt. For
any 2-bit error I think you had a 90% chance of telling which byte was
corrupt - something like that anyway.
Of course, that's no use if your hacker feeds their corrupt stream
through your parity mechanism, or if 0x00000000 is valid when read from
disk.
Cheers,
Wol
^ permalink raw reply
* Re: [PATCH] [md] raid5: check faulty flag for array status during recovery.
From: Eric Mei @ 2015-02-19 22:49 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, eric.mei
In-Reply-To: <20150220085147.03bb2247@notabene.brown>
Hi Neil, You are absolutely right we need RCU lock for this. Thank you
so much!
Eric
On 2015-02-19 2:51 PM, NeilBrown wrote:
> On Tue, 6 Jan 2015 15:24:24 -0700 Eric Mei <meijia@gmail.com> wrote:
>
>> Hi Neil,
>>
>> In a MDRAID derived work we found and fixed a data corruption bug. We think this also affect vanilla MDRAID, but we didn’t directly prove that by constructing a test to show the corruption. Following is the theoretical analysis, please kindly review and see if I missed something.
>>
>> To rebuild a stripe, MD checks whether array will be optimal after rebuild complete, if that’s true, we’ll mark the WIB bit to be cleared, the purpose is to enable “incremental rebuild”. The code section is like this:
>>
>> /* Need to check if array will still be degraded after recovery/resync
>> * We don't need to check the 'failed' flag as when that gets set,
>> * recovery aborts.
>> */
>> for (i = 0; i < conf->raid_disks; i++)
>> if (conf->disks[i].rdev == NULL)
>> still_degraded = 1;
>>
>> The problem is that only checking rdev == NULL might not be enough. Suppose both 2 drives D0 and D1 failed and marked as Faulty; We immediately removed D0 from array, but because some lingering IO on D1, it remains in array with Faulty flags on. A new drive pulled in, rebuild against D0 starts. Now because no rdev is NULL, MD thinks array will be optimal. If some writes happened before rebuild reaches the region, their dirty bits in WIB will be cleared. When later add D1 back into array, we’ll skip rebuilding those stripes, thus data corruption.
>>
>> The attached patch (against 3.18.0-rc6) is supposed to fix this issue.
>>
>> Thanks
>> Eric
>>
> Hi Eric,
> sorry for the delay, and thanks for the reminder...
>
> The issue you described could only affect RAID6 as it requires the array to
> continue with two failed drives.
>
> However in the RAID6 case I think you are correct - there is a chance of
> corruption if there is a double failure and a delay in removing one device.
>
> Your patch isn't quite safe as conf->disks[i].rdev can become NULL at any
> moment, so it could become NULL between testing and de-referencing.
> So I've modified it as follows.
>
> Thanks,
> NeilBrown
>
>
>
> Author: Eric Mei <eric.mei@seagate.com>
> Date: Tue Jan 6 09:35:02 2015 -0800
>
> raid5: check faulty flag for array status during recovery.
>
> When we have more than 1 drive failure, it's possible we start
> rebuild one drive while leaving another faulty drive in array.
> To determine whether array will be optimal after building, current
> code only check whether a drive is missing, which could potentially
> lead to data corruption. This patch is to add checking Faulty flag.
>
> Signed-off-by: NeilBrown <neilb@suse.de>
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index bc6d7595ad76..022a0d99e110 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -5120,12 +5120,17 @@ static inline sector_t sync_request(struct mddev *mddev, sector_t sector_nr, int
> schedule_timeout_uninterruptible(1);
> }
> /* Need to check if array will still be degraded after recovery/resync
> - * We don't need to check the 'failed' flag as when that gets set,
> - * recovery aborts.
> + * Note in case of > 1 drive failures it's possible we're rebuilding
> + * one drive while leaving another faulty drive in array.
> */
> - for (i = 0; i < conf->raid_disks; i++)
> - if (conf->disks[i].rdev == NULL)
> + rcu_read_lock();
> + for (i = 0; i < conf->raid_disks; i++) {
> + struct md_rdev *rdev = ACCESS_ONCE(conf->disks[i].rdev);
> +
> + if (rdev == NULL || test_bit(Faulty, &rdev->flags))
> still_degraded = 1;
> + }
> + rcu_read_unlock();
>
> bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: What are mdadm maintainers to do? (error recovery redundancy/data loss)
From: Roger Heflin @ 2015-02-20 5:12 UTC (permalink / raw)
To: Chris Murphy; +Cc: Linux RAID
In-Reply-To: <CAJCQCtQWBYRu7DFrdoKzj7G1-UKDJKBqyVn=6p91oagLfqd_Kw@mail.gmail.com>
On Thu, Feb 19, 2015 at 12:12 AM, Chris Murphy <lists@colorremedies.com> wrote:
> On Wed, Feb 18, 2015 at 4:04 AM, Chris <email.bug@arcor.de> wrote:
>>>
>>
>> Hello all,
>>
>
> On a single randomly selective drive, I disagree. In aggregate, that's
> true, eventually it will happen, you just won't know which drive or
> when it'll happen. I have a number of 5+ year old drives that have
> never reported a URE. Meanwhile another drive has so many bad sectors
> I only keep it around for abusive purposes.
And I have seen the same. Not all will fail even of a given type.
It also appears if one was really worried, running smartctl -t long often
(daily or weekly) can result in the disk finding and re-writing or moving
the bad sector. I have a disk that started given me trouble and the bad
block count has risen a few times without an os level error during the
-t long test.
>
>
>
>
> To get to one size fits all, where SCT ERC is disabled (consumer
> drive), and the kernel command timer is increased accordingly, we
> still need the delay reportable to user space. You can't have a by
> default 2-3 minute showstopper without an explanation so that the user
> can tune this back to 30 seconds or get rid of the drive or some other
> mitigation. Otherwise this is a 2-3 minute silent failure. I know a
> huge number of users who would assume this is a crash and force power
> off the system.
>
> The option where SCT ERC is configurable, you could also do this one
> size fits all by setting this to say 50-70 deciseconds, and for read
> failures to cause recovery if raid1+ is used, or cause a read retry
> if it's single, raid0, or linear. In other words, control the retries
> in software for these drives.
This gets more interesting. From what I can tell with my drivers (reds
and seagate video driver) they some allow erc to be set only 7 or higher,
and some allow things to be set lower. I have been setting mine lower
when it allows since I have raid 6 and expect to be able to get the data
from the other disks. This min 7 vs min of lower may be a further
distinction between the green(none), red 7, seagate VX (1.0 allowed).
My has video recordings...when the video pauses I counting how long.
I almost always appear to see the full 7 seconds, so I suspect that if
it does not recover in a short time it appears to be unlikely to recover it
all all. Given the data corruption issue without raid the vendors may
have the though that they cannot really do anything else but retry in the
no raid case.
>
>
>
> I can't agree at all, lacking facts, that this change is marginal for
> non-redundant configurations. I've seen no data how common long
> recovery incidents are, or how much more common data loss would be if
> long recovery were prevented.
>
> The mere fact they exist suggests they're necessary. It may very well
> be that the ECC code or hardware used is so slow that it really does
> take so unbelievably long (really 30 seconds is an eternity, and a
> minute seems outrageous, and 2-3 minutes seems wholly ridiculous as in
> worthy of brutal unrelenting ridicule); but that doesn't even matter
> even if it is true, that's the behavior of the ECC whether we like it
> or not, we can't just willy nilly turn these things off without
> understanding the consequences. Just saying it's marginal doesn't make
> it true.
>
> So if SCT ERC is short, now you have to have a mitigation for the
> possibly higher number of URE's this will result in, in the form of
> kernel instigated read retries on read fail. And in fact, this may be
> false. The retries the drive does internally might be completely
> different than the kernel doing another read. The way data is encoded
> on the drive these days bears no resemblance to discreet 1's and 0's.
Given the drive likely has some ability to adjust the levels of the 0 and 1,
I can see the disk retries possibly playing some games like that trying to get
a better answer. It is worth nothing that 7 seconds does mean around 70
retries of the read (data comes under the head 70 times). I doubt the ECC is
so slow it takes more than 10-20 ms to calculate more extreme failures. So
I am betting on the retries being what is recovering the data.
^ permalink raw reply
* rules and scripts (erc timeout fix)
From: email.bug @ 2015-02-20 16:54 UTC (permalink / raw)
To: linux-raid
Hello all,
enjoy, I tested the scripts set timeouts ok here, but I only have
drives that support erc timeouts (even if some have it disabled by default) none that
would really require setting a long controller timeout.
Cheers,
Chris
smartctl-timeouts README
The smartctl-timeouts scripts adjust the disk timeouts according to use-cases,
fixing common mismatching defaults that have often lead to data loss.
The scripts are to be called by udev rules during device initialization.
Every redundancy providing block device module may ship with proper udev rules
that initialize the timeouts for their possibly redundant devices.
The module may further adjust the actual status according to run-time changes.
NOTE: Correct execution during boot requires that distro package managers
hook smartctl and the smartctl-timeouts scripts into the initramfs.
RATIONALE
The error recovery (ERC) timout *must* be shorter than the controller timeout.
Otherwise read errors will cause controller resets, leading to direct data loss
or, if it is a redundant disk, loss of redundancy and a very high probability
of another read error and data loss when re-establishing the redundancy.
If a drive does not support adjusting its ERC timout, the controller timeout
must be increased above the drive's 'maximal error recovery time.
If you don't want that kind of long device timeout, you should look for a drive
with SCT ERC timout support. (smartctl -l scterc /dev/...)
IMPACT
If possible, the ERC timeout is adjusted to the controller timeout minus 5 seconds,
for all disks that contain possibly redundant data.
The controller timeout is only changed (raising it to LONG_CTRL_WAIT_SECONDS)
for drives without SCTERC support and entirely non-redundant-disks, to allow these
drives to properly finish their error recovery before a reset is triggerd.
Because controller timeouts are only increased selectively (only drives without SCTERC
support and surely non-redundant disks), the scripts won't change any timeouts in
professional, dedicated, redundant setups (e.g. storage servers etc.), except if
LONG_WAIT_ALL_NONREDUND_DISKS is configured to be true.
TODO
* non-redundant-partitions: conditional udev triggering, or a test in the script could
determine if all partions of the disk have been detected already and are all
non-redundant, to call non-redundant-disk in this case.
* parser to read ERC timout values?
- redundant-disk: a previously set "controller timeout - 5 seconds" ERC timeout
(possibly-redundant), could also be reset to 7 seconds, not just a "Disabled" value.
* If a redundancy controlling kernel module is to make dynamic adjustments,
"redundant-partition" needs implementation.
^ permalink raw reply
* Re: udev rules and scripts (erc timeout fix)
From: Chris @ 2015-02-22 10:23 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <B639B66A-F606-43CD-8FCC-D1A7810762D1@gmx.de>
[-- Attachment #1: Type: text/plain, Size: 24224 bytes --]
Hello all,
thanks for giving notice that the attachment didn't come through the
mailinglist.
The following is an improved version with its README, inserted inline.
It has changes to support configuring specific timeouts, and switching
between them.
Cheers,
Chris
--------
# smartctl-timeouts_defaults
# Defaults used by smartctl-timeouts scripts:
NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS="183"
# Should always be set. (e.g. 183)
# Used for disks without SCTERC support, to prevent too early
# resets. This long controller timeout value should be above
# the usual error recovery time of the harddrives without
# SCTERC support. Unfortunately, these values don't seem
# to be readable from the drives nor published.
NONREDUNDANT_UNSURE_RESET_ALL_DISKS=""
# If "true", ERC timout gets disabled for non-redundant disks
# an the value is used as the controller timeout.
# Can be set to "true" to try letting non-redundant disks fully
# complete their error recovery attempt.
# The configuration options below can be left blank or commmented
# out. This results in working with the hardware, kernel, or
# distribution defaults, and doing only necessary adaptions when
# initializing.
# But without configuring specific values, switching between
# the redundancy modes may not work well.
#NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS="63"
# May be set to allow ample ERC time (e.g. 63).
# If blank the current timeout will not be changed, if possible.
# Note that the max. ERC timout is 99 seconds, so an exceeding
# controller timeout won't result in longer error correction
# attempts. Possibly use NONREDUNDANT_UNSURE_RESET_ALL_DISKS if your
# disk will do longer error correction attempts, if the ERC
# timeout is disabled.
#POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS="48"
# May be set to allow some (-5s ) ERC timeout, yet not blocking
# redundant disks for too long.
# If blank the current setting will not be changed, if possible.
#REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS="29"
# May be set to quickly reset blocking disks.
# If blank the current setting will not be changed, if possible.
TIMING_CMD="/usr/sbin/smartctl -l scterc"
set -o nounset -o errexit
-------
# do not edit this file, it will be overwritten on update
# Don't process any events if anaconda is running as anaconda brings up
# raid devices manually
ENV{ANACONDA}=="?*", GOTO="md_inc_end"
# assemble md arrays
SUBSYSTEM!="block", GOTO="md_inc_end"
# handle potential components of arrays (the ones supported by md)
ENV{ID_FS_TYPE}=="linux_raid_member", GOTO="md_inc"
# "noiswmd" on kernel command line stops mdadm from handling
# "isw" (aka IMSM - Intel RAID).
# "nodmraid" on kernel command line stops mdadm from handling
# "isw" or "ddf".
IMPORT{cmdline}="noiswmd"
IMPORT{cmdline}="nodmraid"
ENV{nodmraid}=="?*", GOTO="md_inc_end"
ENV{ID_FS_TYPE}=="ddf_raid_member", GOTO="md_inc"
ENV{noiswmd}=="?*", GOTO="md_inc_end"
ENV{ID_FS_TYPE}=="isw_raid_member", GOTO="md_inc"
GOTO="md_inc_end"
LABEL="md_inc"
# initialize redundancy possibility status
# (only the kernel module could set actual run-time state, and may in the future
# set a dynamic FASTFAIL kernel device property instead of calling smartctl-timeout scripts)
IMPORT{program}="BINDIR/mdadm --examine --export $tempnode"
ENV{MD_LEVEL}=="raid[1-9]*", ENV{REDUNDANT_DEV}="possibly"
ENV{MD_LEVEL}=="raid0", ENV{REDUNDANT_DEV}="false"
# remember you can limit what gets auto/incrementally assembled by
# mdadm.conf(5)'s 'AUTO' and selectively whitelist using 'ARRAY'
ACTION=="add|change", IMPORT{program}="BINDIR/mdadm --incremental --export $tempnode --offroot ${DEVLINKS}"
ACTION=="add|change", ENV{MD_STARTED}=="*unsafe*", ENV{MD_FOREIGN}=="no", ENV{SYSTEMD_WANTS}+="mdadm-last-resort@$env{MD_DEVICE}.timer"
ACTION=="remove", ENV{ID_PATH}=="?*", RUN+="BINDIR/mdadm -If $name --path $env{ID_PATH}"
ACTION=="remove", ENV{ID_PATH}!="?*", RUN+="BINDIR/mdadm -If $name"
LABEL="md_inc_end"
# initialize redundancy status for all surely non-redundant devices
# (The mdadm, btrfs, zfs, lvm, ... devices need too be adjusted by their own packages)
ENV{ID_FS_TYPE}!="linux_raid*|ddf_raid*|isw_raid*|lvm_*|LVM*|btrfs*|zfs*", ENV{REDUNDANT_DEV}="false"
# call initial HDD error correction timeouts adjustment
ENV{DEVTYPE}=="partition", ENV{REDUNDANT_DEV}=="possibly", TEST="/usr/sbin/smartctl", RUN+="BINDIR/smartctl-timeouts_possibly-redundant-partition.sh $parent"
ENV{DEVTYPE}=="partition", ENV{REDUNDANT_DEV}=="false", TEST="/usr/sbin/smartctl", RUN+="BINDIR/smartctl-timeouts_non-redundant-partition.sh $parent"
ENV{DEVTYPE}=="disk", ENV{REDUNDANT_DEV}=="possibly", TEST="/usr/sbin/smartctl", RUN+="BINDIR/smartctl-timeouts_posibly-redundant-disk.sh $devnode"
ENV{DEVTYPE}=="disk", ENV{REDUNDANT_DEV}=="false", TEST="/usr/sbin/smartctl", RUN+="BINDIR/smartctl-timeouts_non-redundant-disk.sh $devnode"
------
#!/bin/sh
# smartctl-timeouts_possibly-redundant-disk.sh
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
. $SCRIPT_DIR/smartctl-timeouts_defaults
HDD_DEV="$1"
echo "Adjusting $HDD_DEV timeouts:"
if ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q Disabled \
&& ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q seconds
then
# ERC timeout is not supported (not disabled and not set):
# * Set the controller timeout to be considerably loooooong.
# - To allow the drive to give up its ERC attempts by itself.
# - Let the drive return a proper read error, so that the redundancy
# provider (md, lvm, btrfs, ...) can re-write the bad block.
# - Disk read errors thus result in long i/o blocking periods with
# no error messages that may not be watched by or reported to the user,
# - but waiting this long should prevent unecessary controller resets of the
# entire drive and the corresponding loss of redundancy/data.
echo "Drive without ERC timeout support, setting NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS (${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS}s)"
echo ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
else
SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT="false"
# reset controller timeout, if a configured value was previously set
if [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ]
then
SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT="true"
echo "resetting controller from another configured value (`cat /sys/block/${HDD_DEV}/device/timeout`s) to ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-30}s"
echo ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-30} >/sys/block/${HDD_DEV}/device/timeout
else
# set possibly-redundant timeout anyway, if configured
if [ ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ] ; then
echo "setting controller timeout to POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS (${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS}s)"
echo ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
fi
fi
if ${TIMING_CMD} /dev/${HDD_DEV} | grep -q Disabled \
|| [ $SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT = "true" ] \
|| [ ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ]
then
# ERC timeout is disabled or configured:
# * set it to controller timeout -5 seconds
# - Allows redundancy provider to read data from another disk and re-write the bad block
# before the controller resets the entire drive and the raid looses redundancy/data completely.
# - Longer than the usual 7s default of dedicated raid drives, to allow as much
# ERC time as possible (good if degraded and for non-redundant partitions on same drive).
ERC_TENTHS=$(expr `cat /sys/block/${HDD_DEV}/device/timeout` \* 10 - 50)
# prevent exceeding max. scterc value
if [ $ERC_TENTHS -gt 999 ] ; then
ERC_TENTHS="999"
fi
echo "maximizing ERC timeout to controller timeout -5 seconds (`expr $ERC_TENTHS / 10`s)"
${TIMING_CMD},$ERC_TENTHS,$ERC_TENTHS /dev/${HDD_DEV} > /dev/null
fi
fi
----------
#! /bin/sh
# smartctl-timeouts_possibly-redundant-partition.sh
# This script sets the timeouts for "mixed drives" that contain redundant
# and non-redundant partitions.
# A single, possibly-redundant partition is enough to set the entire drive's
# timeouts to possibly-redundant settings (with a determined ERC timeout slightly
# below the default or configured controller timout, if possible).
#
# This avoids to risk unknown disk recovery times and needing a very long
# controller timeouts. Where configuring such a ERC timout is possible,
# this means the disk recovery may be terminated quicker than the drive
# would without the timout set, but it ensures that there will be no resets
# leading to data loss and redundancy loss.
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
$SCRIPT_DIR/smartctl-timeouts_possibly-redundant-disk.sh $1
--------------
#!/bin/sh
# smartctl-timeouts_non-redundant-disk.sh
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
. $SCRIPT_DIR/smartctl-timeouts_defaults
HDD_DEV="$1"
echo "Adjusting $HDD_DEV timeouts:"
if [ ${NONREDUNDANT_UNSURE_RESET_ALL_DISKS:-false} = "true" ] ; then
# * disable any ERC timeout
# - Allows the drive to do ERC without imposing a timeout.
${TIMING_CMD},0,0 /dev/${HDD_DEV} > /dev/null
# * Set the controller timeout to be considerably loooooong.
# - To allow the drive to give up its ERC attempts by itself.
# - Let the drive return a proper read error, so that the redundancy
# provider (md, lvm, btrfs, ...) can re-write the bad block.
# - Disk read errors thus result in long i/o blocking periods with
# no error messages that may not be watched by or reported to the user,
# - but waiting this long should prevent unecessary controller resets of the
# entire drive and the corresponding loss of redundancy/data.
echo "NONREDUNDANT_UNSURE_RESET_ALL_DISKS is true, setting NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS (${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS}s)"
echo ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
else
if ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q Disabled \
&& ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q seconds
then
# ERC timeout is not supported (not disabled and not set)
# * Set the controller timeout to be considerably loooooong.
echo "Drive without ERC timeout support, setting NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS (${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS}s)"
echo ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
else
if ${TIMING_CMD} /dev/${HDD_DEV} | grep -q seconds \
|| [ ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ]
then
# reset controller timeout, if a configured value was previously set
if [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ]
then
echo "resetting controller from another configured value (`cat /sys/block/${HDD_DEV}/device/timeout`s) to ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-60}s"
echo ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-60} >/sys/block/${HDD_DEV}/device/timeout
else
# set non-redundant timeout anyway, if configured
if [ ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ] ; then
echo "setting configured NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS (${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS}s)"
echo ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
fi
fi
# An ERC timeout is set or configured:
# * change ERC timout to controller timeout -5 seconds
# - Longer than the usual 7s default of dedicated raid drives, to allow as much
# ERC time as possible.
ERC_TENTHS=$(expr `cat /sys/block/${HDD_DEV}/device/timeout` \* 10 - 50)
# prevent exceeding max. scterc value
if [ $ERC_TENTHS -gt 999 ] ; then
ERC_TENTHS="999"
fi
echo "maximizing ERC timeout to controller timeout -5 seconds (`expr $ERC_TENTHS / 10`s)"
${TIMING_CMD},$ERC_TENTHS,$ERC_TENTHS /dev/${HDD_DEV} > /dev/null
else # ERC timeout disabled
echo "found ERC timeout disabled, setting NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS (${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS}s)"
echo ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
fi
fi
fi
--------------
#!/bin/sh
# smartctl-timeouts_non-redundant-partition.sh
# Because there may also be redundant partitions on this disk we must not
# unconditionally alter the timeouts.
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
. $SCRIPT_DIR/smartctl-timeouts_defaults
HDD_DEV="$1"
REDUNDANT_DISK="unchecked"
# TODO
#for dev in `cd /sys/block/${HDD_DEV} ; ls -d ${HDD_DEV}*` ; do
# if equvalent to udev's ENV{REDUNDANT_DEV}=="yes|possibly"; then
# $REDUNDANT_DISK="possibly"
# fi
#done
#if [ $REDUNDANT_DISK="unchecked" ] ; then
# REDUNDANT_DISK="false"
#fi
if [ $REDUNDANT_DISK = "false" ] \
# TODO && all partitions have been detected by udev already
then
$SCRIPT_DIR/smartctl-timeouts_non-redundant-disk.sh $1
else
$SCRIPT_DIR/smartctl-timeouts_possibly-redundant-disk.sh $1
fi
-------------
#!/bin/sh
# smartctl-timeouts_redundant-disk.sh
# Redundant timouts are NEVER to be triggerd by udev rules!
# Because only the redundancy providing kernel module knows the actual run-time
# redundancy status, can adjust it and call this script dynamically.
# Udev rules can only determine "possibly redundant" devices.
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
. $SCRIPT_DIR/smartctl-timeouts_defaults
HDD_DEV="$1"
echo "Adjusting $HDD_DEV timeouts:"
if ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q Disabled \
&& ! ${TIMING_CMD} /dev/${HDD_DEV} | grep -q seconds
then
# ERC timeout is not supported (not disabled and not set):
# * Set the controller timeout to be considerably loooooong.
# - To allow the drive to give up its ERC attempts by itself.
# - Let the drive return a proper read error, so that the redundancy
# provider (md, lvm, btrfs, ...) can re-write the bad block.
# - Disk read errors thus result in long i/o blocking periods with
# no error messages that may not be watched by or reported to the user,
# - but waiting this long should prevent unecessary controller resets of the
# entire drive and the corresponding loss of redundancy/data.
echo "Drive without ERC timeout support, setting NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS (${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS}s)"
echo ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
else
SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT="false"
# reset controller timeout, if a configured value was previously set
if [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${NONREDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ] \
|| [ `cat /sys/block/${HDD_DEV}/device/timeout` = ${POSSIBLY_REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:--1} ]
then
SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT="true"
echo "resetting controller from another configured value (`cat /sys/block/${HDD_DEV}/device/timeout`s) to ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-30}s"
echo ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-30} >/sys/block/${HDD_DEV}/device/timeout
else
# set possibly-redundant timeout anyway, if configured
if [ ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ] ; then
echo "setting controller timeout to REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS (${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS}s)"
echo ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS} >/sys/block/${HDD_DEV}/device/timeout
else
if [ `cat /sys/block/${HDD_DEV}/device/timeout` -gt 30 ] ; then
echo "reducing controller timout to 30 seconds"
echo 30 >/sys/block/${HDD_DEV}/device/timeout
fi
fi
fi
if ${TIMING_CMD} /dev/${HDD_DEV} | grep -q Disabled \
|| [ $SWITCH_FROM_OTHER_CONFIGURED_SMARTCTL_TIMEOUT = "true" ] \
|| [ ${REDUNDANT_DISK_CONTROLLER_TIMEOUT_SECONDS:-undefined} != "undefined" ] \
# TODO: || [ $(expr `cat /sys/block/${HDD_DEV}/device/timeout` \* 10 - 50) = read of current ERC timeout value ]
then
# ERC timeout is disabled, is configured, or has been "maximized" to the controller timeout -5 seconds:
# * set it to 7 seconds
# - The usual quick 7s default of dedicated raid drives.
# - Allows redundancy provider to quickly read data from another disk and re-write the bad block
# before the controller resets the entire drive and the raid looses redundancy/data completely.
echo "setting ERC timeout to 7 seconds"
${TIMING_CMD},70,70 /dev/${HDD_DEV} > /dev/null
fi
fi
------
#!/bin/sh
# smartctl-timeouts_redundant-partition.sh
# Redundant timouts are NEVER to be triggerd by udev rules!
# Because only the redundancy providing kernel module knows the actual run-time
# redundancy status, can adjust it and call this script dynamically.
# Udev rules can only determine "possibly redundant" devices.
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
. $SCRIPT_DIR/smartctl-timeouts_defaults
HDD_DEV="$1"
REDUNDANT_DISK="unchecked"
# TODO
#for dev in `cd /sys/block/${HDD_DEV} ; ls -d ${HDD_DEV}*` ; do
# if equvalent to udev's ENV{REDUNDANT_DEV}=="false"; then
# $REDUNDANT_DISK="false"
# fi
#done
#if [ $REDUNDANT_DISK="unchecked" ] ; then
# REDUNDANT_DISK="true"
#fi
if [ $REDUNDANT_DISK = "true" ] \
# TODO: && all partitions have been detected by udev already
then
$SCRIPT_DIR/smartctl-timeouts_redundant-disk.sh $1
else
$SCRIPT_DIR/smartctl-timeouts_possibly-redundant-disk.sh $1
fi
---------
smartctl-timeouts README
The smartctl-timeouts scripts adjust controller and disk timeouts according
to redundancy status, and fix commonly mismatching defaults with drives that
have no error recovery timeout configured, which has often lead to data loss.
The scripts are to be called by udev rules during device initialization,
and by kernel modules acording to the run-time redundancy status changes.
Every redundancy providing block device module may ship with proper udev rules
that initialize the timeouts for their possibly redundant devices.
An alternative to these scripts may be to investigate the FASTFAIL
feature in the kernel.
NOTE: Correct execution during boot requires that distro package managers
hook smartctl and the smartctl-timeouts scripts into the initramfs.
RATIONALE
The error recovery (ERC) timeout *must* be shorter than the controller timeout.
Otherwise read errors will cause controller resets, leading to direct data loss
or, if it is a redundant disk, loss of redundancy and a very high probability
of another read error and data loss when re-establishing the redundancy.
If a drive does not support adjusting its ERC timeout, the controller timeout
must be increased above the drive's maximal error recovery time.
If you don't want that kind of long device timeout, you should look for a drive
with SCT ERC timeout support. (smartctl -l scterc /dev/...)
IMPACT (without having specific timeouts configured)
For possibly redundant disks: If supported but simply disabled in the drive,
the ERC timeout is adjusted to the current controller timeout minus 5 seconds.
The controller timeout is only raised (to NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS)
for drives without SCTERC support. As well as for entirely non-redundant-disks,
in an attempt to allow these drives to finish their error recovery regularily
before a reset is triggerd.
As controller timeouts are only increased selectively (only drives without SCTERC
support and surely non-redundant disks), the scripts only adapt mismatching
timeouts, by default. Existing manufacturer or custom ERC timeout settings (as in
professional, dedicated, redundant setups, e.g. storage servers etc.) won't be
changed, except with specific configuration options.
TODO
* non-redundant-partitions: conditional udev triggering, or a test in the script could
determine if all partions of the disk have been detected already and are all
non-redundant, to call non-redundant-disk in this case.
* parser to read ERC timeout values?
- redundant-disk: a previously set "controller timeout - 5 seconds" ERC timeout
(possibly-redundant), could also be reset to 7 seconds, not just a "Disabled" value.
* If a redundancy controlling kernel module is to make dynamic adjustments,
"redundant-partition" needs implementation.
smartctl-timeouts README
The smartctl-timeouts scripts adjust controller and disk timeouts according
to redundancy status, and fix commonly mismatching defaults with drives that
have no error recovery timeout configured, which has often lead to data loss.
The scripts are to be called by udev rules during device initialization,
and by kernel modules acording to the run-time redundancy status changes.
Every redundancy providing block device module may ship with proper udev rules
that initialize the timeouts for their possibly redundant devices.
An alternative to these scripts may be to investigate the FASTFAIL
feature in the kernel.
NOTE: Correct execution during boot requires that distro package managers
hook smartctl and the smartctl-timeouts scripts into the initramfs.
RATIONALE
The error recovery (ERC) timeout *must* be shorter than the controller timeout.
Otherwise read errors will cause controller resets, leading to direct data loss
or, if it is a redundant disk, loss of redundancy and a very high probability
of another read error and data loss when re-establishing the redundancy.
If a drive does not support adjusting its ERC timeout, the controller timeout
must be increased above the drive's maximal error recovery time.
If you don't want that kind of long device timeout, you should look for a drive
with SCT ERC timeout support. (smartctl -l scterc /dev/...)
IMPACT (without having specific timeouts configured)
For possibly redundant disks: If supported but simply disabled in the drive,
the ERC timeout is adjusted to the current controller timeout minus 5 seconds.
The controller timeout is only raised (to NONREDUNDANT_UNSURE_CONTROLLER_RESET_SECONDS)
for drives without SCTERC support. As well as for entirely non-redundant-disks,
in an attempt to allow these drives to finish their error recovery regularily
before a reset is triggerd.
As controller timeouts are only increased selectively (only drives without SCTERC
support and surely non-redundant disks), the scripts only adapt mismatching
timeouts, by default. Existing manufacturer or custom ERC timeout settings (as in
professional, dedicated, redundant setups, e.g. storage servers etc.) won't be
changed, except with specific configuration options.
TODO
* non-redundant-partitions: conditional udev triggering, or a test in the script could
determine if all partions of the disk have been detected already and are all
non-redundant, to call non-redundant-disk in this case.
* parser to read ERC timeout values?
- redundant-disk: a previously set "controller timeout - 5 seconds" ERC timeout
(possibly-redundant), could also be reset to 7 seconds, not just a "Disabled" value.
* If a redundancy controlling kernel module is to make dynamic adjustments,
"redundant-partition" needs implementation.
[-- Attachment #2: smartctl-timeouts_email2.zip --]
[-- Type: application/zip, Size: 9916 bytes --]
^ permalink raw reply
* Optimal chunk size for RAID5?
From: Christer Solskogen @ 2015-02-22 11:31 UTC (permalink / raw)
To: linux-raid
Hi!
I'm about to create a RAID5 with three 4TB disks (ST4000VN000
from Seagate) and I wonder what the optimal chunk size is/should be.
There are so many different views on the internet that I've almost lost
faith in the internets.
Is there even such a thing as optimal chunk size?
Are there any other stuff I should think about?
--
chs
^ permalink raw reply
* Re: Optimal chunk size for RAID5?
From: Roman Mamedov @ 2015-02-22 12:30 UTC (permalink / raw)
To: Christer Solskogen; +Cc: linux-raid
In-Reply-To: <mccemd$8pi$1@ger.gmane.org>
On Sun, 22 Feb 2015 12:31:23 +0100
Christer Solskogen <christer.solskogen@gmail.com> wrote:
> There are so many different views on the internet
...and yet you're asking for some more? :)
> Is there even such a thing as optimal chunk size?
64K should be fine:
http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
--
With respect,
Roman
^ permalink raw reply
* Re: Optimal chunk size for RAID5?
From: Christer Solskogen @ 2015-02-22 12:46 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20150222173047.1ea65d67@natsu>
On 22.02.2015 13:30, Roman Mamedov wrote:
> On Sun, 22 Feb 2015 12:31:23 +0100
> Christer Solskogen <christer.solskogen@gmail.com> wrote:
>
>> There are so many different views on the internet
>
> ...and yet you're asking for some more? :)
>
Heh, yes. I see the irony in that :-) But this is pretty much as the
source as you can get.
>> Is there even such a thing as optimal chunk size?
>
> 64K should be fine:
> http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
>
Okay, even if this is almost 5 years old, it is still applicable?
The disks I have are 4k sectors. Is there anything special I need to
think about or is this handled automagicly these days? (I run a pretty
fresh distro)
--
chs
^ permalink raw reply
* Re: re-add POLICY
From: Chris @ 2015-02-22 13:23 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <loom.20150217T152416-500@post.gmane.org>
Hello,
I just noticed that I somehow overlooked that md3 and md7 on that old ubuntu
system *did* have a write-intent bitmap.
So in my tests action="spare" does not seem to allow automatic re-sync of
arrays without a bitmap.
To quote the man page again on "spare":
"if the device is bare it can become a spare if there is any array that it
is a candidate for based on domains and metadata."
I am fankly not sure I fully understand that. A bare device has no
superblock, so does mdadm only look which array fits onto the device?
Since the partitions on the removed disk contain superblocks they are not
bare, may that by why action=spare does not apply, and an automatic re-sync
may either require a new action="re-sync" or be done by "re-add" as well?
Regards,
Chris
^ permalink raw reply
* Re: Optimal chunk size for RAID5?
From: Alireza Haghdoost @ 2015-02-22 14:33 UTC (permalink / raw)
To: Roman Mamedov; +Cc: Christer Solskogen, Linux RAID
In-Reply-To: <20150222173047.1ea65d67@natsu>
On Sun, Feb 22, 2015 at 6:30 AM, Roman Mamedov <rm@romanrm.net> wrote:
> On Sun, 22 Feb 2015 12:31:23 +0100
> Christer Solskogen <christer.solskogen@gmail.com> wrote:
>
>> There are so many different views on the internet
>
> ...and yet you're asking for some more? :)
>
>> Is there even such a thing as optimal chunk size?
>
> 64K should be fine:
> http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
>
I have seen that people report 64K chunk size results better
performance. However, I was not able to find why mdadm maintainers
decided to switch into 512K default chunk size a few years ago ? Was
that decision related to the write-intent bitmap overhead ?
^ permalink raw reply
* Metadata > 0.90 and auto-assemble
From: Joshua Kinard @ 2015-02-22 18:54 UTC (permalink / raw)
To: linux-raid
Hi,
I tried a while back to use the newer metadata formats on my mdadm RAID5 on a
few machines, and discovered that the kernel auto-assembly will only work with
v0.90 metadata, not 1.0 or greater. Is there a solid reason for this? Based
one what I can find regarding the differences in the metadata formats, and
looking at the existing md code, it seems this is largely just because no one
has had the time or motivation to change the code to support auto-assembly on
the newer metdata formats.
I am told that the "correct" solution is to embed a small initramfs to bring
the RAID arrays online instead, before the real rootfs is loaded. I'd like to
avoid this if possible, as I haven't had to use an initramfs for normal booting
in the past, as long as I stay on metadata 0.90. So I thought I'd ask what the
official stance is on this.
Thanks!,
--
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28
"The past tempts us, the present confuses us, the future frightens us. And our
lives slip away, moment by moment, lost in that vast, terrible in-between."
--Emperor Turhan, Centauri Republic
^ permalink raw reply
* Re: Metadata > 0.90 and auto-assemble
From: Chris Murphy @ 2015-02-22 21:29 UTC (permalink / raw)
Cc: linux-raid
In-Reply-To: <54EA25D3.8020300@gentoo.org>
On Sun, Feb 22, 2015 at 11:54 AM, Joshua Kinard <kumba@gentoo.org> wrote:
> I'd like to
> avoid this if possible, as I haven't had to use an initramfs for normal booting
> in the past, as long as I stay on metadata 0.90. So I thought I'd ask what the
> official stance is on this.
https://raid.wiki.kernel.org/index.php/Autodetect
Official stance is that it's deprecated, but people still use it.
--
Chris Murphy
^ permalink raw reply
* Re: Optimal chunk size for RAID5?
From: NeilBrown @ 2015-02-22 21:53 UTC (permalink / raw)
To: Alireza Haghdoost; +Cc: Roman Mamedov, Christer Solskogen, Linux RAID
In-Reply-To: <CAB-428=7y-KSbCeXo6y6o5Jfzgf-5h6pfYxY+4Z74bqYp=oMUg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]
On Sun, 22 Feb 2015 08:33:02 -0600 Alireza Haghdoost <alireza@cs.umn.edu>
wrote:
> On Sun, Feb 22, 2015 at 6:30 AM, Roman Mamedov <rm@romanrm.net> wrote:
> > On Sun, 22 Feb 2015 12:31:23 +0100
> > Christer Solskogen <christer.solskogen@gmail.com> wrote:
> >
> >> There are so many different views on the internet
> >
> > ...and yet you're asking for some more? :)
> >
> >> Is there even such a thing as optimal chunk size?
> >
> > 64K should be fine:
> > http://louwrentius.com/linux-raid-level-and-chunk-size-the-benchmarks.html
> >
>
> I have seen that people report 64K chunk size results better
> performance. However, I was not able to find why mdadm maintainers
> decided to switch into 512K default chunk size a few years ago ? Was
> that decision related to the write-intent bitmap overhead ?
No, write-intent-bitmap sizing is completely independent from chunk sizes.
I don't remember the detail for the change, but some measurement must have
gone faster with larger chunk size.
single threaded loads tend to prefer large chunk sizes.
multi-threaded small-request random IO tends to prefer smaller chunk sizes.
There is no "Optimal" without reference to a particular work load. Or
particular hardware.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox