* Re: Bad sequential performance of RAID5 with a lot of disk seeks
From: Robin Hill @ 2014-10-07 9:25 UTC (permalink / raw)
To: P. Gautschi; +Cc: linux-raid
In-Reply-To: <20141007095422.Horde.3l6BckPhfY5U8dchaOdLNA1@webmail.gautschi.net>
[-- Attachment #1: Type: text/plain, Size: 3074 bytes --]
On Tue Oct 07, 2014 at 09:54:22AM +0200, P. Gautschi wrote:
> Quoting Robin Hill <robin@robinhill.me.uk>:
> > On Tue Oct 07, 2014 at 06:44:49AM +0200, P. Gautschi wrote:
> >
> >> I've created a RAID5 on 5 identical SATA disks. Doing some
> >> performance measurements
> >> with dd I get a disappointing performance.
> >> A dd with bs=1M on a btrfs created on md0 transfers about 110 MB/s.
> >> (both read and write)
> >> A dd on md0 has the same write speed but only about 20 MB/s on read.
> >> In all of the tests I hear the disk constantly seeking. This was
> >> also the case
> >> during creation of the array.
> >> I also created a RAID4 to make sure that I doesn't get fooled by
> >> the stripe layout of RAID5.
> >> Now I get about 110 MB/s for write and 230 MB/s for read on md0.
> >> But the constant
> >> seeking is still present for both read and write and during
> >> creation of the array.
> >>
> >> Why are the disk perform so many seek operations? I think a
> >> sequential access on md0 should
> >> cause a sequential access on the individual disk.
> >>
> >> I have to add that I did something unusual: I created the RAID4/5
> >> with a chunk size of 4KiB.
> >> The idea of this was that when I'm going to use btrfs with the
> >> default nodesize of 16KiB
> >> all node write will fill a full stripe and there won't be any RMW
> >> at all. (both fortunate
> >> for performance and integrity in a power loss situation.)
> >> Nevertheless I think a sequential access on the array should cause
> >> a sequential access on the
> >> disks for any chunk size if the read/write block size is a exact multiple of
> >> the (numdisks-1)*chunk size.
> >>
> >> Is there any explanation for the seeks and how do I get rid of them?
> >>
> > After creating the arrays did you wait for them to finish syncing? The
> > array is created in degraded mode initially and then rebuilds onto the
> > additional disk (this is the fastest way to do things, unless you know
> > the disks are all zeroed initially). Until this rebuild is complete then
> > it'll be competing with any other disk activity.
> >
> Yes I did wait for the syncing to complete before accessing md0.
> I had the seeks during the syncing and afterwards when reading or
> writing to the array.
>
Hmm, shouldn't be seeking then.
What does the SMART info show for the drives - are there any reallocated
blocks? A large number of those scattered over the disk would certainly
cause seeking for both reads and writes.
It's also worth checking whether there's anything else that would be
accessing the disks in the background (monitoring/indexing/etc).
I can't think of anything else that would be causing reads to seek - SMR
disks or write-intent bitmaps would only affect writes.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: Bad sequential performance of RAID5 with a lot of disk seeks
From: P. Gautschi @ 2014-10-07 7:54 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20141007074347.GA18786@cthulhu.home.robinhill.me.uk>
Yes I did wait for the syncing to complete before accessing md0.
I had the seeks during the syncing and afterwards when reading or
writing to the array.
Patrick
Quoting Robin Hill <robin@robinhill.me.uk>:
> On Tue Oct 07, 2014 at 06:44:49AM +0200, P. Gautschi wrote:
>
>> I've created a RAID5 on 5 identical SATA disks. Doing some
>> performance measurements
>> with dd I get a disappointing performance.
>> A dd with bs=1M on a btrfs created on md0 transfers about 110 MB/s.
>> (both read and write)
>> A dd on md0 has the same write speed but only about 20 MB/s on read.
>> In all of the tests I hear the disk constantly seeking. This was
>> also the case
>> during creation of the array.
>> I also created a RAID4 to make sure that I doesn't get fooled by
>> the stripe layout of RAID5.
>> Now I get about 110 MB/s for write and 230 MB/s for read on md0.
>> But the constant
>> seeking is still present for both read and write and during
>> creation of the array.
>>
>> Why are the disk perform so many seek operations? I think a
>> sequential access on md0 should
>> cause a sequential access on the individual disk.
>>
>> I have to add that I did something unusual: I created the RAID4/5
>> with a chunk size of 4KiB.
>> The idea of this was that when I'm going to use btrfs with the
>> default nodesize of 16KiB
>> all node write will fill a full stripe and there won't be any RMW
>> at all. (both fortunate
>> for performance and integrity in a power loss situation.)
>> Nevertheless I think a sequential access on the array should cause
>> a sequential access on the
>> disks for any chunk size if the read/write block size is a exact multiple of
>> the (numdisks-1)*chunk size.
>>
>> Is there any explanation for the seeks and how do I get rid of them?
>>
> After creating the arrays did you wait for them to finish syncing? The
> array is created in degraded mode initially and then rebuilds onto the
> additional disk (this is the fastest way to do things, unless you know
> the disks are all zeroed initially). Until this rebuild is complete then
> it'll be competing with any other disk activity.
>
> Cheers,
> Robin
> --
> ___
> ( ' } | Robin Hill <robin@robinhill.me.uk> |
> / / ) | Little Jim says .... |
> // !! | "He fallen in de water !!" |
^ permalink raw reply
* Re: Bad sequential performance of RAID5 with a lot of disk seeks
From: Robin Hill @ 2014-10-07 7:43 UTC (permalink / raw)
To: P. Gautschi; +Cc: linux-raid
In-Reply-To: <54336FC1.6080306@gautschi.net>
[-- Attachment #1: Type: text/plain, Size: 2116 bytes --]
On Tue Oct 07, 2014 at 06:44:49AM +0200, P. Gautschi wrote:
> I've created a RAID5 on 5 identical SATA disks. Doing some performance measurements
> with dd I get a disappointing performance.
> A dd with bs=1M on a btrfs created on md0 transfers about 110 MB/s. (both read and write)
> A dd on md0 has the same write speed but only about 20 MB/s on read.
> In all of the tests I hear the disk constantly seeking. This was also the case
> during creation of the array.
> I also created a RAID4 to make sure that I doesn't get fooled by the stripe layout of RAID5.
> Now I get about 110 MB/s for write and 230 MB/s for read on md0. But the constant
> seeking is still present for both read and write and during creation of the array.
>
> Why are the disk perform so many seek operations? I think a sequential access on md0 should
> cause a sequential access on the individual disk.
>
> I have to add that I did something unusual: I created the RAID4/5 with a chunk size of 4KiB.
> The idea of this was that when I'm going to use btrfs with the default nodesize of 16KiB
> all node write will fill a full stripe and there won't be any RMW at all. (both fortunate
> for performance and integrity in a power loss situation.)
> Nevertheless I think a sequential access on the array should cause a sequential access on the
> disks for any chunk size if the read/write block size is a exact multiple of
> the (numdisks-1)*chunk size.
>
> Is there any explanation for the seeks and how do I get rid of them?
>
After creating the arrays did you wait for them to finish syncing? The
array is created in degraded mode initially and then rebuilds onto the
additional disk (this is the fastest way to do things, unless you know
the disks are all zeroed initially). Until this rebuild is complete then
it'll be competing with any other disk activity.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: auto assemble in read-only mode?
From: NeilBrown @ 2014-10-07 7:15 UTC (permalink / raw)
To: Errol Neal; +Cc: Linux-RAID
In-Reply-To: <1412664991898536500@businessgrade.com>
[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]
On Tue, 7 Oct 2014 02:56:31 -0400 Errol Neal <eneal@businessgrade.com> wrote:
> On Mon, 10/06/2014 09:06 AM, Errol Neal <eneal@businessgrade.com> wrote:
> > hi folks. is it possible to assemble an md device in read-only mode?
> > after an array has been assembled, I know I can mark it as readonly with --misc -o /dev/md0 for example, but only after its been assembled it seems..
> >
>
> Hi again. No thoughts here?
Not even 18 hours since you last asked - what do you thing this is? The
Internet :-)
> I'm need to run a RAID1 device on two independent nodes (shared storage) and I'd like to assemble the device on the "slave" node in read-only mode to prevent any sync actions and etc.
Did you try adding the "--readonly" option to your --assemble command?
(Always best to report everything you tried and how it failed - requires less
guess work).
I hope you don't expect too much of a filesystem mounted from such an array
while the other node is writing to the array...
NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: /sys/block/md126 still exists even after stopping the array
From: Francis Moreau @ 2014-10-07 7:05 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, sebastian.riemer
In-Reply-To: <542A5F15.7030100@gmail.com>
Hi Neil,
On 09/30/2014 09:43 AM, Francis Moreau wrote:
> Hi Neil,
>
> On 09/29/2014 11:56 PM, NeilBrown wrote:
>> On Mon, 29 Sep 2014 10:45:17 +0200 Francis Moreau <francis.moro@gmail.com>
>> wrote:
>>
>>>> So what were pids 930 and 459?
>>>> One was presumably the "mdadm -Ss" - probably 930.
>>>> Is 459 the "mdadm --monitor" ?? That might be useful hint.
>>>>
>>>
>>> yes.
>>>
>>> [456] is: /sbin/mdadm --monitor --scan --daemonise --syslog
>>> --pid-file=/run/mdadm/mdadm.pid
>>>
>>> and [930] is 'mdamd -Ss'.
>>
>> Good. Please try the patch below.
>>
>
> After applying your patch, this is what I'm getting in syslog:
>
> Sep 30 03:40:07 localhost kernel: md_open(): md125 opened by mdadm [970]
> Sep 30 03:40:07 localhost kernel: md_release(): md125 released by mdadm
> [970]
> Sep 30 03:40:07 localhost kernel: md_open(): md125 opened by mdadm [972]
> Sep 30 03:40:07 localhost kernel: md_open(): md125 opened by mdadm [970]
> Sep 30 03:40:07 localhost kernel: md_release(): md125 released by mdadm
> [972]
> Sep 30 03:40:07 localhost kernel: md_open(): md125 opened by
> systemd-udevd [971]
> Sep 30 03:40:07 localhost systemd[1]: Cannot add dependency job for unit
> mdmonitor-takeover.service, ignoring: Invalid argument
> Sep 30 03:40:07 localhost systemd[1]: Started Software RAID monitoring
> and management.
> Sep 30 03:40:07 localhost kernel: md_release(): md125 released by
> systemd-udevd [971]
> Sep 30 03:40:08 localhost mdadm[466]: DeviceDisappeared event detected
> on md device /dev/md125
> Sep 30 03:40:08 localhost mdadm[466]: DeviceDisappeared event detected
> on md device /dev/md126
> Sep 30 03:40:08 localhost mdadm[466]: DeviceDisappeared event detected
> on md device /dev/md127
> Sep 30 03:40:08 localhost kernel: md125: detected capacity change from
> 1863254016 to 0
> Sep 30 03:40:08 localhost kernel: md: md125 stopped.
> Sep 30 03:40:08 localhost kernel: md: unbind<vdc3>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdc3)
> Sep 30 03:40:08 localhost kernel: md: unbind<vdb3>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdb3)
> Sep 30 03:40:08 localhost kernel: md_release(): md125 released by mdadm
> [970]
> Sep 30 03:40:08 localhost kernel: md_open(): md127 opened by mdadm [466]
> Sep 30 03:40:08 localhost kernel: md_release(): md127 released by mdadm
> [466]
> Sep 30 03:40:08 localhost kernel: md_open(): md126 opened by mdadm [466]
> Sep 30 03:40:08 localhost kernel: md_release(): md126 released by mdadm
> [466]
> Sep 30 03:40:08 localhost kernel: md_open(): md126 opened by mdadm [970]
> Sep 30 03:40:08 localhost kernel: md_release(): md126 released by mdadm
> [970]
> Sep 30 03:40:08 localhost kernel: md_open(): md126 opened by mdadm [970]
> Sep 30 03:40:08 localhost kernel: md126: detected capacity change from
> 67043328 to 0
> Sep 30 03:40:08 localhost kernel: md: md126 stopped.
> Sep 30 03:40:08 localhost kernel: md: unbind<vdc1>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdc1)
> Sep 30 03:40:08 localhost kernel: md: unbind<vdb1>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdb1)
> Sep 30 03:40:08 localhost kernel: md_open(): md127 opened by mdadm [466]
> Sep 30 03:40:08 localhost kernel: md_release(): md127 released by mdadm
> [466]
> Sep 30 03:40:08 localhost kernel: md_release(): md126 released by mdadm
> [970]
> Sep 30 03:40:08 localhost kernel: md_open(): md127 opened by mdadm [970]
> Sep 30 03:40:08 localhost kernel: md_release(): md127 released by mdadm
> [970]
> Sep 30 03:40:08 localhost kernel: md_open(): md127 opened by mdadm [970]
> Sep 30 03:40:08 localhost kernel: md127: detected capacity change from
> 214564864 to 0
> Sep 30 03:40:08 localhost kernel: md: md127 stopped.
> Sep 30 03:40:08 localhost kernel: md: unbind<vdc2>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdc2)
> Sep 30 03:40:08 localhost kernel: md: unbind<vdb2>
> Sep 30 03:40:08 localhost kernel: md: export_rdev(vdb2)
> Sep 30 03:40:08 localhost kernel: md_release(): md127 released by mdadm
> [970]
>
> The ghost device is no more present so your patch seems to have fixed my
> issue. But I must admit I don't really understand what's going on :-/
>
Since those 'ghost' devices are expected from the MD implementation
point of view, I'm wondering how am I supposed to detect them or maybe
how an application is supposed to recognized online arrays.
My application uses udev to detect et to get information about new
devices. I don't think the information exported by udev is enough to
figure this out. Also please note that since I rely on udev, I can't
really read information on /sys since this information may be out of
sync with the one returned by udev.
Thanks.
^ permalink raw reply
* Re: auto assemble in read-only mode?
From: Errol Neal @ 2014-10-07 6:56 UTC (permalink / raw)
To: eneal; +Cc: Linux-RAID
In-Reply-To: <1412600795583058500@businessgrade.com>
On Mon, 10/06/2014 09:06 AM, Errol Neal <eneal@businessgrade.com> wrote:
> hi folks. is it possible to assemble an md device in read-only mode?
> after an array has been assembled, I know I can mark it as readonly with --misc -o /dev/md0 for example, but only after its been assembled it seems..
>
Hi again. No thoughts here?
I'm need to run a RAID1 device on two independent nodes (shared storage) and I'd like to assemble the device on the "slave" node in read-only mode to prevent any sync actions and etc.
^ permalink raw reply
* Bad sequential performance of RAID5 with a lot of disk seeks
From: P. Gautschi @ 2014-10-07 4:44 UTC (permalink / raw)
To: linux-raid
I've created a RAID5 on 5 identical SATA disks. Doing some performance measurements
with dd I get a disappointing performance.
A dd with bs=1M on a btrfs created on md0 transfers about 110 MB/s. (both read and write)
A dd on md0 has the same write speed but only about 20 MB/s on read.
In all of the tests I hear the disk constantly seeking. This was also the case
during creation of the array.
I also created a RAID4 to make sure that I doesn't get fooled by the stripe layout of RAID5.
Now I get about 110 MB/s for write and 230 MB/s for read on md0. But the constant
seeking is still present for both read and write and during creation of the array.
Why are the disk perform so many seek operations? I think a sequential access on md0 should
cause a sequential access on the individual disk.
I have to add that I did something unusual: I created the RAID4/5 with a chunk size of 4KiB.
The idea of this was that when I'm going to use btrfs with the default nodesize of 16KiB
all node write will fill a full stripe and there won't be any RMW at all. (both fortunate
for performance and integrity in a power loss situation.)
Nevertheless I think a sequential access on the array should cause a sequential access on the
disks for any chunk size if the read/write block size is a exact multiple of
the (numdisks-1)*chunk size.
Is there any explanation for the seeks and how do I get rid of them?
Patrick
^ permalink raw reply
* Re: Cancelling not pausing array check
From: Patrik Horník @ 2014-10-06 22:57 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20141007075944.30be70ae@notabene.brown>
2014-10-06 22:59 GMT+02:00 NeilBrown <neilb@suse.de>:
> On Mon, 6 Oct 2014 15:29:13 +0200 Patrik Horník <patrik@dsl.sk> wrote:
>
>> Hello,
>>
>> is it possible to cancel RAID 6 array check? echo check >
>> /sys/block/mdX/md/sync_action apparently only pauses it and next check
>> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
>>
>> I want to achieve two things:
>>
>> - Start check from the start of the array.
>>
>> - Be sure I can shutdown array without any ill effect because check
>> was interrupted.
>>
>> I guess that position will be cleared when array is stopped and
>> started again, but I dont want to guess and experiment... :)
>
> Why ever not? Guessing and experimenting is AWESOME. We call it "Science".
> I can understand that you might not want to experiment we real data, but
> creating a few loop-back devices with "losetup", forming them into an array,
> and experimenting on that can be very worthwhile.
Yeah, I fully agree. I love guessing and experimenting. In this case I
will be guessing if you write it as it should be written and we will
then test it on your production data :) Do we have a deal?
Regarding loopback devices, I of course sometimes test it. If I am in
some potential trouble I usually go further and testing it on cow dm
snapshots over drives, because you never know if it will work on
devices in your state exactly same as on just created small loopback
devices. But maybe I cannot think about all the factors and the most
effective way to know is to ask.
Patrik
>
> NeilBrown
>
>
>>
>> Thanks.
>>
>> Best regards,
>>
>> Patrik
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Cancelling not pausing array check
From: NeilBrown @ 2014-10-06 20:59 UTC (permalink / raw)
To: Patrik Horník; +Cc: linux-raid
In-Reply-To: <CAAOsTSnYjW-r7VoOL0SiNhsdv5kn9DunrfD3izuLDzaHaBb56Q@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]
On Mon, 6 Oct 2014 15:29:13 +0200 Patrik Horník <patrik@dsl.sk> wrote:
> Hello,
>
> is it possible to cancel RAID 6 array check? echo check >
> /sys/block/mdX/md/sync_action apparently only pauses it and next check
> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
>
> I want to achieve two things:
>
> - Start check from the start of the array.
>
> - Be sure I can shutdown array without any ill effect because check
> was interrupted.
>
> I guess that position will be cleared when array is stopped and
> started again, but I dont want to guess and experiment... :)
Why ever not? Guessing and experimenting is AWESOME. We call it "Science".
I can understand that you might not want to experiment we real data, but
creating a few loop-back devices with "losetup", forming them into an array,
and experimenting on that can be very worthwhile.
NeilBrown
>
> Thanks.
>
> Best regards,
>
> Patrik
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: Cancelling not pausing array check
From: Patrik Horník @ 2014-10-06 17:41 UTC (permalink / raw)
To: Patrik Horník, Neil Brown, linux-raid
In-Reply-To: <20141006135314.GB1928@cthulhu.home.robinhill.me.uk>
2014-10-06 15:53 GMT+02:00 Robin Hill <robin@robinhill.me.uk>:
> On Mon Oct 06, 2014 at 03:39:12PM +0200, Patrik Horník wrote:
>
>> 2014-10-06 15:36 GMT+02:00 Robin Hill <robin@robinhill.me.uk>:
>> > On Mon Oct 06, 2014 at 03:29:13PM +0200, Patrik Horník wrote:
>> >
>> >> Hello,
>> >>
>> >> is it possible to cancel RAID 6 array check? echo check >
>> >> /sys/block/mdX/md/sync_action apparently only pauses it and next check
>> >> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
>> >>
>> >> I want to achieve two things:
>> >>
>> >> - Start check from the start of the array.
>> >>
>> >> - Be sure I can shutdown array without any ill effect because check
>> >> was interrupted.
>> >>
>> >> I guess that position will be cleared when array is stopped and
>> >> started again, but I dont want to guess and experiment... :)
>> >>
>> >
>> > You'd echo idle to the sync_action to pause/stop the check. If you want
>> > to restart from the beginning next time, just echo 0 to sync_min
>> > (similarly, if you just want to check/repair a specific range then set
>> > sync_min and sync_max before the check/repair).
>> >
>> OK, thanks. So that should be fully working and correct way to do it
>> and I cant harm my array by this in any way?
>>
> A check is always safe (well, excepting physical disk issues or issues
> with overloading the I/O subsystem, but those are issues not limited to
> check) as it's read-only.
>
> The sync_min and sync_max values need to be multiples of the chunk size
> (it throws an error otherwise), which ensures a whole stripe is being
> checked/repaired (otherwise you could have issues).
>
> The kernel documentation describes setting sync_min/sync_max, so it's an
> expected operation. Whether there's another way of doing the same thing
> I don't know - I've not heard of one though.
If that is recommended way in kernel documentation, that is of course
all I wanted :)
BTW sync_min is reset to 0 also when you stop and start the array as I
found out...
>
> Cheers,
> Robin
> --
> ___
> ( ' } | Robin Hill <robin@robinhill.me.uk> |
> / / ) | Little Jim says .... |
> // !! | "He fallen in de water !!" |
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: UEFI and mdadm questions.
From: Phil Turmel @ 2014-10-06 15:58 UTC (permalink / raw)
To: Chris Murphy, Linux-RAID Raid
In-Reply-To: <F14F7054-A1A1-4381-ADC8-B76952338589@colorremedies.com>
On 10/05/2014 04:22 PM, Chris Murphy wrote:
>
> On Oct 5, 2014, at 2:18 PM, Phil Turmel <philip@turmel.org> wrote:
[trim /]
>> If your BIOS can be configured to try multiple boot images, it
>> should be possible to have true raid fallback without using
>> motherboard or hardware raid. (Set up md raid1 with metadata v1.0
>> of multiple copies of the EFI FAT partition.) I've been meaning to
>> try this….
>
> Problems with this: a.) new Windows 8 hardware might require you boot
> Windows to get to the feature enabling the firmware setup, because on
> such hardware USB isn't initialized by default.
> http://mjg59.dreamwidth.org/24869.html
>
> I don't know why we don't have free software to initiate this, but I
> haven't come across it so far.
Good to know, but totally immaterial to the boot sequence I'm
recommending. Boot linux of of an EFI FAT and let linux initialize the
USB hardware in its own good time.
Matthew Garrett's post is really all about how to get linux into the
Win8 box in the first place. Once there, manipulate the boot sequence
as you please.
> b.) There's no guarantee the firmware won't write to the ESP, thus
> rendering the individual md raid members out of sync and without
> their metadata being updated, i.e. in effect, the logical device they
> become later, is corrupt. Separately they aren't corrupt, merely out
> of sync, but you don't have an obvious way of knowing which one.
This is a very good point. In fact, I withdraw my recommendation to
raid these partitions. Simply have one on every disk the BIOS could
possibly boot from, and place an EFI bootable kernel in each one (with
embedded initramfs).
> c.) strictly speaking any partition with mdadm metadata should have
> the linux raid partition type GUID set; not the EFI System partition
> type GUID. Those GUIDs are mutually exclusive.
The former is not true at all--mdadm does not care *at all* what
partition types are set. Grub might care, but it's moot if you don't
use Grub. :-)
> This is why I'm still not a fan of using mdadm to raid1 an EFI System
> partition.
One further point: the failure decision tree is nicer if you boot
directly into a kernel.
1) Bios locates and attempts to boot from 1st configured kernel image
2a) Corrupted image or other disk error blocks complete load of kernel
image--bios moves to next EFI choice (possibly on a different disk).
2b) Successful EFI kernel load, boot encounters missing/corrupt root
FS--kernel drops to initramfs rescue shell
versus:
1) Bios locates and attempts to boot from 1st configured grub image
2a) Corrupted image or other disk error blocks complete load of
grub--bios moves to next EFI choice (possibly on a different disk).
2b) Successful EFI grub load, grub encounters corrupt config or grub
module--drop to grub shell
2c) Successful EFI grub load, kernel & initramfs load by grub, boot
encounters missing/corrupt root FS--kernel drops to initramfs rescue shell
I haven't had time to set it up yet, but the clear reduction in points
of failure is compelling. Faster boot is just icing on the cake.
Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Cancelling not pausing array check
From: Robin Hill @ 2014-10-06 13:53 UTC (permalink / raw)
To: Patrik Horník; +Cc: Neil Brown, linux-raid
In-Reply-To: <CAAOsTSny1MUvQktS=nSLLGdvdWd3Ud4M7pPxgWKvHCamem8V4Q@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2009 bytes --]
On Mon Oct 06, 2014 at 03:39:12PM +0200, Patrik Horník wrote:
> 2014-10-06 15:36 GMT+02:00 Robin Hill <robin@robinhill.me.uk>:
> > On Mon Oct 06, 2014 at 03:29:13PM +0200, Patrik Horník wrote:
> >
> >> Hello,
> >>
> >> is it possible to cancel RAID 6 array check? echo check >
> >> /sys/block/mdX/md/sync_action apparently only pauses it and next check
> >> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
> >>
> >> I want to achieve two things:
> >>
> >> - Start check from the start of the array.
> >>
> >> - Be sure I can shutdown array without any ill effect because check
> >> was interrupted.
> >>
> >> I guess that position will be cleared when array is stopped and
> >> started again, but I dont want to guess and experiment... :)
> >>
> >
> > You'd echo idle to the sync_action to pause/stop the check. If you want
> > to restart from the beginning next time, just echo 0 to sync_min
> > (similarly, if you just want to check/repair a specific range then set
> > sync_min and sync_max before the check/repair).
> >
> OK, thanks. So that should be fully working and correct way to do it
> and I cant harm my array by this in any way?
>
A check is always safe (well, excepting physical disk issues or issues
with overloading the I/O subsystem, but those are issues not limited to
check) as it's read-only.
The sync_min and sync_max values need to be multiples of the chunk size
(it throws an error otherwise), which ensures a whole stripe is being
checked/repaired (otherwise you could have issues).
The kernel documentation describes setting sync_min/sync_max, so it's an
expected operation. Whether there's another way of doing the same thing
I don't know - I've not heard of one though.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: Cancelling not pausing array check
From: Patrik Horník @ 2014-10-06 13:39 UTC (permalink / raw)
To: Patrik Horník, Neil Brown, linux-raid
In-Reply-To: <20141006133641.GA1928@cthulhu.home.robinhill.me.uk>
OK, thanks. So that should be fully working and correct way to do it
and I cant harm my array by this in any way?
2014-10-06 15:36 GMT+02:00 Robin Hill <robin@robinhill.me.uk>:
> On Mon Oct 06, 2014 at 03:29:13PM +0200, Patrik Horník wrote:
>
>> Hello,
>>
>> is it possible to cancel RAID 6 array check? echo check >
>> /sys/block/mdX/md/sync_action apparently only pauses it and next check
>> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
>>
>> I want to achieve two things:
>>
>> - Start check from the start of the array.
>>
>> - Be sure I can shutdown array without any ill effect because check
>> was interrupted.
>>
>> I guess that position will be cleared when array is stopped and
>> started again, but I dont want to guess and experiment... :)
>>
>> Thanks.
>>
>> Best regards,
>>
>> Patrik
>
> You'd echo idle to the sync_action to pause/stop the check. If you want
> to restart from the beginning next time, just echo 0 to sync_min
> (similarly, if you just want to check/repair a specific range then set
> sync_min and sync_max before the check/repair).
>
> Cheers,
> Robin
> position.
>
> --
> ___
> ( ' } | Robin Hill <robin@robinhill.me.uk> |
> / / ) | Little Jim says .... |
> // !! | "He fallen in de water !!" |
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Cancelling not pausing array check
From: Robin Hill @ 2014-10-06 13:36 UTC (permalink / raw)
To: Patrik Horník; +Cc: Neil Brown, linux-raid
In-Reply-To: <CAAOsTSnYjW-r7VoOL0SiNhsdv5kn9DunrfD3izuLDzaHaBb56Q@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1174 bytes --]
On Mon Oct 06, 2014 at 03:29:13PM +0200, Patrik Horník wrote:
> Hello,
>
> is it possible to cancel RAID 6 array check? echo check >
> /sys/block/mdX/md/sync_action apparently only pauses it and next check
> continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
>
> I want to achieve two things:
>
> - Start check from the start of the array.
>
> - Be sure I can shutdown array without any ill effect because check
> was interrupted.
>
> I guess that position will be cleared when array is stopped and
> started again, but I dont want to guess and experiment... :)
>
> Thanks.
>
> Best regards,
>
> Patrik
You'd echo idle to the sync_action to pause/stop the check. If you want
to restart from the beginning next time, just echo 0 to sync_min
(similarly, if you just want to check/repair a specific range then set
sync_min and sync_max before the check/repair).
Cheers,
Robin
position.
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: Hot replacing device in RAID 6
From: Patrik Horník @ 2014-10-06 13:30 UTC (permalink / raw)
To: Keith Keller; +Cc: linux-raid
In-Reply-To: <gaf8gbxi4k.ln2@goaway.wombat.san-francisco.ca.us>
Thanks for assistance, it seems that everything worked fine. But I
need yet to check the array to be sure.
2014-10-05 20:30 GMT+02:00 Keith Keller <kkeller@wombat.san-francisco.ca.us>:
> On 2014-10-05, Patrik Horník <patrik@dsl.sk> wrote:
>>
>> - We discussed couple of years ago hot replacing the drive. I see it
>> is now supported in mdadm. Is it rock stable and can be used on
>> production data?
>
> I have used replace a handful of times with great success and no
> problems. I have not had to shut down the array during the rebuild
> operation, however. I suspect (but have no direct evidence) that a
> clean shutdown should be fine. I used kernel 3.9 from ELRepo for
> the replace.
>
> --keith
>
>
> --
> kkeller@wombat.san-francisco.ca.us
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Cancelling not pausing array check
From: Patrik Horník @ 2014-10-06 13:29 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid
Hello,
is it possible to cancel RAID 6 array check? echo check >
/sys/block/mdX/md/sync_action apparently only pauses it and next check
continuing from saved position. (I guess /sys/block/mdX/md/sync_min)
I want to achieve two things:
- Start check from the start of the array.
- Be sure I can shutdown array without any ill effect because check
was interrupted.
I guess that position will be cleared when array is stopped and
started again, but I dont want to guess and experiment... :)
Thanks.
Best regards,
Patrik
^ permalink raw reply
* auto assemble in read-only mode?
From: Errol Neal @ 2014-10-06 13:06 UTC (permalink / raw)
To: Linux-RAID
hi folks. is it possible to assemble an md device in read-only mode?
after an array has been assembled, I know I can mark it as readonly with --misc -o /dev/md0 for example, but only after its been assembled it seems..
tia,
^ permalink raw reply
* Re: UEFI and mdadm questions.
From: Francis Moreau @ 2014-10-06 8:43 UTC (permalink / raw)
To: Chris Murphy, Linux-RAID
In-Reply-To: <E82F134B-634B-46B9-A835-846CA89728FB@colorremedies.com>
Hello Chris,
On 10/03/2014 07:04 AM, Chris Murphy wrote:
>
> On Oct 1, 2014, at 12:33 PM, "Wilson, Jonathan" <piercing_male@hotmail.com> wrote:
>
>> From what I can tell with UEFI I need to set up a UEFI partition with a
>> FAT format.
>
> It's a particular kind of FAT, that's defined as EFI FAT, the idea being that if the originating FAT ever changes, EFI FAT won't.
>
>>
>> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
>> Raid 1 40G.
>>
>> Obviously Grub installs to the mbr, and then installs a bit into
>> Biosboot which can read raids, hence it can read and boot from /boot.
>
> BIOSboot applies to GPT disks on BIOS computers, not MBR. On MBR disks, the GRUB stage1 code jumps to stage2 code in the MBR gap which is the region between the MBR and the first partition's starting LBA.
>
>>
>> Further, from what I can tell, into the UEFI partition can go either a
>> kernel & initramfs with UEFI support, or a "loader" that then loads the
>> kernel.
>
> No. An OSLoader is required, it's an EFI application. Its job is to load a kernel and initramfs. The kernel and initramfs could be on the ESP (EFI System partition) but this is fraught with limitations. The expectation is that the kernel and initramfs are on some other partition of the same disk. Of course if you're using GRUB it doesn't care and will find a kernel/initramfs off another disk also, or even off md raid.
>
I think the kernel can be compiled as an EFI application, therefore
OSLoader is not always required. The advantage of this is that you can
get rid of grub entirely (cool !). But the downside is that you can't
customize/edit the kernel command line at boot time.
> In the case of GRUB, it directly understands pretty much every filesystem used on linux, so it can read kernel+initramfs from ext4, or ext4 on mdraid, or on LVM, or on Btrfs (including multiple devices).
>
>>
>> What I am unsure about are...
>>
>> 1) Can the loader/kernel understand md raid? so the / can be in a bog
>> standard raid1 v1.2?
>
> If the EFI OSLoader is GRUB2, then yes.
>
>
>>
>> 2) I'm guessing I would no longer need the /boot as that would be
>> replaced by what ever was in the UEFI partition?
>
> The ESP isn't a replacement for /boot. But you can have a unified /boot and / on a single file system, and GRUB2 can boot from it, including if that volume is on an md device. GRUB2 even boots off md raid6 (it's sorta crazy and badass at the same time, but in my testing it does work with the BIOS being the limiting factor as to how many drives are recognized at this stage).
>
>
>
>>
>> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
>> changes occur they are replicated to the drives I have set up as /boot
>> raid 1, and I installed the mbr portion of boot loader manually on each
>> disk and tested pulling one, and then booting from another.. it
>> worked :-)
>> So I would like to keep things, if not simple, at least less likely to
>> have problems because I forgot to install duplicates of the UEFI on all
>> the disks… so can I use a .90v raid1 on the UEFI partition,
>
> That solution isn't that simple because it requires too many things that aren't supported by any distribution. And also it increases the chances of corruption because of course the EFI firmware only sees these partition/volumes as single devices, not as md devices that need to be assembled first into a logical device. Some EFI firmwares do reportedly write to the ESP, so if they write to one and not the other, while also not changing the metadata on either partition, there's no way to know which one is valid now. So that's just a mess. And even if your firmware doesn't write to the ESP ever, what about dual boot? That too would be proscribed because the ESP is shared among multiple OS's. So the idea is just way off the rails of anything standardized or widely supported. Will it work? Yes. But do you want to support something this non-standard?
>
> An alternative is a single static grub.cfg on the EFI System partition that points to the real grub.cfg on an mdraid1 / or /boot. Since the ESP's are identical, and never need updating again, this is much safer. You can also dispense with this ridiculously bad idea most all linux distros have right now of persistently mounting the ESP at /boot/efi (and persistently mount it rw as well! really bad!).
>
> The way to do this is with the grub configfile command. The most straightforward way to do it is let grub-mkconfig create a grub.cfg for you, and then copy past the parts that tell GRUB how to find /. For example it needs to know the mduuid so it knows there's an mdraid volume to assembled, and also the volume uuid so it knows what filesystem volume it should find the file one, and then "configfile /boot/grub/grub.cfg" and it'll load that grub.cfg. And it's that grub.cfg that your system should update - of course you have to figure out how your system knows to update the grub.cfg so that it does it correctly without stepping on your ESP grub.cfg (for one comment out the /boot/efi line in fstab and then unmount the ESP). This is distribution specific.
>
> Either way it's a bit complicated to figure out initially.
>
After thinking about it for a while, I'm not sure about the point of
having ESP mirrored or at least I'm not sure it really worths the
trouble: ESP and /boot are very static partitions and should be written
mostly during kernel upgrade. So even if something bad happens in those
partitions, the system should be able to run without any issues.
The annoying part is that if you decide/have to reboot and ESP is
damaged you won't be able to boot your system again. But that said if
you created initially a copy of /boot on each disk, you should be able
to boot again even though those copies are not uptodate.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: UEFI and mdadm questions.
From: Francis Moreau @ 2014-10-06 6:14 UTC (permalink / raw)
To: Wilson, Jonathan; +Cc: Linux-RAID
In-Reply-To: <BLU436-SMTP10330CBEBC0CA1E27853BE498B80@phx.gbl>
Hello,
On 10/01/2014 06:33 PM, Wilson, Jonathan wrote:
> From what I can tell with UEFI I need to set up a UEFI partition with a
> FAT format.
>
> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
> Raid 1 40G.
>
> Obviously Grub installs to the mbr, and then installs a bit into
> Biosboot which can read raids, hence it can read and boot from /boot.
>
> Further, from what I can tell, into the UEFI partition can go either a
> kernel & initramfs with UEFI support, or a "loader" that then loads the
> kernel.
>
> What I am unsure about are...
>
> 1) Can the loader/kernel understand md raid? so the / can be in a bog
> standard raid1 v1.2?
>
> 2) I'm guessing I would no longer need the /boot as that would be
> replaced by what ever was in the UEFI partition?
>
> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
> changes occur they are replicated to the drives I have set up as /boot
> raid 1, and I installed the mbr portion of boot loader manually on each
> disk and tested pulling one, and then booting from another.. it
> worked :-)
> So I would like to keep things, if not simple, at least less likely to
> have problems because I forgot to install duplicates of the UEFI on all
> the disks... so can I use a .90v raid1 on the UEFI partition, then
> format it as fat... so that all copies of what ever is in the UEFI
> partition are replicated across the multiple raid1 disks, instead of
> having to remember to copy what ever is in there manually to ach disk?
>
There was already a post on that subject you can find here:
http://comments.gmane.org/gmane.linux.raid/45611
It might answer some of your questions.
Bye.
^ permalink raw reply
* Re: MD Raid1 hangs system on read error (3.10)
From: Matthijs Kooijman @ 2014-10-06 5:32 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <20141006110833.5028706c@notabene.brown>
[-- Attachment #1: Type: text/plain, Size: 581 bytes --]
Hi Neil,
> You are running 3.10.3. It contains a bug that was fixed in 3.10.32.
>
> http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=9f2d289933e60ec726a7a9522e2dcdfdc82c58de
>
> So if you upgrade your kernel, the problem should be gone.
Great, I was hoping things were so simple :-)
I already thought I tried upgrading the kernel at some point, but it
seems I upgraded some older kernel package I had installed. Just
confirmed that all the crashes happened using 3.10.3, so you're most
certainly right.
Thanks!
Gr.
Matthijs
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply
* Re: MD Raid1 hangs system on read error (3.10)
From: NeilBrown @ 2014-10-06 0:08 UTC (permalink / raw)
To: Matthijs Kooijman; +Cc: linux-raid
In-Reply-To: <20141005092700.GA1452@login.tika.stderr.nl>
[-- Attachment #1: Type: text/plain, Size: 1187 bytes --]
On Sun, 5 Oct 2014 11:27:01 +0200 Matthijs Kooijman <matthijs@stdin.nl> wrote:
> Hey folks,
>
> a few times now I've found my system being locked up after a read error
> from a hard disk. It looks like the MD code that handles the read error
> messes up and causes a GPF.
>
> After this happens, the system becomes completely unresponsive - it
> responds to ping and opens TCP connections, but no data comes out. The
> serial console also gives no response.
>
> After rebooting, the disk in question showed a pending sector. In the
> most recent occurence, I found that the array was also resyncing. I'm
> not sure if this also happened in the earlier occurences (but since the
> pending sector didn't disappear in the next day, I'd expect no resync
> happened before). The read errors always happened during a routine check
> of the array.
That last sentence is the clue I needed - thanks.
You are running 3.10.3. It contains a bug that was fixed in 3.10.32.
http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=9f2d289933e60ec726a7a9522e2dcdfdc82c58de
So if you upgrade your kernel, the problem should be gone.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: 2 disk raid 5 failure
From: NeilBrown @ 2014-10-05 23:50 UTC (permalink / raw)
To: Jean-Paul Sergent; +Cc: linux-raid
In-Reply-To: <CANuKpt=ujnZh1R+g8O6QxGW4=CLOJAePB-+tFaTKz+XaNVvOYw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 526 bytes --]
On Sun, 5 Oct 2014 02:55:25 -0700 Jean-Paul Sergent <jpsergent@gmail.com>
wrote:
> Great, so its mostly media, movies and stuff. So no real tragedy if I
> loose it. but I'll go ahead and do both then copy the few essential
> files I think I have off of it.
>
> Do you know why version 3.3 had those problems and 3.3.2 works?
Because I introduced a bug when I added support for --replace, and my test
suite didn't find it. Someone else did and reported it, so I fixed it.
Just the normal stuff :-)
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply
* Re: not enough operational mirrors
From: Ian Young @ 2014-10-05 21:43 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <CANs+QMwarDWxRkxfrN4JXfH7W1uBSWPDkms7YC=DCa1qJNv0Gg@mail.gmail.com>
I've received two replacement drives and added them to the array. One
of them finished synchronizing and became an active member. The
other, sdf, has been treated as a spare. After running a smartctl
test on each of the drives, I found that sde has errors, preventing
the sync process from making sdf an active member. I have tried a
couple of recommendations I read on various sites, such as stopping
the array and recreating it with the "--assume-clean" option (not
possible because a process is using the array) and growing the array
one disk larger (not possible because this is RAID 10). Should I try
to repair the bad blocks or is there a way to force sde and sdf to
sync first?
[root@localhost ~]# smartctl -l selftest /dev/sde
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.6.10-4.fc18.x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining
LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: read failure 90% 11822
1187144704
# 2 Short offline Completed: read failure 90% 11814
1187144704
On Tue, Sep 23, 2014 at 10:07 AM, Ian Young <ian@duffrecords.com> wrote:
> I booted from a live CD so I could use version 3.1.10 of xfs_repair
> (versions < 3.1.8 reportedly have a bug when using ag_stride), then
> ran the following command:
>
> xfs_repair -P -o bhash=16384 -o ihash=16384 -o ag_stride=16
> /dev/mapper/vg_raid10-srv
>
> It stopped after a few seconds, saying:
>
> xfs_repair: read failed: Input/output error
> XFS: failed to find log head
> zero_log: cannot find log head/tail (xlog_find_tail=5), zeroing it anyway
> xfs_repair: libxfs_device_zero write failed: Input/output error
>
> However, I was able to mount the volume after that and my data was
> still there! Thanks for pointing me in the right direction with the
> RAID.
>
> On Mon, Sep 22, 2014 at 5:55 PM, Ian Young <ian@duffrecords.com> wrote:
>> It's XFS. I'm running:
>>
>> xfs_repair -n /dev/mapper/vg_raid10-srv
>>
>> I expect it will take hours or days as this volume is 8.15 TiB.
>>
>> On Mon, Sep 22, 2014 at 4:53 PM, NeilBrown <neilb@suse.de> wrote:
>>> On Mon, 22 Sep 2014 10:17:46 -0700 Ian Young <ian@duffrecords.com> wrote:
>>>
>>>> I forced the three good disks and the one that was behind by two
>>>> events to assemble:
>>>>
>>>> mdadm --assemble --force /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sde2
>>>>
>>>> Then I added the other two disks and let it sync overnight:
>>>>
>>>> mdadm --add --force /dev/md0 /dev/sdd2
>>>> mdadm --add --force /dev/md0 /dev/sdf2
>>>>
>>>> I rebooted the system in recovery mode and the root filesystem is
>>>> back! However, / is read-only and my /srv partition, which is the
>>>> largest and has most of my data, can't mount. When I try to examine
>>>> the array, it says "no md superblock detected on /dev/md0." On top of
>>>> the software RAID, I have four logical volumes. Here is the full LVM
>>>> configuration:
>>>>
>>>> http://pastebin.com/gzdZq5DL
>>>>
>>>> How do I recover the superblock?
>>>
>>> What sort of filesystem is it? ext4??
>>>
>>> Try "fsck -n" and see if it finds anything.
>>>
>>> The fact that LVM found everything suggests that the array is mostly
>>> working. Maybe just one superblock got corrupted somehow. If 'fsck' doesn't
>>> get you anywhere you might need to ask on a forum dedicated to the particular
>>> filesystem.
>>>
>>> NeilBrown
>>>
>>>
>>>>
>>>> On Sun, Sep 21, 2014 at 10:47 PM, NeilBrown <neilb@suse.de> wrote:
>>>> > On Sun, 21 Sep 2014 22:32:19 -0700 Ian Young <ian@duffrecords.com> wrote:
>>>> >
>>>> >> My 6-drive software RAID 10 array failed. The individual drives
>>>> >> failed one at a time over the past few months but it's been an
>>>> >> extremely busy summer and I didn't have the free time to RMA the
>>>> >> drives and rebuild the array. Now I'm wishing I had acted sooner
>>>> >> because three of the drives are marked as removed and the array
>>>> >> doesn't have enough mirrors to start. I followed the recovery
>>>> >> instructions at raid.wiki.kernel.org and, before making things any
>>>> >> worse, saved the status using mdadm --examine and consulted this
>>>> >> mailing list. Here's the status:
>>>> >>
>>>> >> http://pastebin.com/KkV8e8Gq
>>>> >>
>>>> >> I can see that the event counts on sdd2 and sdf2 are significantly far
>>>> >> behind, so we can consider that data too old. sdc2 is only behind by
>>>> >> two events, so any data loss there should be minimal. If I can make
>>>> >> the array start with sd[abce]2 I think that will be enough to mount
>>>> >> the filesystem, back up my data, and start replacing drives. How do I
>>>> >> do that?
>>>> >
>>>> > Use the "--force" option with "--assemble".
>>>> >
>>>> > NeilBrown
>>>
^ permalink raw reply
* Re: UEFI and mdadm questions.
From: Chris Murphy @ 2014-10-05 20:22 UTC (permalink / raw)
To: Linux-RAID Raid
In-Reply-To: <54318B86.7060506@turmel.org>
On Oct 5, 2014, at 2:18 PM, Phil Turmel <philip@turmel.org> wrote:
> On 10/03/2014 01:04 AM, Chris Murphy wrote:
>>
>> No. An OSLoader is required, it's an EFI application. Its job is to
>> load a kernel and initramfs. The kernel and initramfs could be on the
>> ESP (EFI System partition) but this is fraught with limitations. The
>> expectation is that the kernel and initramfs are on some other
>> partition of the same disk. Of course if you're using GRUB it doesn't
>> care and will find a kernel/initramfs off another disk also, or even
>> off md raid.
>
> An option to consider is to compile a kernel using the EFI stub option,
> a pre-set command line, and an embedded initramfs. Then the kernel can
> boot directly from the EFI FAT partition with *no bootloader*. The
> embedded initramfs can support any raid/lvm/partitioning scheme
> what-so-ever.
Sure but then it means the kernel+initramfs is on FAT32. Ick. And there's no advantage if using maintstream storage options like md raid, lvm, btrfs, xfs, ext234, reiserfs, ufs, zfs, hfs+, ntfs and more, because GRUB2 already understands these things so the kernel can be located on them.
For appliances, there could be some advantage since there'd only be one kernel present at a time, no fallback necessary. My Android phone, for example, has 35 partitions.
>
> If your BIOS can be configured to try multiple boot images, it should be
> possible to have true raid fallback without using motherboard or
> hardware raid. (Set up md raid1 with metadata v1.0 of multiple copies
> of the EFI FAT partition.) I've been meaning to try this….
Problems with this: a.) new Windows 8 hardware might require you boot Windows to get to the feature enabling the firmware setup, because on such hardware USB isn't initialized by default.
http://mjg59.dreamwidth.org/24869.html
I don't know why we don't have free software to initiate this, but I haven't come across it so far.
b.) There's no guarantee the firmware won't write to the ESP, thus rendering the individual md raid members out of sync and without their metadata being updated, i.e. in effect, the logical device they become later, is corrupt. Separately they aren't corrupt, merely out of sync, but you don't have an obvious way of knowing which one.
c.) strictly speaking any partition with mdadm metadata should have the linux raid partition type GUID set; not the EFI System partition type GUID. Those GUIDs are mutually exclusive.
This is why I'm still not a fan of using mdadm to raid1 an EFI System partition.
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Hot replacing device in RAID 6
From: Keith Keller @ 2014-10-05 18:30 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <CAAOsTSmn3MDn-a6usUTpmBObwYngvFNNRUXXF6zhmDKyUTHU_w@mail.gmail.com>
On 2014-10-05, Patrik Horník <patrik@dsl.sk> wrote:
>
> - We discussed couple of years ago hot replacing the drive. I see it
> is now supported in mdadm. Is it rock stable and can be used on
> production data?
I have used replace a handful of times with great success and no
problems. I have not had to shut down the array during the rebuild
operation, however. I suspect (but have no direct evidence) that a
clean shutdown should be fine. I used kernel 3.9 from ELRepo for
the replace.
--keith
--
kkeller@wombat.san-francisco.ca.us
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox