* One question about man mdadm
From: Xiao Ni @ 2014-11-27 7:27 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <2101015878.5310500.1417073140617.JavaMail.zimbra@redhat.com>
Hi all
I'm trying to learn about the layout about raid10. I read
the man about mdadm.
It says "See md(4) for more detail about ’near’, ’offset’, and ’far’"
What's md(4) mean? Where can I find the detail information about this?
Best Regards
Xiao
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Raid 10 far 2 layout clarification.
From: Wilson, Jonathan @ 2014-11-26 22:26 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <BLU436-SMTP2133418C1B434E36DB60B7198700@phx.gbl>
On Wed, 2014-11-26 at 18:30 +0000, Wilson, Jonathan wrote:
> I just wanted to double check something before I return a failing SSD.
>
> I set up a 2 partition raid 10 in far 2 mode... from what I understood
> it allows a mirror and stripe to exist on a 2 drive set up, so that it
> gets the benefits of stripe with the redundancy of mirror.
I should clarify, two partitions on two different SSD's not the same
SSD :-/ /sda4 and /sdb4
>
> I just wanted to double check this was correct and if so confirm that I
> can fail and remove one of the drives.
>
> Obvioulsy I will have to drop one of the drives anyway/somehow but if it
> wont hold up/keep running after failing one drive then I will have to
> deal with it in another way, probably a quick clone onto a spare disk as
> a stop gap measure until I get the replacement, as this array is my OS
> disk.
>
> Jon.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re:
From: NeilBrown @ 2014-11-26 20:49 UTC (permalink / raw)
To: Travis Williams; +Cc: linux-raid
In-Reply-To: <CABGuBc=AJZzxR9W7frK_bdhaq24p1evsOBAdB1Otb4M2hbfqWg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]
On Wed, 26 Nov 2014 12:38:44 -0600 Travis Williams <travis@euppc.com> wrote:
> Hello all,
>
> I feel as though I must be missing something that I have had no luck
> finding all morning.
>
> When setting up arrays with spares in a spare-group, I'm having no
> luck finding a way to get that information from mdadm or mdstat. This
> becomes an issue when trying to write out configs and the like, or
> simply trying to get a feel for how arrays are setup on a system.
>
> Many tutorials/documentation/etc etc list using `mdadm --scan --detail
> >> /etc/mdadm/mdadm.conf` as a way to write out the running config for
> initialization at reboot. There is never any of the spare-group
> information listed in that output. Is there another way to see what
> spare-group is included in a currently running array?
>
> It also isn't listed in `mdadm --scan`, or by `cat /proc/mdstat`
>
> I've primarily noticed this with Debian 7, with mdadm v3.2.5 - 18th
> May 2012. kernel 3.2.0-4.
>
> When I modify the mdadm.conf myself and add the 'spare-group' setting
> myself, the arrays work as expected, but I haven't been able to find a
> way to KNOW that they are currently running that way without failing
> drives out to see. This nearly burned me after a restart in one
> instance that I caught out of dumb luck before anything of value was
> lost.
>
mdadm.conf is the primary location for spare-group information.
When "mdadm --monitor" is run, it reads that file and uses that information.
If you change the spare-group information in mdadm.conf, it would make sense
to restart "mdadm --monitor" so that it uses the updated information.
mdadm --scan --detail >> /etc/mdadm.conf
was only even meant to be a starting point - a guide. You are still
responsible for your mdadm.conf file.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: LVM RAID1 syncing component
From: NeilBrown @ 2014-11-26 20:41 UTC (permalink / raw)
To: Joe Lawrence; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1411242255300.12886@jlaw-desktop.mno.stratus.com>
[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]
On Mon, 24 Nov 2014 23:07:32 -0500 Joe Lawrence <joe.lawrence@stratus.com>
wrote:
> Does anyone know how its possible to determine which side of an LVM RAID 1
> is the stale partner during RAID resync?
>
> In ordinary MD RAID, I believe you can check
> /sys/block/md0/md/dev-XXX/state,
Why do you believe that?
During a resync (after an unclean shutdown) the devices are indistinguishable.
RAID1 reads all drives and if there is a difference it chooses one data block
to write to the others - always the one with the lowest index number.
So with md or LVM it is the same: first "first" is "copied" to the "second".
NeilBrown
> but LVM RAID seems to hide those files
> when leveraging the MD code. I've looked though pvs/vgs/lvs manpages, but
> can't figure anything out there either.
>
> Thanks,
>
> -- Joe
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* (unknown),
From: Travis Williams @ 2014-11-26 18:38 UTC (permalink / raw)
To: linux-raid
Hello all,
I feel as though I must be missing something that I have had no luck
finding all morning.
When setting up arrays with spares in a spare-group, I'm having no
luck finding a way to get that information from mdadm or mdstat. This
becomes an issue when trying to write out configs and the like, or
simply trying to get a feel for how arrays are setup on a system.
Many tutorials/documentation/etc etc list using `mdadm --scan --detail
>> /etc/mdadm/mdadm.conf` as a way to write out the running config for
initialization at reboot. There is never any of the spare-group
information listed in that output. Is there another way to see what
spare-group is included in a currently running array?
It also isn't listed in `mdadm --scan`, or by `cat /proc/mdstat`
I've primarily noticed this with Debian 7, with mdadm v3.2.5 - 18th
May 2012. kernel 3.2.0-4.
When I modify the mdadm.conf myself and add the 'spare-group' setting
myself, the arrays work as expected, but I haven't been able to find a
way to KNOW that they are currently running that way without failing
drives out to see. This nearly burned me after a restart in one
instance that I caught out of dumb luck before anything of value was
lost.
Thanks,
-Travis
^ permalink raw reply
* Raid 10 far 2 layout clarification.
From: Wilson, Jonathan @ 2014-11-26 18:30 UTC (permalink / raw)
To: linux-raid
I just wanted to double check something before I return a failing SSD.
I set up a 2 partition raid 10 in far 2 mode... from what I understood
it allows a mirror and stripe to exist on a 2 drive set up, so that it
gets the benefits of stripe with the redundancy of mirror.
I just wanted to double check this was correct and if so confirm that I
can fail and remove one of the drives.
Obvioulsy I will have to drop one of the drives anyway/somehow but if it
wont hold up/keep running after failing one drive then I will have to
deal with it in another way, probably a quick clone onto a spare disk as
a stop gap measure until I get the replacement, as this array is my OS
disk.
Jon.
^ permalink raw reply
* Re: mdadm raid5 single drive fail, single drive out of sync terror
From: Robin Hill @ 2014-11-26 16:38 UTC (permalink / raw)
To: Robison, Jon (CMG-Atlanta); +Cc: linux-raid
In-Reply-To: <5475FC0E.8090701@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4842 bytes --]
On Wed Nov 26, 2014 at 11:13:02AM -0500, Robison, Jon (CMG-Atlanta) wrote:
> On 11/26/14 10:49 AM, Robin Hill wrote:
> > On Wed Nov 26, 2014 at 10:08:12AM -0500, Jon Robison wrote:
> >
> >> Hi all!
> >>
> >> I upgraded to mdadm-3.3-7.fc20.x86_64, and my raid5 array would no
> >> longer recognize /dev/sdb1 in my raid 5 array (which is normally
> >> /dev/sd[b-f]1). I `mdadm --detail --scan`, which resulted in a degraded
> >> array, then added /dev/sdb1, and it started rebuilding happily until 25%
> >> or so, when another failure seemed to occur.
> >>
> >> I am convinced the data is fine on /dev/sd[c-f]1, and that somehow I
> >> just need to inform mdadm about that, but they got out of sync and
> >> /dev/sde1 thinks the array is AAAAA while the others think its AAA.. .
> >> The drives also seem to think e is bad because f said e was bad or some
> >> weird stuff, and sde1 is behind by ~50 events or so. That error hasn't
> >> shown itself recently. I fear sdb is bad and sde is going to go soon.
> >>
> >> Results of `mdadm --examine /dev/sd[b-f]1` are here
> >> http://dpaste.com/2Z7CPVY
> >>
> >> I'm scared and alone. Everything is off and sitting as above, though e
> >> 50 events behind and out of synch. New drives coming Friday and backup
> >> is of course a bit old. I'm petrified to execute `mdadm --create
> >> --assume-clean --level=5 --raid-devices=5 /dev/md0 /dev/sdf1 /dev/sdd1
> >> /dev/sdc1 /dev/sde1 missing`, but that seems my next option unless ya'll
> >> know better. I tried `mdadm --assemble -f /dev/md0 /dev/sdf1 /dev/sdd1
> >> /dev/sdc1 /dev/sde1` and it said something like can't start with only 3
> >> devices (which I wouldn't expect because examine still shows 4, just
> >> that they are out of sync and I thought that was -f's express purpose in
> >> assemble mode). Anyone have any suggestions? Thanks!
> > It looks like this is a bug in 3.3 (the checkin logs show something
> > similar anyway). I'd advise getting 3.3.1 or 3.3.2 and retrying the
> > forced assembly.
> >
> > If it failed during the rebuild, that would suggest there's an
> > unreadable block on sde though, which means you'll hit the same issue
> > again when you try to rebuild sdb. You'll need to:
> > - image sde to a new disk (via ddrescue)
> > - assemble the array
> > - add another new disk in to rebuild
> > - once the rebuild has completed, force a fsck on the array
> > (fsck -f /dev/md0) as the unreadable block may have caused some
> > filesystem corruption. It may also cause some file corruption, but
> > that's not something that can be easily checked.
> >
> > These read errors can be picked up and fixed by running regular array
> > checks (echo check > /sys/block/md0/md/sync_action). Most distributions
> > have these set up in cron, so make sure that's in there and enabled.
> >
> > The failed disks may actually be okay (sde particularly), so I'd advise
> > checking SMART stats and running full badblocks write tests on them. If
> > the badblocks tests run okay and there's no increase in reallocated
> > sectors reported in SMART, they should be perfectly okay for re-use.
> >
> > Cheers,
> > Robin
> Thanks you two, I'll check the logs on the machine later. I'm hopeful
> about new mdadm, rawhide appears to have 3.3.1 at least... maybe I'll
> livecd with 3.3.2..? When I checked yesterday, SMART said everything
> (including sdb and sde) was ok, I didn't do badblocks though. dmesg
> didn't seem to have anything meaningful, though I'll attach later.
>
The full badblocks write test is destructive, so should only be done
once you've got everything recovered from the disks. There is a safe
read-write mode, but that won't do as thorough a test.
> I'm inclined to wait for the disks to come on Friday. When I add them to
> the backup machine, it should only be ~500GB off, so it could rsync that
> for a few hours in degraded mode. I'd rather have the 500GB and risk
> however many bad blocks. Does that sound logical or will rsyncing with
> potential sde bad blocks ruin the whole target filesystem?
>
If the unreadable block contains filesystem metadata or file data which
needs synching, the array will fail when the processing hits it. I'd
expect that to just cause the rsync process to stop, but I wouldn't want
to count on it. I'd run the rsync without deletions first (which should
be safe - worst case is that the file being synced gets corrupted), then
run it with deletions only if everything worked the first time.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: mdadm raid5 single drive fail, single drive out of sync terror
From: Robison, Jon (CMG-Atlanta) @ 2014-11-26 16:13 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20141126154922.GA12222@cthulhu.home.robinhill.me.uk>
On 11/26/14 10:49 AM, Robin Hill wrote:
> On Wed Nov 26, 2014 at 10:08:12AM -0500, Jon Robison wrote:
>
>> Hi all!
>>
>> I upgraded to mdadm-3.3-7.fc20.x86_64, and my raid5 array would no
>> longer recognize /dev/sdb1 in my raid 5 array (which is normally
>> /dev/sd[b-f]1). I `mdadm --detail --scan`, which resulted in a degraded
>> array, then added /dev/sdb1, and it started rebuilding happily until 25%
>> or so, when another failure seemed to occur.
>>
>> I am convinced the data is fine on /dev/sd[c-f]1, and that somehow I
>> just need to inform mdadm about that, but they got out of sync and
>> /dev/sde1 thinks the array is AAAAA while the others think its AAA.. .
>> The drives also seem to think e is bad because f said e was bad or some
>> weird stuff, and sde1 is behind by ~50 events or so. That error hasn't
>> shown itself recently. I fear sdb is bad and sde is going to go soon.
>>
>> Results of `mdadm --examine /dev/sd[b-f]1` are here
>> http://dpaste.com/2Z7CPVY
>>
>> I'm scared and alone. Everything is off and sitting as above, though e
>> 50 events behind and out of synch. New drives coming Friday and backup
>> is of course a bit old. I'm petrified to execute `mdadm --create
>> --assume-clean --level=5 --raid-devices=5 /dev/md0 /dev/sdf1 /dev/sdd1
>> /dev/sdc1 /dev/sde1 missing`, but that seems my next option unless ya'll
>> know better. I tried `mdadm --assemble -f /dev/md0 /dev/sdf1 /dev/sdd1
>> /dev/sdc1 /dev/sde1` and it said something like can't start with only 3
>> devices (which I wouldn't expect because examine still shows 4, just
>> that they are out of sync and I thought that was -f's express purpose in
>> assemble mode). Anyone have any suggestions? Thanks!
> It looks like this is a bug in 3.3 (the checkin logs show something
> similar anyway). I'd advise getting 3.3.1 or 3.3.2 and retrying the
> forced assembly.
>
> If it failed during the rebuild, that would suggest there's an
> unreadable block on sde though, which means you'll hit the same issue
> again when you try to rebuild sdb. You'll need to:
> - image sde to a new disk (via ddrescue)
> - assemble the array
> - add another new disk in to rebuild
> - once the rebuild has completed, force a fsck on the array
> (fsck -f /dev/md0) as the unreadable block may have caused some
> filesystem corruption. It may also cause some file corruption, but
> that's not something that can be easily checked.
>
> These read errors can be picked up and fixed by running regular array
> checks (echo check > /sys/block/md0/md/sync_action). Most distributions
> have these set up in cron, so make sure that's in there and enabled.
>
> The failed disks may actually be okay (sde particularly), so I'd advise
> checking SMART stats and running full badblocks write tests on them. If
> the badblocks tests run okay and there's no increase in reallocated
> sectors reported in SMART, they should be perfectly okay for re-use.
>
> Cheers,
> Robin
Thanks you two, I'll check the logs on the machine later. I'm hopeful
about new mdadm, rawhide appears to have 3.3.1 at least... maybe I'll
livecd with 3.3.2..? When I checked yesterday, SMART said everything
(including sdb and sde) was ok, I didn't do badblocks though. dmesg
didn't seem to have anything meaningful, though I'll attach later.
I'm inclined to wait for the disks to come on Friday. When I add them to
the backup machine, it should only be ~500GB off, so it could rsync that
for a few hours in degraded mode. I'd rather have the 500GB and risk
however many bad blocks. Does that sound logical or will rsyncing with
potential sde bad blocks ruin the whole target filesystem?
^ permalink raw reply
* Re: mdadm raid5 single drive fail, single drive out of sync terror
From: Robin Hill @ 2014-11-26 15:49 UTC (permalink / raw)
To: Jon Robison; +Cc: linux-raid
In-Reply-To: <5475ECDC.6070309@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3164 bytes --]
On Wed Nov 26, 2014 at 10:08:12AM -0500, Jon Robison wrote:
> Hi all!
>
> I upgraded to mdadm-3.3-7.fc20.x86_64, and my raid5 array would no
> longer recognize /dev/sdb1 in my raid 5 array (which is normally
> /dev/sd[b-f]1). I `mdadm --detail --scan`, which resulted in a degraded
> array, then added /dev/sdb1, and it started rebuilding happily until 25%
> or so, when another failure seemed to occur.
>
> I am convinced the data is fine on /dev/sd[c-f]1, and that somehow I
> just need to inform mdadm about that, but they got out of sync and
> /dev/sde1 thinks the array is AAAAA while the others think its AAA.. .
> The drives also seem to think e is bad because f said e was bad or some
> weird stuff, and sde1 is behind by ~50 events or so. That error hasn't
> shown itself recently. I fear sdb is bad and sde is going to go soon.
>
> Results of `mdadm --examine /dev/sd[b-f]1` are here
> http://dpaste.com/2Z7CPVY
>
> I'm scared and alone. Everything is off and sitting as above, though e
> 50 events behind and out of synch. New drives coming Friday and backup
> is of course a bit old. I'm petrified to execute `mdadm --create
> --assume-clean --level=5 --raid-devices=5 /dev/md0 /dev/sdf1 /dev/sdd1
> /dev/sdc1 /dev/sde1 missing`, but that seems my next option unless ya'll
> know better. I tried `mdadm --assemble -f /dev/md0 /dev/sdf1 /dev/sdd1
> /dev/sdc1 /dev/sde1` and it said something like can't start with only 3
> devices (which I wouldn't expect because examine still shows 4, just
> that they are out of sync and I thought that was -f's express purpose in
> assemble mode). Anyone have any suggestions? Thanks!
It looks like this is a bug in 3.3 (the checkin logs show something
similar anyway). I'd advise getting 3.3.1 or 3.3.2 and retrying the
forced assembly.
If it failed during the rebuild, that would suggest there's an
unreadable block on sde though, which means you'll hit the same issue
again when you try to rebuild sdb. You'll need to:
- image sde to a new disk (via ddrescue)
- assemble the array
- add another new disk in to rebuild
- once the rebuild has completed, force a fsck on the array
(fsck -f /dev/md0) as the unreadable block may have caused some
filesystem corruption. It may also cause some file corruption, but
that's not something that can be easily checked.
These read errors can be picked up and fixed by running regular array
checks (echo check > /sys/block/md0/md/sync_action). Most distributions
have these set up in cron, so make sure that's in there and enabled.
The failed disks may actually be okay (sde particularly), so I'd advise
checking SMART stats and running full badblocks write tests on them. If
the badblocks tests run okay and there's no increase in reallocated
sectors reported in SMART, they should be perfectly okay for re-use.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: mdadm raid5 single drive fail, single drive out of sync terror
From: Phil Turmel @ 2014-11-26 15:47 UTC (permalink / raw)
To: Jon Robison, linux-raid
In-Reply-To: <5475ECDC.6070309@gmail.com>
Good morning Jon,
On 11/26/2014 10:08 AM, Jon Robison wrote:
> Hi all!
>
> I upgraded to mdadm-3.3-7.fc20.x86_64, and my raid5 array would no
> longer recognize /dev/sdb1 in my raid 5 array (which is normally
> /dev/sd[b-f]1). I `mdadm --detail --scan`, which resulted in a degraded
> array, then added /dev/sdb1, and it started rebuilding happily until 25%
> or so, when another failure seemed to occur.
Well, failures during rebuild of a raid5 are common. In my experience,
including helping on this list, most often due to timeout mismatch and a
failure to regularly scrub.
> I am convinced the data is fine on /dev/sd[c-f]1, and that somehow I
> just need to inform mdadm about that, but they got out of sync and
> /dev/sde1 thinks the array is AAAAA while the others think its AAA.. .
> The drives also seem to think e is bad because f said e was bad or some
> weird stuff, and sde1 is behind by ~50 events or so. That error hasn't
> shown itself recently. I fear sdb is bad and sde is going to go soon.
Please show your dmesg from the start of the problem. Also show
"smartctl -x /dev/sdX" for each of the member devices. Also show an
excerpt from "ls -l /dev/disk/by-id/" that shows the device vs. serial
number relationship for your drives.
> Results of `mdadm --examine /dev/sd[b-f]1` are here
> http://dpaste.com/2Z7CPVY
Just put the results in the email in the future. Kernel.org tolerates
relatively large messages.
> I'm scared and alone. Everything is off and sitting as above, though e
> 50 events behind and out of synch. New drives coming Friday and backup
> is of course a bit old. I'm petrified to execute `mdadm --create
> --assume-clean --level=5 --raid-devices=5 /dev/md0 /dev/sdf1 /dev/sdd1
> /dev/sdc1 /dev/sde1 missing`,
You should be petrified of any '--create' operation. What you've shown
above would certainly *not* work, thanks to your data offsets.
> but that seems my next option unless ya'll
> know better. I tried `mdadm --assemble -f /dev/md0 /dev/sdf1 /dev/sdd1
> /dev/sdc1 /dev/sde1` and it said something like can't start with only 3
> devices (which I wouldn't expect because examine still shows 4, just
> that they are out of sync and I thought that was -f's express purpose in
> assemble mode). Anyone have any suggestions? Thanks!
Show the contents of /proc/mdstat, then show the results of:
mdadm --stop /dev/md0
mdadm --assemble --force --verbose /dev/md0 /dev/sd[cdef]1
Phil
^ permalink raw reply
* mdadm raid5 single drive fail, single drive out of sync terror
From: Jon Robison @ 2014-11-26 15:08 UTC (permalink / raw)
To: linux-raid
Hi all!
I upgraded to mdadm-3.3-7.fc20.x86_64, and my raid5 array would no
longer recognize /dev/sdb1 in my raid 5 array (which is normally
/dev/sd[b-f]1). I `mdadm --detail --scan`, which resulted in a degraded
array, then added /dev/sdb1, and it started rebuilding happily until 25%
or so, when another failure seemed to occur.
I am convinced the data is fine on /dev/sd[c-f]1, and that somehow I
just need to inform mdadm about that, but they got out of sync and
/dev/sde1 thinks the array is AAAAA while the others think its AAA.. .
The drives also seem to think e is bad because f said e was bad or some
weird stuff, and sde1 is behind by ~50 events or so. That error hasn't
shown itself recently. I fear sdb is bad and sde is going to go soon.
Results of `mdadm --examine /dev/sd[b-f]1` are here
http://dpaste.com/2Z7CPVY
I'm scared and alone. Everything is off and sitting as above, though e
50 events behind and out of synch. New drives coming Friday and backup
is of course a bit old. I'm petrified to execute `mdadm --create
--assume-clean --level=5 --raid-devices=5 /dev/md0 /dev/sdf1 /dev/sdd1
/dev/sdc1 /dev/sde1 missing`, but that seems my next option unless ya'll
know better. I tried `mdadm --assemble -f /dev/md0 /dev/sdf1 /dev/sdd1
/dev/sdc1 /dev/sde1` and it said something like can't start with only 3
devices (which I wouldn't expect because examine still shows 4, just
that they are out of sync and I thought that was -f's express purpose in
assemble mode). Anyone have any suggestions? Thanks!
^ permalink raw reply
* Re: LVM RAID1 syncing component
From: Joe Lawrence @ 2014-11-26 13:20 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-raid, linux-lvm
In-Reply-To: <CAJCQCtQauJQkKjYL6-hxYkquCHL2yXSo4034jfiJxUPO7Nw2dQ@mail.gmail.com>
On Tue, 25 Nov 2014 22:42:38 -0700
Chris Murphy <lists@colorremedies.com> wrote:
> On Mon, Nov 24, 2014 at 9:07 PM, Joe Lawrence <joe.lawrence@stratus.com> wrote:
> > Does anyone know how its possible to determine which side of an LVM RAID 1
> > is the stale partner during RAID resync?
> >
> > In ordinary MD RAID, I believe you can check
> > /sys/block/md0/md/dev-XXX/state, but LVM RAID seems to hide those files
> > when leveraging the MD code. I've looked though pvs/vgs/lvs manpages, but
> > can't figure anything out there either.
>
> Rather indirectly: iotop which will show you which devices are mostly
> being read from and written to.
>
> # lvs -a -o copy_percent
> Anything less than 100% is syncing. I think.
>
From the manpages I see the following attribute bits:
* lvs, lv_attr bit Volume Health: (p)artial
* vgs, vg_attr bit (p)artial: one or more physical volumes belonging
to the volume group are missing from the system
* pvs, pv_attr bit (m)issing
along with the lvs copy_percent (is this similar to sync_percent) that
you mentioned. That's about it.
Since there seems to be no real underlying MD device, I'm assuming that
ioctls are out of the question as well.
-- Joe
^ permalink raw reply
* Re: Is it possible to grow a (far) RAID 10?
From: cvb @ 2014-11-26 7:32 UTC (permalink / raw)
To: NeilBrown; +Cc: Carsten Aulbert, linux RAID, linux-raid-owner
In-Reply-To: <20141126081915.58d7fa71@notabene.brown>
Am 25.11.2014 um 22:19 schrieb NeilBrown:
> But I'm fairly sure that "far" RAID10 arrays cannot be reshaped at all.
And for near it would work?
I'll try to volunteer for adjusting the RAID growing section on the RAID
wiki (also regarding GPT).
Chris
^ permalink raw reply
* Re: Split RAID: Proposal for archival RAID using incremental batch checksum
From: Anshuman Aggarwal @ 2014-11-26 6:24 UTC (permalink / raw)
To: NeilBrown; +Cc: Mdadm
In-Reply-To: <20141125095052.51f8eadc@notabene.brown>
On 25 November 2014 at 04:20, NeilBrown <neilb@suse.de> wrote:
> On Mon, 24 Nov 2014 12:59:47 +0530 Anshuman Aggarwal
> <anshuman.aggarwal@gmail.com> wrote:
>
>> On 3 November 2014 at 11:22, NeilBrown <neilb@suse.de> wrote:
>> > On Thu, 30 Oct 2014 20:30:40 +0530 Anshuman Aggarwal
>> > <anshuman.aggarwal@gmail.com> wrote:
>> >
>> >> Would chunksize==disksize work? Wouldn't that lead to the entire
>> >> parity be invalidated for any write to any of the disks (assuming md
>> >> operates at a chunk level)...also please see my reply below
>> >
>> > Operating at a chunk level would be a very poor design choice. md/raid5
>> > operates in units of 1 page (4K).
>>
>> It appears that my requirement may be met by a partitionable md raid 4
>> array where the partitions are all on individual underlying block
>> devices not striped across the block devices. Is that currently
>> possible with md raid? I dont' see how but such an enhancement could
>> do all that I had outlined earlier
>>
>> Is this possible to implement using RAID4 and MD already?
>
> Nearly. RAID4 currently requires the chunk size to be a power of 2.
> Rounding down the size of your drives to match that could waste nearly half
> the space. However it should work as a proof-of-concept.
>
> RAID0 supports non-power-of-2 chunk sizes. Doing the same thing for
> RAID4/5/6 would be quite possible.
>
>> can the
>> partitions be made to write to individual block devices such that
>> parity updates don't require reading all devices?
>
> md/raid4 will currently tries to minimize total IO requests when performing
> an update, but prefer spreading the IO over more devices if the total number
> of requests is the same.
>
> So for a 4-drive RAID4, Updating a single block can be done by:
> read old data block, read parity, write data, write parity - 4 IO requests
> or
> read other 2 data blocks, write data, write parity - 4 IO requests.
>
> In this case it will prefer the second, which is not what you want.
> With 5-drive RAID4, the second option will require 5 IO requests, so the first
> will be chosen.
> It is quite trivial to flip this default for testing
>
> - if (rmw < rcw && rmw > 0) {
> + if (rmw <= rcw && rmw > 0) {
>
>
> If you had 5 drives, you could experiment with no code changes.
> Make the chunk size the largest power of 2 that fits in the device, and then
> partition to align the partitions on those boundaries.
If the chunk size is almost the same as the device size, I assume the
entire chunk is not invalidated for parity on writing to a single
block? i.e. if only 1 block is updated only that blocks parity will be
read and written and not for the whole chunk? If thats' the case, what
purpose does a chunk serve in md raid ? If that's not the case, it
wouldn't work because a single block updation would lead to parity
being written for the entire chunk, which is the size of the device
I do have more than 5 drives though they are in use currently. I will
create a small testing partition on each device of the same size and
run the test on that after ensuring that the drives do go to sleep.
>
> NeilBrown
>
Thanks,
Anshuman
>
>>
>> To illustrate:
>> -----------------RAID - 4 ---------------------
>> |
>> Device 1 Device 2 Device 3 Parity
>> A1 B1 C1 P1
>> A2 B2 C2 P2
>> A3 B3 C3 P3
>>
>> Each device gets written to independently (via a layer of block
>> devices)...so Data on Device 1 is written as A1, A2, A3 contiguous
>> blocks leading to updation of P1, P2 P3 (without causing any reads on
>> devices 2 and 3 using XOR for the parity).
>>
>> In RAID4, IIUC data gets striped and all devices become a single block device.
>>
>>
>> >
>> >
>> >>
>> >> On 29 October 2014 14:55, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
>> >> > Right on most counts but please see comments below.
>> >> >
>> >> > On 29 October 2014 14:35, NeilBrown <neilb@suse.de> wrote:
>> >> >> Just to be sure I understand, you would have N + X devices. Each of the N
>> >> >> devices contains an independent filesystem and could be accessed directly if
>> >> >> needed. Each of the X devices contains some codes so that if at most X
>> >> >> devices in total died, you would still be able to recover all of the data.
>> >> >> If more than X devices failed, you would still get complete data from the
>> >> >> working devices.
>> >> >>
>> >> >> Every update would only write to the particular N device on which it is
>> >> >> relevant, and all of the X devices. So N needs to be quite a bit bigger
>> >> >> than X for the spin-down to be really worth it.
>> >> >>
>> >> >> Am I right so far?
>> >> >
>> >> > Perfectly right so far. I typically have a N to X ratio of 4 (4
>> >> > devices to 1 data) so spin down is totally worth it for data
>> >> > protection but more on that below.
>> >> >
>> >> >>
>> >> >> For some reason the writes to X are delayed... I don't really understand
>> >> >> that part.
>> >> >
>> >> > This delay is basically designed around archival devices which are
>> >> > rarely read from and even more rarely written to. By delaying writes
>> >> > on 2 criteria ( designated cache buffer filling up or preset time
>> >> > duration from last write expiring) we can significantly reduce the
>> >> > writes on the parity device. This assumes that we are ok to lose a
>> >> > movie or two in case the parity disk is not totally up to date but are
>> >> > more interested in device longevity.
>> >> >
>> >> >>
>> >> >> Sounds like multi-parity RAID6 with no parity rotation and
>> >> >> chunksize == devicesize
>> >> > RAID6 would present us with a joint device and currently only allows
>> >> > writes to that directly, yes? Any writes will be striped.
>> >
>> > If the chunksize equals the device size, then you need a very large write for
>> > it to be striped.
>> >
>> >> > In any case would md raid allow the underlying device to be written to
>> >> > directly? Also how would it know that the device has been written to
>> >> > and hence parity has to be updated? What about the superblock which
>> >> > the FS would not know about?
>> >
>> > No, you wouldn't write to the underlying device. You would carefully
>> > partition the RAID5 so each partition aligns exactly with an underlying
>> > device. Then write to the partition.
>> >
>> >> >
>> >> > Also except for the delayed checksum writing part which would be
>> >> > significant if one of the objectives is to reduce the amount of
>> >> > writes. Can we delay that in the code currently for RAID6? I
>> >> > understand the objective of RAID6 is to ensure data recovery and we
>> >> > are looking at a compromise in this case.
>> >
>> > "simple matter of programming"
>> > Of course there would be a limit to how much data can be buffered in memory
>> > before it has to be flushed out.
>> > If you are mostly storing movies, then they are probably too large to
>> > buffer. Why not just write them out straight away?
>> >
>> > NeilBrown
>> >
>> >
>> >
>> >> >
>> >> > If feasible, this can be an enhancement to MD RAID as well where N
>> >> > devices are presented instead of a single joint device in case of
>> >> > raid6 (maybe the multi part device can be individual disks?)
>> >> >
>> >> > It will certainly solve my problem of where to store the metadata. I
>> >> > was currently hoping to just store it as a configuration file to be
>> >> > read by the initramfs since in this case worst case scenario the
>> >> > checksum goes out of sync and is rebuilt from scratch.
>> >> >
>> >> >>
>> >> >> I wouldn't use device-mapper myself, but you are unlikely to get an entirely
>> >> >> impartial opinion from me on that topic.
>> >> >
>> >> > I haven't hacked around the kernel internals much so far so will have
>> >> > to dig out that history. I will welcome any particular links/mail
>> >> > threads I should look at for guidance (with both yours and opposing
>> >> > points of view)
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> >> the body of a message to majordomo@vger.kernel.org
>> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: LVM RAID1 syncing component
From: Chris Murphy @ 2014-11-26 5:42 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <alpine.DEB.2.02.1411242255300.12886@jlaw-desktop.mno.stratus.com>
On Mon, Nov 24, 2014 at 9:07 PM, Joe Lawrence <joe.lawrence@stratus.com> wrote:
> Does anyone know how its possible to determine which side of an LVM RAID 1
> is the stale partner during RAID resync?
>
> In ordinary MD RAID, I believe you can check
> /sys/block/md0/md/dev-XXX/state, but LVM RAID seems to hide those files
> when leveraging the MD code. I've looked though pvs/vgs/lvs manpages, but
> can't figure anything out there either.
Rather indirectly: iotop which will show you which devices are mostly
being read from and written to.
# lvs -a -o copy_percent
Anything less than 100% is syncing. I think.
--
Chris Murphy
^ permalink raw reply
* Re: Is it possible to grow a (far) RAID 10?
From: NeilBrown @ 2014-11-25 21:19 UTC (permalink / raw)
To: cvb; +Cc: Carsten Aulbert, linux RAID, linux-raid-owner
In-Reply-To: <81fe7d20d731393c86a6ca557d71adaa@kruemel.org>
[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]
On Tue, 25 Nov 2014 12:01:58 +0100 cvb@kruemel.org wrote:
> Hi
>
> Am 2014-11-25 10:25, schrieb Carsten Aulbert:
> > Without much thinking, I would simply follow the wiki page, mark a
> > device as failed, remove it, repartition it, add it again, and wait
> > for sync to complete. Repeat for all 4 devices and finally --grow with
> > mdadm (followed by xfs resizing) - and of course, all online, while
> > machine is in flight.
>
> That is exactoy what I did with my RAID 10 a bit more than a year ago.
> It basically worked, but there was one caveat... some part (the
> ---grow?) which required a certain kernel version for which I had booted
> a live CD back then...
>
> The thing is, I can't exactly remember what the caveat was. And as I'm
> about to grow the RAID again (having replaced all hard drives already),
> I have been looking for the related mails in the archive of this list,
> but I can't seem to find them...
>
> So I guess we'll have to hope that Neil replies again.
I recommend creating some loop-back block devices and experimenting.
But I'm fairly sure that "far" RAID10 arrays cannot be reshaped at all.
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* [PATCH] Change way of printing name of a process
From: Pawel Baldysiak @ 2014-11-25 12:56 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, pawel.baldysiak, artur.paszkiewicz
Sometimes mdadm prints messages with wrong name "mdmon",
and vice versa.
This patch solves this problem by changing method of determining
process name.
Now "Name" will be set in const at start of a program,
previously was hardcoded as #define.
Signed-off-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
---
Grow.c | 18 +++++++++---------
Incremental.c | 4 ++--
Monitor.c | 4 ++--
ReadMe.c | 2 +-
mdadm.c | 7 ++++---
mdadm.h | 6 +++---
mdmon.c | 2 ++
mdmon.h | 3 +--
super-intel.c | 2 +-
sysfs.c | 8 ++++----
xmalloc.c | 12 ++++++++----
11 files changed, 37 insertions(+), 31 deletions(-)
diff --git a/Grow.c b/Grow.c
index a0f7526..41b1e5d 100644
--- a/Grow.c
+++ b/Grow.c
@@ -1919,7 +1919,7 @@ size_change_error:
int err;
err = remove_disks_for_takeover(st, sra, array.layout);
if (err) {
- dprintf(Name": Array cannot be reshaped\n");
+ dprintf("%s: Array cannot be reshaped\n", Name);
if (cfd > -1)
close(cfd);
rv = 1;
@@ -2133,7 +2133,7 @@ static int verify_reshape_position(struct mdinfo *info, int level)
char *ep;
unsigned long long position = strtoull(buf, &ep, 0);
- dprintf(Name": Read sync_max sysfs entry is: %s\n", buf);
+ dprintf("%s: Read sync_max sysfs entry is: %s\n", Name, buf);
if (!(ep == buf || (*ep != 0 && *ep != '\n' && *ep != ' '))) {
position *= get_data_disks(level,
info->new_layout,
@@ -3494,8 +3494,8 @@ int reshape_container(char *container, char *devname,
return 1;
default: /* parent */
if (!freeze_reshape)
- printf(Name ": multi-array reshape continues"
- " in background\n");
+ printf("%s: multi-array reshape continues"
+ " in background\n", Name);
return 0;
case 0: /* child */
map_fork();
@@ -3557,8 +3557,8 @@ int reshape_container(char *container, char *devname,
fd = open_dev(mdstat->devnm);
if (fd < 0) {
- printf(Name ": Device %s cannot be opened for reshape.",
- adev);
+ printf("%s: Device %s cannot be opened for reshape.",
+ Name, adev);
break;
}
@@ -3573,8 +3573,8 @@ int reshape_container(char *container, char *devname,
* This is possibly interim until the behaviour of
* reshape_array is resolved().
*/
- printf(Name ": Multiple reshape execution detected for "
- "device %s.", adev);
+ printf("%s: Multiple reshape execution detected for "
+ "device %s.", Name, adev);
close(fd);
break;
}
@@ -4611,7 +4611,7 @@ int Grow_restart(struct supertype *st, struct mdinfo *info, int *fdlist, int cnt
st->ss->free_super(st);
offsets[j] = dinfo.data_offset * 512;
}
- printf(Name ": restoring critical section\n");
+ printf("%s: restoring critical section\n", Name);
if (restore_stripes(fdlist, offsets,
info->array.raid_disks,
diff --git a/Incremental.c b/Incremental.c
index 13b68bc..38c9989 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -1710,9 +1710,9 @@ int IncrementalRemove(char *devname, char *id_path, int verbose)
char buf[32];
if (!id_path)
- dprintf(Name ": incremental removal without --path <id_path> "
+ dprintf("%s: incremental removal without --path <id_path> "
"lacks the possibility to re-add new device in this "
- "port\n");
+ "port\n", Name);
if (strchr(devname, '/')) {
pr_err("incremental removal requires a "
diff --git a/Monitor.c b/Monitor.c
index 971d2ec..7a5203c 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -381,14 +381,14 @@ static void alert(char *event, char *dev, char *disc, struct alert_info *info)
if (info->mailfrom)
fprintf(mp, "From: %s\n", info->mailfrom);
else
- fprintf(mp, "From: " Name " monitoring <root>\n");
+ fprintf(mp, "From: %s monitoring <root>\n", Name);
fprintf(mp, "To: %s\n", info->mailaddr);
fprintf(mp, "Subject: %s event on %s:%s\n\n",
event, dev, hname);
fprintf(mp,
"This is an automatically generated"
- " mail message from " Name "\n");
+ " mail message from %s\n", Name);
fprintf(mp, "running on %s\n\n", hname);
fprintf(mp,
diff --git a/ReadMe.c b/ReadMe.c
index 445c388..87a4916 100644
--- a/ReadMe.c
+++ b/ReadMe.c
@@ -30,7 +30,7 @@
#ifndef VERS_DATE
#define VERS_DATE "21st August 2014"
#endif
-char Version[] = Name " - v" VERSION " - " VERS_DATE "\n";
+char Version[] = "mdadm - v" VERSION " - " VERS_DATE "\n";
/*
* File: ReadMe.c
diff --git a/mdadm.c b/mdadm.c
index c856fcd..475c71c 100644
--- a/mdadm.c
+++ b/mdadm.c
@@ -38,6 +38,7 @@ static int misc_list(struct mddev_dev *devlist,
struct mddev_ident *ident,
char *dump_directory,
struct supertype *ss, struct context *c);
+const char Name[] = "mdadm";
int main(int argc, char *argv[])
{
@@ -771,12 +772,12 @@ int main(int argc, char *argv[])
if (strcmp(c.update,"?") == 0 ||
strcmp(c.update, "help") == 0) {
outf = stdout;
- fprintf(outf, Name ": ");
+ fprintf(outf, "%s: ", Name);
} else {
outf = stderr;
fprintf(outf,
- Name ": '--update=%s' is invalid. ",
- c.update);
+ "%s: '--update=%s' is invalid. ",
+ Name, c.update);
}
fprintf(outf, "Valid --update options are:\n"
" 'sparc2.2', 'super-minor', 'uuid', 'name', 'resync',\n"
diff --git a/mdadm.h b/mdadm.h
index fc1fd31..c8d079c 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -180,6 +180,8 @@ extern __off64_t lseek64 __P ((int __fd, __off64_t __offset, int __whence));
#define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0]))
+extern const char Name[];
+
/* general information that might be extracted from a superblock */
struct mdinfo {
mdu_array_info_t array;
@@ -262,8 +264,6 @@ struct createinfo {
struct supertype *supertype;
};
-#define Name "mdadm"
-
enum mode {
ASSEMBLE=1,
BUILD,
@@ -1457,7 +1457,7 @@ static inline int xasprintf(char **strp, const char *fmt, ...) {
return ret;
}
-#define pr_err(fmt ...) fprintf(stderr, Name ": " fmt)
+#define pr_err(fmt, args...) fprintf(stderr, "%s: "fmt, Name, ##args)
#define cont_err(fmt ...) fprintf(stderr, " " fmt)
void *xmalloc(size_t len);
diff --git a/mdmon.c b/mdmon.c
index 27045a1..ee12b7c 100644
--- a/mdmon.c
+++ b/mdmon.c
@@ -67,6 +67,8 @@
#include "mdadm.h"
#include "mdmon.h"
+char const Name[] = "mdmon";
+
struct active_array *discard_this;
struct active_array *pending_discard;
diff --git a/mdmon.h b/mdmon.h
index 5a8e120..aa750c6 100644
--- a/mdmon.h
+++ b/mdmon.h
@@ -18,8 +18,7 @@
* 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
*/
-#undef pr_err
-#define pr_err(fmt ...) fprintf(stderr, "mdmon: " fmt)
+extern const char Name[];
enum array_state { clear, inactive, suspended, readonly, read_auto,
clean, active, write_pending, active_idle, bad_word};
diff --git a/super-intel.c b/super-intel.c
index 4c53019..19f6d33 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -1386,7 +1386,7 @@ static int imsm_check_attributes(__u32 attributes)
}
if (not_supported)
- dprintf(Name "(IMSM): Unknown attributes : %x\n", not_supported);
+ dprintf("%s (IMSM): Unknown attributes : %x\n", Name, not_supported);
ret_val = 0;
}
diff --git a/sysfs.c b/sysfs.c
index 9a1d856..ceab27f 100644
--- a/sysfs.c
+++ b/sysfs.c
@@ -413,8 +413,8 @@ int sysfs_set_str(struct mdinfo *sra, struct mdinfo *dev,
n = write(fd, val, strlen(val));
close(fd);
if (n != strlen(val)) {
- dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
- val, fname, strerror(errno));
+ dprintf("%s: failed to write '%s' to '%s' (%s)\n",
+ Name, val, fname, strerror(errno));
return -1;
}
return 0;
@@ -450,8 +450,8 @@ int sysfs_uevent(struct mdinfo *sra, char *event)
n = write(fd, event, strlen(event));
close(fd);
if (n != (int)strlen(event)) {
- dprintf(Name ": failed to write '%s' to '%s' (%s)\n",
- event, fname, strerror(errno));
+ dprintf("%s: failed to write '%s' to '%s' (%s)\n",
+ Name, event, fname, strerror(errno));
return -1;
}
return 0;
diff --git a/xmalloc.c b/xmalloc.c
index 8d42a7c..75ae4e2 100644
--- a/xmalloc.c
+++ b/xmalloc.c
@@ -37,7 +37,8 @@ void *xmalloc(size_t len)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -47,7 +48,8 @@ void *xrealloc(void *ptr, size_t len)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -57,7 +59,8 @@ void *xcalloc(size_t num, size_t size)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
@@ -67,6 +70,7 @@ char *xstrdup(const char *str)
char *msg;
if (rv)
return rv;
- msg = Name ": memory allocation failure - aborting\n";
+ msg = ": memory allocation failure - aborting\n";
+ write(2, Name, strlen(Name));
exit(4+!!write(2, msg, strlen(msg)));
}
^ permalink raw reply related
* Re: Is it possible to grow a (far) RAID 10?
From: cvb @ 2014-11-25 11:01 UTC (permalink / raw)
To: Carsten Aulbert; +Cc: linux RAID, linux-raid-owner
In-Reply-To: <54744AEE.10908@aei.mpg.de>
Hi
Am 2014-11-25 10:25, schrieb Carsten Aulbert:
> Without much thinking, I would simply follow the wiki page, mark a
> device as failed, remove it, repartition it, add it again, and wait
> for sync to complete. Repeat for all 4 devices and finally --grow with
> mdadm (followed by xfs resizing) - and of course, all online, while
> machine is in flight.
That is exactoy what I did with my RAID 10 a bit more than a year ago.
It basically worked, but there was one caveat... some part (the
---grow?) which required a certain kernel version for which I had booted
a live CD back then...
The thing is, I can't exactly remember what the caveat was. And as I'm
about to grow the RAID again (having replaced all hard drives already),
I have been looking for the related mails in the archive of this list,
but I can't seem to find them...
So I guess we'll have to hope that Neil replies again.
Chris
^ permalink raw reply
* Is it possible to grow a (far) RAID 10?
From: Carsten Aulbert @ 2014-11-25 9:25 UTC (permalink / raw)
To: linux RAID
Hi
after browsing various search results I'm not sure if a RAID0 (or a
RAID10) can be grown at all, especially as
https://raid.wiki.kernel.org/index.php/Growing
only mentions levels 1/4/5/6 while the mdadm man-page suggests raid0 but
not raid 10 - so consider me confused.
However, if at all possible, here is what I have/plan:
We have four 100GB SSDs partitioned only to use 50% of that, e.g.
parted -s /dev/sdc print
Model: ATA INTEL SSDSC2BA10 (scsi)
Disk /dev/sdc: 100GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 2097kB 50.0GB 50.0GB primary raid
These are assembled like this:
# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Thu Oct 9 14:11:36 2014
Raid Level : raid10
Array Size : 97615616 (93.09 GiB 99.96 GB)
Used Dev Size : 48807808 (46.55 GiB 49.98 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Tue Nov 25 10:18:33 2014
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : far=2
Chunk Size : 64K
Name : einstein-db1.atlas.local:0
UUID : dcd28f40:a020f822:d87a5b91:31bedccf
Events : 38
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1
2 8 49 2 active sync /dev/sdd1
3 8 65 3 active sync /dev/sde1
But life always tells you your initial thoughts are wrong, so we would
like to expand this to use 75% of each SSD.
Without much thinking, I would simply follow the wiki page, mark a
device as failed, remove it, repartition it, add it again, and wait for
sync to complete. Repeat for all 4 devices and finally --grow with mdadm
(followed by xfs resizing) - and of course, all online, while machine is
in flight.
The question now is, will this really work with RAID10 or would one need
to change it to RAID0 first, then perform this exercise and convert back
to RAID10 afterwards (and possibly lose all data because I will
inadvertently will make a serious typo somewhere ;)).
cheers
Carsten
--
Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics
Callinstrasse 38, 30167 Hannover, Germany
phone/fax: +49 511 762-17185 / -17193
https://wiki.atlas.aei.uni-hannover.de/foswiki/bin/view/ATLAS/WebHome
^ permalink raw reply
* LVM RAID1 syncing component
From: Joe Lawrence @ 2014-11-25 4:07 UTC (permalink / raw)
To: linux-raid; +Cc: Joe Lawrence
Does anyone know how its possible to determine which side of an LVM RAID 1
is the stale partner during RAID resync?
In ordinary MD RAID, I believe you can check
/sys/block/md0/md/dev-XXX/state, but LVM RAID seems to hide those files
when leveraging the MD code. I've looked though pvs/vgs/lvs manpages, but
can't figure anything out there either.
Thanks,
-- Joe
^ permalink raw reply
* Re: homehost and 0.90 format
From: NeilBrown @ 2014-11-25 4:02 UTC (permalink / raw)
To: Boylan, Ross; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <F1F13E14A610474196571953929C02096432BD@ex08.net.ucsf.edu>
[-- Attachment #1: Type: text/plain, Size: 4169 bytes --]
On Tue, 25 Nov 2014 03:31:32 +0000 "Boylan, Ross" <Ross.Boylan@ucsf.edu>
wrote:
> Thanks; I didn't know one could do that.
>
> But am I correct in my understanding that if 2 arrays have the same homehost they will have the same end (last half?) of the UUID? That would mean the 2 devices I'm looking at can't have the same homehost.
Yes, that is correct.
>
> I am able to guess the homehost for one of them using your trick; none of the possibilities have tried have matched the other.
Can't help you there, sorry.
You can always look at the content of the array, device what host it *should*
belong to, and the assemble the array with
--update=homehost --homehost=real-host
to "fix" the uuid.
NeilBrown
>
> Ross
> ________________________________________
> From: NeilBrown [neilb@suse.de]
> Sent: Monday, November 24, 2014 7:08 PM
> To: Boylan, Ross
> Cc: linux-raid@vger.kernel.org
> Subject: Re: homehost and 0.90 format
>
> On Tue, 11 Nov 2014 03:14:34 +0000 "Boylan, Ross" <Ross.Boylan@ucsf.edu>
> wrote:
>
> > I am trying to figure out which of my RAID devices is considered home. I haven't switched machines, but I have several operating systems, with different hostnames, on the machine. 0.90 format metadata.
> >
> > Even 2 RAID devices created on the same OS don't seem to have any UUID in common. i thought that the end of the UUID's would be the same. For example, inhttp://marc.info/?l=linux-raid&m=123999237229707 Neil Brown says
> > > Note that 0.90 metadata does contain homehost information to some
> > > extent. When homehost is set, the last few bytes of the uuid is set
> > > from a hash of the homehost name. That makes it possible to test if a
> > > 0.90 array was created for 'this' host, but not to find out what host
> > > it was created for.
> >
> > mdadm.conf has
> > HOMEHOST <system>
> >
> > The 2 UUIDS are
> > 8691ad98:af955756:ae5efbfa:dfd20ce6
> > 6d89d3b5:19a0acce:319e1a9c:58606385
> >
> > Thanks,
> > Ross Boylan
>
> You need to guess-and-test.
> e.g.
>
> mdadm --detail /dev/md7 --homehost=foo | grep UUID
>
> If it says "(local to host foo)", then "foo" is the correct name.
> If not, then try a different name.
>
> NeilBrown
>
>
> >
> > Details:
> >
> > root@wheezy4:~# mdadm --detail /dev/md7
> > /dev/md7:
> > Version : 0.90
> > Creation Time : Fri Oct 3 17:20:13 2014
> > Raid Level : raid1
> > Array Size : 48828352 (46.57 GiB 50.00 GB)
> > Used Dev Size : 48828352 (46.57 GiB 50.00 GB)
> > Raid Devices : 1
> > Total Devices : 1
> > Preferred Minor : 7
> > Persistence : Superblock is persistent
> >
> > Update Time : Tue Nov 4 17:48:40 2014
> > State : clean
> > Active Devices : 1
> > Working Devices : 1
> > Failed Devices : 0
> > Spare Devices : 0
> >
> > UUID : 8691ad98:af955756:ae5efbfa:dfd20ce6
> > Events : 0.1
> >
> > Number Major Minor RaidDevice State
> > 0 8 20 0 active sync /dev/sdb4
> > root@wheezy4:~# mdadm --detail /dev/md8
> > /dev/md8:
> > Version : 0.90
> > Creation Time : Tue Sep 30 13:27:56 2014
> > Raid Level : raid1
> > Array Size : 970702784 (925.73 GiB 994.00 GB)
> > Used Dev Size : 970702784 (925.73 GiB 994.00 GB)
> > Raid Devices : 2
> > Total Devices : 1
> > Preferred Minor : 8
> > Persistence : Superblock is persistent
> >
> > Update Time : Mon Nov 10 18:16:11 2014
> > State : clean, degraded
> > Active Devices : 1
> > Working Devices : 1
> > Failed Devices : 0
> > Spare Devices : 0
> >
> > UUID : 6d89d3b5:19a0acce:319e1a9c:58606385
> > Events : 0.790792
> >
> > Number Major Minor RaidDevice State
> > 0 8 19 0 active sync /dev/sdb3
> > 1 0 0 1 removed
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: raid10 - won't rebuild - assigns all added disks as spares
From: Jonathan Molyneux @ 2014-11-25 3:44 UTC (permalink / raw)
To: NeilBrown; +Cc: linux RAID
In-Reply-To: <20141125132810.3b4aa867@notabene.brown>
Thanks Neil,
> echo recover > /sys/block/md1/md/sync_action
That did the trick.
Regards
Jonathan
On 25/11/2014 1:28 PM, NeilBrown wrote:
> On Tue, 25 Nov 2014 12:49:12 +1100 Jonathan Molyneux
> <jonathan@infinitedepth.com.au> wrote:
>
>> Hi Everyone,
>>
>> Have a strange situation that hasn't happened before.
>> Running Debian 7.7 with kernel version 3.2.63-2+deb7u1.
>> Have a raid10 that runs the server (boot's off a raid1) that after
>> replacing a failed disk, just won't rebuild.
>>
>> This is what it looks like without the disk (failed & removed):
>> md1 : active raid10 sda2[6] sdc2[4] sdb2[1]
>> 1952987136 blocks super 1.2 512K chunks 2 far-copies [4/3] [UUU_]
>> bitmap: 8/15 pages [32KB], 65536KB chunk
>>
>> Then when the disk is added:
>> md1 : active raid10 sdd2[5](S) sda2[6] sdc2[4] sdb2[1]
>> 1952987136 blocks super 1.2 512K chunks 2 far-copies [4/3] [UUU_]
>> bitmap: 8/15 pages [32KB], 65536KB chunk
>>
>> Nothing unusual is being spat out in dmesg.
>> When removing the disk:
>> [313434.073997] md: unbind<sdd2>
>> [313434.138307] md: export_rdev(sdd2)
>> When adding the disk:
>> [313468.056484] md: bind<sdd2>
>>
>> This is a strange one that I haven't had before.
>> Any thoughts on how to kick the rebuild off without needing a reboot ?
> I'm sure I've seen this bug before... and fixed it.
> I don't remember the details and cannot find anything obvious in change logs.
>
> You could try
>
> echo recover > /sys/block/md1/md/sync_action
>
> Alternately, if you are re-adding a disk that had just been removed, you could
>
> mdadm /dev/md1 --remove /dev/sdd2
> mdadm --zero /dev/sdd2
> mdadm /dev/md1 --add /dev/sdd2
>
> that will force a full recovery instead of just a bitmap-based recovery.
> That will of course take longer than a bitmap-based recover, but seeing the
> bitmap based recovery isn't starting, that could still be an improvement.
>
> NeilBrown
^ permalink raw reply
* RE: homehost and 0.90 format
From: Boylan, Ross @ 2014-11-25 3:31 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <20141125140821.6ece0afb@notabene.brown>
Thanks; I didn't know one could do that.
But am I correct in my understanding that if 2 arrays have the same homehost they will have the same end (last half?) of the UUID? That would mean the 2 devices I'm looking at can't have the same homehost.
I am able to guess the homehost for one of them using your trick; none of the possibilities have tried have matched the other.
Ross
________________________________________
From: NeilBrown [neilb@suse.de]
Sent: Monday, November 24, 2014 7:08 PM
To: Boylan, Ross
Cc: linux-raid@vger.kernel.org
Subject: Re: homehost and 0.90 format
On Tue, 11 Nov 2014 03:14:34 +0000 "Boylan, Ross" <Ross.Boylan@ucsf.edu>
wrote:
> I am trying to figure out which of my RAID devices is considered home. I haven't switched machines, but I have several operating systems, with different hostnames, on the machine. 0.90 format metadata.
>
> Even 2 RAID devices created on the same OS don't seem to have any UUID in common. i thought that the end of the UUID's would be the same. For example, inhttp://marc.info/?l=linux-raid&m=123999237229707 Neil Brown says
> > Note that 0.90 metadata does contain homehost information to some
> > extent. When homehost is set, the last few bytes of the uuid is set
> > from a hash of the homehost name. That makes it possible to test if a
> > 0.90 array was created for 'this' host, but not to find out what host
> > it was created for.
>
> mdadm.conf has
> HOMEHOST <system>
>
> The 2 UUIDS are
> 8691ad98:af955756:ae5efbfa:dfd20ce6
> 6d89d3b5:19a0acce:319e1a9c:58606385
>
> Thanks,
> Ross Boylan
You need to guess-and-test.
e.g.
mdadm --detail /dev/md7 --homehost=foo | grep UUID
If it says "(local to host foo)", then "foo" is the correct name.
If not, then try a different name.
NeilBrown
>
> Details:
>
> root@wheezy4:~# mdadm --detail /dev/md7
> /dev/md7:
> Version : 0.90
> Creation Time : Fri Oct 3 17:20:13 2014
> Raid Level : raid1
> Array Size : 48828352 (46.57 GiB 50.00 GB)
> Used Dev Size : 48828352 (46.57 GiB 50.00 GB)
> Raid Devices : 1
> Total Devices : 1
> Preferred Minor : 7
> Persistence : Superblock is persistent
>
> Update Time : Tue Nov 4 17:48:40 2014
> State : clean
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 0
> Spare Devices : 0
>
> UUID : 8691ad98:af955756:ae5efbfa:dfd20ce6
> Events : 0.1
>
> Number Major Minor RaidDevice State
> 0 8 20 0 active sync /dev/sdb4
> root@wheezy4:~# mdadm --detail /dev/md8
> /dev/md8:
> Version : 0.90
> Creation Time : Tue Sep 30 13:27:56 2014
> Raid Level : raid1
> Array Size : 970702784 (925.73 GiB 994.00 GB)
> Used Dev Size : 970702784 (925.73 GiB 994.00 GB)
> Raid Devices : 2
> Total Devices : 1
> Preferred Minor : 8
> Persistence : Superblock is persistent
>
> Update Time : Mon Nov 10 18:16:11 2014
> State : clean, degraded
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 0
> Spare Devices : 0
>
> UUID : 6d89d3b5:19a0acce:319e1a9c:58606385
> Events : 0.790792
>
> Number Major Minor RaidDevice State
> 0 8 19 0 active sync /dev/sdb3
> 1 0 0 1 removed
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: homehost and 0.90 format
From: NeilBrown @ 2014-11-25 3:08 UTC (permalink / raw)
To: Boylan, Ross; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <F1F13E14A610474196571953929C020963C639@ex08.net.ucsf.edu>
[-- Attachment #1: Type: text/plain, Size: 3027 bytes --]
On Tue, 11 Nov 2014 03:14:34 +0000 "Boylan, Ross" <Ross.Boylan@ucsf.edu>
wrote:
> I am trying to figure out which of my RAID devices is considered home. I haven't switched machines, but I have several operating systems, with different hostnames, on the machine. 0.90 format metadata.
>
> Even 2 RAID devices created on the same OS don't seem to have any UUID in common. i thought that the end of the UUID's would be the same. For example, inhttp://marc.info/?l=linux-raid&m=123999237229707 Neil Brown says
> > Note that 0.90 metadata does contain homehost information to some
> > extent. When homehost is set, the last few bytes of the uuid is set
> > from a hash of the homehost name. That makes it possible to test if a
> > 0.90 array was created for 'this' host, but not to find out what host
> > it was created for.
>
> mdadm.conf has
> HOMEHOST <system>
>
> The 2 UUIDS are
> 8691ad98:af955756:ae5efbfa:dfd20ce6
> 6d89d3b5:19a0acce:319e1a9c:58606385
>
> Thanks,
> Ross Boylan
You need to guess-and-test.
e.g.
mdadm --detail /dev/md7 --homehost=foo | grep UUID
If it says "(local to host foo)", then "foo" is the correct name.
If not, then try a different name.
NeilBrown
>
> Details:
>
> root@wheezy4:~# mdadm --detail /dev/md7
> /dev/md7:
> Version : 0.90
> Creation Time : Fri Oct 3 17:20:13 2014
> Raid Level : raid1
> Array Size : 48828352 (46.57 GiB 50.00 GB)
> Used Dev Size : 48828352 (46.57 GiB 50.00 GB)
> Raid Devices : 1
> Total Devices : 1
> Preferred Minor : 7
> Persistence : Superblock is persistent
>
> Update Time : Tue Nov 4 17:48:40 2014
> State : clean
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 0
> Spare Devices : 0
>
> UUID : 8691ad98:af955756:ae5efbfa:dfd20ce6
> Events : 0.1
>
> Number Major Minor RaidDevice State
> 0 8 20 0 active sync /dev/sdb4
> root@wheezy4:~# mdadm --detail /dev/md8
> /dev/md8:
> Version : 0.90
> Creation Time : Tue Sep 30 13:27:56 2014
> Raid Level : raid1
> Array Size : 970702784 (925.73 GiB 994.00 GB)
> Used Dev Size : 970702784 (925.73 GiB 994.00 GB)
> Raid Devices : 2
> Total Devices : 1
> Preferred Minor : 8
> Persistence : Superblock is persistent
>
> Update Time : Mon Nov 10 18:16:11 2014
> State : clean, degraded
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 0
> Spare Devices : 0
>
> UUID : 6d89d3b5:19a0acce:319e1a9c:58606385
> Events : 0.790792
>
> Number Major Minor RaidDevice State
> 0 8 19 0 active sync /dev/sdb3
> 1 0 0 1 removed
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: MD component device renaming with udev and MD on full disk
From: NeilBrown @ 2014-11-25 2:55 UTC (permalink / raw)
To: Jason Keltz; +Cc: linux-raid
In-Reply-To: <546FB1B7.3050304@cse.yorku.ca>
[-- Attachment #1: Type: text/plain, Size: 3467 bytes --]
On Fri, 21 Nov 2014 16:42:15 -0500 Jason Keltz <jas@cse.yorku.ca> wrote:
> Hi.
>
> I have two questions about MD..
>
> 1) I've written a udev rule to remap /dev/sdX devices on my system to
> /dev/cXeYsZpA (controller, enclosure, slot, partition) mapping. When I
> reboot the system, I see that all the devices in /dev are appropriately
> renamed. If I look at /proc/mdstat, it still has the kernel names of
> the devices (/dev/sdX) even though those devices no longer exist. If I
> do an mdadm --detail /dev/mdX the system reports the proper device name
> makeup. I manually failed a device, and I got the correct device name
> in the email. I'm just wondering what command I would execute to make
> /proc/mdstat update the devices in its output?
Names in /dev/ are link symlinks. They point to the device (which is
identified by a pair of numbers: e.g. 8,1), but they aren't the device itself.
block device 8,1 is always "sda1" to the kernel, whether you access it through
a block-special file called "/dev/sda1" or "/dev/box-of-bits".
The names reported by /proc/mdstat are the kernel's internal names for the
devices. You cannot get it to use the names that you have created in /dev.
"mdadm -D" is the best way to describe an array in terms of names that you
have chosen.
>
> 2) Unrelated to 1) -- the argument re: using MD on full devices versus
> partitions has been around for a long time. I've been experimenting
> with using it on full devices. One of the arguments that I've read for
> not using full devices is that apparently, if you have two devices that
> are identical, but one of them is slightly smaller than the other due to
> say, bad sectors, then these disks can't be used together in one MD
> because they are different sizes. I'm wondering how valid that argument
> is? Surely it would make sense if MD was using full devices for it to
> actually stop short of the end of the disk for situations like these...
There is no validity at all in that argument.
md is quite happy using devices of different sizes and will ignore extra
space provided on the larger device.
If you had an array with two devices of size X, and got a new device of size
"X - delta", then it is true that the new device cannot be used in the array.
I think I have read a suggestion of creating partitions smaller than the
device so that when the "X - delta" device is used, it is still big enough
for that smaller partition.
There are two reasons why this is an invalid argument for using partitions:
1/ modern drives have standardised sizes. If you have a size-X device, then
you will not find a size-"X - delta" device unless "delta" is very big.
2/ you can use the --size option of mdadm to make the array use less than
the total space on the device to allow for later reconfigurations.
There are other valid reasons for preferring whole devices or partitions,
such as some tool or other might get confused by one of the arrangements
(e.g. lilo or and installer). And some people might be more comfortable with
one than the other.
But the size issue is completely irrelevant.
NeilBrown
>
> Thanks for any help you can provide.
>
> Jason.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox