* How to initialize "composite" RAID @ 2010-09-10 22:14 Mike Hartman 2010-09-10 22:28 ` Wolfgang Denk 0 siblings, 1 reply; 14+ messages in thread From: Mike Hartman @ 2010-09-10 22:14 UTC (permalink / raw) To: linux-raid This is unrelated to my other RAID thread, but I discovered this issue when I was forced to hard restart due to the other one. My main raid (md0) is a RAID 5 composite that looks like this: - partition on hard drive A (1.5TB) - partition on hard drive B (1.5TB) - partition on hard drive C (1.5TB) - partition on RAID 1 (md1) (1.5TB) md1 is a RAID 0 used to combine two 750GB drives I already had so that they could be fit into the larger RAID 5 (since all the RAID 5 components need to be the same size). This seems to be a fairly standard approach that is more or less endorsed by the various RAID tutorials I've read through, and it works fine when I start all my arrays manually, with md1 started before md0. But when the system boots up it tries to start all my arrays automatically and the timeline looks like: Detecting md0. Can't start md0 because it's missing a component (md1) and thus wouldn't be in a clean state. Detecting md1. md1 started. Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), which works fine at that point because md1 is up. But aside from the fact that I don't want to do that manually every time I reboot, since md0 was started without the md1 component and then had it re-added it decides the array needs to go through a resync, which takes 10 hours. And that will only get worse as I continue to add more drives. Is there any way to exercise more control over the array initialization order while still having everything start automatically at bootup? Right now I've done no setup like that at all - it all just works. I've been keeping /etc/mdadm.conf updated, but as I understand it that's more for my own reference than the system's. Mike ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 22:14 How to initialize "composite" RAID Mike Hartman @ 2010-09-10 22:28 ` Wolfgang Denk 2010-09-10 22:33 ` Mike Hartman 2010-09-10 22:37 ` Neil Brown 0 siblings, 2 replies; 14+ messages in thread From: Wolfgang Denk @ 2010-09-10 22:28 UTC (permalink / raw) To: Mike Hartman; +Cc: linux-raid Dear Mike Hartman, In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: > This is unrelated to my other RAID thread, but I discovered this issue > when I was forced to hard restart due to the other one. > > My main raid (md0) is a RAID 5 composite that looks like this: > > - partition on hard drive A (1.5TB) > - partition on hard drive B (1.5TB) > - partition on hard drive C (1.5TB) > - partition on RAID 1 (md1) (1.5TB) I guess this is a typo and you mean RAID 0 ? > md1 is a RAID 0 used to combine two 750GB drives I already had so that ...as used here? > Detecting md0. Can't start md0 because it's missing a component (md1) > and thus wouldn't be in a clean state. > Detecting md1. md1 started. > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), > which works fine at that point because md1 is up. Did you try changing your configurations uch that md0 is the RAID 0 and md1 is the RAID 5 array? Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de If today is the first day of the rest of your life, what the hell was yesterday? ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 22:28 ` Wolfgang Denk @ 2010-09-10 22:33 ` Mike Hartman 2010-09-10 22:37 ` Neil Brown 1 sibling, 0 replies; 14+ messages in thread From: Mike Hartman @ 2010-09-10 22:33 UTC (permalink / raw) To: Wolfgang Denk; +Cc: linux-raid On Fri, Sep 10, 2010 at 6:28 PM, Wolfgang Denk <wd@denx.de> wrote: > Dear Mike Hartman, > > In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: >> This is unrelated to my other RAID thread, but I discovered this issue >> when I was forced to hard restart due to the other one. >> >> My main raid (md0) is a RAID 5 composite that looks like this: >> >> - partition on hard drive A (1.5TB) >> - partition on hard drive B (1.5TB) >> - partition on hard drive C (1.5TB) >> - partition on RAID 1 (md1) (1.5TB) > > I guess this is a typo and you mean RAID 0 ? > >> md1 is a RAID 0 used to combine two 750GB drives I already had so that > > ...as used here? > Definitely, sorry about that. >> Detecting md0. Can't start md0 because it's missing a component (md1) >> and thus wouldn't be in a clean state. >> Detecting md1. md1 started. >> Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), >> which works fine at that point because md1 is up. > > Did you try changing your configurations uch that md0 is the RAID 0 > and md1 is the RAID 5 array? > That's the kind of simple solution I had in mind, I just have no idea how to do it. The arrays are already created and I don't have enough space to back up all the data elsewhere (I know - *cringe*). Is there a way to do that swap "in place"? I created everything by following the current directions in the Linux RAID wiki and I didn't see anything there about specifying the device name or anything like that. > Best regards, > > Wolfgang Denk > > -- > DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de > If today is the first day of the rest of your life, what the hell was > yesterday? > -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 22:28 ` Wolfgang Denk 2010-09-10 22:33 ` Mike Hartman @ 2010-09-10 22:37 ` Neil Brown 2010-09-10 22:45 ` Mike Hartman 1 sibling, 1 reply; 14+ messages in thread From: Neil Brown @ 2010-09-10 22:37 UTC (permalink / raw) To: Wolfgang Denk; +Cc: Mike Hartman, linux-raid On Sat, 11 Sep 2010 00:28:14 +0200 Wolfgang Denk <wd@denx.de> wrote: > Dear Mike Hartman, > > In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: > > This is unrelated to my other RAID thread, but I discovered this issue > > when I was forced to hard restart due to the other one. > > > > My main raid (md0) is a RAID 5 composite that looks like this: > > > > - partition on hard drive A (1.5TB) > > - partition on hard drive B (1.5TB) > > - partition on hard drive C (1.5TB) > > - partition on RAID 1 (md1) (1.5TB) > > I guess this is a typo and you mean RAID 0 ? > > > md1 is a RAID 0 used to combine two 750GB drives I already had so that > > ...as used here? > > > Detecting md0. Can't start md0 because it's missing a component (md1) > > and thus wouldn't be in a clean state. > > Detecting md1. md1 started. > > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), > > which works fine at that point because md1 is up. > > Did you try changing your configurations uch that md0 is the RAID 0 > and md1 is the RAID 5 array? > Or just swap the order of the two lines in /etc/mdadm.conf. NeilBrown ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 22:37 ` Neil Brown @ 2010-09-10 22:45 ` Mike Hartman 2010-09-10 23:07 ` Neil Brown 0 siblings, 1 reply; 14+ messages in thread From: Mike Hartman @ 2010-09-10 22:45 UTC (permalink / raw) To: Neil Brown; +Cc: Wolfgang Denk, linux-raid On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: > On Sat, 11 Sep 2010 00:28:14 +0200 > Wolfgang Denk <wd@denx.de> wrote: > >> Dear Mike Hartman, >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: >> > This is unrelated to my other RAID thread, but I discovered this issue >> > when I was forced to hard restart due to the other one. >> > >> > My main raid (md0) is a RAID 5 composite that looks like this: >> > >> > - partition on hard drive A (1.5TB) >> > - partition on hard drive B (1.5TB) >> > - partition on hard drive C (1.5TB) >> > - partition on RAID 1 (md1) (1.5TB) >> >> I guess this is a typo and you mean RAID 0 ? >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that >> >> ...as used here? >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) >> > and thus wouldn't be in a clean state. >> > Detecting md1. md1 started. >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), >> > which works fine at that point because md1 is up. >> >> Did you try changing your configurations uch that md0 is the RAID 0 >> and md1 is the RAID 5 array? >> > > Or just swap the order of the two lines in /etc/mdadm.conf. > > NeilBrown > I thought about trying that, but I was under the impression that the autodetect process didn't refer to that file at all. I take it I was mistaken? If so that sounds like the simplest fix. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 22:45 ` Mike Hartman @ 2010-09-10 23:07 ` Neil Brown 2010-09-10 23:36 ` Mike Hartman 0 siblings, 1 reply; 14+ messages in thread From: Neil Brown @ 2010-09-10 23:07 UTC (permalink / raw) To: Mike Hartman; +Cc: Wolfgang Denk, linux-raid On Fri, 10 Sep 2010 18:45:54 -0400 Mike Hartman <mike@hartmanipulation.com> wrote: > On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: > > On Sat, 11 Sep 2010 00:28:14 +0200 > > Wolfgang Denk <wd@denx.de> wrote: > > > >> Dear Mike Hartman, > >> > >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: > >> > This is unrelated to my other RAID thread, but I discovered this issue > >> > when I was forced to hard restart due to the other one. > >> > > >> > My main raid (md0) is a RAID 5 composite that looks like this: > >> > > >> > - partition on hard drive A (1.5TB) > >> > - partition on hard drive B (1.5TB) > >> > - partition on hard drive C (1.5TB) > >> > - partition on RAID 1 (md1) (1.5TB) > >> > >> I guess this is a typo and you mean RAID 0 ? > >> > >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that > >> > >> ...as used here? > >> > >> > Detecting md0. Can't start md0 because it's missing a component (md1) > >> > and thus wouldn't be in a clean state. > >> > Detecting md1. md1 started. > >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), > >> > which works fine at that point because md1 is up. > >> > >> Did you try changing your configurations uch that md0 is the RAID 0 > >> and md1 is the RAID 5 array? > >> > > > > Or just swap the order of the two lines in /etc/mdadm.conf. > > > > NeilBrown > > > > I thought about trying that, but I was under the impression that the > autodetect process didn't refer to that file at all. I take it I was > mistaken? If so that sounds like the simplest fix. Depends what you mean by the "auto detect" process. If you are referring to in-kernel auto-detect triggered by the 0xFD partition type, then just don't use that. You cannot control the order in which arrays are assembled. You could swap the name md1 and md0 (Which isn't too hard using --assemble --update=super-minor) but it probably wouldn't make any change to behaviour. Get disable in-kernel autodetect and let mdadm assemble the arrays for you. It has a much better chance of getting it right. NeilBrown ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 23:07 ` Neil Brown @ 2010-09-10 23:36 ` Mike Hartman 2010-09-11 0:23 ` Neil Brown 0 siblings, 1 reply; 14+ messages in thread From: Mike Hartman @ 2010-09-10 23:36 UTC (permalink / raw) To: Neil Brown; +Cc: Wolfgang Denk, linux-raid >On Fri, Sep 10, 2010 at 7:07 PM, Neil Brown <neilb@suse.de> wrote: > On Fri, 10 Sep 2010 18:45:54 -0400 > Mike Hartman <mike@hartmanipulation.com> wrote: > >> On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: >> > On Sat, 11 Sep 2010 00:28:14 +0200 >> > Wolfgang Denk <wd@denx.de> wrote: >> > >> >> Dear Mike Hartman, >> >> >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: >> >> > This is unrelated to my other RAID thread, but I discovered this issue >> >> > when I was forced to hard restart due to the other one. >> >> > >> >> > My main raid (md0) is a RAID 5 composite that looks like this: >> >> > >> >> > - partition on hard drive A (1.5TB) >> >> > - partition on hard drive B (1.5TB) >> >> > - partition on hard drive C (1.5TB) >> >> > - partition on RAID 1 (md1) (1.5TB) >> >> >> >> I guess this is a typo and you mean RAID 0 ? >> >> >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that >> >> >> >> ...as used here? >> >> >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) >> >> > and thus wouldn't be in a clean state. >> >> > Detecting md1. md1 started. >> >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), >> >> > which works fine at that point because md1 is up. >> >> >> >> Did you try changing your configurations uch that md0 is the RAID 0 >> >> and md1 is the RAID 5 array? >> >> >> > >> > Or just swap the order of the two lines in /etc/mdadm.conf. >> > >> > NeilBrown >> > >> >> I thought about trying that, but I was under the impression that the >> autodetect process didn't refer to that file at all. I take it I was >> mistaken? If so that sounds like the simplest fix. > > Depends what you mean by the "auto detect" process. > > If you are referring to in-kernel auto-detect triggered by the 0xFD partition > type, then just don't use that. You cannot control the order in which arrays > are assembled. You could swap the name md1 and md0 (Which isn't too hard > using --assemble --update=super-minor) but it probably wouldn't make any > change to behaviour. I'm not using the 0xFD partition type - the partitions my RAIDs are composed of are all 0xDA, as suggested in the linux raid wiki. (I'd provide the link but the site seems to be down at the moment.) I believe that type is suggested specifically to avoid triggering the kernel auto-detect. I followed the directions on the wiki for creating the arrays, creating the file system, etc (including keeping my /etc/mdadm.conf updated) and nothing ever really called out what to do to get it all mounted automatically at boot. I was going to worry about getting them built now and getting them automated later, but when a bug (mentioned in another thread) forced me to reboot I was surprised to see that they were autodetected (more or less) anyway. So I'm not sure if it's the kernel doing it or mdadm or what. I don't see any kind of entry for mdadm when I run "rc-update show", so if it's mdadm doing the detecting and not the kernel I have no idea what's kicking it off. Is there something I could look for in the logs that would indicate how the RAIDs are actually getting assembled? > > Get disable in-kernel autodetect and let mdadm assemble the arrays for you. > It has a much better chance of getting it right. Assuming it's the kernel doing the assembling now, what are the specific settings in the config I need to turn off? How would I get mdadm to do the assembling? Just put the same commands I use when doing it manually into a script run during the boot process? Or is there already some kind of mechanism in place for this? > > NeilBrown > Sorry for all the questions. When the wiki addresses a topic it does a good job, but if it's not mentioned it's pretty hard to find good info on it anywhere. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-10 23:36 ` Mike Hartman @ 2010-09-11 0:23 ` Neil Brown 2010-09-11 2:30 ` Mike Hartman 0 siblings, 1 reply; 14+ messages in thread From: Neil Brown @ 2010-09-11 0:23 UTC (permalink / raw) To: Mike Hartman; +Cc: Wolfgang Denk, linux-raid On Fri, 10 Sep 2010 19:36:18 -0400 Mike Hartman <mike@hartmanipulation.com> wrote: > >On Fri, Sep 10, 2010 at 7:07 PM, Neil Brown <neilb@suse.de> wrote: > > On Fri, 10 Sep 2010 18:45:54 -0400 > > Mike Hartman <mike@hartmanipulation.com> wrote: > > > >> On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: > >> > On Sat, 11 Sep 2010 00:28:14 +0200 > >> > Wolfgang Denk <wd@denx.de> wrote: > >> > > >> >> Dear Mike Hartman, > >> >> > >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: > >> >> > This is unrelated to my other RAID thread, but I discovered this issue > >> >> > when I was forced to hard restart due to the other one. > >> >> > > >> >> > My main raid (md0) is a RAID 5 composite that looks like this: > >> >> > > >> >> > - partition on hard drive A (1.5TB) > >> >> > - partition on hard drive B (1.5TB) > >> >> > - partition on hard drive C (1.5TB) > >> >> > - partition on RAID 1 (md1) (1.5TB) > >> >> > >> >> I guess this is a typo and you mean RAID 0 ? > >> >> > >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that > >> >> > >> >> ...as used here? > >> >> > >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) > >> >> > and thus wouldn't be in a clean state. > >> >> > Detecting md1. md1 started. > >> >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), > >> >> > which works fine at that point because md1 is up. > >> >> > >> >> Did you try changing your configurations uch that md0 is the RAID 0 > >> >> and md1 is the RAID 5 array? > >> >> > >> > > >> > Or just swap the order of the two lines in /etc/mdadm.conf. > >> > > >> > NeilBrown > >> > > >> > >> I thought about trying that, but I was under the impression that the > >> autodetect process didn't refer to that file at all. I take it I was > >> mistaken? If so that sounds like the simplest fix. > > > > Depends what you mean by the "auto detect" process. > > > > If you are referring to in-kernel auto-detect triggered by the 0xFD partition > > type, then just don't use that. You cannot control the order in which arrays > > are assembled. You could swap the name md1 and md0 (Which isn't too hard > > using --assemble --update=super-minor) but it probably wouldn't make any > > change to behaviour. > > I'm not using the 0xFD partition type - the partitions my RAIDs are > composed of are all 0xDA, as suggested in the linux raid wiki. (I'd > provide the link but the site seems to be down at the moment.) I > believe that type is suggested specifically to avoid triggering the > kernel auto-detect. Good. So mdadm must be doing the assembly. What are the conrents of /etc/mdadm.conf (or /etc/mdadm/mdadm.conf)? If you stop both arrays, then run mdadm --assemble --scan --verbose what is reported, and what happens? The kernel logs should give you some idea of what is happening at boot - look for "md" or "raid". NeilBrown > > I followed the directions on the wiki for creating the arrays, > creating the file system, etc (including keeping my /etc/mdadm.conf > updated) and nothing ever really called out what to do to get it all > mounted automatically at boot. I was going to worry about getting them > built now and getting them automated later, but when a bug (mentioned > in another thread) forced me to reboot I was surprised to see that > they were autodetected (more or less) anyway. So I'm not sure if it's > the kernel doing it or mdadm or what. I don't see any kind of entry > for mdadm when I run "rc-update show", so if it's mdadm doing the > detecting and not the kernel I have no idea what's kicking it off. > > Is there something I could look for in the logs that would indicate > how the RAIDs are actually getting assembled? > > > > > Get disable in-kernel autodetect and let mdadm assemble the arrays for you. > > It has a much better chance of getting it right. > > Assuming it's the kernel doing the assembling now, what are the > specific settings in the config I need to turn off? How would I get > mdadm to do the assembling? Just put the same commands I use when > doing it manually into a script run during the boot process? Or is > there already some kind of mechanism in place for this? > > > > > NeilBrown > > > > Sorry for all the questions. When the wiki addresses a topic it does a > good job, but if it's not mentioned it's pretty hard to find good info > on it anywhere. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 0:23 ` Neil Brown @ 2010-09-11 2:30 ` Mike Hartman 2010-09-11 4:42 ` Neil Brown 0 siblings, 1 reply; 14+ messages in thread From: Mike Hartman @ 2010-09-11 2:30 UTC (permalink / raw) To: Neil Brown; +Cc: Wolfgang Denk, linux-raid [-- Attachment #1: Type: text/plain, Size: 5285 bytes --] On Fri, Sep 10, 2010 at 8:23 PM, Neil Brown <neilb@suse.de> wrote: > On Fri, 10 Sep 2010 19:36:18 -0400 > Mike Hartman <mike@hartmanipulation.com> wrote: > >> >On Fri, Sep 10, 2010 at 7:07 PM, Neil Brown <neilb@suse.de> wrote: >> > On Fri, 10 Sep 2010 18:45:54 -0400 >> > Mike Hartman <mike@hartmanipulation.com> wrote: >> > >> >> On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: >> >> > On Sat, 11 Sep 2010 00:28:14 +0200 >> >> > Wolfgang Denk <wd@denx.de> wrote: >> >> > >> >> >> Dear Mike Hartman, >> >> >> >> >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: >> >> >> > This is unrelated to my other RAID thread, but I discovered this issue >> >> >> > when I was forced to hard restart due to the other one. >> >> >> > >> >> >> > My main raid (md0) is a RAID 5 composite that looks like this: >> >> >> > >> >> >> > - partition on hard drive A (1.5TB) >> >> >> > - partition on hard drive B (1.5TB) >> >> >> > - partition on hard drive C (1.5TB) >> >> >> > - partition on RAID 1 (md1) (1.5TB) >> >> >> >> >> >> I guess this is a typo and you mean RAID 0 ? >> >> >> >> >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that >> >> >> >> >> >> ...as used here? >> >> >> >> >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) >> >> >> > and thus wouldn't be in a clean state. >> >> >> > Detecting md1. md1 started. >> >> >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), >> >> >> > which works fine at that point because md1 is up. >> >> >> >> >> >> Did you try changing your configurations uch that md0 is the RAID 0 >> >> >> and md1 is the RAID 5 array? >> >> >> >> >> > >> >> > Or just swap the order of the two lines in /etc/mdadm.conf. >> >> > >> >> > NeilBrown >> >> > >> >> >> >> I thought about trying that, but I was under the impression that the >> >> autodetect process didn't refer to that file at all. I take it I was >> >> mistaken? If so that sounds like the simplest fix. >> > >> > Depends what you mean by the "auto detect" process. >> > >> > If you are referring to in-kernel auto-detect triggered by the 0xFD partition >> > type, then just don't use that. You cannot control the order in which arrays >> > are assembled. You could swap the name md1 and md0 (Which isn't too hard >> > using --assemble --update=super-minor) but it probably wouldn't make any >> > change to behaviour. >> >> I'm not using the 0xFD partition type - the partitions my RAIDs are >> composed of are all 0xDA, as suggested in the linux raid wiki. (I'd >> provide the link but the site seems to be down at the moment.) I >> believe that type is suggested specifically to avoid triggering the >> kernel auto-detect. > > Good. > > So mdadm must be doing the assembly. > > What are the conrents of /etc/mdadm.conf (or /etc/mdadm/mdadm.conf)? ARRAY /dev/md0 metadata=1.2 name=odin:0 UUID=714c307e:71626854:2c2cc6c8:c67339a0 ARRAY /dev/md1 metadata=1.2 name=odin:1 UUID=e51aa0b8:e8157c6a:c241acef:a2e1fb62 > > If you stop both arrays, then run > > mdadm --assemble --scan --verbose > > what is reported, and what happens? I REALLY want to avoid that if possible. It's only 44% of the way through the resync that was started due to the last time it tried to start them automatically. Assuming it still won't detect them properly, I'd be back to a 10+ hour wait before everything was stable. > > The kernel logs should give you some idea of what is happening at boot - look > for "md" or "raid". Everything that seems related to "md" or "raid" since the last boot is attached (raid_md.log). > > NeilBrown > > >> >> I followed the directions on the wiki for creating the arrays, >> creating the file system, etc (including keeping my /etc/mdadm.conf >> updated) and nothing ever really called out what to do to get it all >> mounted automatically at boot. I was going to worry about getting them >> built now and getting them automated later, but when a bug (mentioned >> in another thread) forced me to reboot I was surprised to see that >> they were autodetected (more or less) anyway. So I'm not sure if it's >> the kernel doing it or mdadm or what. I don't see any kind of entry >> for mdadm when I run "rc-update show", so if it's mdadm doing the >> detecting and not the kernel I have no idea what's kicking it off. >> >> Is there something I could look for in the logs that would indicate >> how the RAIDs are actually getting assembled? >> >> > >> > Get disable in-kernel autodetect and let mdadm assemble the arrays for you. >> > It has a much better chance of getting it right. >> >> Assuming it's the kernel doing the assembling now, what are the >> specific settings in the config I need to turn off? How would I get >> mdadm to do the assembling? Just put the same commands I use when >> doing it manually into a script run during the boot process? Or is >> there already some kind of mechanism in place for this? >> >> > >> > NeilBrown >> > >> >> Sorry for all the questions. When the wiki addresses a topic it does a >> good job, but if it's not mentioned it's pretty hard to find good info >> on it anywhere. > > [-- Attachment #2: raid_md.log --] [-- Type: text/x-log, Size: 25669 bytes --] Sep 11 04:41:38 odin raid6: int32x1 783 MB/s Sep 11 04:41:38 odin raid6: int32x2 778 MB/s Sep 11 04:41:38 odin raid6: int32x4 607 MB/s Sep 11 04:41:38 odin raid6: int32x8 573 MB/s Sep 11 04:41:38 odin raid6: mmxx1 2476 MB/s Sep 11 04:41:38 odin raid6: mmxx2 2873 MB/s Sep 11 04:41:38 odin raid6: sse1x1 1810 MB/s Sep 11 04:41:38 odin raid6: sse1x2 2273 MB/s Sep 11 04:41:38 odin raid6: sse2x1 3408 MB/s Sep 11 04:41:38 odin raid6: sse2x2 4132 MB/s Sep 11 04:41:38 odin raid6: using algorithm sse2x2 (4132 MB/s) Sep 11 04:41:38 odin md: linear personality registered for level -1 Sep 11 04:41:38 odin md: raid0 personality registered for level 0 Sep 11 04:41:38 odin md: raid1 personality registered for level 1 Sep 11 04:41:38 odin md: raid10 personality registered for level 10 Sep 11 04:41:38 odin md: raid6 personality registered for level 6 Sep 11 04:41:38 odin md: raid5 personality registered for level 5 Sep 11 04:41:38 odin md: raid4 personality registered for level 4 Sep 11 04:41:38 odin raid6test: testing the 4-disk case... Sep 11 04:41:38 odin raid6test: test_disks(0, 1): faila= 0(D) failb= 1(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 2): faila= 0(D) failb= 2(P) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 3): faila= 0(D) failb= 3(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 2): faila= 1(D) failb= 2(P) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 3): faila= 1(D) failb= 3(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 3): faila= 2(P) failb= 3(Q) OK Sep 11 04:41:38 odin raid6test: testing the 5-disk case... Sep 11 04:41:38 odin raid6test: test_disks(0, 1): faila= 0(D) failb= 1(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 2): faila= 0(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 3): faila= 0(D) failb= 3(P) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 4): faila= 0(D) failb= 4(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 2): faila= 1(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 3): faila= 1(D) failb= 3(P) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 4): faila= 1(D) failb= 4(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 3): faila= 2(D) failb= 3(P) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 4): faila= 2(D) failb= 4(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 4): faila= 3(P) failb= 4(Q) OK Sep 11 04:41:38 odin raid6test: testing the 11-disk case... Sep 11 04:41:38 odin raid6test: test_disks(0, 1): faila= 0(D) failb= 1(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 2): faila= 0(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 3): faila= 0(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 4): faila= 0(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 5): faila= 0(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 6): faila= 0(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 7): faila= 0(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 8): faila= 0(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 9): faila= 0(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 10): faila= 0(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 2): faila= 1(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 3): faila= 1(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 4): faila= 1(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 5): faila= 1(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 6): faila= 1(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 7): faila= 1(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 8): faila= 1(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 9): faila= 1(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 10): faila= 1(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 3): faila= 2(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 4): faila= 2(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 5): faila= 2(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 6): faila= 2(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 7): faila= 2(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 8): faila= 2(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 9): faila= 2(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 10): faila= 2(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 4): faila= 3(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 5): faila= 3(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 6): faila= 3(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 7): faila= 3(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 8): faila= 3(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 9): faila= 3(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 10): faila= 3(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 5): faila= 4(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 6): faila= 4(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 7): faila= 4(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 8): faila= 4(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 9): faila= 4(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 10): faila= 4(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 6): faila= 5(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 7): faila= 5(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 8): faila= 5(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 9): faila= 5(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 10): faila= 5(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 7): faila= 6(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 8): faila= 6(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 9): faila= 6(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 10): faila= 6(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 8): faila= 7(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 9): faila= 7(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 10): faila= 7(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 9): faila= 8(D) failb= 9(P) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 10): faila= 8(D) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 10): faila= 9(P) failb= 10(Q) OK Sep 11 04:41:38 odin raid6test: testing the 12-disk case... Sep 11 04:41:38 odin raid6test: test_disks(0, 1): faila= 0(D) failb= 1(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 2): faila= 0(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 3): faila= 0(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 4): faila= 0(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 5): faila= 0(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 6): faila= 0(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 7): faila= 0(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 8): faila= 0(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 9): faila= 0(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 10): faila= 0(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 11): faila= 0(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 2): faila= 1(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 3): faila= 1(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 4): faila= 1(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 5): faila= 1(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 6): faila= 1(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 7): faila= 1(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 8): faila= 1(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 9): faila= 1(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 10): faila= 1(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 11): faila= 1(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 3): faila= 2(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 4): faila= 2(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 5): faila= 2(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 6): faila= 2(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 7): faila= 2(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 8): faila= 2(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 9): faila= 2(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 10): faila= 2(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 11): faila= 2(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 4): faila= 3(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 5): faila= 3(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 6): faila= 3(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 7): faila= 3(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 8): faila= 3(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 9): faila= 3(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 10): faila= 3(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 11): faila= 3(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 5): faila= 4(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 6): faila= 4(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 7): faila= 4(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 8): faila= 4(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 9): faila= 4(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 10): faila= 4(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 11): faila= 4(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 6): faila= 5(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 7): faila= 5(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 8): faila= 5(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 9): faila= 5(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 10): faila= 5(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 11): faila= 5(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 7): faila= 6(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 8): faila= 6(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 9): faila= 6(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 10): faila= 6(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 11): faila= 6(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 8): faila= 7(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 9): faila= 7(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 10): faila= 7(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 11): faila= 7(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 9): faila= 8(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 10): faila= 8(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 11): faila= 8(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 10): faila= 9(D) failb= 10(P) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 11): faila= 9(D) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 11): faila= 10(P) failb= 11(Q) OK Sep 11 04:41:38 odin raid6test: testing the 16-disk case... Sep 11 04:41:38 odin raid6test: test_disks(0, 1): faila= 0(D) failb= 1(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 2): faila= 0(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 3): faila= 0(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 4): faila= 0(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 5): faila= 0(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 6): faila= 0(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 7): faila= 0(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 8): faila= 0(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 9): faila= 0(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 10): faila= 0(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 11): faila= 0(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 12): faila= 0(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 13): faila= 0(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 14): faila= 0(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(0, 15): faila= 0(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 2): faila= 1(D) failb= 2(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 3): faila= 1(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 4): faila= 1(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 5): faila= 1(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 6): faila= 1(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 7): faila= 1(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 8): faila= 1(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 9): faila= 1(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 10): faila= 1(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 11): faila= 1(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 12): faila= 1(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 13): faila= 1(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 14): faila= 1(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(1, 15): faila= 1(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 3): faila= 2(D) failb= 3(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 4): faila= 2(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 5): faila= 2(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 6): faila= 2(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 7): faila= 2(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 8): faila= 2(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 9): faila= 2(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 10): faila= 2(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 11): faila= 2(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 12): faila= 2(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 13): faila= 2(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 14): faila= 2(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(2, 15): faila= 2(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 4): faila= 3(D) failb= 4(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 5): faila= 3(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 6): faila= 3(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 7): faila= 3(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 8): faila= 3(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 9): faila= 3(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 10): faila= 3(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 11): faila= 3(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 12): faila= 3(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 13): faila= 3(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 14): faila= 3(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(3, 15): faila= 3(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 5): faila= 4(D) failb= 5(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 6): faila= 4(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 7): faila= 4(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 8): faila= 4(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 9): faila= 4(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 10): faila= 4(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 11): faila= 4(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 12): faila= 4(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 13): faila= 4(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 14): faila= 4(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(4, 15): faila= 4(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 6): faila= 5(D) failb= 6(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 7): faila= 5(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 8): faila= 5(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 9): faila= 5(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 10): faila= 5(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 11): faila= 5(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 12): faila= 5(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 13): faila= 5(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 14): faila= 5(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(5, 15): faila= 5(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 7): faila= 6(D) failb= 7(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 8): faila= 6(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 9): faila= 6(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 10): faila= 6(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 11): faila= 6(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 12): faila= 6(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 13): faila= 6(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 14): faila= 6(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(6, 15): faila= 6(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 8): faila= 7(D) failb= 8(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 9): faila= 7(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 10): faila= 7(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 11): faila= 7(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 12): faila= 7(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 13): faila= 7(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 14): faila= 7(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(7, 15): faila= 7(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 9): faila= 8(D) failb= 9(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 10): faila= 8(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 11): faila= 8(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 12): faila= 8(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 13): faila= 8(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 14): faila= 8(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(8, 15): faila= 8(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 10): faila= 9(D) failb= 10(D) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 11): faila= 9(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 12): faila= 9(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 13): faila= 9(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 14): faila= 9(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(9, 15): faila= 9(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 11): faila= 10(D) failb= 11(D) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 12): faila= 10(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 13): faila= 10(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 14): faila= 10(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(10, 15): faila= 10(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(11, 12): faila= 11(D) failb= 12(D) OK Sep 11 04:41:38 odin raid6test: test_disks(11, 13): faila= 11(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(11, 14): faila= 11(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(11, 15): faila= 11(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(12, 13): faila= 12(D) failb= 13(D) OK Sep 11 04:41:38 odin raid6test: test_disks(12, 14): faila= 12(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(12, 15): faila= 12(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(13, 14): faila= 13(D) failb= 14(P) OK Sep 11 04:41:38 odin raid6test: test_disks(13, 15): faila= 13(D) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: test_disks(14, 15): faila= 14(P) failb= 15(Q) OK Sep 11 04:41:38 odin raid6test: Sep 11 04:41:38 odin raid6test: complete (257 tests, 0 failures) Sep 11 04:41:38 odin md: Waiting for all devices to be available before autodetect Sep 11 04:41:38 odin md: If you don't use raid, use raid=noautodetect Sep 11 04:41:38 odin md: Autodetecting RAID arrays. Sep 11 04:41:38 odin md: Scanned 0 and added 0 devices. Sep 11 04:41:38 odin md: autorun ... Sep 11 04:41:38 odin md: ... autorun DONE. Sep 11 04:41:38 odin md: md0 stopped. Sep 11 04:41:38 odin md: bind<sdh1> Sep 11 04:41:38 odin md: bind<sdj1> Sep 11 04:41:38 odin md: bind<sdi1> Sep 11 04:41:38 odin md: md1 stopped. Sep 11 04:41:38 odin md: bind<sdk1> Sep 11 04:41:38 odin md: bind<sdg1> Sep 11 04:41:38 odin md/raid0:md1: looking at sdg1 Sep 11 04:41:38 odin md/raid0:md1: comparing sdg1(1465141760) with sdg1(1465141760) Sep 11 04:41:38 odin md/raid0:md1: END Sep 11 04:41:38 odin md/raid0:md1: ==> UNIQUE Sep 11 04:41:38 odin md/raid0:md1: 1 zones Sep 11 04:41:38 odin md/raid0:md1: looking at sdk1 Sep 11 04:41:38 odin md/raid0:md1: comparing sdk1(1465141760) with sdg1(1465141760) Sep 11 04:41:38 odin md/raid0:md1: EQUAL Sep 11 04:41:38 odin md/raid0:md1: FINAL 1 zones Sep 11 04:41:38 odin md/raid0:md1: done. Sep 11 04:41:38 odin md/raid0:md1: md_size is 2930283520 sectors. Sep 11 04:41:38 odin ******* md1 configuration ********* Sep 11 04:41:38 odin zone0=[sdg1/sdk1/] Sep 11 04:41:38 odin zone offset=0kb device offset=0kb size=1465141760kb Sep 11 04:41:38 odin ********************************** Sep 11 04:41:38 odin Sep 11 04:41:38 odin md1: detected capacity change from 0 to 1500305162240 Sep 11 04:41:38 odin md1: detected capacity change from 0 to 1500305162240 Sep 11 04:41:38 odin md1: p1 Sep 11 04:45:16 odin md: md0 stopped. Sep 11 04:45:16 odin md: unbind<sdi1> Sep 11 04:45:16 odin md: export_rdev(sdi1) Sep 11 04:45:16 odin md: unbind<sdj1> Sep 11 04:45:16 odin md: export_rdev(sdj1) Sep 11 04:45:16 odin md: unbind<sdh1> Sep 11 04:45:16 odin md: export_rdev(sdh1) Sep 11 04:45:19 odin md: md0 stopped. Sep 11 04:45:19 odin md: bind<sdh1> Sep 11 04:45:19 odin md: bind<sdj1> Sep 11 04:45:19 odin md: bind<md1p1> Sep 11 04:45:19 odin md: bind<sdi1> Sep 11 04:45:19 odin md/raid:md0: not clean -- starting background reconstruction Sep 11 04:45:19 odin md/raid:md0: device sdi1 operational as raid disk 0 Sep 11 04:45:19 odin md/raid:md0: device md1p1 operational as raid disk 3 Sep 11 04:45:19 odin md/raid:md0: device sdj1 operational as raid disk 2 Sep 11 04:45:19 odin md/raid:md0: device sdh1 operational as raid disk 1 Sep 11 04:45:19 odin md/raid:md0: allocated 4221kB Sep 11 04:45:19 odin md/raid:md0: raid level 6 active with 4 out of 4 devices, algorithm 2 Sep 11 04:45:19 odin RAID conf printout: Sep 11 04:45:19 odin --- level:6 rd:4 wd:4 Sep 11 04:45:19 odin disk 0, o:1, dev:sdi1 Sep 11 04:45:19 odin disk 1, o:1, dev:sdh1 Sep 11 04:45:19 odin disk 2, o:1, dev:sdj1 Sep 11 04:45:19 odin disk 3, o:1, dev:md1p1 Sep 11 04:45:19 odin md0: Warning: Device md1p1 is misaligned Sep 11 04:45:19 odin md0: detected capacity change from 0 to 3000003723264 Sep 11 04:45:19 odin md: resync of RAID array md0 Sep 11 04:45:19 odin md: minimum _guaranteed_ speed: 1000 KB/sec/disk. Sep 11 04:45:19 odin md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync. Sep 11 04:45:19 odin md: using 128k window, over a total of 1464845568 blocks. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 2:30 ` Mike Hartman @ 2010-09-11 4:42 ` Neil Brown 2010-09-11 4:58 ` Mike Hartman 0 siblings, 1 reply; 14+ messages in thread From: Neil Brown @ 2010-09-11 4:42 UTC (permalink / raw) To: Mike Hartman; +Cc: Wolfgang Denk, linux-raid On Fri, 10 Sep 2010 22:30:30 -0400 Mike Hartman <mike@hartmanipulation.com> wrote: > On Fri, Sep 10, 2010 at 8:23 PM, Neil Brown <neilb@suse.de> wrote: > > On Fri, 10 Sep 2010 19:36:18 -0400 > > Mike Hartman <mike@hartmanipulation.com> wrote: > > > >> >On Fri, Sep 10, 2010 at 7:07 PM, Neil Brown <neilb@suse.de> wrote: > >> > On Fri, 10 Sep 2010 18:45:54 -0400 > >> > Mike Hartman <mike@hartmanipulation.com> wrote: > >> > > >> >> On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: > >> >> > On Sat, 11 Sep 2010 00:28:14 +0200 > >> >> > Wolfgang Denk <wd@denx.de> wrote: > >> >> > > >> >> >> Dear Mike Hartman, > >> >> >> > >> >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: > >> >> >> > This is unrelated to my other RAID thread, but I discovered this issue > >> >> >> > when I was forced to hard restart due to the other one. > >> >> >> > > >> >> >> > My main raid (md0) is a RAID 5 composite that looks like this: > >> >> >> > > >> >> >> > - partition on hard drive A (1.5TB) > >> >> >> > - partition on hard drive B (1.5TB) > >> >> >> > - partition on hard drive C (1.5TB) > >> >> >> > - partition on RAID 1 (md1) (1.5TB) > >> >> >> > >> >> >> I guess this is a typo and you mean RAID 0 ? > >> >> >> > >> >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that > >> >> >> > >> >> >> ...as used here? > >> >> >> > >> >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) > >> >> >> > and thus wouldn't be in a clean state. > >> >> >> > Detecting md1. md1 started. > >> >> >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), > >> >> >> > which works fine at that point because md1 is up. > >> >> >> > >> >> >> Did you try changing your configurations uch that md0 is the RAID 0 > >> >> >> and md1 is the RAID 5 array? > >> >> >> > >> >> > > >> >> > Or just swap the order of the two lines in /etc/mdadm.conf. > >> >> > > >> >> > NeilBrown > >> >> > > >> >> > >> >> I thought about trying that, but I was under the impression that the > >> >> autodetect process didn't refer to that file at all. I take it I was > >> >> mistaken? If so that sounds like the simplest fix. > >> > > >> > Depends what you mean by the "auto detect" process. > >> > > >> > If you are referring to in-kernel auto-detect triggered by the 0xFD partition > >> > type, then just don't use that. You cannot control the order in which arrays > >> > are assembled. You could swap the name md1 and md0 (Which isn't too hard > >> > using --assemble --update=super-minor) but it probably wouldn't make any > >> > change to behaviour. > >> > >> I'm not using the 0xFD partition type - the partitions my RAIDs are > >> composed of are all 0xDA, as suggested in the linux raid wiki. (I'd > >> provide the link but the site seems to be down at the moment.) I > >> believe that type is suggested specifically to avoid triggering the > >> kernel auto-detect. > > > > Good. > > > > So mdadm must be doing the assembly. > > > > What are the conrents of /etc/mdadm.conf (or /etc/mdadm/mdadm.conf)? > > ARRAY /dev/md0 metadata=1.2 name=odin:0 UUID=714c307e:71626854:2c2cc6c8:c67339a0 > ARRAY /dev/md1 metadata=1.2 name=odin:1 UUID=e51aa0b8:e8157c6a:c241acef:a2e1fb62 > > > > > If you stop both arrays, then run > > > > mdadm --assemble --scan --verbose > > > > what is reported, and what happens? > > I REALLY want to avoid that if possible. It's only 44% of the way > through the resync that was started due to the last time it tried to > start them automatically. Assuming it still won't detect them > properly, I'd be back to a 10+ hour wait before everything was stable. If you cleanly stop and restart an array, the resync will pick up from where it left off. But you don't need to do that, the other info you gave is sufficient. > > > > > The kernel logs should give you some idea of what is happening at boot - look > > for "md" or "raid". > > Everything that seems related to "md" or "raid" since the last boot is > attached (raid_md.log). The log shows md0 being assembled from 3 of 4 components and then *not* started. Then md1 is assembled. Then 4 minutes later (presumably when you intervened) md0 disassembled and re-assembled from all 4 devices. The reason it then started resync has nothing to do with the order in which the array was assembled, but probably more to do with how it was shutdown. The array was already marked 'dirty' as in 'needs a resync' before the system booted. If you can used mdadm-3.1.2 or later you will find that mdadm will start md0 properly after it has started md1. Or you can just swap the order of the lines in mdadm.conf. If you add a bitmap (mdadm --grow /dev/md0 --bitmap=internal) after the current resync finished, then any subsequent resync due to an unclean shutdown will be much faster. I don't know why it was marked dirty. Presumably because the system wasn't shut down properly, but I have not details and so cannot make a useful guess. NeilBrown > > > > > NeilBrown > > > > > >> > >> I followed the directions on the wiki for creating the arrays, > >> creating the file system, etc (including keeping my /etc/mdadm.conf > >> updated) and nothing ever really called out what to do to get it all > >> mounted automatically at boot. I was going to worry about getting them > >> built now and getting them automated later, but when a bug (mentioned > >> in another thread) forced me to reboot I was surprised to see that > >> they were autodetected (more or less) anyway. So I'm not sure if it's > >> the kernel doing it or mdadm or what. I don't see any kind of entry > >> for mdadm when I run "rc-update show", so if it's mdadm doing the > >> detecting and not the kernel I have no idea what's kicking it off. > >> > >> Is there something I could look for in the logs that would indicate > >> how the RAIDs are actually getting assembled? > >> > >> > > >> > Get disable in-kernel autodetect and let mdadm assemble the arrays for you. > >> > It has a much better chance of getting it right. > >> > >> Assuming it's the kernel doing the assembling now, what are the > >> specific settings in the config I need to turn off? How would I get > >> mdadm to do the assembling? Just put the same commands I use when > >> doing it manually into a script run during the boot process? Or is > >> there already some kind of mechanism in place for this? > >> > >> > > >> > NeilBrown > >> > > >> > >> Sorry for all the questions. When the wiki addresses a topic it does a > >> good job, but if it's not mentioned it's pretty hard to find good info > >> on it anywhere. > > > > -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 4:42 ` Neil Brown @ 2010-09-11 4:58 ` Mike Hartman 2010-09-11 5:09 ` Neil Brown 2010-09-11 22:13 ` John Robinson 0 siblings, 2 replies; 14+ messages in thread From: Mike Hartman @ 2010-09-11 4:58 UTC (permalink / raw) To: Neil Brown; +Cc: Wolfgang Denk, linux-raid On Sat, Sep 11, 2010 at 12:42 AM, Neil Brown <neilb@suse.de> wrote: > On Fri, 10 Sep 2010 22:30:30 -0400 > Mike Hartman <mike@hartmanipulation.com> wrote: > >> On Fri, Sep 10, 2010 at 8:23 PM, Neil Brown <neilb@suse.de> wrote: >> > On Fri, 10 Sep 2010 19:36:18 -0400 >> > Mike Hartman <mike@hartmanipulation.com> wrote: >> > >> >> >On Fri, Sep 10, 2010 at 7:07 PM, Neil Brown <neilb@suse.de> wrote: >> >> > On Fri, 10 Sep 2010 18:45:54 -0400 >> >> > Mike Hartman <mike@hartmanipulation.com> wrote: >> >> > >> >> >> On Fri, Sep 10, 2010 at 6:37 PM, Neil Brown <neilb@suse.de> wrote: >> >> >> > On Sat, 11 Sep 2010 00:28:14 +0200 >> >> >> > Wolfgang Denk <wd@denx.de> wrote: >> >> >> > >> >> >> >> Dear Mike Hartman, >> >> >> >> >> >> >> >> In message <AANLkTim9TnyTGMWnRr65SrmJDrLN=Maua_QnVLLDerwS@mail.gmail.com> you wrote: >> >> >> >> > This is unrelated to my other RAID thread, but I discovered this issue >> >> >> >> > when I was forced to hard restart due to the other one. >> >> >> >> > >> >> >> >> > My main raid (md0) is a RAID 5 composite that looks like this: >> >> >> >> > >> >> >> >> > - partition on hard drive A (1.5TB) >> >> >> >> > - partition on hard drive B (1.5TB) >> >> >> >> > - partition on hard drive C (1.5TB) >> >> >> >> > - partition on RAID 1 (md1) (1.5TB) >> >> >> >> >> >> >> >> I guess this is a typo and you mean RAID 0 ? >> >> >> >> >> >> >> >> > md1 is a RAID 0 used to combine two 750GB drives I already had so that >> >> >> >> >> >> >> >> ...as used here? >> >> >> >> >> >> >> >> > Detecting md0. Can't start md0 because it's missing a component (md1) >> >> >> >> > and thus wouldn't be in a clean state. >> >> >> >> > Detecting md1. md1 started. >> >> >> >> > Then I use mdadm to stop md0 and restart it (mdadm --assemble md0), >> >> >> >> > which works fine at that point because md1 is up. >> >> >> >> >> >> >> >> Did you try changing your configurations uch that md0 is the RAID 0 >> >> >> >> and md1 is the RAID 5 array? >> >> >> >> >> >> >> > >> >> >> > Or just swap the order of the two lines in /etc/mdadm.conf. >> >> >> > >> >> >> > NeilBrown >> >> >> > >> >> >> >> >> >> I thought about trying that, but I was under the impression that the >> >> >> autodetect process didn't refer to that file at all. I take it I was >> >> >> mistaken? If so that sounds like the simplest fix. >> >> > >> >> > Depends what you mean by the "auto detect" process. >> >> > >> >> > If you are referring to in-kernel auto-detect triggered by the 0xFD partition >> >> > type, then just don't use that. You cannot control the order in which arrays >> >> > are assembled. You could swap the name md1 and md0 (Which isn't too hard >> >> > using --assemble --update=super-minor) but it probably wouldn't make any >> >> > change to behaviour. >> >> >> >> I'm not using the 0xFD partition type - the partitions my RAIDs are >> >> composed of are all 0xDA, as suggested in the linux raid wiki. (I'd >> >> provide the link but the site seems to be down at the moment.) I >> >> believe that type is suggested specifically to avoid triggering the >> >> kernel auto-detect. >> > >> > Good. >> > >> > So mdadm must be doing the assembly. >> > >> > What are the conrents of /etc/mdadm.conf (or /etc/mdadm/mdadm.conf)? >> >> ARRAY /dev/md0 metadata=1.2 name=odin:0 UUID=714c307e:71626854:2c2cc6c8:c67339a0 >> ARRAY /dev/md1 metadata=1.2 name=odin:1 UUID=e51aa0b8:e8157c6a:c241acef:a2e1fb62 >> >> > >> > If you stop both arrays, then run >> > >> > mdadm --assemble --scan --verbose >> > >> > what is reported, and what happens? >> >> I REALLY want to avoid that if possible. It's only 44% of the way >> through the resync that was started due to the last time it tried to >> start them automatically. Assuming it still won't detect them >> properly, I'd be back to a 10+ hour wait before everything was stable. > > If you cleanly stop and restart an array, the resync will pick up from where > it left off. But you don't need to do that, the other info you gave is > sufficient. > >> >> > >> > The kernel logs should give you some idea of what is happening at boot - look >> > for "md" or "raid". >> >> Everything that seems related to "md" or "raid" since the last boot is >> attached (raid_md.log). > > The log shows md0 being assembled from 3 of 4 components and then *not* > started. > Then md1 is assembled. > Then 4 minutes later (presumably when you intervened) md0 disassembled and > re-assembled from all 4 devices. > > The reason it then started resync has nothing to do with the order in which > the array was assembled, but probably more to do with how it was shutdown. > The array was already marked 'dirty' as in 'needs a resync' before the system > booted. > > > If you can used mdadm-3.1.2 or later you will find that mdadm will start md0 > properly after it has started md1. Or you can just swap the order of the > lines in mdadm.conf. > I'm using mdadm 3.1.3 but I went ahead and swapped the lines in mdadm.conf anyway. > If you add a bitmap (mdadm --grow /dev/md0 --bitmap=internal) after the > current resync finished, then any subsequent resync due to an unclean > shutdown will be much faster. I read somewhere (I think in the wiki) that an intent bitmap only works properly on ext2 and ext3 and can cause trouble on other file systems. Can I use one on ext4 (what I'm using)? I'm hoping/assuming what I read just predates the common use of ext4. Will I need to remove the bitmap before adding another disk and growing the array to use it? If I don't, will it speed up that operation any? > > I don't know why it was marked dirty. Presumably because the system wasn't > shut down properly, but I have not details and so cannot make a useful guess. > > NeilBrown > Thanks for all the help Neil. I feel much more confident in my understanding of what mdadm is doing now. > > >> >> > >> > NeilBrown >> > >> > >> >> >> >> I followed the directions on the wiki for creating the arrays, >> >> creating the file system, etc (including keeping my /etc/mdadm.conf >> >> updated) and nothing ever really called out what to do to get it all >> >> mounted automatically at boot. I was going to worry about getting them >> >> built now and getting them automated later, but when a bug (mentioned >> >> in another thread) forced me to reboot I was surprised to see that >> >> they were autodetected (more or less) anyway. So I'm not sure if it's >> >> the kernel doing it or mdadm or what. I don't see any kind of entry >> >> for mdadm when I run "rc-update show", so if it's mdadm doing the >> >> detecting and not the kernel I have no idea what's kicking it off. >> >> >> >> Is there something I could look for in the logs that would indicate >> >> how the RAIDs are actually getting assembled? >> >> >> >> > >> >> > Get disable in-kernel autodetect and let mdadm assemble the arrays for you. >> >> > It has a much better chance of getting it right. >> >> >> >> Assuming it's the kernel doing the assembling now, what are the >> >> specific settings in the config I need to turn off? How would I get >> >> mdadm to do the assembling? Just put the same commands I use when >> >> doing it manually into a script run during the boot process? Or is >> >> there already some kind of mechanism in place for this? >> >> >> >> > >> >> > NeilBrown >> >> > >> >> >> >> Sorry for all the questions. When the wiki addresses a topic it does a >> >> good job, but if it's not mentioned it's pretty hard to find good info >> >> on it anywhere. >> > >> > > > -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 4:58 ` Mike Hartman @ 2010-09-11 5:09 ` Neil Brown 2010-09-11 5:15 ` Mike Hartman 2010-09-11 22:13 ` John Robinson 1 sibling, 1 reply; 14+ messages in thread From: Neil Brown @ 2010-09-11 5:09 UTC (permalink / raw) To: Mike Hartman; +Cc: Wolfgang Denk, linux-raid On Sat, 11 Sep 2010 00:58:21 -0400 Mike Hartman <mike@hartmanipulation.com> wrote: > > If you add a bitmap (mdadm --grow /dev/md0 --bitmap=internal) after the > > current resync finished, then any subsequent resync due to an unclean > > shutdown will be much faster. > > I read somewhere (I think in the wiki) that an intent bitmap only > works properly on ext2 and ext3 and can cause trouble on other file > systems. Can I use one on ext4 (what I'm using)? I'm hoping/assuming > what I read just predates the common use of ext4. That is completely wrong. An intent bitmap works properly no matter what filesystem is on top. It does impose a small performance penalty for writes which is very work-load-dependant, and it is not impossible that different filesystems would suffer more or less from this, but I doubt you would notice. If you can find it again let me know and I'll try to get it fixed. > > Will I need to remove the bitmap before adding another disk and > growing the array to use it? If I don't, will it speed up that > operation any? You, you will need to remove the bitmap before growing the array. I really should fix that but it doesn't seem to rise to the top of my to-do list... And no, a bitmap would have no effect on a reshape operation even if it were allowed to be present. NeilBrown ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 5:09 ` Neil Brown @ 2010-09-11 5:15 ` Mike Hartman 0 siblings, 0 replies; 14+ messages in thread From: Mike Hartman @ 2010-09-11 5:15 UTC (permalink / raw) To: Neil Brown; +Cc: Wolfgang Denk, linux-raid On Sat, Sep 11, 2010 at 1:09 AM, Neil Brown <neilb@suse.de> wrote: > On Sat, 11 Sep 2010 00:58:21 -0400 > Mike Hartman <mike@hartmanipulation.com> wrote: > >> > If you add a bitmap (mdadm --grow /dev/md0 --bitmap=internal) after the >> > current resync finished, then any subsequent resync due to an unclean >> > shutdown will be much faster. >> >> I read somewhere (I think in the wiki) that an intent bitmap only >> works properly on ext2 and ext3 and can cause trouble on other file >> systems. Can I use one on ext4 (what I'm using)? I'm hoping/assuming >> what I read just predates the common use of ext4. > > That is completely wrong. An intent bitmap works properly no matter what > filesystem is on top. > It does impose a small performance penalty for writes which is very > work-load-dependant, and it is not impossible that different filesystems > would suffer more or less from this, but I doubt you would notice. > > If you can find it again let me know and I'll try to get it fixed. I found it. Looks like I misremembered what it said though - it's referring to external bitmaps only. In the man page: "Note: external bitmaps are only known to work on ext2 and ext3. Storing bitmap files on other filesystems may result in serious problems." Thanks again! > >> >> Will I need to remove the bitmap before adding another disk and >> growing the array to use it? If I don't, will it speed up that >> operation any? > > You, you will need to remove the bitmap before growing the array. I really > should fix that but it doesn't seem to rise to the top of my to-do list... > > And no, a bitmap would have no effect on a reshape operation even if it were > allowed to be present. > > NeilBrown > > -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: How to initialize "composite" RAID 2010-09-11 4:58 ` Mike Hartman 2010-09-11 5:09 ` Neil Brown @ 2010-09-11 22:13 ` John Robinson 1 sibling, 0 replies; 14+ messages in thread From: John Robinson @ 2010-09-11 22:13 UTC (permalink / raw) To: Mike Hartman; +Cc: linux-raid On 11/09/2010 05:58, Mike Hartman wrote: [...] > I'm using mdadm 3.1.3 but I went ahead and swapped the lines in > mdadm.conf anyway. Having done that you probably ought to refresh your initrd so it contains your updated mdadm.conf. Cheers, John. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2010-09-11 22:13 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-09-10 22:14 How to initialize "composite" RAID Mike Hartman 2010-09-10 22:28 ` Wolfgang Denk 2010-09-10 22:33 ` Mike Hartman 2010-09-10 22:37 ` Neil Brown 2010-09-10 22:45 ` Mike Hartman 2010-09-10 23:07 ` Neil Brown 2010-09-10 23:36 ` Mike Hartman 2010-09-11 0:23 ` Neil Brown 2010-09-11 2:30 ` Mike Hartman 2010-09-11 4:42 ` Neil Brown 2010-09-11 4:58 ` Mike Hartman 2010-09-11 5:09 ` Neil Brown 2010-09-11 5:15 ` Mike Hartman 2010-09-11 22:13 ` John Robinson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).