Patch for boot-time assembly of v1.x-metadata-based soft (MD) arrays

Patch for boot-time assembly of v1.x-metadata-based soft (MD) arrays

[Abe Skolnik]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

Dear Mr./Dr./Prof. Brown et al,

I recently had the unpleasant experience of creating an MD array for
the purpose of booting off it and then not being able to do so.  Since
I had already made changes to the array's contents relative to that
which I cloned it from, I did not want to reformat the array and
re-clone it just to bring it down to the old 0.90 metadata format so
that I would be able to boot off it, so I searched for a solution, and
I found it.

First I tried the patch (written by Neil Brown) which can be seen at...
  <http://www.issociate.de/board/post/277868/>

That patch did not work as-is, but with some more hacking, I got it
working.  I then cleaned up my work and added relevant comments.

I know that Mr./Dr./Prof. Brown is against in-kernel boot-time MD
assembly and prefers init[rd/ramfs], but I prefer in-kernel assembly,
and I think several other people do too.  Since this patch does not
(AFAIK) disable the init[rd/ramfs] way of bringing up MDs in boot-time,
I hope that this patch will be accepted and submitted up-stream for
future inclusion in the mainline kernel.org kernel distribution.
This way kernel users can choose their MD assembly strategy at will
without having to restrict themselves to the old metadata format.

I hope that this message finds all those who read it doing well and
feeling fine.

Sincerely,

Abe Skolnik

P.S.  Mr./Dr./Prof. Brown, in case you read this:  thanks!
      And if you want your name removed from the code, just say so.

[-- Attachment #2: 3723671779-add-ability-to-start-MDs-with-persistent-superblock-v1.x---update-against-2.6.22.4_.patch --]
[-- Type: application/octet-stream, Size: 6477 bytes --]

--- init/do_mounts_md.c___plain-vanilla_kernel.org_2.6.22.4	Sun Aug 26 01:03:04 2007
+++ init/do_mounts_md.c	Sun Aug 26 09:02:21 2007
@@ -7,26 +7,61 @@
  * When md (and any require personalities) are compiled into the kernel
  * (not a module), arrays can be assembles are boot time using with AUTODETECT
  * where specially marked partitions are registered with md_autodetect_dev(),
  * and with MD_BOOT where devices to be collected are given on the boot line
  * with md=.....
+ *
  * The code for that is here.
  */
 
+
+/*
+ * Boot parameters
+ * ---------------
+ * example #1: md=0,1,/dev/a,/dev/b -> md0, level 1
+ *      means: request boot-time assembly of md0 that was created in the ancient
+ *             non-persistent-superblock fashion using RAID level 1,
+ *             use /dev/a and /dev/b as components
+ *
+ * example #2: md=0,v?,/dev/a,/dev/b/,/dev/...
+ *      means: request boot-time assembly of md0, persistent superblock format
+ *             version ? instead of the default of requiring _specifically_
+ *             version 0.90 (no auto-detect of version yet), RAID level is
+ *             auto-detected from the superblock(s), use /dev/a and /dev/b and
+ *             /dev/... as components; ? may be "1" (synonym for "1.0"), "1.0",
+ *             "1.1", or "1.2".
+ */
+ 
+ /*
+  * User notes
+  * ----------
+  * #1: for now, the kernel tries v0.90 first regardless, so, for v1.x metadata,
+  *     expect spurious "invalid raid superblock" errors, even when it works.
+  *
+  * #2: the order of component specification seems to matter for some reason.
+  *     If at first you don't succeed, re-order your parameters and try again.
+  *     I (Abe) suggest trying _all_ the possible orderings of your RAID's
+  *     components before giving up on being able to assemble at boot-time
+  *     without an initrd/initramfs.
+  *
+  */
+  
 static int __initdata raid_noautodetect, raid_autopart;
 
 static struct {
 	int minor;
 	int partitioned;
 	int level;
 	int chunk;
+	int vers_major, vers_minor;
 	char *device_names;
 } md_setup_args[256] __initdata;
 
 static int md_setup_ents __initdata;
 
 extern int mdp_major;
+
 /*
  * Parse the command-line parameters given our kernel, but do not
  * actually try to invoke the MD device now; that is handled by
  * md_setup_drive after the low-level disk drivers have initialised.
  *
@@ -42,11 +77,15 @@
  *             elements in device-list are read by name_to_kdev_t so can be
  *             a hex number or something like /dev/hda1 /dev/sdb
  * 2001-06-03: Dave Cinege <dcinege@psychosis.com>
  *		Shifted name_to_kdev_t() and related operations to md_set_drive()
  *		for later execution. Rewrote section to make devfs compatible.
+ *
+ * 2007-8-26: patched by Abe Skolnik based on initial code by Neil Brown
+ *            to allow boot-time assembling of MDs based on v1.x superblocks
  */
+
 static int __init md_setup(char *str)
 {
 	int minor, level, factor, fault, partitioned = 0;
 	char *pername = "";
 	char *str1;
@@ -72,12 +111,17 @@
 		printk(KERN_WARNING "md: md=%s%d - too many md initialisations\n", partitioned?"d":"", minor);
 		return 0;
 	}
 	if (ent >= md_setup_ents)
 		md_setup_ents++;
+		
+	md_setup_args[ent].vers_major = 0;
+	md_setup_args[ent].vers_minor = 90;
+
 	switch (get_option(&str, &level)) {	/* RAID level */
-	case 2: /* could be 0 or -1.. */
+	case 2:
+	case 1:
 		if (level == 0 || level == LEVEL_LINEAR) {
 			if (get_option(&str, &factor) != 2 ||	/* Chunk Size */
 					get_option(&str, &fault) != 2) {
 				printk(KERN_WARNING "md: Too few arguments supplied to md=.\n");
 				return 0;
@@ -88,19 +132,47 @@
 				pername = "linear";
 			else
 				pername = "raid0";
 			break;
 		}
-		/* FALL THROUGH */
-	case 1: /* the first device is numeric */
-		str = str1;
-		/* FALL THROUGH */
-	case 0:
+	  break;
+
+	case 0: // a number was not found at the current position, i.e. no RAID level was specified after the device minor number
+  
 		md_setup_args[ent].level = LEVEL_NONE;
 		pername="super-block";
 	}
 
+   if (str[0]=='v' && str[1] == '1') {
+
+     md_setup_args[ent].vers_major = 1;
+     md_setup_args[ent].vers_minor = 0; // the default sub-version is zero
+
+     if (str[2]=='.') {
+       switch (str[3]) {
+         case '0':
+           md_setup_args[ent].vers_minor = 0;
+           break;
+         case '1':
+           md_setup_args[ent].vers_minor = 1;
+           break;
+         case '2': 
+           md_setup_args[ent].vers_minor = 2;
+           break;
+         default:
+           // maybe printk an error msg.?
+       } // end of switch (str[3])
+     } // end of if (str[2]=='.')
+
+     // wind the string pointer past the comma after the version param.
+     while (str[0]!=',' && str[0] /* check for null too */ ) ++str;
+     if (str[0]) ++str; // push past the comma if we are not at null
+     
+   } // end of if (str[0]=='v' && str[1] == '1')
+
+	 printk(KERN_INFO "md: superblock version is assumed to be %d.%d ===\n",md_setup_args[ent].vers_major,md_setup_args[ent].vers_minor);
+
 	printk(KERN_INFO "md: Will configure md%d (%s) from %s, below.\n",
 		minor, pername, str);
 	md_setup_args[ent].device_names = str;
 	md_setup_args[ent].partitioned = partitioned;
 	md_setup_args[ent].minor = minor;
@@ -120,10 +192,11 @@
 		int fd;
 		int err = 0;
 		char *devname;
 		mdu_disk_info_t dinfo;
 		char name[16];
+		mdu_array_info_t ainfo;
 
 		minor = md_setup_args[ent].minor;
 		partitioned = md_setup_args[ent].partitioned;
 		devname = md_setup_args[ent].device_names;
 
@@ -171,21 +244,25 @@
 		if (fd < 0) {
 			printk(KERN_ERR "md: open failed - cannot start "
 					"array %s\n", name);
 			continue;
 		}
-		if (sys_ioctl(fd, SET_ARRAY_INFO, 0) == -EBUSY) {
+
+		ainfo.raid_disks = 0; /* flag just setting version */
+		ainfo.major_version = md_setup_args[ent].vers_major;
+		ainfo.minor_version = md_setup_args[ent].vers_minor;
+
+		if (sys_ioctl(fd, SET_ARRAY_INFO, (long)&ainfo) == -EBUSY) {
 			printk(KERN_WARNING
 			       "md: Ignoring md=%d, already autodetected. (Use raid=noautodetect)\n",
 			       minor);
 			sys_close(fd);
 			continue;
 		}
 
 		if (md_setup_args[ent].level != LEVEL_NONE) {
 			/* non-persistent */
-			mdu_array_info_t ainfo;
 			ainfo.level = md_setup_args[ent].level;
 			ainfo.size = 0;
 			ainfo.nr_disks =0;
 			ainfo.raid_disks =0;
 			while (devices[ainfo.raid_disks])
@@ -277,5 +354,6 @@
 			sys_close(fd);
 		}
 	}
 	md_setup_drive();
 }
+

Re: Patch for boot-time assembly of v1.x-metadata-based soft (MD) arrays

[Justin Piszcz]

On Sun, 26 Aug 2007, Abe Skolnik wrote:

> Dear Mr./Dr./Prof. Brown et al,
>
> I recently had the unpleasant experience of creating an MD array for
> the purpose of booting off it and then not being able to do so.  Since
> I had already made changes to the array's contents relative to that
> which I cloned it from, I did not want to reformat the array and
> re-clone it just to bring it down to the old 0.90 metadata format so
> that I would be able to boot off it, so I searched for a solution, and
> I found it.
>
> First I tried the patch (written by Neil Brown) which can be seen at...
>  <http://www.issociate.de/board/post/277868/>
>
> That patch did not work as-is, but with some more hacking, I got it
> working.  I then cleaned up my work and added relevant comments.
>
> I know that Mr./Dr./Prof. Brown is against in-kernel boot-time MD
> assembly and prefers init[rd/ramfs], but I prefer in-kernel assembly,
> and I think several other people do too.  Since this patch does not
> (AFAIK) disable the init[rd/ramfs] way of bringing up MDs in boot-time,
> I hope that this patch will be accepted and submitted up-stream for
> future inclusion in the mainline kernel.org kernel distribution.
> This way kernel users can choose their MD assembly strategy at will
> without having to restrict themselves to the old metadata format.
>
> I hope that this message finds all those who read it doing well and
> feeling fine.
>
> Sincerely,
>
> Abe Skolnik
>
> P.S.  Mr./Dr./Prof. Brown, in case you read this:  thanks!
>      And if you want your name removed from the code, just say so.
>

> but I prefer in-kernel assembly,
> and I think several other people do too.
I concur with this statement, why go through the hassle of init[rd/ramfs] 
if we can just have it done in the kernel?

Justin.

Re: Patch for boot-time assembly of v1.x-metadata-based soft (MD) arrays

[Dan Williams]

On 8/26/07, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>
>
> On Sun, 26 Aug 2007, Abe Skolnik wrote:
>
> > Dear Mr./Dr./Prof. Brown et al,
> >
> > I recently had the unpleasant experience of creating an MD array for
> > the purpose of booting off it and then not being able to do so.  Since
> > I had already made changes to the array's contents relative to that
> > which I cloned it from, I did not want to reformat the array and
> > re-clone it just to bring it down to the old 0.90 metadata format so
> > that I would be able to boot off it, so I searched for a solution, and
> > I found it.
> >
> > First I tried the patch (written by Neil Brown) which can be seen at...
> >  <http://www.issociate.de/board/post/277868/>
> >
> > That patch did not work as-is, but with some more hacking, I got it
> > working.  I then cleaned up my work and added relevant comments.
> >
> > I know that Mr./Dr./Prof. Brown is against in-kernel boot-time MD
> > assembly and prefers init[rd/ramfs], but I prefer in-kernel assembly,
> > and I think several other people do too.  Since this patch does not
> > (AFAIK) disable the init[rd/ramfs] way of bringing up MDs in boot-time,
> > I hope that this patch will be accepted and submitted up-stream for
> > future inclusion in the mainline kernel.org kernel distribution.
> > This way kernel users can choose their MD assembly strategy at will
> > without having to restrict themselves to the old metadata format.
> >
> > I hope that this message finds all those who read it doing well and
> > feeling fine.
> >
> > Sincerely,
> >
> > Abe Skolnik
> >
> > P.S.  Mr./Dr./Prof. Brown, in case you read this:  thanks!
> >      And if you want your name removed from the code, just say so.
> >
>
> > but I prefer in-kernel assembly,
> > and I think several other people do too.
> I concur with this statement, why go through the hassle of init[rd/ramfs]
> if we can just have it done in the kernel?
>

Because you can rely on the configuration file to be certain about
which disks to pull in and which to ignore.  Without the config file
the auto-detect routine may not always do the right thing because it
will need to make assumptions.

So I turn the question around, why go through the exercise of trying
to improve an auto-detect routine which can never be perfect when the
explicit configuration can be specified by a config file?

I believe the real issue is the need to improve the distributions'
initramfs build-scripts and relieve the hassle of handling MD details.

> Justin.

Regards,
Dan

Re: Patch for boot-time assembly of v1.x-metadata-based soft (MD) arrays

[Mr. James W. Laferriere]

On Sun, 26 Aug 2007, Justin Piszcz wrote:
> On Sun, 26 Aug 2007, Abe Skolnik wrote:
>> Dear Mr./Dr./Prof. Brown et al,
>> 
>> I recently had the unpleasant experience of creating an MD array for
>> the purpose of booting off it and then not being able to do so.  Since
>> I had already made changes to the array's contents relative to that
>> which I cloned it from, I did not want to reformat the array and
>> re-clone it just to bring it down to the old 0.90 metadata format so
>> that I would be able to boot off it, so I searched for a solution, and
>> I found it.
>> 
>> First I tried the patch (written by Neil Brown) which can be seen at...
>>  <http://www.issociate.de/board/post/277868/>
>> 
>> That patch did not work as-is, but with some more hacking, I got it
>> working.  I then cleaned up my work and added relevant comments.
>> 
>> I know that Mr./Dr./Prof. Brown is against in-kernel boot-time MD
>> assembly and prefers init[rd/ramfs], but I prefer in-kernel assembly,
>> and I think several other people do too.  Since this patch does not
>> (AFAIK) disable the init[rd/ramfs] way of bringing up MDs in boot-time,
>> I hope that this patch will be accepted and submitted up-stream for
>> future inclusion in the mainline kernel.org kernel distribution.
>> This way kernel users can choose their MD assembly strategy at will
>> without having to restrict themselves to the old metadata format.
>> 
>> I hope that this message finds all those who read it doing well and
>> feeling fine.
>> 
>> Sincerely,
>> 
>> Abe Skolnik
>> 
>> P.S.  Mr./Dr./Prof. Brown, in case you read this:  thanks!
>>      And if you want your name removed from the code, just say so.
>> 
>
>> but I prefer in-kernel assembly,
>> and I think several other people do too.
> I concur with this statement, why go through the hassle of init[rd/ramfs] if 
> we can just have it done in the kernel?
>
> Justin.

 	Motion seconded !-) .  I too agree with Abe & Justin .
 	I am not in favor with the init[rd/ramfs] layer of abstractions .

 		Twyl ,  JimL
-- 
+-----------------------------------------------------------------+
| James   W.   Laferriere | System   Techniques | Give me VMS     |
| Network        Engineer | 663  Beaumont  Blvd |  Give me Linux  |
| babydr@baby-dragons.com | Pacifica, CA. 94044 |   only  on  AXP |
+-----------------------------------------------------------------+