New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Thomas Horsten]

Hi,

I've started a mailing list for those interested in the development of a Linux 
driver supporting Medley RAID (the BIOS/software RAID used by the Silicon 
Image 3112 and some CMD chipsets).

My main goal is that I want autodetection for these drives to work in 2.6 
(like the ataraid framework in 2.4), but also to leverage the code in the 
existing MD and DM drivers as far as possilble. This currently works in the 
current 2.4 kernels, thanks to Arjan van de Ven's ataraid framework, which my 
Medley driver for 2.4 uses.

The reason I insist on autodetection is that I think it's important that if 
the BIOS will reckognise the drive without additional intervention, so will 
Linux. This will make the entry route for newbies much simpler.

If you are interested in taking part in the discussions/development of the 2.6 
driver for Medley RAID (and other BIOS "wholedisk" RAID arrays, such as 
Highpoint), I would ask you to join this list.

On the new list we'll also welcome support questions for the 2.4 Medley driver 
and other related issues. I expect this to be a low volume list.

You can find more information about how to subscribe at 
http://lists.infowares.com/cgi-bin/listinfo/medley

More informateion about the 2.4 driver, and the patch source, is at 
http://www.infowares.com/linux/

// Thomas

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Neil Brown]

On Monday February 9, thomas@horsten.com wrote:
> Hi,
> 
> I've started a mailing list for those interested in the development of a Linux 
> driver supporting Medley RAID (the BIOS/software RAID used by the Silicon 
> Image 3112 and some CMD chipsets).

Why a new list?  Why not just start a discussion on linux-raid (or
linux-kernel) and if there is enough interest/traffic to annoy the
spectators, then fork off a new list.

NeilBrown

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Thomas Horsten]

On Monday 09 February 2004 02:47, Neil Brown wrote:

> Why a new list?  Why not just start a discussion on linux-raid (or
> linux-kernel) and if there is enough interest/traffic to annoy the
> spectators, then fork off a new list.

There already is, in my personal mailbox :) Hence a list for this topic.

// Thomas

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 361 bytes --]

On Mon, 2004-02-09 at 02:23, Thomas Horsten wrote:

> 
> The reason I insist on autodetection is that I think it's important that if 
> the BIOS will reckognise the drive without additional intervention, so will 
> Linux. This will make the entry route for newbies much simpler.


do you call running devicemapper tools from the initrd autodetection ?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Thomas Horsten]

On Mon, 9 Feb 2004, Arjan van de Ven wrote:

> > The reason I insist on autodetection is that I think it's important that if
> > the BIOS will reckognise the drive without additional intervention, so will
> > Linux. This will make the entry route for newbies much simpler.
>
> do you call running devicemapper tools from the initrd autodetection ?

Probably not. I am working with several ways of doing it, and that's why I
wanted to have a discussion about this.

Ideally I'd want something like the MD autodetect code, so that the whole
thing can be set up by the kernel at boot-time if the necessary drivers
are compiled in (by reading the Medley superblock the same way it's done
for 0xfe partitions). And if the drivers are modules, then use the drive
mapper tools from userspace to set it up.

This would be the most flexible solution, since it'd allow you to boot
without an initrd if you wanted that, or to use the initrd and map the
drives manually if you preferred that solution.

Having autodetection at kernel level would make it possible to boot from a
kernel on a floppy disk without initrd support, and in general make a
system easier to set up.

But the reason I wanted this discussion is to figure out the best way to
go about it, and if there are some good arguments against autodetecting in
the kernel I'll listen to them.

// Thomas

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

On Mon, Feb 09, 2004 at 12:01:55PM +0000, Thomas Horsten wrote:
> On Mon, 9 Feb 2004, Arjan van de Ven wrote:
> 
> > > The reason I insist on autodetection is that I think it's important that if
> > > the BIOS will reckognise the drive without additional intervention, so will
> > > Linux. This will make the entry route for newbies much simpler.
> >
> > do you call running devicemapper tools from the initrd autodetection ?
> 
> Probably not. I am working with several ways of doing it, and that's why I
> wanted to have a discussion about this.
> 
> Ideally I'd want something like the MD autodetect code, so that the whole
> thing can be set up by the kernel at boot-time if the necessary drivers
> are compiled in (by reading the Medley superblock the same way it's done
> for 0xfe partitions).

I (and I suspect a lot of other folks) rather get rid of such autodetect and
move it to userspace. Either via initrd or initramfs.

> Having autodetection at kernel level would make it possible to boot from a
> kernel on a floppy disk without initrd support, and in general make a
> system easier to set up.

initrd/initramfs is increasingly becoming mandatory sort of, and it's
actually easy if not even default to set up. (Eg on Fedora / Red Hat even
just typing make install will auto-create this for you)

> But the reason I wanted this discussion is to figure out the best way to
> go about it, and if there are some good arguments against autodetecting in
> the kernel I'll listen to them.

It doesn't really belong there.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Thomas Horsten]

On Mon, 9 Feb 2004, Arjan van de Ven wrote:

> On Mon, Feb 09, 2004 at 12:01:55PM +0000, Thomas Horsten wrote:
> > Ideally I'd want something like the MD autodetect code, so that the whole
> > thing can be set up by the kernel at boot-time if the necessary drivers
> > are compiled in (by reading the Medley superblock the same way it's done
> > for 0xfe partitions).
>
> I (and I suspect a lot of other folks) rather get rid of such autodetect and
> move it to userspace. Either via initrd or initramfs.

The question is, where do we draw the line between kernel and userspace
setup of devices. For example, why is the frame buffer device detected in
the kernel, it could just as well be done in the initrd.

My gut feeling is that if it is provided by the BIOS and reliable
autodetection is possible, it should be autodetected. Why require the user
to discover and supply information that the kernel could easily and
reliably find out by itself? Besides, if there is no autodetection, the
drive will come up with an invalid partition table (since the partition
tables for these arrays are stored as the first block of the first disk in
the array). If there is an extended partition, the kernel may try to read
past the end of the disk, causing a lot of spurious error messages that
confuse the user.

If the user tries to mount any of these partitions, or edit the partition
table, they will probably corrupt the disk.

What I have in mind currently is a solution that uses the md and dm
frameworks. Basically an option that adds support for reading and parsing
the Medley superblock, and create an md raiddev based on it (using the
standard RAID0/RAID1 personalities). Then a dm drive needs to be
registered, to support the partition table on the whole disk array.

For the autodetection to work, when compiled into the kernel I would put a
call into gendisk.c, just before it calls add_disk to parse the partition
table. That way, if the Medley superblock is detected it will not try to
parse the partition table.

The only thing I don't like about this solution is that we are relying on
both the md and dm framework. I can't see an easy way around this, since
we need to parse the partition table on the RAID. If we just create a
separate md device for each partition, the user won't be able to change
the partition table.

> > Having autodetection at kernel level would make it possible to boot from a
> > kernel on a floppy disk without initrd support, and in general make a
> > system easier to set up.
>
> initrd/initramfs is increasingly becoming mandatory sort of, and it's
> actually easy if not even default to set up. (Eg on Fedora / Red Hat even
> just typing make install will auto-create this for you)

It's sort of mandatory for generic systems like distro installer disks,
livecd's etc. But I've never needed to use it on any of my own systems
after I compile a cutom kernel. I guess a lot of people are like me, and
prefer to keep it simple with the essential drivers to get my specific
system running compiled in, and the rest as modules.

> > But the reason I wanted this discussion is to figure out the best way to
> > go about it, and if there are some good arguments against autodetecting in
> > the kernel I'll listen to them.
>
> It doesn't really belong there.

But where to draw the line?

I think if it can be implemented cleanly (as cleanly as the current md
detection at least), and if it can be nearly 100% reliable, it doesn't add
much complexity to the kernel and removes a lot from userspace.

// Thomas

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Jeff Garzik]

Thomas Horsten wrote:
> My gut feeling is that if it is provided by the BIOS and reliable
> autodetection is possible, it should be autodetected. Why require the user
> to discover and supply information that the kernel could easily and
> reliably find out by itself? Besides, if there is no autodetection, the

The autodetection will occur, reliably and without additional user 
information, from initramfs.

As Arjan said, we are moving this type of stuff out of the kernel.

	Jeff

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Dominik Kubla]

On Mon, Feb 09, 2004 at 11:57:02AM -0500, Jeff Garzik wrote:
> Thomas Horsten wrote:
> >My gut feeling is that if it is provided by the BIOS and reliable
> >autodetection is possible, it should be autodetected. Why require the user
> >to discover and supply information that the kernel could easily and
> >reliably find out by itself? Besides, if there is no autodetection, the
> 
> The autodetection will occur, reliably and without additional user 
> information, from initramfs.
> 
> As Arjan said, we are moving this type of stuff out of the kernel.
> 
> 	Jeff

Is there any existing code using initramfs? So far all i have seen is
very incomplete.  I would love to move to initramfs and get rid of
mkinitrd, but too many pieces seem to be still missing...

Regards,
  Dominik Kubla
-- 
The more cordial the buyer's secretary, the greater the odds that the
competition already has the order.

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Neil Brown]

On Monday February 9, thomas@horsten.com wrote:
> The only thing I don't like about this solution is that we are relying on
> both the md and dm framework. I can't see an easy way around this, since
> we need to parse the partition table on the RAID. If we just create a
> separate md device for each partition, the user won't be able to change
> the partition table.

2.6.3-rc1-mm1 supports partitioning of md arrays though a new major
number.  I'm hopeful that this will get into 2.6.4 at least.

NeilBrown

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Thomas Horsten]

On Mon, 9 Feb 2004, Arjan van de Ven wrote:

> It doesn't really belong there.

I didn't really know about how initramfs was supposed to work, now I've
read up on it and I agree that it is the best solution.

I'll do the ataraid detection userside, but I am still not sure whether
some additions to the device mapper might be needed.

For Medley RAID it's pretty straightforward to map striped sets, but how
to deal with ataraids like Highpoint where they use non-standard zone
models for the RAID0 sets?

// Thomas

Re: New mailing list for 2.6 Medley RAID (Silicon Image 3112 etc.) BIOS RAID development

[Neil Brown]

On Monday February 9, thomas@horsten.com wrote:
> 
> For Medley RAID it's pretty straightforward to map striped sets, but how
> to deal with ataraids like Highpoint where they use non-standard zone
> models for the RAID0 sets?

Can you point me at documentation of code that gives the details of
this "non-standard zone model" ??

NeilBrown