intel fakeraid (imsm) linux kernel support

intel fakeraid (imsm) linux kernel support

[K. Posern]

[-- Attachment #1: Type: text/plain, Size: 2382 bytes --]

Hi all,

I am trying to find out if it is possible to:

	a) Have the linux kernel autodetect / auto-assemble the fakeraid (to 
avoid having an initrd (for speed and KISS))

	b) Boot linux kernel from a fakeraid this: grub2? lilo? ... anything?	

About (a):
Is it possible to autodetect any intel fakeraid (imsm)?
If yes: What RAID-Levels are supported? Especially what would I need to 
do to get my RAID-0 auto-assembled?
If no: Are there plans?

About (b):
What boot-loader support to assemble the intel fakeraid to read the 
kernel from it - e.g. are able to assemble the /boot partition on the 
RAID (especially: level 0)?
According to this wiki entry:
	http://wiki.neuralbs.com/index.php/Gentoo_Quick_Install_notes#GRUB2
grub2 should support imsm (but grep for imsm in the Bazaar checkout does 
not yield any hit) <<< I also contacted the grub2 mailing-list for 
clarification
I learned today that Ubuntu seems to be able to boot from a RAID-0 with 
grub2 (v1.98-1ubuntu7) and seemingly installed to /dev/mapper/isw_*.
So far I only tried to install vanilla grub 1.98 to the mdadm v3 devices 
/dev/md12[67], without success.
In the documentation of grub2 (afaik) they don't distinguish between 
mdadm/dmraid software raid, they just state that grub2 works with 
software raid.

Side-Question:
To be able to have the kernel auto-assemble / autodetect any [e.g. 
mdadm] linux software RAID-0 you need *partions*, right?
--> You can't assemble a software raid-0 using two complete drives:
	/dev/sda
	/dev/sdb
But you need to create e.g.:
	/dev/sda1 <<< type set to FD
	/dev/sdb1 <<< type set to FD


It would be great and very kind if someone could clarify these things 
for me (just a link to read the answers somewhere would work too ;).
Thanks a lot!


Knuth

P.S.:
I adore Intels engagement for linux. Unfortunately is there no website 
(yet?) "intellinuxraid.org" (like there is: intellinuxgraphics.org).

The only official information from intel (google: intel raid linux) I 
could dig up so fare states: imsm is supported with mdadm >=v3 and 
dmraid >=v1. That leaves a lot of questions (eg. I never saw any details 
on how the intel software raid format works - e.g. I noticed that the 
first physical device of my RAID-0 seems to have the primary paritions, 
all the other RAID-0 devices are unpartitioned).



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3644 bytes --]

Re: [mdadm git pull] "--assemble --scan" support for imsm

[Dan Williams]

[ added linux-raid with permission ]

On 8/16/2010 5:17 PM, K. Posern wrote:
> Dear Mr. Williams,
>
> I am a bit desperate that's why I dare to contact you like this:
> As I did not find any answer to my below 2 questions in man-pages, in
> the source-code (quick look with grep+less), on irc channels and of
> course with google, I am hoping that you can give me any hint or refer
> me to somebody who might.
>
> I adore Intels engagement for linux. Unfortunately is there no website
> (yet?): "intellinuxraid.org" (like there is: intellinuxgraphics.org).

There is no website.  My personal thought is that we should extend the 
content at raid.wiki.kernel.org.

> The only official information from intel I could dig up so fare states:
> imsm is supported with mdadm >=v3 and dmraid >=v1
>
> But I am trying to find out if it is possible to:
>
> a) Autoassemble the RAID (so avoid having an initrd)

No, the auto-assemble I mentioned in the pull request will not avoid 
having an initrd.  You are thinking of "in-kernel autodetect" which is 
not supported with the imsm metadata format (or any metadata format 
besides the md-v0.90 format).  In general assembling arrays in userspace 
is much more flexible which is why you see the in-kernel autodetect code 
has been put in a no-new-features/maintenance-only mode.

I know Fedora and OpenSUSE have added support for this configuration to 
their initrd tools.

> b) BOOT linux from this: grub2? lilo? ... anything?

One of the primary benefits of using the Intel(R) RST (mdadm-imsm) 
format is that you can use the platform bios option-rom to boot from 
raid.  So grub, grub2, lilo, or any other bootloader can boot from raid. 
  This is because the option-rom installs a bios int13 handler that 
allows the bootloader to read from a raid volume as if it were a 
standalone disk.

> About (a):
> The git pull request seems to indicate thate there is auto-assembly
> support for imsm software raids?!
> Is this true? - again: grep

The "mdadm -As" command will search all available disks and 
automatically assemble all the imsm arrays that it finds.

> What RAID-Levels (for an imsm fakeraid) are supported?

I think the 'fakeraid' moniker is tired and misplaced when referring to 
this new functionality in md/mdadm.  There is nothing fake about Linux 
software RAID.  I prefer "bios-raid" or even "software raid".

Current platforms ship support for raid levels 0, 1, 10, and 5.

> Especially: What about RAID-0?

Supported.

>
> About (b):
> According to "Kyron" and his wiki:
> http://wiki.neuralbs.com/index.php/Gentoo_Quick_Install_notes#GRUB2
> grub2 should support imsm (but grep for imsm in the Bazaar checkout does
> not yield any hit) <<< I contacted the grub2 mailing-list for clarification

Like I mentioned above, the bootloader does not need any special code to 
boot from a bios-raid array because it can use standard int13 services.

> I really hope you can help me. Most importantly with question (a).
>
>
> Yours sincerely,
>
> K. Posern
>

Regards,
Dan

Re: intel fakeraid (imsm) linux kernel support

[Dan Williams]

On Tue, Aug 17, 2010 at 9:19 AM, K. Posern <quickhelp@gmail.com> wrote:
> Hi all,
>
> I am trying to find out if it is possible to:
>
>        a) Have the linux kernel autodetect / auto-assemble the fakeraid (to
> avoid having an initrd (for speed and KISS))
>
>        b) Boot linux kernel from a fakeraid this: grub2? lilo? ... anything?

Hi, I think I answered these in my other response:

http://marc.info/?l=linux-raid&m=128206374710903&w=2

...but you specifically ask about partition-type-0xfd below:

[..]
> Side-Question:
> To be able to have the kernel auto-assemble / autodetect any [e.g. mdadm]
> linux software RAID-0 you need *partions*, right?

You need partitions and the v0.90 metadata format.

> --> You can't assemble a software raid-0 using two complete drives:

You can, but not if you want to autodetect it.  But just use an initrd
and be happy :-)

>        /dev/sda
>        /dev/sdb
> But you need to create e.g.:
>        /dev/sda1 <<< type set to FD
>        /dev/sdb1 <<< type set to FD
>
>
> It would be great and very kind if someone could clarify these things for me
> (just a link to read the answers somewhere would work too ;).
> Thanks a lot!

One of the biggest differences between the native-md (0.90 and 1.x)
formats and external (imsm and ddf) formats, is that the latter only
use complete drives.  For example if you wanted two raid0 arrays on
the same set of disks using a native-md format you would need to
partition the disks first and then:

mdadm -C /dev/md0 /dev/sd[a-d]1 -n 4 - l 0
mdadm -C /dev/md1 /dev/sd[a-d]2 -n 4 -l 0

...with the external formats you create a container of disks and then
carve multiple volumes out of that set:
mdadm -C /dev/md/imsm /dev/sd[a-d] -n 4 -e imsm # create an imsm container
mdadm -C /dev/md0 /dev/md/imsm -n 4 -l 0 -z $size # create a raid0
array in the container
mdadm -C /dev/md1 /dev/md/imsm -n 4 -l 0 # use the remaining space for
a second raid0 volume

The bios (on an enabled platform) will see /dev/md0 and /dev/md1 as
bootable devices, not the individual component disks (/dev/sd[a-d]).

--
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [mdadm git pull] "--assemble --scan" support for imsm

[Jiang, Dave]

> -----Original Message-----
> From: Williams, Dan J
> Sent: Tuesday, August 17, 2010 9:49 AM
> To: K. Posern
> Cc: linux-raid@vger.kernel.org; Jiang, Dave
> Subject: Re: [mdadm git pull] "--assemble --scan" support for imsm
> 
> [ added linux-raid with permission ]
> 
> On 8/16/2010 5:17 PM, K. Posern wrote:
> > Dear Mr. Williams,
> >
> > I am a bit desperate that's why I dare to contact you like this:
> > As I did not find any answer to my below 2 questions in man-pages, in
> > the source-code (quick look with grep+less), on irc channels and of
> > course with google, I am hoping that you can give me any hint or
> refer
> > me to somebody who might.
> >
> > I adore Intels engagement for linux. Unfortunately is there no
> website
> > (yet?): "intellinuxraid.org" (like there is: intellinuxgraphics.org).
> 
> There is no website.  My personal thought is that we should extend the
> content at raid.wiki.kernel.org.

I have updated raid.wiki with the external metadata information a few weeks back. Please take a look and let me know if additional information is required. Thanks!

Re: intel fakeraid (imsm) linux kernel support

[K. Posern]

[-- Attachment #1: Type: text/plain, Size: 4786 bytes --]

Dear Mr. Williams,

Thank you *very* much... you added very valueable pieces to my imsm 
RAID(-0) puzzle.

I just used the term "fakeraid" because it is *short* :) ... I did not 
mean it negative. Plus a lot of people use this term:
	https://secure.wikimedia.org/wikipedia/en/wiki/RAID#Firmware.2Fdriver-based_RAID_.28.22FakeRAID.22.29
	The Ubuntu FakeRAID HowTo
	ArchLinux FakeRaid Howto

I would really like a new point on the raid-wikis first page named along 
these lines:
	"[Intel] Firmware-based/BIOS-supported RAID ("FakeRAID")"
And I can do this if you want (aready created account yesterday).

On there should be a
	- short description about what what a "fakeraid" is
	- then for INTEL the info you get when you google for "intel linux raid"
	- the info from Dan's answer below. Especially the part with "the 
option-rom installs a bios int13 handler" (because there /is/ confusion 
about that out there :).
	- Also the answer to the belows question: "How to install a bootloader" 
should be on there.

And of course the more technical details the better. Eg.
	- that (and maybe why) the primary partitions setup within the imsm 
raid(-0) will be created on the first physical drive (/dev/sda), but not 
on the other drives.
	- maybe even more in depth details on the external metadata format
	- maybe some commonly used commandlines for mdadm and dmraid to check 
the raid status ... or at least links to their howtos/wikis...

You mentioned:
  > One of the primary benefits of using the Intel(R) RST (mdadm-imsm)
  > format is that you can use the platform bios option-rom to boot from
  > raid.
2 questions:
	a) What does RST stand for?
	b) Does the (mdadm-imsm) mean, that the device-mappers "dmraid -f isw" 
does not the same (or does intel just officially only support mdadm imsm?)

And finally: Can anyone tell me which solution:
	dmraid -f isw
	mdadm with imsm container support
is better in the sense:
	- faster (higher throughput)
	- less load (e.g. on cpu)
	- more stable
	- more features
I already posted this question to the dm-devel@redhat.com mailing list 
but (afaik) never got an answer.

Also: What I find odd with the dmraid solution: In fdisk it shows you 
e.g. /dev/dm-0p6, but there is no /dev/dm-0p6, there is only /dev/dm-0 
to /dev/dm-11 and many /dev/mapper/LONGNAME symlinks to the /dev/dm-*. 
But the /dev/mapper/Volume_HASH_06 points to /dev/dm-4 ?!.... IMHO: 
total mess.

--------------

"How to install a bootloader"?

You mentioned:
 > One of the primary benefits of using the Intel(R) RST (mdadm-imsm)
 > format is that you can use the platform bios option-rom to boot from
 > raid. So grub, grub2, lilo, or any other bootloader can boot from raid.
 > This is because the option-rom installs a bios int13 handler that allows
 > the bootloader to read from a raid volume as if it were a standalone 
disk.

Which sounds great, but I can send you the chatlog from the #grub 
channel this morning... they don't seem to have an idea about that.

Could you/someone be so kind to provide me with the steps to manually 
install grub (legacy v0.97) or grub2 (v1.98) on a linux that accesses 
the raid via mdadm-imsm (/dev/md126, /dev/md127)?

Because for me it seems: "Intel" says: grub, gub2: No problem. "Grub" 
says: ???
I know that this is not a grub mailing list, but maybe just a small 
hint? Like: What should be in /boot/grub/device.map?
/dev/md126?
Do I install grub to the container (/dev/md127) or to the volume 
(/dev/md126)?
E.g.
- grub legacy gave me a "Error 22" on "setup (hd0)" with "/dev/md126 as 
(hd0) in device.map
- grub2: 1st it seems unclear what should be in the /dev/md, because 
grub assembles /dev/mdadm raids (independent of the BIOS), 2nd it seems 
unclear how this internal mdadm assembly interferes with "accessing the 
RAID via the int13 handler as a standalone disk"
- lilo: I would prefer a grub based solution

Again: Maybe this information is somewhere out there, but that's the 
problem: There are bits and pieces [about Intel BIOS supported RAID 
imsm] all over the place --> That's why I would love to (help to) create 
this wiki page I mentioned earlier to just assemble the most basic 
things in one place.
.. Also that next time a guy like me (with a lot of half-knowledge ;) 
comes along and asks many "stupid" questions about imsm integration you 
can hopefully just point him to this wiki page.
Does this make sense to you too?


Any hint will be greatly appreciated!

K. Posern

--

On 17/08/10 12:49, Dan Williams wrote:
> [ added linux-raid with permission ]
>
> On 8/16/2010 5:17 PM, K. Posern wrote:
>> Dear Mr. Williams,
>
... <relevant parts pasted above>
> Regards,
> Dan




[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3644 bytes --]