mdadm file system type check

mdadm file system type check

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

To whom it may concern,

	It seems mdadm does not check, warn, abort, or etc if a partition has
an incorrect file system type. This has come up for me on a few
occasions while building servers with software raid. On one occasion I
had a machine fully up and running on raid partitions that were ext3 not
linux auto raid/fd.

	If possible would really be nice if it could do a check for that. Since
one usually has to start over when they discover the problem and go to
correct it. Since raidtools is not deprecated, others migrating from it
on Gentoo are running into it at as well. At the present time mostly
developers. Really would not want users running into it :)

	Anyway if that could be addressed that would be great. Not sure if this
is the right place for this or not. If not please advise, thank you.

-- 
William L. Thomson Jr.
Gentoo/Java

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mdadm file system type check

[Neil Brown]

On Friday March 16, wltjr@gentoo.org wrote:
> To whom it may concern,
> 
> 	It seems mdadm does not check, warn, abort, or etc if a partition has
> an incorrect file system type. This has come up for me on a few
> occasions while building servers with software raid. On one occasion I
> had a machine fully up and running on raid partitions that were ext3 not
> linux auto raid/fd.

It would be very awkward for mdadm to get at the partition type
information. 
And there is very little software that actually cares.  mdadm
certainly doesn't care what the partition type is.

> 
> 	If possible would really be nice if it could do a check for that. Since
> one usually has to start over when they discover the problem and go to
> correct it. Since raidtools is not deprecated, others migrating from it
> on Gentoo are running into it at as well. At the present time mostly
> developers. Really would not want users running into it :)

I don't understand why you would have to 'start over'.  If you
discover that the partition type isn't what you want for some reason,
just change it.

(and if raidtools isn't deprecated, it should be.  raidstart certainly
is deprecated).

> 
> 	Anyway if that could be addressed that would be great. Not sure if this
> is the right place for this or not. If not please advise, thank you.

This is the right place, thanks.
But maybe I don't really understand what the problem is.
Could you give more details?  Exactly what didn't work the way you
expected it to?

Thanks,
NeilBrown

Re: mdadm file system type check

[berk walker]

William L. Thomson Jr. wrote:
> To whom it may concern,
>
> 	It seems mdadm does not check, warn, abort, or etc if a partition has
> an incorrect file system type. This has come up for me on a few
> occasions while building servers with software raid. On one occasion I
> had a machine fully up and running on raid partitions that were ext3 not
> linux auto raid/fd.
>
> 	If possible would really be nice if it could do a check for that. Since
> one usually has to start over when they discover the problem and go to
> correct it. Since raidtools is not deprecated, others migrating from it
> on Gentoo are running into it at as well. At the present time mostly
> developers. Really would not want users running into it :)
>
> 	Anyway if that could be addressed that would be great. Not sure if this
> is the right place for this or not. If not please advise, thank you.
>
>   
Mdadm has no need to know or care about file systems.  It exists on the 
hardware, /dev level, upon which file systems, if any, are built.

b-

Re: mdadm file system type check

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]

On Sat, 2007-03-17 at 12:30 +1100, Neil Brown wrote:
> 
> It would be very awkward for mdadm to get at the partition type
> information. 
> And there is very little software that actually cares.  mdadm
> certainly doesn't care what the partition type is.

I got it reversed. :) There was an error when the file system was not
linux auto raid. More on that below.

> I don't understand why you would have to 'start over'.  If you
> discover that the partition type isn't what you want for some reason,
> just change it.

It might have been that simple, it was quite some time ago. But it was a
memorable event since it stuck in my head as one of the things to check
when having problems with mdadm. But obviously not a clear memory.

> This is the right place, thanks.

Good to know ;)

> But maybe I don't really understand what the problem is.
> Could you give more details?  Exactly what didn't work the way you
> expected it to?

Instead of passing along an interpretation, here are some IRC log
snippets that pertain from #gentoo-dev @ freenode.net

kingtaco|work: livecd ~ # mdadm --create --level=1 --raid-devices=2 /dev/md0 /dev/sda1 /dev/sdb1
kingtaco|work: mdadm: /dev/sda1 is too small: 0K
kingtaco|work: mdadm: create aborted

Now despite suggesting it pretty early on, it took a bit of time to
realize that error was existing only because of file system type. Once
he realized the problem and changed fs type. All was well, and he was
able to create the arrays and move on.

kingtaco|work: wltjr, see, raidtools would have let me create an array eventhough the type wasn't fs
kingtaco|work: er, fd
kingtaco|work: mdadm just isn't that smart
kingtaco|work: (and that fixed it)

So ot to sure about the error or why they would get that just because of
file system type.

-- 
William L. Thomson Jr.
Gentoo/Java

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mdadm file system type check

[Neil Brown]

On Friday March 16, wltjr@gentoo.org wrote:
> 
> Instead of passing along an interpretation, here are some IRC log
> snippets that pertain from #gentoo-dev @ freenode.net
> 
> kingtaco|work: livecd ~ # mdadm --create --level=1 --raid-devices=2 /dev/md0 /dev/sda1 /dev/sdb1
> kingtaco|work: mdadm: /dev/sda1 is too small: 0K
> kingtaco|work: mdadm: create aborted
> 
> Now despite suggesting it pretty early on, it took a bit of time to
> realize that error was existing only because of file system type. Once
> he realized the problem and changed fs type. All was well, and he was
> able to create the arrays and move on.
> 
> kingtaco|work: wltjr, see, raidtools would have let me create an array eventhough the type wasn't fs
> kingtaco|work: er, fd
> kingtaco|work: mdadm just isn't that smart
> kingtaco|work: (and that fixed it)
> 
> So ot to sure about the error or why they would get that just because of
> file system type.


This failure has nothing do to with the partition type. As I
mentioned, mdadm doesn't see the partition type at all.

When you change the partition type, fdisk (or whatever) normally tells
the kernel to reread the partition table.  It would seem that the
partition table hadn't been read properly before this point, so
/dev/sda1 didn't exist.  After running fdisk that partition table had
been read by the kernel and everything was fine.
I.e. it was using fdisk to write the partition table that 'fixed' the
problem, not changing the partition type.  And the problem was not
causes by mdadm but by something else not setting up the partitions
properly (or removing them maybe).

NeilBrown

Re: mdadm file system type check

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]

On Sat, 2007-03-17 at 16:40 +1100, Neil Brown wrote:
> On Friday March 16, wltjr@gentoo.org wrote:
> > 
> > Instead of passing along an interpretation, here are some IRC log
> > snippets that pertain from #gentoo-dev @ freenode.net
> > 
> > kingtaco|work: livecd ~ # mdadm --create --level=1 --raid-devices=2 /dev/md0 /dev/sda1 /dev/sdb1
> > kingtaco|work: mdadm: /dev/sda1 is too small: 0K
> > kingtaco|work: mdadm: create aborted
>
> This failure has nothing do to with the partition type. As I
> mentioned, mdadm doesn't see the partition type at all.

Not to contradict but only thing that was changed and the problem was fs
type.

> When you change the partition type, fdisk (or whatever) normally tells
> the kernel to reread the partition table.  It would seem that the
> partition table hadn't been read properly before this point, so
> /dev/sda1 didn't exist.

Not to sure about that, part that I did not post after the above snippet
still left, and before the other one I removed.

wltjr: kingtaco|work: likely something else, partition off, wrong fs type, or etc
kingtaco|work: /dev/sda1 1 17 136521 83 Linux
kingtaco|work: /dev/sdb1 1 17 136521 83 Linux

So they for sure existed 83/ext3, and were changed to fd.

>   After running fdisk that partition table had
> been read by the kernel and everything was fine.

That's likely, but the kernel was aware of the partitions prior to the
fs type change.

> I.e. it was using fdisk to write the partition table that 'fixed' the
> problem, not changing the partition type.  And the problem was not
> causes by mdadm but by something else not setting up the partitions
> properly (or removing them maybe).

Well we can rule it out to funky partition creation or etc. But the
partitions were just created via fdisk. With the next logical step being
to setup the array, format, etc. So that the partitions being off, and
re-running fdisk updated kernel or etc is all possible. But kinda
doubtful or odd at the least.

Unless udev or etc was doing something once the partition type changed.
So the problem wasn't with mdadm but something else that detected the
change in fs type.

-- 
William L. Thomson Jr.
Gentoo/Java

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mdadm file system type check

[Chris Lindley]

What I think the OP is getting at is that MDADM will create an array
with partitions whose type is not set to FD (Linux Raid Auto), but are
perhaps 83.

The issue with that is that upon a reboot mdadm will not be able to
start the array.  If you use MDADM to manually reassemble the array then
it will work fine. But until you reset the partition type to be FD, you
will have to run this step every time you reboot the machine.

Please note that I'm not disagreeing or agreeing whether the OP's idea
was a good idea, just trying to clarify the issue!

Cheers
Ferg

-- 
                               
  scotgate.org AIM#fergycool jabber#fergy IRC#Ferg MSN#fergycool
  Climb up it, kayak down it + make sure it runs on GNU/Linux
  "cease to exist, giving my goodbye, drive my car into the ocean,
  you think I'm dead, but i sail away, on a wave of mutilation!"





On Fri, 2007-03-16 at 20:39 -0500, berk walker wrote:
> 
> William L. Thomson Jr. wrote:
> > To whom it may concern,
> >
> > 	It seems mdadm does not check, warn, abort, or etc if a partition has
> > an incorrect file system type. This has come up for me on a few
> > occasions while building servers with software raid. On one occasion I
> > had a machine fully up and running on raid partitions that were ext3 not
> > linux auto raid/fd.
> >
> > 	If possible would really be nice if it could do a check for that. Since
> > one usually has to start over when they discover the problem and go to
> > correct it. Since raidtools is not deprecated, others migrating from it
> > on Gentoo are running into it at as well. At the present time mostly
> > developers. Really would not want users running into it :)
> >
> > 	Anyway if that could be addressed that would be great. Not sure if this
> > is the right place for this or not. If not please advise, thank you.
> >
> >   
> Mdadm has no need to know or care about file systems.  It exists on the 
> hardware, /dev level, upon which file systems, if any, are built.
> 
> b-
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm file system type check

[Bill Davidsen]

William L. Thomson Jr. wrote:
> On Sat, 2007-03-17 at 12:30 +1100, Neil Brown wrote:
>   
>> It would be very awkward for mdadm to get at the partition type
>> information. 
>> And there is very little software that actually cares.  mdadm
>> certainly doesn't care what the partition type is.
>>     
>
> I got it reversed. :) There was an error when the file system was not
> linux auto raid. More on that below.
>
>   
>> I don't understand why you would have to 'start over'.  If you
>> discover that the partition type isn't what you want for some reason,
>> just change it.
>>     
>
> It might have been that simple, it was quite some time ago. But it was a
> memorable event since it stuck in my head as one of the things to check
> when having problems with mdadm. But obviously not a clear memory.
>
>   
>> This is the right place, thanks.
>>     
>
> Good to know ;)
>
>   
>> But maybe I don't really understand what the problem is.
>> Could you give more details?  Exactly what didn't work the way you
>> expected it to?
>>     
>
> Instead of passing along an interpretation, here are some IRC log
> snippets that pertain from #gentoo-dev @ freenode.net
>
> kingtaco|work: livecd ~ # mdadm --create --level=1 --raid-devices=2 /dev/md0 /dev/sda1 /dev/sdb1
> kingtaco|work: mdadm: /dev/sda1 is too small: 0K
> kingtaco|work: mdadm: create aborted
>
> Now despite suggesting it pretty early on, it took a bit of time to
> realize that error was existing only because of file system type. Once
> he realized the problem and changed fs type. All was well, and he was
> able to create the arrays and move on.
>   
First, please learn the difference between file system type (user data 
ON the device), and partition type (a byte in the partition table). It 
will at least let people understand your question...

Second, mdadm cares not a bit about the partition type, although some 
init scripts might. The reason it didn't work was because the partition 
was zero size or missing, and would not work regardless of the partition 
type.
> kingtaco|work: wltjr, see, raidtools would have let me create an array eventhough the type wasn't fs
> kingtaco|work: er, fd
> kingtaco|work: mdadm just isn't that smart
> kingtaco|work: (and that fixed it)
>
> So ot to sure about the error or why they would get that just because of
> file system type.
>
>   
If raidtools creates or resizes partitions, that's certainly not 
documented, nor is it in any way "smart," since it's guessing what you 
want to do and where you want to do it.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: mdadm file system type check

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]

On Sat, 2007-03-17 at 13:10 -0500, Bill Davidsen wrote:
>
> First, please learn the difference between file system type (user data 
> ON the device), and partition type (a byte in the partition table).

Which it's technical name would be a partition system ID. However older
versions of fdisk did refer to the t option as file system type.
http://www.redhat.com/docs/manuals/linux/RHL-7.3-Manual/ref-guide/s1-filesystem-ext3-create.html

Given it's quite old and all newer docs refer to it mainly as partition
system id. At least that's what's shown in current fdisks

t   change a partition's system id

Sorry for the miss-clarification. For the record, simply changing the
partitions system id from 83 -> fd allowed the commands that previously
failed to succeed and create the array.

> Second, mdadm cares not a bit about the partition type,

That would be my understanding of it so far, and experience with it.

>  although some 
> init scripts might.

FYI, this was a new install running on a livecd. So no init scripts were
involved.

>  The reason it didn't work was because the partition 
> was zero size or missing, and would not work regardless of the partition 
> type.

Negative, it's one of the things I suggested for them to take a look at.
Even provided the snippet of the IRC log that I left out containing that
information in another post. To save you from having to dig it out. Here
it is again :)

wltjr: kingtaco|work: likely something else, partition off, wrong fs type, or etc
kingtaco|work: /dev/sda1 1 17 136521 83 Linux
kingtaco|work: /dev/sdb1 1 17 136521 83 Linux

Also I really would not be reporting this, or wasting any one's time if
it was not something abnormal. Thus reporting it, and my
miss-clarification of stuff surely did not help explain the situation :)

-- 
William L. Thomson Jr.
Gentoo/Java

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mdadm file system type check

[Nix]

On 17 Mar 2007, Chris Lindley told this:

> What I think the OP is getting at is that MDADM will create an array
> with partitions whose type is not set to FD (Linux Raid Auto), but are
> perhaps 83.
>
> The issue with that is that upon a reboot mdadm will not be able to
> start the array.

I think you mean that the Linux kernel's auto-assembly code won't be
able to start the array. mdadm doesn't care.

>                   If you use MDADM to manually reassemble the array then
> it will work fine. But until you reset the partition type to be FD, you
> will have to run this step every time you reboot the machine.

That's what initramfs/initrd is good at :)

-- 
`In the future, company names will be a 32-character hex string.'
  --- Bruce Schneier on the shortage of company names