metadata 1.2

metadata 1.2

[Piergiorgio Sartor]

Hi all,

quick question: what's the use case for metadata 1.2?

I read that the 4K offset is intended to leave space
for the MBR and bootloader, I assume.
The subtext here seems to be a partionless configuration
with RAID, but I'm not aware of any bootloader which
can boot from RAID and fit into 4K.

Clarifications?

Thanks a lot in advance,

bye,

-- 

piergiorgio

Re: metadata 1.2

[Michael Evans]

On Fri, Mar 26, 2010 at 12:43 PM, Piergiorgio Sartor
<piergiorgio.sartor@nexgo.de> wrote:
> Hi all,
>
> quick question: what's the use case for metadata 1.2?
>
> I read that the 4K offset is intended to leave space
> for the MBR and bootloader, I assume.
> The subtext here seems to be a partionless configuration
> with RAID, but I'm not aware of any bootloader which
> can boot from RAID and fit into 4K.
>
> Clarifications?
>
> Thanks a lot in advance,
>
> bye,
>
> --
>
> piergiorgio
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Even grub 'legacy' 0.9x can perform in the following configuration:

Several whole device in a RAID-1 units which are partitioned via some
other means internally.  Grub can map the package it needs to load in
to memory as a series of absolute block locations and embed that
information within even the normal boot-loader area, or boot-loader
area and 1/2 more sectors with ease.

This may be more difficult with raid-10 or raid-0 sets, and
considerably more difficult with raid-456 (grub would not have
redundancy at it's level of operation, but you could use an external
system and still effect data /recovery/ at which point it'd then
work...); however I see no reason it would be technically impossible,
though I don't expect it to have been a current usage consideration.

Oh, it's also possible to chainload, so the 1.2 format offers the same
benefits on partitions if you need to operate within the constraints
of other existing boot-loaders.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata 1.2

[Piergiorgio Sartor]

Hi,

> Even grub 'legacy' 0.9x can perform in the following configuration:
> 
> Several whole device in a RAID-1 units which are partitioned via some
> other means internally.  Grub can map the package it needs to load in
> to memory as a series of absolute block locations and embed that
> information within even the normal boot-loader area, or boot-loader
> area and 1/2 more sectors with ease.
> 
> This may be more difficult with raid-10 or raid-0 sets, and
> considerably more difficult with raid-456 (grub would not have
> redundancy at it's level of operation, but you could use an external
> system and still effect data /recovery/ at which point it'd then
> work...); however I see no reason it would be technically impossible,
> though I don't expect it to have been a current usage consideration.

well, I just tried grub2, but it seems the raid.mod
is a bit bigger than 5K, so it is not possible to put
it in the 4K left.
Not to mention LVM on RAID, which makes, in lvm.mod
and raid.mod, more or less 12K.
Actually, grub-install itself complains that the
embedding is not possible.

The "default" core.img of grub2 is around about 28K,
which is a little bit less than the 32K normal
partiniong leaves between the MBR and the actual data.

Bottom line is that, it seems to me, it is impossible
to have a partionless system with RAID, let's say, 5.
I wondered why 4K?

So, would it be better to have a metadata 1.3 with a 32K
offset for the superblock, so there could be enough space
for something like grub2?

Actually, I would like to propose metadata 2.0, or
dual RAID, where the superblock is located somewhere
in the disk(s), the space before will have one type
of RAID and the space after possibly another type.
The use case will be: RAID-1 before, RAID-x after, so
a clever bootmanager could sit in the space before,
the system will be partionless, the hot-plug add,
which seems a topic, will be by far easier and we
will get rid off the legacy partioning thing (LVM
or mdp will do the trick).

> Oh, it's also possible to chainload, so the 1.2 format offers the same
> benefits on partitions if you need to operate within the constraints
> of other existing boot-loaders.

Wouldn't this still require partions somewhere?

bye,

-- 

piergiorgio

Re: metadata 1.2

[Michael Evans]

On Sat, Mar 27, 2010 at 2:16 AM, Piergiorgio Sartor
<piergiorgio.sartor@nexgo.de> wrote:
> Hi,
>
>> Even grub 'legacy' 0.9x can perform in the following configuration:
>>
>> Several whole device in a RAID-1 units which are partitioned via some
>> other means internally.  Grub can map the package it needs to load in
>> to memory as a series of absolute block locations and embed that
>> information within even the normal boot-loader area, or boot-loader
>> area and 1/2 more sectors with ease.
>>
>> This may be more difficult with raid-10 or raid-0 sets, and
>> considerably more difficult with raid-456 (grub would not have
>> redundancy at it's level of operation, but you could use an external
>> system and still effect data /recovery/ at which point it'd then
>> work...); however I see no reason it would be technically impossible,
>> though I don't expect it to have been a current usage consideration.
>
> well, I just tried grub2, but it seems the raid.mod
> is a bit bigger than 5K, so it is not possible to put
> it in the 4K left.
> Not to mention LVM on RAID, which makes, in lvm.mod
> and raid.mod, more or less 12K.
> Actually, grub-install itself complains that the
> embedding is not possible.
>
> The "default" core.img of grub2 is around about 28K,
> which is a little bit less than the 32K normal
> partiniong leaves between the MBR and the actual data.
>
> Bottom line is that, it seems to me, it is impossible
> to have a partionless system with RAID, let's say, 5.
> I wondered why 4K?
>
> So, would it be better to have a metadata 1.3 with a 32K
> offset for the superblock, so there could be enough space
> for something like grub2?
>
> Actually, I would like to propose metadata 2.0, or
> dual RAID, where the superblock is located somewhere
> in the disk(s), the space before will have one type
> of RAID and the space after possibly another type.
> The use case will be: RAID-1 before, RAID-x after, so
> a clever bootmanager could sit in the space before,
> the system will be partionless, the hot-plug add,
> which seems a topic, will be by far easier and we
> will get rid off the legacy partioning thing (LVM
> or mdp will do the trick).
>
>> Oh, it's also possible to chainload, so the 1.2 format offers the same
>> benefits on partitions if you need to operate within the constraints
>> of other existing boot-loaders.
>
> Wouldn't this still require partions somewhere?
>
> bye,
>
> --
>
> piergiorgio
>

I forget the commands, but the key thing for grub2 is stuffing all of
the modules you need in to a single loadable image, and invoking the
grub2 install command to embed THAT image.  It's then supposed to
deduce the blocks on disk that the image occupies and load it.  That
should work for raid1 and cases where the image is on a single real
device.  I don't know how it would cope with possibly bios-shuffled
real devices though.  I suspect supporting that wouldn't be worth it.

In any event, the 4k offset code is still useful for raid 1 arrays,
and supporting secondary MBR points that can be chained by grub/other
bootloaders when combined with partitioning.  It MAY also work if you
create the file just right and it's either beneath the chunk size, or
structured in such a way that it's duplicated (redundantly) across all
of the data devices; however even then you'd need to examine the raw
device and map out the parity block; it should wind up as two load
segments though.  Such support is just one potential solution, not
something I'd expect to exist at this moment.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata 1.2

[Piergiorgio Sartor]

Hi,

> I forget the commands, but the key thing for grub2 is stuffing all of
> the modules you need in to a single loadable image, and invoking the
> grub2 install command to embed THAT image.  It's then supposed to

exactly, THAT image is 28K and grub-install complains
there is no space, basically...
Actually, the complain is different, but the meaning
is that grub2 cannot do it.

The problem is grub2 wants to have its own things
outside the filesystem/lvm/raid containers.
Which is OK, but it requires also space somewhere
in the disk.
Standard option is to use the 32K left by tools
like fdisk, which put the beginning of the first
partition, by default, at sector 63.

Of course it is a bit silly to have a single
partition on a disk just to leave 32K free for
the bootmanager.
Not that this makes any damage, but it would be
more clean, i.e. less tools to deal with, to
avoid the partitioning completely.
This would require enough space for the bootmanager
to put its stuff.

That's why I was surprised that 1.2 leaves *only* 4K
without, apparently, any chance of having more space.

> In any event, the 4k offset code is still useful for raid 1 arrays,

How about changing it to something else?

Thanks,

bye,

-- 

piergiorgio

Re: metadata 1.2

[Michael Evans]

On Sat, Mar 27, 2010 at 2:47 AM, Piergiorgio Sartor
<piergiorgio.sartor@nexgo.de> wrote:
> Hi,
>
>> I forget the commands, but the key thing for grub2 is stuffing all of
>> the modules you need in to a single loadable image, and invoking the
>> grub2 install command to embed THAT image.  It's then supposed to
>
> exactly, THAT image is 28K and grub-install complains
> there is no space, basically...
> Actually, the complain is different, but the meaning
> is that grub2 cannot do it.
>
> The problem is grub2 wants to have its own things
> outside the filesystem/lvm/raid containers.
> Which is OK, but it requires also space somewhere
> in the disk.
> Standard option is to use the 32K left by tools
> like fdisk, which put the beginning of the first
> partition, by default, at sector 63.
>
> Of course it is a bit silly to have a single
> partition on a disk just to leave 32K free for
> the bootmanager.
> Not that this makes any damage, but it would be
> more clean, i.e. less tools to deal with, to
> avoid the partitioning completely.
> This would require enough space for the bootmanager
> to put its stuff.
>
> That's why I was surprised that 1.2 leaves *only* 4K
> without, apparently, any chance of having more space.
>
>> In any event, the 4k offset code is still useful for raid 1 arrays,
>
> How about changing it to something else?
>
> Thanks,
>
> bye,
>
> --
>
> piergiorgio
>

That's an entirely different question: Why only 4k and not more space?

The answer is that ~4k is the 'standard' offset for virtually every
filesystem that actually has an offset.  If you look at hexdumps from
the first few K of an LVM, msdos, ext*, and many other standard
filesystems you'll find that the general trend is to either leave no
space, or approximately 4k of space.  In this way the two 'at the
start' options clearly identify the devices as both NOT empty and NOT
another driver's container.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata 1.2

[Luca Berra]

On Sat, Mar 27, 2010 at 10:16:58AM +0100, Piergiorgio Sartor wrote:
>So, would it be better to have a metadata 1.3 with a 32K
>offset for the superblock, so there could be enough space
>for something like grub2?
this was discussed nearly one year ago.
look for the thread "md extension to support booting from raid whole
disks". Unfortunately that thread turned to an holy war. But before
that, Neil said he planned on offering an option to reserve some space,
i believe after the superblock, to host non-raid data.

>Actually, I would like to propose metadata 2.0, or
>dual RAID, where the superblock is located somewhere
the issue would be, how do you locate that piece of metadata?

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: metadata 1.2

[Piergiorgio Sartor]

Hi,

> That's an entirely different question: Why only 4k and not more space?

well, this is somehow following.
Since a bootmananger does not fit, why is this
space reserved at all...

I might understand the bootchain case, anyhow.

Thanks,

bye,

-- 

piergiorgio

Re: metadata 1.2

[Piergiorgio Sartor]

Hi,

> this was discussed nearly one year ago.
> look for the thread "md extension to support booting from raid whole
> disks". Unfortunately that thread turned to an holy war. But before
> that, Neil said he planned on offering an option to reserve some space,
> i believe after the superblock, to host non-raid data.

interesting, I'll have a look.

> >Actually, I would like to propose metadata 2.0, or
> >dual RAID, where the superblock is located somewhere
> the issue would be, how do you locate that piece of metadata?

That would be the easiest part.
Since the bootmanager should know how to boot, it will
know also where to look for the superblock and pass this
information to the kernel.
Or it would be possible to have some "fixed" place to look.

I think a reasonable solution is possible.

bye,

-- 

piergiorgio