Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andrew Clausen]

Ben LaHaise wrote:
> The work-in-progress patch for-demonstration-purposes-only below consists
> of 3 major components, and is meant to start discussion about the future
> direction of device naming and its interaction block layer.  The main
> motivations here are the wasting of minor numbers for partitions, and the
> duplication of code between user and kernel space in areas such as
> partition detection, uuid location, lvm setup, mount by label, journal
> replay, and so on...

(1) these issues are independent.  The partition parsing could
be done in user space, today, by blkpg, if I read the code correctly
;-)  (there's an ioctl for [un]registering partitions)  Never
tried it though ;-)

(2) what about bootstrapping?  how do you find the root device?
Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.

(3) how does this work for LVM and RAID?

(4) <propaganda>libparted already has a fair bit of partition
scanning code, etc.  Should be trivial to hack it up... That said,
it should be split up into .so modules... 200k is a bit heavy just
for mounting partitions (most of the bulk is file system stuff).
</propaganda>

(5) what happens to /etc/fstab?  User-space ([u]mount?) translates
/dev/hda1 into /dev/hda/offset=63,limit=1235823, and back?

Andrew Clausen

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Alexander Viro]

On Sat, 19 May 2001, Andrew Clausen wrote:

> (1) these issues are independent.  The partition parsing could
> be done in user space, today, by blkpg, if I read the code correctly
> ;-)  (there's an ioctl for [un]registering partitions)  Never
> tried it though ;-)

ioctls are even more evil than encoding limits into the name. Cease
and desist, please.

> (2) what about bootstrapping?  how do you find the root device?
> Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.

Ben's patch makes initrd mandatory.

> (3) how does this work for LVM and RAID?

It doesn't
 
> (4) <propaganda>libparted already has a fair bit of partition
> scanning code, etc.  Should be trivial to hack it up... That said,
> it should be split up into .so modules... 200k is a bit heavy just
> for mounting partitions (most of the bulk is file system stuff).
> </propaganda>

We will be much better off providing a sane API from the kernel. And
dropping the layout-aware code from fdisk, parted, yodda, yodda.

Libraries do not remove code duplication. You still need to relink the
stuff you keep statically linked, etc. Otherwise you get version skew.
Big way. Besides, you can't use library from a script, etc.

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andrew Clausen]

Alexander Viro wrote:
> On Sat, 19 May 2001, Andrew Clausen wrote:
> 
> > (1) these issues are independent.  The partition parsing could
> > be done in user space, today, by blkpg, if I read the code correctly
> > ;-)  (there's an ioctl for [un]registering partitions)  Never
> > tried it though ;-)
> 
> ioctls are even more evil than encoding limits into the name.

Why?  Encoding sounds funky... you don't get normal
ls behaviour, etc.

I think I'd prefer something like /dev/hda/table, where
table is something like /proc/partitions for /dev/hda, but
it's read/write ;-)

> Cease and desist, please.

Hehe
 
> > (4) <propaganda>libparted already has a fair bit of partition
> > scanning code, etc.  Should be trivial to hack it up... That said,
> > it should be split up into .so modules... 200k is a bit heavy just
> > for mounting partitions (most of the bulk is file system stuff).
> > </propaganda>
> 
> We will be much better off providing a sane API from the kernel. And
> dropping the layout-aware code from fdisk, parted, yodda, yodda.

What about partition editing on other OSs?  There's no reason
why fdisk/parted/etc. should be Linux only.  Why should the kernel
need to know how to write partition tables?

Also, different partition table formats have different alignment
constraints (which is relevant for creating partitions).  These
mainly need to be respected for other braindead OS's and/or BIOSes.

Communicating those between user/kernel space doesn't excite me.

Any ideas?
 
> Libraries do not remove code duplication. You still need to relink the
> stuff you keep statically linked, etc. Otherwise you get version skew.
> Big way. Besides, you can't use library from a script, etc.

Libtool & friends deals with version skew (ugly, but it works...)

You can write wrappers for libraries.

That said, a kernel API would be nice, if it was doable...

Andrew Clausen

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Ben LaHaise]

On Sat, 19 May 2001, Andrew Clausen wrote:

> (1) these issues are independent.  The partition parsing could
> be done in user space, today, by blkpg, if I read the code correctly
> ;-)  (there's an ioctl for [un]registering partitions)  Never
> tried it though ;-)

I tried to imply that through the use of the the word component.  Yes,
they're independant, but the code is pretty meaningless without a
demonstration of how it's used. ;-)

> (2) what about bootstrapping?  how do you find the root device?
> Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.

root= becomes a parameter to mount, and initrd becomes mandatory.  I'd be
all for including all of the bits needed to build the initrd boot code in
the tree, but it's completely in the air.

> (3) how does this work for LVM and RAID?

It's not done yet, but similar techniques would be applied.  I envision
that a raid device would support operations such as
open("/dev/md0/slot=5,hot-add=/dev/sda")

> (4) <propaganda>libparted already has a fair bit of partition
> scanning code, etc.  Should be trivial to hack it up... That said,
> it should be split up into .so modules... 200k is a bit heavy just
> for mounting partitions (most of the bulk is file system stuff).
> </propaganda>

Good.  Less work to do.

> (5) what happens to /etc/fstab?  User-space ([u]mount?) translates
> /dev/hda1 into /dev/hda/offset=63,limit=1235823, and back?

I'd just create a symlink to /dev/hda1 at mount time, although that really
isn't what the user wants to see: the label or uuid is more useful.

		-ben

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Alexander Viro]

On Sat, 19 May 2001, Ben LaHaise wrote:

> It's not done yet, but similar techniques would be applied.  I envision
> that a raid device would support operations such as
> open("/dev/md0/slot=5,hot-add=/dev/sda")

Think for a moment and you'll see why it's not only ugly as hell, but simply
won't work.

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Ben LaHaise]

On Sat, 19 May 2001, Alexander Viro wrote:

> On Sat, 19 May 2001, Ben LaHaise wrote:
>
> > It's not done yet, but similar techniques would be applied.  I envision
> > that a raid device would support operations such as
> > open("/dev/md0/slot=5,hot-add=/dev/sda")
>
> Think for a moment and you'll see why it's not only ugly as hell, but simply
> won't work.

Yeah, I shouldn't be replying to email anymore in my bleery-eyed state.
=) Of course slash seperated data doesn't work, so it would have to be
hot-add=<filedescriptor> or somesuch.  Gah, that's why the options are all
parsed from a single lookup name anyways...

		-ben (who's going to sleep)

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Alexander Viro]

On Sat, 19 May 2001, Andrew Clausen wrote:

> Alexander Viro wrote:
> > On Sat, 19 May 2001, Andrew Clausen wrote:
> > 
> > > (1) these issues are independent.  The partition parsing could
> > > be done in user space, today, by blkpg, if I read the code correctly
> > > ;-)  (there's an ioctl for [un]registering partitions)  Never
> > > tried it though ;-)
> > 
> > ioctls are even more evil than encoding limits into the name.
> 
> Why?  Encoding sounds funky... you don't get normal
> ls behaviour, etc.

ioctls are evil, period. At least with these names you can use normal
scripting and don't need any special tools. Every ioctl means a binary
that has no business to exist.

> What about partition editing on other OSs?  There's no reason
> why fdisk/parted/etc. should be Linux only.  Why should the kernel
> need to know how to write partition tables?

It needs to read them. Writing doesn't add much. I'd rather see
trivial partitioning tools that consist only of UI code in case
of Linux.

> Also, different partition table formats have different alignment
> constraints (which is relevant for creating partitions).  These
> mainly need to be respected for other braindead OS's and/or BIOSes.
> 
> Communicating those between user/kernel space doesn't excite me.

So don't communicate them.
 
> Libtool & friends deals with version skew (ugly, but it works...)

With statically linked binaries? How?
 
> You can write wrappers for libraries.

Uh-huh. And you can write them for ioctls. We had been busily doing that
for years. Results are not pretty, to put it very mildly.

OK, let me put it that way: I know how to do it in the kernel with
no code duplication and less impact on userland. BTW, most of the
code can very well sit in the userland, but that's another story
(userland filesystems). Anyway, there's only one way to settle such
stuff - sit down and write the patch. Which is what I'm going to do.

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Alexander Viro]

On Sat, 19 May 2001, Ben LaHaise wrote:

> On Sat, 19 May 2001, Alexander Viro wrote:
> 
> > On Sat, 19 May 2001, Ben LaHaise wrote:
> >
> > > It's not done yet, but similar techniques would be applied.  I envision
> > > that a raid device would support operations such as
> > > open("/dev/md0/slot=5,hot-add=/dev/sda")
> >
> > Think for a moment and you'll see why it's not only ugly as hell, but simply
> > won't work.
> 
> Yeah, I shouldn't be replying to email anymore in my bleery-eyed state.
> =) Of course slash seperated data doesn't work, so it would have to be
> hot-add=<filedescriptor> or somesuch.  Gah, that's why the options are all
> parsed from a single lookup name anyways...

That's why you want to use write(2) to pass that information instead of
encoding it into open(2).

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andrew Morton]

Alexander Viro wrote:
> 
> > (2) what about bootstrapping?  how do you find the root device?
> > Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.
> 
> Ben's patch makes initrd mandatory.
> 

Can this be fixed?  I've *never* had to futz with initrd.
Probably most systems are the same.  It seems a step
backward to make it necessary.

-

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Alexander Viro]

On Sat, 19 May 2001, Andrew Morton wrote:

> Alexander Viro wrote:
> > 
> > > (2) what about bootstrapping?  how do you find the root device?
> > > Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.
> > 
> > Ben's patch makes initrd mandatory.
> > 
> 
> Can this be fixed?  I've *never* had to futz with initrd.
> Probably most systems are the same.  It seems a step
> backward to make it necessary.

Well, if you remove partition table parsing from the kernel - you've
got to boot with root on unpartitioned device (e.g. /dev/ram0) and
either stay that way or bring the userland code that understands
partitioning on that device...

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andrew Clausen]

Alexander Viro wrote:
> ioctls are evil, period. At least with these names you can use normal
> scripting and don't need any special tools. Every ioctl means a binary
> that has no business to exist.

Special names are butt-ugly.

ioctl's can be replaced with games on /proc or whatever, which are
better than special names.

> > What about partition editing on other OSs?  There's no reason
> > why fdisk/parted/etc. should be Linux only.  Why should the kernel
> > need to know how to write partition tables?
> 
> It needs to read them. Writing doesn't add much.

Wrong.  When you read, you throw out 90% of the useless crap.
When you write, you need to know about it, and provide
interfaces for it.

> I'd rather see
> trivial partitioning tools that consist only of UI code in case
> of Linux.

Some stuff friendly partition tools should have, IMHO:
(1) ability to predict what's going to happen.  That way, you can
play around until it looks nice, and hit the friendly commit
button.
(2) ability to do data recovery (eg: probe for signatures where
it expects the start of partitions to occur.  You can be
intelligent/quick about it, by knowing about alignment stuff,
for example)
(3) ability to convert between partition table types (and
even LVM ;-)  This can be tricky because of alignment stuff.

So:
(1) could be done in-kernel by being able to discard changes,
and re-reading, I guess.
(2) and (3) really only need alignment stuff.

Also, you need to be able to deal with legacy stuff, like
setting magic flags for booting.

> > Also, different partition table formats have different alignment
> > constraints (which is relevant for creating partitions).  These
> > mainly need to be respected for other braindead OS's and/or BIOSes.
> >
> > Communicating those between user/kernel space doesn't excite me.
> 
> So don't communicate them.

So, what do you do?

Sometimes, you want to force alignment violations (eg: recovering
an accidently deleted partition)

The real problem happens when you want to resize file systems, and
you need to simultaneously satisfy resizer and partition table
constraints.  (there are currently no resizers like this, but
an ext2-resize-the-start and NTFS-resize-the-start would definitely
be like this... when I get time to write them.  It's pure luck
that you don't need this for FAT, but this causes all sorts of
headaches for Linux...)

Anyway, you have one constraint in user space, and one in the
kernel... how do you find the intersection?
 
> > Libtool & friends deals with version skew (ugly, but it works...)
> 
> With statically linked binaries? How?

Why do we need them?
 
> > You can write wrappers for libraries.
> 
> Uh-huh. And you can write them for ioctls. We had been busily doing that
> for years. Results are not pretty, to put it very mildly.

If you can get everything into a nice file system interface,
then you've convinced me.

> BTW, most of the
> code can very well sit in the userland, but that's another story
> (userland filesystems). Anyway, there's only one way to settle such
> stuff - sit down and write the patch. Which is what I'm going to do.

Have fun.

So, my patch will be about 50 lines in parted, to call blkpg,
and provide a "kernelread" command... But, philosophy essay to
write... :-(  (you have to wait until Monday)

Then you can rm -r fs/partitions

But, I don't see how patches will settle anything, when we're
arguing over interfaces & stuff needed for partition tools.  Or
are you writing patches for Parted as well?

Andrew Clausen

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andries.Brouwer]

Andrew Morton writes:

    > > (2) what about bootstrapping?  how do you find the root device?
    > > Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.
    > 
    > Ben's patch makes initrd mandatory.

    Can this be fixed?  I've *never* had to futz with initrd.
    Probably most systems are the same.  It seems a step
    backward to make it necessary.

I don't think so. It is necessary, and it is good.

But it is easy to make the transition painless.
Instead of the current choice between INITRD (yes/no)
we have INITRD (default built-in / external).
The built-in version can then slowly become smaller and die.

Andries

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Aaron Lehmann]

On Sat, May 19, 2001 at 01:30:14PM +0200, Andries.Brouwer@cwi.nl wrote:
> I don't think so. It is necessary, and it is good.
> 
> But it is easy to make the transition painless.
> Instead of the current choice between INITRD (yes/no)
> we have INITRD (default built-in / external).
> The built-in version can then slowly become smaller and die.

initrd is an unnecessary pain in the ass for most people. It had
better not become mandatory.

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andries.Brouwer]

> initrd is an unnecessary pain in the ass for most people.
> It had better not become mandatory.

You would not notice the difference, only your kernel would be
a bit smaller and the RRPART ioctl disappears.

[Besides: we have lived with DOS-type partition tables for ten years,
but they will not last another ten years. Very soon disk partitions
will look very different. It will be good to move knowledge about
these things out of the kernel before this happens.]

Andries

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Aaron Lehmann]

On Sat, May 19, 2001 at 08:05:02PM +0200, Andries.Brouwer@cwi.nl wrote:
> > initrd is an unnecessary pain in the ass for most people.
> > It had better not become mandatory.
> 
> You would not notice the difference, only your kernel would be
> a bit smaller and the RRPART ioctl disappears.

Would I not notice the difference as a user, as a sysadmin, as a
kernel builder, as a kernel hacker, or all of the above?

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Richard Gooch]

Andries Brouwer writes:
> Andrew Morton writes:
> 
>     > > (2) what about bootstrapping?  how do you find the root device?
>     > > Do you do "root=/dev/hda/offset=63,limit=1235823"?  Bit nasty.
>     > 
>     > Ben's patch makes initrd mandatory.
> 
>     Can this be fixed?  I've *never* had to futz with initrd.
>     Probably most systems are the same.  It seems a step
>     backward to make it necessary.
> 
> I don't think so. It is necessary, and it is good.

It most certainly is not. This attitude of pushing more and more stuff
out of the kernel and into initrd is really annoying. Initrd is messy,
nasty and opaque. It makes the boot process more complex. There is no
way in hell I want to be forced to use it.

Removing N lines from the kernel at the cost of adding N+k lines to
user-space is a *loss*, not a gain. I want my *system* to be simple
and transparent.

> But it is easy to make the transition painless.

No, initrd is fundamentally painful. Let go of this ideology of
removing code from the kernel at all costs. A super-slim kernel which
requires a more complex to administer user-space is too high a cost.
The benefits of removing partition support from the kernel are
basically zero.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Andries.Brouwer]

    From aaronl@vitelus.com Sat May 19 20:07:23 2001

    > > initrd is an unnecessary pain in the ass for most people.
    > > It had better not become mandatory.
    > 
    > You would not notice the difference, only your kernel would be
    > a bit smaller and the RRPART ioctl disappears.

    Would I not notice the difference as a user, as a sysadmin, as a
    kernel builder, as a kernel hacker, or all of the above?

All of the above.

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Linus Torvalds]

On Sat, 19 May 2001, Brad Boyer wrote:
> 
> If I understand the status of stuff correctly, I think this would make it
> a lot more painful to admin if it became a requirement to use initrd on
> everything just to be able to boot.

Don't get too hung up on initrd. Symbolic links really _are_ workable ways
to basically cache the information across boots, and the real problems are
elsewhere.

		Linus

Re: [RFD w/info-PATCH] device arguments from lookup, partion code inuserspace

[Brad Boyer]

Aaron Lehmann wrote:
> On Sat, May 19, 2001 at 08:05:02PM +0200, Andries.Brouwer@cwi.nl wrote:
> > > initrd is an unnecessary pain in the ass for most people.
> > > It had better not become mandatory.
> > 
> > You would not notice the difference, only your kernel would be
> > a bit smaller and the RRPART ioctl disappears.
> 
> Would I not notice the difference as a user, as a sysadmin, as a
> kernel builder, as a kernel hacker, or all of the above?

If I understand the status of stuff correctly, I think this would make it
a lot more painful to admin if it became a requirement to use initrd on
everything just to be able to boot. If you've ever seen the way some of
the bootloaders for alterate platforms (like ppc and 68k) handle booting,
you'd see what a pain it can be to have anything more than a simple string
of options getting passed to the kernel. It's particularly bad on some
of the embedded ppc platforms. I suspect that if this happened, it would
never be allowed into many people's trees, because it would be worth their
effort to maintain different code so they don't have to squeeze an initrd
on flash along with their kernel and root filesystem. If I'm missing the
boat here, please tell me, but it sure seems like a bad idea to me.

	Brad Boyer