Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Christoph Hellwig]

On Thu, Apr 30, 2009 at 03:23:42PM +0200, Kay Sievers wrote:
> From: Kay Sievers <kay.sievers@vrfy.org>
> Subject: driver-core: devtmpfs - driver core maintained /dev tmpfs

Umm, guys this needs much broader discussion than just sneaking in
a patch under the covers.

It basically does re-introduce devfs under a different name, and from
looking at the implementation it might not be quite as bad a Gooch's
original, but it's certainly worse than Adam Richters rewrite the we
never ended up merging.

Now we might want to revisit the decision to leave all the device name
handling to a userspace daemon, because it provded to be quite fragile
under certain circumstances, and you apparently see performance issues.

> Devtmpfs lets the kernel create a tmpfs very early at kernel
> initialization, before any driver core device is registered. Every
> device with a major/minor will have a device node created in this
> tmpfs instance. After the rootfs is mounted by the kernel, the
> populated tmpfs is mounted at /dev. In initramfs, it can be moved
> to the manually mounted root filesystem before /sbin/init is
> executed.

That for example is something that is not acceptable.  We really don't
want the kernel to mess with the initial namespace in such a major way.

> The tmpfs instance can be changed and altered by userspace at any time,
> and in any way needed - just like today's udev-mounted tmpfs. Unmodified
> udev versions will run just fine on top of it, and will recognize an
> already existing kernel-created device node and use it.
> The default node permissions are root:root 0600. Only if none of these
> values have been changed by userspace, the driver core will remove the
> device node when the device goes away. If the device node was altered
> by udev, by applying the appropriate permissions and ownership, it will
> need to be removed by udev - just as it usually works today.

That's some really, really odd lifetime rules.


Counter-proposal:  Re-introduce a proper mini-devfs.  All nodes in there
are kernel-created and not changeable which sorts out that whole
mess of both drivers and userspace messing with tree topology we had
both in original devfs and this new devtmpfs.  Single-instance so it can be
populated before it's actually mounted somewhere, that way the kernel
doesn't have to do any policy devicision on where it's mounted.  Mount
point would usually be /dev/something so /dev can remaining udev-managed
tmpfs or even manually maintained and symlinks can point into
/dev/something.

> +static char *bsg_nodename(struct device *dev)
> +{
> +	return kasprintf(GFP_KERNEL, "bsg/%s", dev_name(dev));
> +}
> +
>  static int __init bsg_init(void)
>  {
>  	int ret, i;
> @@ -1082,6 +1087,7 @@ static int __init bsg_init(void)
>  		ret = PTR_ERR(bsg_class);
>  		goto destroy_kmemcache;
>  	}
> +	bsg_class->nodename = bsg_nodename;

And adding this gunk to every driver is really ugly.  Must say
late-devfs version of the same defintively was more pretty.

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Kay Sievers]

On Sat, May 2, 2009 at 09:16, Christoph Hellwig <hch@infradead.org> wrote:
> >After the rootfs is mounted by the kernel, the
>> populated tmpfs is mounted at /dev. In initramfs, it can be moved
>> to the manually mounted root filesystem before /sbin/init is
>> executed.
>
> That for example is something that is not acceptable.  We really don't
> want the kernel to mess with the initial namespace in such a major way.

There is nothing like "mess around", it's not mounted at all, until
the kernel mounts the root filesystem at /, then devtmpfs is mounted
the first time, and only if it's compiled in because you asked for it.
Also, just try:
  egrep 'mknod|create_dev' init/*.c
and see what we currently do.

> Counter-proposal:  Re-introduce a proper mini-devfs.  All nodes in there
> are kernel-created and not changeable which sorts out that whole
> mess of both drivers and userspace messing with tree topology we had
> both in original devfs and this new devtmpfs.  Single-instance so it can be
> populated before it's actually mounted somewhere, that way the kernel
> doesn't have to do any policy devicision on where it's mounted.

That sounds worse than devtpfs, and does not help for most of the
mentioned problems we are trying to solve here.

> Mount
> point would usually be /dev/something so /dev can remaining udev-managed
> tmpfs or even manually maintained and symlinks can point into
> /dev/something.

And that would solve what? init=/bin/sh would still not work, you can
not bring your box up with that, and you have some pretty useless
unchangeable stuff hanging around in a /dev subdirectory?

>> @@ -1082,6 +1087,7 @@ static int __init bsg_init(void)
>>               ret = PTR_ERR(bsg_class);
>>               goto destroy_kmemcache;
>>       }
>> +     bsg_class->nodename = bsg_nodename;
>
> And adding this gunk to every driver is really ugly.  Must say
> late-devfs version of the same defintively was more pretty.

There are only a very few places who need this, there nothing ever
like "every driver". It's a very few subsystems, not even the drivers,
if they have more than one. Most device nodes do not have a
subdirectory and don't have that at all.

Thanks,
Kay

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Alan Jenkins]

On 5/2/09, Kay Sievers <kay.sievers@vrfy.org> wrote:
> On Sat, May 2, 2009 at 09:16, Christoph Hellwig <hch@infradead.org> wrote:
>> >After the rootfs is mounted by the kernel, the
>>> populated tmpfs is mounted at /dev. In initramfs, it can be moved
>>> to the manually mounted root filesystem before /sbin/init is
>>> executed.
>>
>> That for example is something that is not acceptable.  We really don't
>> want the kernel to mess with the initial namespace in such a major way.
>
> There is nothing like "mess around", it's not mounted at all, until
> the kernel mounts the root filesystem at /, then devtmpfs is mounted
> the first time, and only if it's compiled in because you asked for it.
> Also, just try:
>   egrep 'mknod|create_dev' init/*.c
> and see what we currently do.
>
>> Counter-proposal:  Re-introduce a proper mini-devfs.  All nodes in there
>> are kernel-created and not changeable which sorts out that whole
>> mess of both drivers and userspace messing with tree topology we had
>> both in original devfs and this new devtmpfs.  Single-instance so it can
>> be
>> populated before it's actually mounted somewhere, that way the kernel
>> doesn't have to do any policy devicision on where it's mounted.
>
> That sounds worse than devtpfs, and does not help for most of the
> mentioned problems we are trying to solve here.

On a narrow issue: do you really object to moving the "mount dev -t
devfs2 /dev" into userspace (and therefore giving it a user-visible
name)??  That would address Cristophs particular objection about
"messing around" with the initial namespace.  It means I can be 100%
sure I can boot an old initramfs with this option enabled.  And it
gives a nice clean way for new initramfs' to test for this feature -
when they try to mount it, it fails.  It would seem to make for a
rather smoother migration path.

It shouldn't mean too many more LOC, you're already doing the "single
instance" thing.

Thanks
Alan

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Kay Sievers]

On Sat, May 2, 2009 at 22:22, Alan Jenkins
<sourcejedi.lkml@googlemail.com> wrote:

> On a narrow issue: do you really object to moving the "mount dev -t
> devfs2 /dev" into userspace (and therefore giving it a user-visible
> name)??  That would address Cristophs particular objection about
> "messing around" with the initial namespace.

An argument which does not stand at all, there is no mess, it is not
mounted at all until we are ready with initializing the kernel. And
instead of creating some random nodes in /dev like we do today, we
mount it, and it contains a node for every device. I hardly see any of
the mentioned "namespace mess" here, it's just the simplest, most
robust, and most efficient thing we can do. :)

> It means I can be 100%
> sure I can boot an old initramfs with this option enabled.

Oh, it does not change anything for an existing initramfs, if that
option enabled. After initramfs found and mounted the real rootfs at
/root, your are totally free to call:
  mount --move /dev/ /root/dev
or not to do that, like we do today.

> And it
> gives a nice clean way for new initramfs' to test for this feature -
> when they try to mount it, it fails.  It would seem to make for a
> rather smoother migration path.

I think that is all covered just fine.

One thing that I tried to solve by doing a kernel mounted fs, is that
/dev on the rootfs is completely empty, it is that way on some distros
today, and if you do init=/bin/sh, it will not work, because
/dev/console is missing.

Another thing, why I would like to avoid a new fstype is that
userspace checks if /dev is a tmpfs to find out if it's a dynamic /dev
- nothing really that should prevent us from doing a new filesystem,
but we should need a good reason to do it, I think.

Thanks,
Kay

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Alan Jenkins]

On 5/2/09, Kay Sievers <kay.sievers@vrfy.org> wrote:
> On Sat, May 2, 2009 at 22:22, Alan Jenkins
> <sourcejedi.lkml@googlemail.com> wrote:
>
>> On a narrow issue: do you really object to moving the "mount dev -t
>> devfs2 /dev" into userspace (and therefore giving it a user-visible
>> name)??  That would address Cristophs particular objection about
>> "messing around" with the initial namespace.
>
> An argument which does not stand at all, there is no mess, it is not
> mounted at all until we are ready with initializing the kernel. And
> instead of creating some random nodes in /dev like we do today, we
> mount it, and it contains a node for every device. I hardly see any of
> the mentioned "namespace mess" here, it's just the simplest, most
> robust, and most efficient thing we can do. :)
>
>> It means I can be 100%
>> sure I can boot an old initramfs with this option enabled.
>
> Oh, it does not change anything for an existing initramfs, if that
> option enabled. After initramfs found and mounted the real rootfs at
> /root, your are totally free to call:
>   mount --move /dev/ /root/dev
> or not to do that, like we do today.

Sorry, you're right.  I should go to bed :-).

It would matter if you had a different naming scheme for /dev than the
kernel, and you were trying to get away with a static /dev.  I can't
believe anyone important does that though :-).

>> And it
>> gives a nice clean way for new initramfs' to test for this feature -
>> when they try to mount it, it fails.  It would seem to make for a
>> rather smoother migration path.
>
> I think that is all covered just fine.

Oh, I see.

grep "/dev" /proc/mounts > /dev/null

> One thing that I tried to solve by doing a kernel mounted fs, is that
> /dev on the rootfs is completely empty, it is that way on some distros
> today, and if you do init=/bin/sh, it will not work, because
> /dev/console is missing.
>
> Another thing, why I would like to avoid a new fstype is that
> userspace checks if /dev is a tmpfs to find out if it's a dynamic /dev
> - nothing really that should prevent us from doing a new filesystem,
> but we should need a good reason to do it, I think.

I thought udev was documented somewhere as compatible with a non-tmpfs
/dev, in a "just because you could" sort of way.  I've seen something
test for tmpfs... nevermind, it's probably something different.  (Just
the init script that checks whether /dev has been mounted that way by
an initramfs, or the user decided to do without intramfs'  so the
rootfs gets to mount it instead).

Good night
Alan

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Michael Tokarev]

Alan Jenkins wrote:
> On 5/2/09, Kay Sievers <kay.sievers@vrfy.org> wrote:
[]
>>> And it
>>> gives a nice clean way for new initramfs' to test for this feature -
>>> when they try to mount it, it fails.  It would seem to make for a
>>> rather smoother migration path.
>> I think that is all covered just fine.
> 
> Oh, I see.
> 
> grep "/dev" /proc/mounts > /dev/null

Oh no.  This one will match all your /dev/sda1 etc too :)
What yo want to do is either

   grep -w /dev /proc/mount

or

  awk '$2 == "/dev" { exit 0; } END { exit 1; }' /proc/mounts

or something similar.  To distinguish between the following
two cases:

/dev/sda1 / ext3 rw,noatime,data=ordered 0 0
devfs /dev tmpfs rw,nosuid,size=512k,mode=755 0 0

/mjt

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Alan Jenkins]

On 5/2/09, Alan Jenkins <sourcejedi.lkml@googlemail.com> wrote:
> On 5/2/09, Kay Sievers <kay.sievers@vrfy.org> wrote:
>> On Sat, May 2, 2009 at 09:16, Christoph Hellwig <hch@infradead.org>
>> wrote:
>>> >After the rootfs is mounted by the kernel, the
>>>> populated tmpfs is mounted at /dev. In initramfs, it can be moved
>>>> to the manually mounted root filesystem before /sbin/init is
>>>> executed.
>>>
>>> That for example is something that is not acceptable.  We really don't
>>> want the kernel to mess with the initial namespace in such a major way.
>>
>> There is nothing like "mess around", it's not mounted at all, until
>> the kernel mounts the root filesystem at /, then devtmpfs is mounted
>> the first time, and only if it's compiled in because you asked for it.
>> Also, just try:
>>   egrep 'mknod|create_dev' init/*.c
>> and see what we currently do.
>>
>>> Counter-proposal:  Re-introduce a proper mini-devfs.  All nodes in there
>>> are kernel-created and not changeable which sorts out that whole
>>> mess of both drivers and userspace messing with tree topology we had
>>> both in original devfs and this new devtmpfs.  Single-instance so it can
>>> be
>>> populated before it's actually mounted somewhere, that way the kernel
>>> doesn't have to do any policy devicision on where it's mounted.
>>
>> That sounds worse than devtpfs, and does not help for most of the
>> mentioned problems we are trying to solve here.
>
> On a narrow issue: do you really object to moving the "mount dev -t
> devfs2 /dev" into userspace (and therefore giving it a user-visible
> name)??  That would address Cristophs particular objection about
> "messing around" with the initial namespace.  It means I can be 100%
> sure I can boot an old initramfs with this option enabled.  And it
> gives a nice clean way for new initramfs' to test for this feature -
> when they try to mount it, it fails.  It would seem to make for a
> rather smoother migration path.
>
> It shouldn't mean too many more LOC, you're already doing the "single
> instance" thing.

Also, AFAICS this would avoid a memory leak on umount.

In it's original form, if you unmount it, you can't get it back.  But
it doesn't get destroyed either; all the tmpfs nodes just hang around
in limbo, right?


It'd be even nicer if it somehow avoided consuming memory when it
isn't used.  I guess that requires more code, looks more like a "real"
devfs, and as C. says is probably more sane if exported as a read
only.  Hopefully it would make it possible to remove the code
afterwards, but that sounds like even more work.

But is read-only so bad?  You just have to copy it over to a tmpfs and
then mount that on top of /dev.  That's atomic, so it won't interfere
with parallel early init.  I sympathize, devtmpfs is a really neat
hack that does exactly what udev needs.  But you have to admit, it
doesn't fit in _quite_ as well with the kernel status quo.

Thanks
Alan

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Greg KH]

On Sat, May 02, 2009 at 10:41:50PM +0100, Alan Jenkins wrote:
> 
> But is read-only so bad?  You just have to copy it over to a tmpfs and
> then mount that on top of /dev.  That's atomic, so it won't interfere
> with parallel early init.

The copy would not be atomic.

> I sympathize, devtmpfs is a really neat hack that does exactly what
> udev needs.  But you have to admit, it doesn't fit in _quite_ as well
> with the kernel status quo.

I disagree, it mirrors exactly what we are doing today from userspace,
which is quite the "status quo" in that distros have been doing that for
years now :)

thanks,

greg k-h

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Kay Sievers]

On Sat, May 2, 2009 at 23:41, Alan Jenkins
<sourcejedi.lkml@googlemail.com> wrote:
> On 5/2/09, Alan Jenkins <sourcejedi.lkml@googlemail.com> wrote:
>> On 5/2/09, Kay Sievers <kay.sievers@vrfy.org> wrote:
>>> On Sat, May 2, 2009 at 09:16, Christoph Hellwig <hch@infradead.org>
>>> wrote:
>>>> >After the rootfs is mounted by the kernel, the
>>>>> populated tmpfs is mounted at /dev. In initramfs, it can be moved
>>>>> to the manually mounted root filesystem before /sbin/init is
>>>>> executed.
>>>>
>>>> That for example is something that is not acceptable.  We really don't
>>>> want the kernel to mess with the initial namespace in such a major way.
>>>
>>> There is nothing like "mess around", it's not mounted at all, until
>>> the kernel mounts the root filesystem at /, then devtmpfs is mounted
>>> the first time, and only if it's compiled in because you asked for it.
>>> Also, just try:
>>>   egrep 'mknod|create_dev' init/*.c
>>> and see what we currently do.
>>>
>>>> Counter-proposal:  Re-introduce a proper mini-devfs.  All nodes in there
>>>> are kernel-created and not changeable which sorts out that whole
>>>> mess of both drivers and userspace messing with tree topology we had
>>>> both in original devfs and this new devtmpfs.  Single-instance so it can
>>>> be
>>>> populated before it's actually mounted somewhere, that way the kernel
>>>> doesn't have to do any policy devicision on where it's mounted.
>>>
>>> That sounds worse than devtpfs, and does not help for most of the
>>> mentioned problems we are trying to solve here.
>>
>> On a narrow issue: do you really object to moving the "mount dev -t
>> devfs2 /dev" into userspace (and therefore giving it a user-visible
>> name)??  That would address Cristophs particular objection about
>> "messing around" with the initial namespace.  It means I can be 100%
>> sure I can boot an old initramfs with this option enabled.  And it
>> gives a nice clean way for new initramfs' to test for this feature -
>> when they try to mount it, it fails.  It would seem to make for a
>> rather smoother migration path.
>>
>> It shouldn't mean too many more LOC, you're already doing the "single
>> instance" thing.
>
> Also, AFAICS this would avoid a memory leak on umount.
>
> In it's original form, if you unmount it, you can't get it back.  But
> it doesn't get destroyed either; all the tmpfs nodes just hang around
> in limbo, right?

Sure you can't get it back if you drop it, but what would be the point
of un-mounting it and wanting it back? It's just like every other
tmpfs too. :)

> It'd be even nicer if it somehow avoided consuming memory when it
> isn't used.

It's probably easy to make it destroy itself, and free everything, if
it's un-mounted, if that solves your concerns about built-in but not
used.

> I guess that requires more code, looks more like a "real"
> devfs, and as C. says is probably more sane if exported as a read
> only.  Hopefully it would make it possible to remove the code
> afterwards, but that sounds like even more work.

Then you should probably not enable it, or have a boot option to
disable it. What's the point in removing it later, it is exactly what
you want, and what everybody has today, a /dev on tmpfs, just that the
kernel will puts the default nodes in there.

> But is read-only so bad?  You just have to copy it over to a tmpfs and
> then mount that on top of /dev.  That's atomic, so it won't interfere
> with parallel early init.  I sympathize, devtmpfs is a really neat
> hack that does exactly what udev needs.  But you have to admit, it
> doesn't fit in _quite_ as well with the kernel status quo.

I think it fits quite nice as a "device node companion" to sysfs. And
I don't really see the point of a read-only mount, it should be the
/dev that is used on the real system, until you shut it down.

Thanks,
Kay

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Jeff Garzik]

Christoph Hellwig wrote:
> On Thu, Apr 30, 2009 at 03:23:42PM +0200, Kay Sievers wrote:
>> From: Kay Sievers <kay.sievers@vrfy.org>
>> Subject: driver-core: devtmpfs - driver core maintained /dev tmpfs
> 
> Umm, guys this needs much broader discussion than just sneaking in
> a patch under the covers.
> 
> It basically does re-introduce devfs under a different name, and from
> looking at the implementation it might not be quite as bad a Gooch's
> original, but it's certainly worse than Adam Richters rewrite the we
> never ended up merging.

I was interested in this Richter devfs rewrite, since I was unfamiliar 
with it.

For the benefit of the thread, here is a URL that people can examine:

	http://marc.info/?l=linux-kernel&m=104138806530375&w=2

Regards,

	Jeff

Re: [PATCH] driver-core: devtmpfs - driver core maintained /dev tmpfs

[Kay Sievers]

On Sat, May 2, 2009 at 18:59, Jeff Garzik <jeff@garzik.org> wrote:
> Christoph Hellwig wrote:

>> It basically does re-introduce devfs under a different name, and from
>> looking at the implementation it might not be quite as bad a Gooch's
>> original, but it's certainly worse than Adam Richters rewrite the we
>> never ended up merging.
>
> I was interested in this Richter devfs rewrite, since I was unfamiliar with
> it.
>
> For the benefit of the thread, here is a URL that people can examine:
>
>        http://marc.info/?l=linux-kernel&m=104138806530375&w=2

And it did:
  - the crazy devfs naming scheme with:
      /dev/ide/host0/bus0/target0/lun0/part1, /dev/tty/0,
    and it did not even create our current names by default
  - path lookup interception and called modprobe behind your back
  - call_usermodhelper() from the kernel to name devices
  - introduced a new filesystem type and a bunch of
    new datatypes
  - and so on ...

I don't think anybody else besides Christoph would call devtmpfs
"seriously worse" than this. :) None of these "features" are needed
today, or even close to even worth to think about it.

And for the "policy in kernel department" which will be the next naive
argument: The kernel carries the policy today for 98% of the devices,
if you change any driver given name, it will no longer show up in /dev
with the current name. That's the reality since years, and will not be
different anytime soon, there is no real naming policy besides the
current kernel supplied names.

Now that we managed to define a common set of /dev names we all share
across distros, it's time to let the kernel know about the remaining
2% and let it pre-create the device nodes itself for userspace to pick
up, with all the earlier mentioned opportunities it offers.

The final policy will still live in udev which sets ownership and
mode, and possibly overwrites the node name. Nothing really has
changed, it can just be made faster, and it is much more reliable if
stuff in userspace goes wrong.

Thanks,
Kay