Mounting only inserted disks

Mounting only inserted disks

[Kai Hendry]

Hi there,

I used some udev automounting scripts in Webconverger:
https://wiki.archlinux.org/index.php/Udev#Auto_mounting_USB_devices

However since users live boot Webconverger and a system that might
already have an operating system installed like Windows, they
surprisingly found that once they reboot back into Windows, a chkdsk
resulted IIUC.

So I've removed the offending scripts:
http://anonscm.debian.org/gitweb/?pÞbian-live/config-webc.git;a=commitdiff;hê18dff8af2b6800c580b70ce4219aacae9dae73;hpecb0929dd60e0e708e55af81f5bd91bc6173fe6

Now I'm thinking is there a way to only mount disks that were
*inserted* only once the machine has booted?


Kind regards,

Re: Mounting only inserted disks

[Kay Sievers]

On Sun, Jan 29, 2012 at 04:45, Kai Hendry <hendry@webconverger.com> wrote:
> I used some udev automounting scripts in Webconverger:
> https://wiki.archlinux.org/index.php/Udev#Auto_mounting_USB_devices

It is very wrong to recommend anything like this to users. It's just
an entirely flawed concept.

> However since users live boot Webconverger and a system that might
> already have an operating system installed like Windows, they
> surprisingly found that once they reboot back into Windows, a chkdsk
> resulted IIUC.
>
> So I've removed the offending scripts:
> http://anonscm.debian.org/gitweb/?pÞbian-live/config-webc.git;a=commitdiff;hê18dff8af2b6800c580b70ce4219aacae9dae73;hpecb0929dd60e0e708e55af81f5bd91bc6173fe6
>
> Now I'm thinking is there a way to only mount disks that were
> *inserted* only once the machine has booted?

No, it is intentional, that udev rules can not distinguish between
bootup from device hotplug; they look exactly the same.

Udev's rule engine is not the right place to hookup mounting of
arbitrary filesystems, or configure non-trivial network settings like
DHCP, or start system daemons, or run any other programs that runs for
an unpredictable amount of time. Udev rules should only be used to
identify or initially configure hardware, but never to execute system
management jobs or things that involve policy or need error handling
like filesystem checking or mounting. Running such programs from udev
rules will block related events, and might render the entire system
unusable. To ensure timely event execution, udev forcefully kills all
programs it has executed from rules, and which take longer than 30 or
60 seconds to finish, and mounting and checking disks can take much
longer than that.

Udev can send events to services which can act on device changes
though. An auto-mounter service can listen to block device events and
take the appropriate actions, such a service will not block udev's
operations for an unpredictable time. Udisks and systemd for example
work like that.

Mounting filesystems is just not simple enough to do that in udev, you
need a real service to do that properly. Udev rules are just not the
right tool for the job, and very likely never will be.

Kay

Re: Mounting only inserted disks

[Kai Hendry]

It's very unfortunate that udev can't distinguish between bootup from
device hotplug. :(

On 29 January 2012 12:23, Kay Sievers <kay.sievers@vrfy.org> wrote:
> Udev's rule engine is not the right place to hookup mounting of
> arbitrary filesystems, or configure non-trivial network settings like
> DHCP, or start system daemons, or run any other programs that runs for
> an unpredictable amount of time. Udev rules should only be used to
> identify or initially configure hardware, but never to execute system
> management jobs or things that involve policy or need error handling
> like filesystem checking or mounting. Running such programs from udev
> rules will block related events, and might render the entire system
> unusable. To ensure timely event execution, udev forcefully kills all
> programs it has executed from rules, and which take longer than 30 or
> 60 seconds to finish, and mounting and checking disks can take much
> longer than that.

I don't expect mounting a disk (or getting an IP for that matter) will
take longer longer than 30 seconds!

> Udev can send events to services which can act on device changes
> though. An auto-mounter service can listen to block device events and
> take the appropriate actions, such a service will not block udev's
> operations for an unpredictable time. Udisks and systemd for example
> work like that.
> Mounting filesystems is just not simple enough to do that in udev, you
> need a real service to do that properly. Udev rules are just not the
> right tool for the job, and very likely never will be.

Crikey. udisks & systemd has a crazy amount of deps and SLOC. Seriously?

I think you guys need to please start sucking less.

Kind regards,

Re: Mounting only inserted disks

[Kay Sievers]

On Sun, Jan 29, 2012 at 11:26, Kai Hendry <hendry@webconverger.com> wrote:
> It's very unfortunate that udev can't distinguish between bootup from
> device hotplug. :(
>
> On 29 January 2012 12:23, Kay Sievers <kay.sievers@vrfy.org> wrote:
>> Udev's rule engine is not the right place to hookup mounting of
>> arbitrary filesystems, or configure non-trivial network settings like
>> DHCP, or start system daemons, or run any other programs that runs for
>> an unpredictable amount of time. Udev rules should only be used to
>> identify or initially configure hardware, but never to execute system
>> management jobs or things that involve policy or need error handling
>> like filesystem checking or mounting. Running such programs from udev
>> rules will block related events, and might render the entire system
>> unusable. To ensure timely event execution, udev forcefully kills all
>> programs it has executed from rules, and which take longer than 30 or
>> 60 seconds to finish, and mounting and checking disks can take much
>> longer than that.
>
> I don't expect mounting a disk (or getting an IP for that matter) will
> take longer longer than 30 seconds!

Reality might not always be what you expect. Mounting dumb filesystems
without fsck is broken, even filesystems which do no require it can
take that long in the kernel to get mounted. Many really large
filesystems can take very  long to mount.

>> Udev can send events to services which can act on device changes
>> though. An auto-mounter service can listen to block device events and
>> take the appropriate actions, such a service will not block udev's
>> operations for an unpredictable time. Udisks and systemd for example
>> work like that.
>> Mounting filesystems is just not simple enough to do that in udev, you
>> need a real service to do that properly. Udev rules are just not the
>> right tool for the job, and very likely never will be.
>
> Crikey. udisks & systemd has a crazy amount of deps and SLOC. Seriously?

Nobody said you need to use it. But you have to get the model right,
and that's certainly what they are doing.

Write your own that does only what you need, if you want; but do it
like they do, with a model that can reliably work, and does not do the
wrong thing like your udev hack.

> I think you guys need to please start sucking less.

I think you need to understand the problem before you pretend you have
solved it and push it to users. System management tasks are not as
easy as you think. What you do with udev here is a completely broken
hack which risks data loss for people who use your tool.

You can do that on your own box, nobody will object to that, but
please do not do that in tools you offer others to use. Other people
are far better of with the tools that people provide that 'suck too
much' in your eyes.

Thanks,
Kay

Re: Mounting only inserted disks

[Karl O. Pinc]

On 01/28/2012 10:23:43 PM, Kay Sievers wrote:

> Mounting filesystems is just not simple enough to do that in udev, 
> you
> need a real service to do that properly. Udev rules are just not the
> right tool for the job, and very likely never will be.

It occurs to me that udev could provide an RUN-like key that
would not only run but would also detach the new process,
as a favor to long-running RUNed processes so that
each does not have to code it's own detach.

Regards,

Karl <kop@meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

P.S. FYI.  Based on this thread I have filed a bug
against the Debian usbmount program, which uses
udev to trigger mounting.
http://bugs.debian.org/cgi-bin/bugreport.cgi?buge8028

Re: Mounting only inserted disks

[Marco d'Itri]

On Jan 30, "Karl O. Pinc" <kop@meme.com> wrote:

> It occurs to me that udev could provide an RUN-like key that
> would not only run but would also detach the new process,
> as a favor to long-running RUNed processes so that
> each does not have to code it's own detach.
Here:

RUN+="/bin/sh -c '/usr/bin/program &'"

HTH.

-- 
ciao,
Marco

Re: Mounting only inserted disks

[Kay Sievers]

On Mon, Jan 30, 2012 at 22:24, Karl O. Pinc <kop@meme.com> wrote:
> On 01/28/2012 10:23:43 PM, Kay Sievers wrote:
>
>> Mounting filesystems is just not simple enough to do that in udev,
>> you
>> need a real service to do that properly. Udev rules are just not the
>> right tool for the job, and very likely never will be.
>
> It occurs to me that udev could provide an RUN-like key that
> would not only run but would also detach the new process,
> as a favor to long-running RUNed processes so that
> each does not have to code it's own detach.

No, udev is not supposed to ever directly start long running system
services or daemons. It has intentionally no support for that.
Services who need to detach from udev rules are simply broken.

Udev is not an init system or service manager, and should not become
one. Systemd or upstart, or whatever else that works like these tools,
should be used if automatic start of services, based on kernel device
events, is needed.

Udev has a pretty advanced netlink-based event-delivery and filter
mechanism, and kernel device enumerator for services to use to. It's
trivial to use from almost any programming language. And that's the
right point to hook a tool into kernel device events, and not by
adding udev rules to fork off tools to run complex operations.

It is just plain wrong to continue anything like the dirty hacks which
are mentioned earlier in this mail thread. They not only risk data
loss and filesystem damage for users, they are also conceptually wrong
for many other reasons.

Services need to be started as a direct child processes of and not
inherit udev's execution context. Services need to handle all sorts of
failures, guard against races caused by running multiple instances of
the same tool, need to offer job control, support manual and automatic
restarting of the service, handle device surprise removal, must not
hook into early boot operations and block or delay basic system setup,
and and and ... None of that is provided by udev and very likely never
will. It takes a lot more safeguards to reliably run a system
management task, than what udev can provide here.

Kay

Re: Mounting only inserted disks

[Allin Cottrell]

On Mon, 30 Jan 2012, Kay Sievers wrote:

> It is just plain wrong to continue anything like the dirty 
> hacks which are mentioned earlier in this mail thread.

I see your point, but could you comment on the use of a udev 
rule that aims solely to auto-mount detachable media such as 
USB sticks? I've been using a mounting rule which starts

KERNEL!="sd[b-z][0-9]", GOTO=<get out>

on a system where the only hard drive is /dev/sda, and sdb and 
higher will only be USB memory devices. This has worked fine, 
although in one respect the rule is now out of date, in that 
it includes

  IMPORT{program}="/sbin/blkid -o udev -p %N"

which I guess should be substituted by

  IMPORT{builtin}="blkid --noraid" (?)

Would this be considered kosher in context?

--
Allin Cottrell
Department of Economics
Wake Forest University, NC

Re: Mounting only inserted disks

[Kay Sievers]

On Mon, Jan 30, 2012 at 23:26, Allin Cottrell <cottrell@wfu.edu> wrote:
> On Mon, 30 Jan 2012, Kay Sievers wrote:
>
>> It is just plain wrong to continue anything like the dirty hacks which are
>> mentioned earlier in this mail thread.
>
> I see your point, but could you comment on the use of a udev rule that aims
> solely to auto-mount detachable media such as USB sticks? I've been using a
> mounting rule which starts
>
> KERNEL!="sd[b-z][0-9]", GOTO=<get out>
>
> on a system where the only hard drive is /dev/sda, and sdb and higher will
> only be USB memory devices.

No, that's not fine. Using the kernel device name to implement
device-is-removable policy is wrong. No real filesystem must ever be
mounted from a udev rule.

The time the kernel takes to mount an on-disk filesystem is
unpredictable. The mount command must not be killed by udev if the we
run into the timeout. Things need to clean up if anything goes wrong.
Many filesystems require an fsck before they are mounted.

Using udev for a local hack is fine, sure, but please never distribute
broken stuff like that to users who might just naively run it. It
risks _their_ data, not yours; and that's where things get nasty, and
where one needs to make sure that dirty hacks like that do not leak
into the public.

There is a reason, why tools like systemd and udisks are larger
projects. You can probably strip all that down, remove the user
interfaces, and implement a simple automounter in 1000+ lines of code,
but surely not in udev rules.

> This has worked fine, although in one respect
> the rule is now out of date, in that it includes
>
>  IMPORT{program}="/sbin/blkid -o udev -p %N"
>
> which I guess should be substituted by
>
>  IMPORT{builtin}="blkid --noraid" (?)

It does not really matter. The external /sbin/blkid will continue to work.

> Would this be considered kosher in context?

No, not at all. :)

Kay