OOM & [OT] util-linux-2.12e

OOM & [OT] util-linux-2.12e

[Andries.Brouwer]

Just released (on ftp.win.tue.nl in /pub/linux-local/utils/util-linux)
util-linux-2.12e.

The reason for this release were complaints that mount and umount
OOM the kernel when the number of mounts is large.
And indeed - I tried with 30000 mounts and the OOM-killer killed
everything in sight, including X's console, making X exit, killing
all remaining processes.

The new versions have been polished a little bit so as not to waste
too much memory, and now survive the 30000 mount/umount test for me.
Further polishing is needed for the case of large numbers of mounts;
when /etc/mtab is not a symlink to /proc/mounts then umount -a has
quadratic behaviour (it updates mtab after each unmount) and that
gets terribly slow.

About OOM: I am still of the opinion that the default state of the
kernel must be one where OOM does not occur and malloc() tells us
that we are out of memory. A system that suddenly decides to kill
all processes is really very poor and unreliable.
Users can enable other behaviours if they don't care about reliability.

About mount: I wondered whether I should rewrite [u]mounts's handling
of /etc/mtab so as to be a bit faster. But it seems a waste of time -
/proc/mounts has many advantages: automatically up-to-date, correct
also when namespaces are used, much faster. On the other hand, /etc/mtab
contains mount options that are sometimes needed later.
If it were possible to store the mount options in the kernel, making
them visible in /proc/mounts, then we could forget /etc/mtab altogether.

People have asked repeatedly for a way to mark lines in /etc/fstab
so as to make clear that such lines are managed by some GUI or other
external program. Labels like "kudzu".
In this release I added a comment convention for /etc/fstab: options
can have a part starting with \; - that part is ignored by mount
but can be used by other programs managing fstab.

If we would put the mount options in /proc/mounts, and introduced
a comment convention (say, the part starting with \: is ignored by
the kernel but can be used by programs reading /proc/mounts),
then /etc/mtab can die. Comments? Better solutions?

About util-linux and stuff: I have maintained various packages
for ten years or so - it may be time to pass things on to someone else.
Write to aeb@cwi.nl if you are interested in taking over or
co-maintaining kbd or man or man-pages or util-linux.

Andries

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Andries.Brouwer@cwi.nl wrote:

> then /etc/mtab can die. Comments? Better solutions?

Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
introduced. So, kill it today from your mount binary! TODAY. ...

Then discuss what is still missing from /proc/self/mounts:

- the 'user' option for umount must be handled in some way
- loop mounts do not map to the real filename, users cant open the
  device node to run losetup on it. I think it will also get relative
  path names.
- fix all broken apps that still rely on mtab. like GNU df(1)

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Andreas Schwab]

Olaf Hering <olh@suse.de> writes:

> - fix all broken apps that still rely on mtab. like GNU df(1)

df does not rely on /etc/mtab.  It relies on getmntent.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Andreas :)

 * Andreas Schwab <schwab@suse.de> dixit:
> > - fix all broken apps that still rely on mtab. like GNU df(1)
> df does not rely on /etc/mtab.  It relies on getmntent.

    Then my GNU df has any problem :???

    $ df --version

df (coreutils) 5.2.1
Written by Torbjorn Granlund, David MacKenzie, and Paul Eggert.

Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    $ mv /etc/mtab /etc/nomtab
    $ df
df: cannot read table of mounted filesystems: No such file or directory

    $ mv /etc/nomtab /etc/mtab
    $ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             78106388  58601124  19505264  76% /
tmpfs                    32768         0     32768   0% /dev/shm
/dev/backup           19530636  15707348   3823288  81% /var/backup

    IMHO, it depeds on mtab. Maybe it is configurable thru
'./configure' at build time?

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Andreas Schwab]

DervishD <lkml@dervishd.net> writes:

>     Hi Andreas :)
>
>  * Andreas Schwab <schwab@suse.de> dixit:
>> > - fix all broken apps that still rely on mtab. like GNU df(1)
>> df does not rely on /etc/mtab.  It relies on getmntent.
>
>     Then my GNU df has any problem :???

No, if any then getmntent.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: OOM & [OT] util-linux-2.12e

[Paulo Marques]

Andreas Schwab wrote:
> DervishD <lkml@dervishd.net> writes:
> 
> 
>>    Hi Andreas :)
>>
>> * Andreas Schwab <schwab@suse.de> dixit:
>>
>>>>- fix all broken apps that still rely on mtab. like GNU df(1)
>>>
>>>df does not rely on /etc/mtab.  It relies on getmntent.
>>
>>    Then my GNU df has any problem :???
> 
> 
> No, if any then getmntent.

I don't get this. From "man getmntent" it seems that getmntent is just a 
parser for /etc/mtab, and that you must call "setmntent" with the 
filename you want to parse.

So if you do "setmntent("/etc/mtab",...)" you're explicitly saying that 
you want getmntent to use /etc/mtab. This is just a open/read in disguise.

Am I missing something?

-- 
Paulo Marques - www.grupopie.com

To err is human, but to really foul things up requires a computer.
Farmers' Almanac, 1978

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Paulo :)

 * Paulo Marques <pmarques@grupopie.com> dixit:
> >>>>- fix all broken apps that still rely on mtab. like GNU df(1)
> >>>df does not rely on /etc/mtab.  It relies on getmntent.
> >>   Then my GNU df has any problem :???
> >No, if any then getmntent.
> I don't get this. From "man getmntent" it seems that getmntent is just a 
> parser for /etc/mtab, and that you must call "setmntent" with the 
> filename you want to parse.

    From the code of coreutils 5.2.1, lib/mountlist.c, df uses the
default name for the mounted filesystems table. Under glibc, it uses
_PATH_MOUNTED that is, effectively, "/etc/mtab". BTW, 'MOUNTED' is
shown in glibc headers as a deprecated alias.

> So if you do "setmntent("/etc/mtab",...)" you're explicitly saying
> that you want getmntent to use /etc/mtab. This is just a open/read
> in disguise.
> Am I missing something?

    Maybe: df (well, lib/mountlist.c) shouldn't try to 'detect' which
kernel are you running, if one with /etc/mtab or a newer one that
only supports /proc/mounts (which needs procfs support, BTW, and that
can be a problem too). Is glibc who should do all that, and try to
open /etc/mtab and /proc/mounts, in that order. Being a userspace
app, df should not mess with kernel interface changes. In the worst
case, it *could* try to open /etc/mtab (with setmntent, I mean) and
if that fails, open /proc/mounts if the operating system is Linux
(which can be easily decided at './configure' time).

    I'm really not sure about what should be fixed. IMHO, getmntent
should only try to open _PATH_MOUNTED. Any other desired behaviour
should be set using setmntent. Just my 0,02 EUR.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Andreas Schwab]

Paulo Marques <pmarques@grupopie.com> writes:

> Andreas Schwab wrote:
>> DervishD <lkml@dervishd.net> writes:
>> 
>>>    Hi Andreas :)
>>>
>>> * Andreas Schwab <schwab@suse.de> dixit:
>>>
>>>>>- fix all broken apps that still rely on mtab. like GNU df(1)
>>>>
>>>>df does not rely on /etc/mtab.  It relies on getmntent.
>>>
>>>    Then my GNU df has any problem :???
>> No, if any then getmntent.
>
> I don't get this. From "man getmntent" it seems that getmntent is just a
> parser for /etc/mtab, and that you must call "setmntent" with the filename
> you want to parse.
>
> So if you do "setmntent("/etc/mtab",...)" you're explicitly saying that
> you want getmntent to use /etc/mtab. This is just a open/read in disguise.
>
> Am I missing something?

No.  You are of course right, but the df sources don't reference /etc/mtab
directly, but use _PATH_MOUNTED from <paths.h>.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Andreas :)

 * Andreas Schwab <schwab@suse.de> dixit:
> >> > - fix all broken apps that still rely on mtab. like GNU df(1)
> >> df does not rely on /etc/mtab.  It relies on getmntent.
> >     Then my GNU df has any problem :???
> No, if any then getmntent.

    I've seen the code: my excuses. Is the libc which needs fixing,
and it probably have been fixed, I don't have the latest version. 

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

>  On Mon, Sep 20, Andries.Brouwer@cwi.nl wrote:
> 
> > then /etc/mtab can die. Comments? Better solutions?
> 
> Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
> introduced. So, kill it today from your mount binary! TODAY. ...

How do you distinguish between manual and automatic loop device setup?
How do you filter /proc/mounts for chroot environments?

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> Hi,
> 
> On Mon, 20 Sep 2004, Olaf Hering wrote:
> 
> >  On Mon, Sep 20, Andries.Brouwer@cwi.nl wrote:
> > 
> > > then /etc/mtab can die. Comments? Better solutions?
> > 
> > Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
> > introduced. So, kill it today from your mount binary! TODAY. ...
> 
> How do you distinguish between manual and automatic loop device setup?

-v

> How do you filter /proc/mounts for chroot environments?

you have a chroot enviroment without /proc mounted?
Then just create /proc/mounts?

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

> > How do you distinguish between manual and automatic loop device setup?
> 
> -v

???

> > How do you filter /proc/mounts for chroot environments?
> 
> you have a chroot enviroment without /proc mounted?
> Then just create /proc/mounts?

$ cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext3 rw 0 0
proc /proc proc rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/hda5 /boot ext2 rw 0 0
/dev/hda11 /home ext3 rw 0 0
/dev/hda12 /test ext3 rw 0 0
none /test/proc proc rw 0 0
/dev/hda11 /test/home ext3 rw 0 0
/dev/hda6 / ext3 rw 0 0
none /proc proc rw 0 0
/dev/hda11 /home ext3 rw 0 0
$ mount
/dev/hda6 on / type ext3 (rw)
proc on /proc type proc (rw)
/dev/hda11 on /home type ext3 (rw)
$ ln -sf /proc/mounts /etc/mtab
$ df
Filesystem           1k-blocks      Used Available Use% Mounted on
rootfs                 2480352   1222760   1232392  50% /
/dev/root              2480352   1222760   1232392  50% /
df: `/dev/pts': No such file or directory
/dev/hda5              2480352   1222760   1232392  50% /boot
/dev/hda11            15874464  14187420   1647044  90% /home
df: `/test': No such file or directory
df: `/test/proc': No such file or directory
df: `/test/home': No such file or directory
/dev/hda6              2480352   1222760   1232392  50% /
/dev/hda11            15874464  14187420   1647044  90% /home

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> Hi,
> 
> On Mon, 20 Sep 2004, Olaf Hering wrote:
> 
> > > How do you distinguish between manual and automatic loop device setup?
> > 
> > -v

What do you mean by auto vs. manual? I dont understand what you mean
here.

> /dev/hda6 / ext3 rw 0 0

df should start with the last / entry. But this will work only if the
chroot is on its own mount point, not if you chroot into a subdir of a
mount point. It could just ignore the error.

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

> > > > How do you distinguish between manual and automatic loop device setup?
> > > 
> > > -v
> 
> What do you mean by auto vs. manual? I dont understand what you mean
> here.

$ mount -oloop image /mnt

vs

$ losetup image /dev/loop0
$ mount /dev/loop0 /mnt

What should umount do, when called with /mnt?

> > /dev/hda6 / ext3 rw 0 0
> 
> df should start with the last / entry. But this will work only if the
> chroot is on its own mount point, not if you chroot into a subdir of a
> mount point. It could just ignore the error.

Relying on any specifc ordering in /proc/mounts is broken.

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> Hi,
> 
> On Mon, 20 Sep 2004, Olaf Hering wrote:
> 
> > > > > How do you distinguish between manual and automatic loop device setup?
> > > > 
> > > > -v
> > 
> > What do you mean by auto vs. manual? I dont understand what you mean
> > here.
> 
> $ mount -oloop image /mnt
> 
> vs
> 
> $ losetup image /dev/loop0
> $ mount /dev/loop0 /mnt
> 
> What should umount do, when called with /mnt?

I have /dev/loop0 in /proc/mounts, umount does nothing wrong here.
What do you see?

> Relying on any specifc ordering in /proc/mounts is broken.

Thats most likely true.  You could bind mount a mtab file.
This specific case is tricky.

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

> > $ mount -oloop image /mnt
> > 
> > vs
> > 
> > $ losetup image /dev/loop0
> > $ mount /dev/loop0 /mnt
> > 
> > What should umount do, when called with /mnt?
> 
> I have /dev/loop0 in /proc/mounts, umount does nothing wrong here.

What exactly is that "nothing wrong"?

> > Relying on any specifc ordering in /proc/mounts is broken.
> 
> Thats most likely true.  You could bind mount a mtab file.
> This specific case is tricky.

This is a rather common case and IMO it can expect a clean solution.

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> Hi,
> 
> On Mon, 20 Sep 2004, Olaf Hering wrote:
> 
> > > $ mount -oloop image /mnt
> > > 
> > > vs
> > > 
> > > $ losetup image /dev/loop0
> > > $ mount /dev/loop0 /mnt
> > > 
> > > What should umount do, when called with /mnt?
> > 
> > I have /dev/loop0 in /proc/mounts, umount does nothing wrong here.
> 
> What exactly is that "nothing wrong"?

It umounts /mnt for me.


-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

> > > > $ mount -oloop image /mnt
> > > > 
> > > > vs
> > > > 
> > > > $ losetup image /dev/loop0
> > > > $ mount /dev/loop0 /mnt
> > > > 
> > > > What should umount do, when called with /mnt?
> > > 
> > > I have /dev/loop0 in /proc/mounts, umount does nothing wrong here.
> > 
> > What exactly is that "nothing wrong"?
> 
> It umounts /mnt for me.

What happens to /dev/loop0?

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> What happens to /dev/loop0?

I dont know, whats supposed to happen? losetup -d?

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Roman Zippel]

Hi,

On Mon, 20 Sep 2004, Olaf Hering wrote:

> > What happens to /dev/loop0?
> 
> I dont know, whats supposed to happen? losetup -d?

Yes, depending on how it was mounted, but that information isn't in 
/proc/mounts.
(BTW how difficult was it to find this out yourself? Have you even tried 
it?)

bye, Roman

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Roman Zippel wrote:

> Hi,
> 
> On Mon, 20 Sep 2004, Olaf Hering wrote:
> 
> > > What happens to /dev/loop0?
> > 
> > I dont know, whats supposed to happen? losetup -d?
> 
> Yes, depending on how it was mounted, but that information isn't in 
> /proc/mounts.
> (BTW how difficult was it to find this out yourself? Have you even tried 
> it?)

Sure I have tried it. I wonder why umount doesnt do the losetup? I have
never looked at the umount sources, nor have I played with all the
possible ways of loop mount.


-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Andreas Schwab]

Olaf Hering <olh@suse.de> writes:

> Sure I have tried it. I wonder why umount doesnt do the losetup?

umount should only call losetup if mount did it too.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Andreas Schwab wrote:

> Olaf Hering <olh@suse.de> writes:
> 
> > Sure I have tried it. I wonder why umount doesnt do the losetup?
> 
> umount should only call losetup if mount did it too.

we could reuse -n for that.

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[CaT]

On Mon, Sep 20, 2004 at 01:38:44PM +0200, Roman Zippel wrote:
> > What do you mean by auto vs. manual? I dont understand what you mean
> > here.
> 
> $ mount -oloop image /mnt
> 
> vs
> 
> $ losetup image /dev/loop0
> $ mount /dev/loop0 /mnt
> 
> What should umount do, when called with /mnt?

Does the kernel crash and burn if you pass the filesystem an option it
does not know about on a mount? If not then just have mount pass all
the options it gets to the kernel, the fs weeds out what it likes and 
the full thing gets stored for use in /proc/self/mounts.

That would mean the above would have loop stored for the first and
not for the second and so umount would know what to do with each
case.

(from another bit of this thread) This would also mean that the user
keyword can be stored too.

-- 
    Red herrings strewn hither and yon.

Re: OOM & [OT] util-linux-2.12e

[Andreas Schwab]

CaT <cat@zip.com.au> writes:

> Does the kernel crash and burn if you pass the filesystem an option it
> does not know about on a mount?

It will refuse to mount.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: OOM & [OT] util-linux-2.12e

[CaT]

On Mon, Sep 20, 2004 at 12:23:30PM +0200, Roman Zippel wrote:
> > > then /etc/mtab can die. Comments? Better solutions?
> > 
> > Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
> > introduced. So, kill it today from your mount binary! TODAY. ...
> 
> How do you distinguish between manual and automatic loop device setup?
> How do you filter /proc/mounts for chroot environments?

My apologies if something like this is already in the source. Just
throwing out an idea to the wolves. ;)

>From /proc/self/mounts I take it that that's meant to be a list of 
mountpoints the requesting process sees. So... why not have instances
of mountlists. The default is what the kernel starts with and each
process inherits. The chroot creates a 2nd instance so that any
processes spawned from that one see the mountlist from that instance.

Each process would, obviously, inherit the mountlist instance from
its parent. 

The hassle happens when you want to deal with the process that began
an instance dieing without taking its children out with it. One way
to deal with it would be to introduce a counter, but that would 
require locking to keep it serialised.

-- 
    Red herrings strewn hither and yon.

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Olaf :)

 * Olaf Hering <olh@suse.de> dixit:
> > then /etc/mtab can die. Comments? Better solutions?
> Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
> introduced. So, kill it today from your mount binary! TODAY. ...

    Bad idea... ;))) I upgraded my 'mount' yesterday. I was using a
mount from Debian, from 1998 more or less, that worked flawlessly
except for the '--bind' feature and things like those. I used
/etc/mtab as a symlink to /proc/mounts, and all worked OK except for
the double root entry and the need to manually call losetup to delete
unused /dev/loop entries.

    But after the upgrade I no longer could umount a filesystem that
I mounted as 'user', because the device is a symlink and the 'user'
option is not stored in /proc/mounts. So my problems were:

> - the 'user' option for umount must be handled in some way
> - loop mounts do not map to the real filename, users cant open the
>   device node to run losetup on it. I think it will also get relative
>   path names.
> - fix all broken apps that still rely on mtab. like GNU df(1)

    Exactly!!! I will add the double root filesystem entry (one from
the device, the other one being 'rootfs / rootfs rw 0 0'). Until all
these problems are solved, is not a good idea to remove /etc/mtab :(

    I have another issue with a chroot environment I have, which has
its own copy of proc and devpts mounted. Those appear in /proc/mounts
twice, but not in /etc/mtab (they appear in the /etc/mtab copy of the
chroot env).

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Helge Hafting]

DervishD wrote:

>    Hi Olaf :)
>
> * Olaf Hering <olh@suse.de> dixit:
>  
>
>>>then /etc/mtab can die. Comments? Better solutions?
>>>      
>>>
>>Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
>>introduced. So, kill it today from your mount binary! TODAY. ...
>>    
>>
>
>    Bad idea... ;))) I upgraded my 'mount' yesterday. I was using a
>mount from Debian, from 1998 more or less, that worked flawlessly
>except for the '--bind' feature and things like those. I used
>/etc/mtab as a symlink to /proc/mounts, and all worked OK except for
>the double root entry and the need to manually call losetup to delete
>unused /dev/loop entries.
>
>    But after the upgrade I no longer could umount a filesystem that
>I mounted as 'user', because the device is a symlink and the 'user'
>option is not stored in /proc/mounts. So my problems were:
>  
>
Using a mtab that is a link to /proc/mounts fails with quota too.
Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
mount options.  These appear in a normal /etc/mtab but not in /proc/mounts,

so the tools gets the mistaken impression that no fs actually use quotas.

Helge Hafting

Re: OOM & [OT] util-linux-2.12e

[Olaf Hering]

 On Mon, Sep 20, Helge Hafting wrote:

> Using a mtab that is a link to /proc/mounts fails with quota too.
> Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
> mount options.  These appear in a normal /etc/mtab but not in /proc/mounts,

I have never played with quota. But: does the kernel or a userland tool
if quota is active for a mount point? smells like a kernel bug.

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, nÜRNBERG

Re: OOM & [OT] util-linux-2.12e

[Helge Hafting]

Olaf Hering wrote:

> On Mon, Sep 20, Helge Hafting wrote:
>
>  
>
>>Using a mtab that is a link to /proc/mounts fails with quota too.
>>Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
>>mount options.  These appear in a normal /etc/mtab but not in /proc/mounts,
>>    
>>
>
>I have never played with quota. But: does the kernel or a userland tool
>if quota is active for a mount point? smells like a kernel bug.
>  
>
The kernel must know that quota is in use, or it'd be unable to
refuse the syscalls when someone tries to go over his quota.

 From "man mount":
       grpquota / noquota / quota / usrquota
              These  options are accepted but ignored.  (However, quota 
utili‐
              ties may react to such strings in /etc/fstab.)

quota utilities indeed react to such strings in /etc/mtab too.

Qutas aren't actually enabled when the mount options are used,
they are enabled when the "quotaon" tool runs.  I guess it uses
some special syscall or ioctl to really turn quota on.

Doing it at mount time instead, byt actually using those options,
seems saner to me.  But I guess they had their reasons. . .

Helge Hafting

Re: OOM & [OT] util-linux-2.12e

[Herbert Poetzl]

On Mon, Sep 20, 2004 at 03:32:59PM +0200, Helge Hafting wrote:
> Olaf Hering wrote:
> 
> >On Mon, Sep 20, Helge Hafting wrote:
> >
> >>Using a mtab that is a link to /proc/mounts fails with quota too.
> >>Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
> >>mount options.  These appear in a normal /etc/mtab but not in 
> >>/proc/mounts,
> >
> >I have never played with quota. But: does the kernel or a userland tool
> >if quota is active for a mount point? smells like a kernel bug.

- to make the quota active (enable it), the mount option
  is required

- to display an enabled quota as mount option, the quota
  on that 'mount point' has to be enabled 

chicken egg thing, eh?

besides that, not every mountpoint can support quota
and quota should (must) not be enabled at mount time
because before the quota is enabled, the quota hash
has to be initialized to the current usage ...

> The kernel must know that quota is in use, or it'd be unable to
> refuse the syscalls when someone tries to go over his quota.

yep, after 'enabling' quota on a specific filesystem
the kernel knows which quota is enabled ...

> From "man mount":
>       grpquota / noquota / quota / usrquota
>              These  options are accepted but ignored.  (However, quota 
> utili???
>              ties may react to such strings in /etc/fstab.)
> 
> quota utilities indeed react to such strings in /etc/mtab too.
> 
> Qutas aren't actually enabled when the mount options are used,
> they are enabled when the "quotaon" tool runs.  I guess it uses
> some special syscall or ioctl to really turn quota on.

yes, and basically the 'mount options' are not stored
by the kernel in any way, so it would be simple to 
remove that check in the quota tools and replace them
by something else ...

> Doing it at mount time instead, byt actually using those options,
> seems saner to me.  But I guess they had their reasons. . .

yes, quota calculation, see above ...

HTH,
Herbert

> Helge Hafting
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: OOM & [OT] util-linux-2.12e

[Helge Hafting]

Herbert Poetzl wrote:

>On Mon, Sep 20, 2004 at 03:32:59PM +0200, Helge Hafting wrote:
>  
>
>>Olaf Hering wrote:
>>
>>    
>>
>>>On Mon, Sep 20, Helge Hafting wrote:
>>>
>>>      
>>>
>>>>Using a mtab that is a link to /proc/mounts fails with quota too.
>>>>Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
>>>>mount options.  These appear in a normal /etc/mtab but not in 
>>>>/proc/mounts,
>>>>        
>>>>
>>>I have never played with quota. But: does the kernel or a userland tool
>>>if quota is active for a mount point? smells like a kernel bug.
>>>      
>>>
>
>- to make the quota active (enable it), the mount option
>  is required
>
>- to display an enabled quota as mount option, the quota
>  on that 'mount point' has to be enabled 
>
>chicken egg thing, eh?
>  
>
Chicken egg by design perhaps, but I can't see
why it have to be that way.

>besides that, not every mountpoint can support quota
>and quota should (must) not be enabled at mount time
>because before the quota is enabled, the quota hash
>has to be initialized to the current usage ...
>  
>
A better way:
Enable the quota at mount time.  If the system isn't ready
for that, i.e. the quota files aren't created/updated - then
refuse the mount.  (Alternatively, mount without quota and
log a complaint.) The administrator can then mount
without quota, run checkquota, and  mount /fs -o remount,usrquota
to turn quota on.

In other words:
quotaon becomes mount -o remount,usrquota
quotaoff becomes mount -o remount,nousrquota
(And/or grpquota of course)

Does it have to be any more complicated than that?


>>Doing it at mount time instead, byt actually using those options,
>>seems saner to me.  But I guess they had their reasons. . .
>>    
>>
>
>yes, quota calculation, see above ...
>  
>
What I don't get is why we have to mount with quota options
that aren't acutally used, and then turn quota on.
Why not mount without quota, and then remount with
quota options when enabling quota for the first time?

The common case should be a fs that was shut down
cleanly and was mounted with quota the last time it
was used.  So it should be able to mount directly
with quota on, because all the on-disk quota information
is valid and up to date.

Helge Hafting

Re: OOM & [OT] util-linux-2.12e

[Herbert Poetzl]

On Tue, Sep 21, 2004 at 09:20:04AM +0200, Helge Hafting wrote:
> Herbert Poetzl wrote:
> 
> >On Mon, Sep 20, 2004 at 03:32:59PM +0200, Helge Hafting wrote:
> > 
> >>Olaf Hering wrote:
> >>
> >>>On Mon, Sep 20, Helge Hafting wrote:
> >>>
> >>>>Using a mtab that is a link to /proc/mounts fails with quota too.
> >>>>Quta tools read /etc/mtab looking for "usrquota" and or "grpquota"
> >>>>mount options.  These appear in a normal /etc/mtab but not in 
> >>>>/proc/mounts,
> >>>>       
> >>>I have never played with quota. But: does the kernel or a userland tool
> >>>if quota is active for a mount point? smells like a kernel bug.
> >
> >- to make the quota active (enable it), the mount option
> > is required
> >
> >- to display an enabled quota as mount option, the quota
> > on that 'mount point' has to be enabled 
> >
> >chicken egg thing, eh?
> >
> Chicken egg by design perhaps, but I can't see
> why it have to be that way.
> 
> >besides that, not every mountpoint can support quota
> >and quota should (must) not be enabled at mount time
> >because before the quota is enabled, the quota hash
> >has to be initialized to the current usage ...
> > 
> >
> A better way:
> Enable the quota at mount time.  If the system isn't ready
> for that, i.e. the quota files aren't created/updated - then
> refuse the mount.  (Alternatively, mount without quota and
> log a complaint.) The administrator can then mount
> without quota, run checkquota, and  mount /fs -o remount,usrquota
> to turn quota on.
> 
> In other words:
> quotaon becomes mount -o remount,usrquota
> quotaoff becomes mount -o remount,nousrquota
> (And/or grpquota of course)
> 
> Does it have to be any more complicated than that?

probably not, but I guess you have to discuss
this with the quota maintainer ... ;)

best,
Herbert

> >>Doing it at mount time instead, byt actually using those options,
> >>seems saner to me.  But I guess they had their reasons. . .
> >
> >yes, quota calculation, see above ...
> > 
> What I don't get is why we have to mount with quota options
> that aren't acutally used, and then turn quota on.
> Why not mount without quota, and then remount with
> quota options when enabling quota for the first time?
> 
> The common case should be a fs that was shut down
> cleanly and was mounted with quota the last time it
> was used.  So it should be able to mount directly
> with quota on, because all the on-disk quota information
> is valid and up to date.
> 
> Helge Hafting
> 

Re: OOM & [OT] util-linux-2.12e

[Tonnerre]

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

Salut,

On Mon, Sep 20, 2004 at 12:59:50PM +0200, DervishD wrote:
>     Bad idea... ;))) I upgraded my 'mount' yesterday. I was using a
> mount from Debian, from 1998 more or less, that worked flawlessly
> except for the '--bind' feature and things like those. I used
> /etc/mtab as a symlink to /proc/mounts, and all worked OK except for
> the double root entry and the need to manually call losetup to delete
> unused /dev/loop entries.

I  keep a  mvmount and  bindmount program  on my  farm for  that exact
purpose. :)

			   Tonnerre

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: OOM & [OT] util-linux-2.12e

[David Gómez]

Hi Olaf ;),

> Andries, /etc/mtab is obsolete since the day when /proc/self/mounts was
> introduced. So, kill it today from your mount binary! TODAY. ...
> 
> Then discuss what is still missing from /proc/self/mounts:

I agree with your approach Olaf. Better _fix_ first mount to remove
totally mtab support, and after put in /proc/self/mount the additional
info that previously only existed in /etc/mtab. Or i'm afraid that
it will never be removed...

Heck, i should learn myself some kernel programming and add the
new options to /proc/self/mounts. I'm really tired of having an
/etc/mtab file in my system...

bye

-- 
David Gómez

"The question of whether computers can think is just like the question of
whether submarines can swim." -- Edsger W. Dijkstra

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Andries :)

 * Andries.Brouwer@cwi.nl <Andries.Brouwer@cwi.nl> dixit:
> If we would put the mount options in /proc/mounts, and introduced
> a comment convention (say, the part starting with \: is ignored by
> the kernel but can be used by programs reading /proc/mounts),
> then /etc/mtab can die. Comments? Better solutions?

    If you add a comment convention to /proc/mounts so you can use it
as a substitute for /etc/mtab, you will probably break the apps that
use /etc/mtab. I was wondering, then... does the kernel *read*
/proc/mounts contents? If the answer is no, then you can add all
syntactic noise you want to /proc/mounts, exporting options needed
for userspace programs, with no problem. You can make /proc/mounts to
look like /etc/mtab. That will solve most of the problems.

    If the kernel needs to read /proc/mounts, then you have a
problem: you will need /etc/mtab as long as you have to use loop
devices, user mounts, etc.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Xavier Bestel]

Le lun 20/09/2004 à 13:06, DervishD a écrit :
> does the kernel *read* /proc/mounts contents?

/proc/mounts is kernel-generated on the fly (it's alive only during the
read() call).

	Xav

Re: OOM & [OT] util-linux-2.12e

[DervishD]

    Hi Xavier :)

 * Xavier Bestel <xavier.bestel@free.fr> dixit:
> > does the kernel *read* /proc/mounts contents?
> /proc/mounts is kernel-generated on the fly (it's alive only during the
> read() call).

    Then you can cripple it with any extra contents you want, am I
wrong? The kernel won't mind...

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: OOM & [OT] util-linux-2.12e

[Xavier Bestel]

Le lun 20/09/2004 à 13:59, DervishD a écrit :
>     Hi Xavier :)
> 
>  * Xavier Bestel <xavier.bestel@free.fr> dixit:
> > > does the kernel *read* /proc/mounts contents?
> > /proc/mounts is kernel-generated on the fly (it's alive only during the
> > read() call).
> 
>     Then you can cripple it with any extra contents you want, am I
> wrong? The kernel won't mind...

Sure.

	Xav

Re: OOM & [OT] util-linux-2.12e

[Herbert Poetzl]

On Mon, Sep 20, 2004 at 02:32:32PM +0200, Xavier Bestel wrote:
> Le lun 20/09/2004 à 13:59, DervishD a écrit :
> >     Hi Xavier :)
> > 
> >  * Xavier Bestel <xavier.bestel@free.fr> dixit:
> > > > does the kernel *read* /proc/mounts contents?
> > > /proc/mounts is kernel-generated on the fly (it's alive only during the
> > > read() call).
> > 
> >     Then you can cripple it with any extra contents you want, am I
> > wrong? The kernel won't mind...
> 
> Sure.


well, yes and no, actually all those funny 'mount' options
provided by the kernel 'on the fly' are just bits which
get set (or cleared) after 'parsing' the mount options
at mount time ...

this means:

 - bad mount option, no mount
 - ignored mount option, no flag
 - known mount option, flag is set (or cleared)

of course it would be possible to 'extend' the kernel
by some 'important' flags which are _not_ used by the
kernel itself ...

HTH,
Herbert

> 	Xav
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: OOM & [OT] util-linux-2.12e

[Jari Ruusu]

Andries.Brouwer@cwi.nl wrote:
> People have asked repeatedly for a way to mark lines in /etc/fstab
> so as to make clear that such lines are managed by some GUI or other
> external program. Labels like "kudzu".
> In this release I added a comment convention for /etc/fstab: options
> can have a part starting with \; - that part is ignored by mount
> but can be used by other programs managing fstab.

How about implementing /etc/fstab option parsing code that is compatible
with existing libc /etc/fstab parsing code:

defaults,noauto,comment=kudzu,rw
                ^^^^^^^^^^^^^

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: OOM & [OT] util-linux-2.12e

[Andries Brouwer]

On Tue, Sep 21, 2004 at 07:45:21PM +0300, Jari Ruusu wrote:

> How about implementing /etc/fstab option parsing code that is compatible
> with existing libc /etc/fstab parsing code:
> 
> defaults,noauto,comment=kudzu,rw
>                 ^^^^^^^^^^^^^

Is there such libc parsing code? Can you tell me which libc?
Which file? Invoked for what function calls?

Andries

Re: OOM & [OT] util-linux-2.12e

[Jari Ruusu]

Andries Brouwer wrote:
> On Tue, Sep 21, 2004 at 07:45:21PM +0300, Jari Ruusu wrote:
> > How about implementing /etc/fstab option parsing code that is compatible
> > with existing libc /etc/fstab parsing code:
> >
> > defaults,noauto,comment=kudzu,rw
> >                 ^^^^^^^^^^^^^
> 
> Is there such libc parsing code? Can you tell me which libc?
> Which file? Invoked for what function calls?

man setmntent

SYNOPSIS
       #include <stdio.h>
       #include <mntent.h>
       FILE *setmntent(const char *filename, const char *type);
       struct mntent *getmntent(FILE *filep);
       int addmntent(FILE *filep, const struct mntent *mnt);
       int endmntent(FILE *filep);
       char *hasmntopt(const struct mntent *mnt, const char *opt);

Mount is not the only piece of code that parses fstab. Even swapon and
swapoff programs that are part of util-linux were broken by this change. The
'comment=fubar' mount option requires two line change to mount.c, and most
of all, does not break any existing fstab parsing code.

Your fstab options comment change means that all code that parses fstab
needs to be modified to understand the new comment separator sequence. If
they are not modified, they will mis-parse the comment separator sequence
and mis-parse options beyond the comment separator sequence.

Not directly related to above, but you need to release new version of
util-linux soon anyway. You intruduced this type of gems to util-linux-2.12e

--- util-linux-2.12d/mount/lomount.c	Sun Jul 11 20:26:41 2004
+++ util-linux-2.12e/mount/lomount.c	Fri Sep 17 01:28:58 2004

+	memset(&loopinfo64, 0, sizeof(loopinfo64));
 
 	close (fd);
---------------^^
+
+	if (i) {
+		ioctl (fd, LOOP_CLR_FD, 0);
-----------------------^^
+		return 1;
+	}
+
 	if (verbose > 1)
 		printf(_("set_loop(%s,%s,%llu): success\n"),
 		       device, file, offset);
 	return 0;
-
- fail:
-	(void) ioctl (fd, LOOP_CLR_FD, 0);
-	close (fd);
-	return 1;
 }

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: OOM & [OT] util-linux-2.12e

[Andries Brouwer]

On Wed, Sep 22, 2004 at 08:05:31PM +0300, Jari Ruusu wrote:
> > > How about implementing /etc/fstab option parsing code that is compatible
> > > with existing libc /etc/fstab parsing code:
> > >
> > > defaults,noauto,comment=kudzu,rw
> > >                 ^^^^^^^^^^^^^
> > 
> > Is there such libc parsing code? Can you tell me which libc?
> > Which file? Invoked for what function calls?
> 
> man setmntent

I thought you were pointing me at a comment convention.
But as far as I can see there is none today.

If on the other hand the above is the suggestion for a
comment convention, then I like it, but it has the disadvantage
that it takes part of the namespace of filesystem mount options.
My version did not do that.

> Not directly related to above, but you need to release new version of
> util-linux soon anyway.

OK - done.

> You intruduced this type of gems to util-linux-2.12e
>  
>  	close (fd);
> +
> +	if (i) {
> +		ioctl (fd, LOOP_CLR_FD, 0);

A merge error. Thanks! Fixed.

Andries

[PS - you write "type of", vaguely suggesting that you could
point out more flaws. If you can, please do.]