Does mandatory locking need to be set when the file is locked?

Does mandatory locking need to be set when the file is locked?

[Joseph D. Wagner]

I'm writing a program that changes the permissions on a file to enable 
mandatory locking, does its thing to the file, and then resets the 
permissions.

Do do the mandatory locking permissions have to be set before the lock is 
set? Or will the kernel dynamically recognize the change on the fly so that 
an active lock becomes mandatory?

TIA.

Joseph D. Wagner

Re: Does mandatory locking need to be set when the file is locked?

[Matthew Wilcox]

On Mon, Dec 15, 2003 at 07:57:57AM +0600, Joseph D. Wagner wrote:
> I'm writing a program that changes the permissions on a file to enable 
> mandatory locking, does its thing to the file, and then resets the 
> permissions.
> 
> Do do the mandatory locking permissions have to be set before the lock is 
> set? Or will the kernel dynamically recognize the change on the fly so that 
> an active lock becomes mandatory?

Please, don't use mandatory locking.  It's racy (by design), buggy
(by design) and adds code to the fast path.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: Does mandatory locking need to be set when the file is locked?

[Joseph D. Wagner]

> Please, don't use mandatory locking.  It's racy (by design), buggy
> (by design) and adds code to the fast path.

What!?!  Are you saying mandatory locking doesn't always work?

Exactly what are the known bugs, and when do they happen?

My program DEPENDS upon mandatory locking, so I'd like a little more info before I abandon that route entirely.

Thanks for your time.

Joseph D. Wagner

Re: Does mandatory locking need to be set when the file is locked?

[Randy.Dunlap]

On Mon, 15 Dec 2003 09:50:18 -0600 "Joseph D. Wagner" <theman@josephdwagner.info> wrote:

| > Please, don't use mandatory locking.  It's racy (by design), buggy
| > (by design) and adds code to the fast path.


provide suggested alternatives...

| What!?!  Are you saying mandatory locking doesn't always work?
| 
| Exactly what are the known bugs, and when do they happen?
| 
| My program DEPENDS upon mandatory locking, so I'd like a little more info before I abandon that route entirely.
| 
| Thanks for your time.


--
~Randy
MOTD:  Always include version info.

Re: Does mandatory locking need to be set when the file is locked?

[Trond Myklebust]

>>>>> " " == Joseph D Wagner <theman@josephdwagner.info> writes:

     > My program DEPENDS upon mandatory locking, so I'd like a little
     > more info before I abandon that route entirely.

Then don't be surprised if it cannot work on networked filesystems.

NFS is, for instance, incapable of supporting mandatory locks in v2
and v3 (v4 will support it for ordinary files).

What's more, mmap() is forbidden on on any file with mandatory
locks (this is true for all filesystems).

Finally, the Linux mandatory locking implementation does not eliminate
all possible races: currently there is no attempt made to exclude one
thread from taking a lock while another is still in the process of
writing to the file, or if it already has a section of that file
mmapped. In that case, the lock call may well succeed, but the
other thread that is inside read/write/mmap will not see it.

So if you want to DEPEND on mandatory locking to work, then you
probably have some kernel debugging to do first.

Cheers,
  Trond

Re: Does mandatory locking need to be set when the file is locked?

[Matthew Wilcox]

On Mon, Dec 15, 2003 at 01:09:11PM -0500, Trond Myklebust wrote:
> Finally, the Linux mandatory locking implementation does not eliminate
> all possible races: currently there is no attempt made to exclude one
> thread from taking a lock while another is still in the process of
> writing to the file, or if it already has a section of that file
> mmapped. In that case, the lock call may well succeed, but the
> other thread that is inside read/write/mmap will not see it.
> 
> So if you want to DEPEND on mandatory locking to work, then you
> probably have some kernel debugging to do first.

And fixing these problems requires some pretty invasive work to
current hot code paths like read/write/mmap/open/close.  When I started
investigating it, I was told the additional overhead for my scheme was
unacceptable and to just forget about it because nobody really used
mandatory locks anyway.

That was enough to put me off.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: Does mandatory locking need to be set when the file is locked?

[Bryan Henderson]

>When I started
>investigating it, I was told the additional overhead for my scheme was
>unacceptable and to just forget about it because nobody really used
>mandatory locks anyway.

I don't mean to be argumentative, but discussions like this lead me to 
suspect that the reason no one is using mandatory locks is _because_ they 
don't work.  So while it's true that fixing them won't fix a bunch of 
broken systems, fixing them would still add valuable function to Linux.  I 
believe people are using mandatory range locks a lot on Windows (where 
they work).  Maybe those people would like to be able to port applications 
to Linux.

So anyone else considering finding a viable way to make mandatory range 
locks work (or even just get closer to working than they are now) should 
not be discouraged.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: Does mandatory locking need to be set when the file is locked?

[Mike Fedyk]

On Tue, Dec 16, 2003 at 11:13:36AM -0800, Bryan Henderson wrote:
> I don't mean to be argumentative, but discussions like this lead me to 
> suspect that the reason no one is using mandatory locks is _because_ they 
> don't work.  So while it's true that fixing them won't fix a bunch of 

I tend to agree.

> So anyone else considering finding a viable way to make mandatory range 
> locks work (or even just get closer to working than they are now) should 
> not be discouraged.

Another problem, is that with standard network filesystems (read NFS < 4)
the suggestive locks aren't propagated unless you use lock files!

What linux needs is to stop using stateless network filesystems.  While they
have their upsides, they do have their downsides.

Re: Does mandatory locking need to be set when the file is locked?

[Trond Myklebust]

På ty , 16/12/2003 klokka 15:33, skreiv Mike Fedyk:
> Another problem, is that with standard network filesystems (read NFS < 4)
> the suggestive locks aren't propagated unless you use lock files!

Huh? If by 'suggestive' locks you mean advisory locks, then the above is
pure FUD.

The Linux NFS implementation has been stateful ever since 2.2.0, when
Olaf added full support for POSIX byte-range locks for v2 and v3 over
the standard NLM protocol.
Nobody has yet used it in order to implement support for BSD locks, but
that is at least partly because it would break with the "standard"
behaviour that BSD locks and POSIX locks should not see each other
(remember those "sendmail" bugs a couple of years ago?).

All NFSv4 adds of value when it comes to locking is support for
mandatory locking, and a faster, more robust protocol than NLM ever was.
Otherwise the combination NFSv3 w/ NLM has been keeping most people's
mailboxes safe for several years now...

Cheers,
  Trond
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Does mandatory locking need to be set when the file is locked?

[Mike Fedyk]

On Tue, Dec 16, 2003 at 04:48:49PM -0500, Trond Myklebust wrote:
> P? ty , 16/12/2003 klokka 15:33, skreiv Mike Fedyk:
> > Another problem, is that with standard network filesystems (read NFS < 4)
> > the suggestive locks aren't propagated unless you use lock files!
> 
> Huh? If by 'suggestive' locks you mean advisory locks, then the above is
> pure FUD.
> 

I stand corrected.

> The Linux NFS implementation has been stateful ever since 2.2.0, when
> Olaf added full support for POSIX byte-range locks for v2 and v3 over
> the standard NLM protocol.
> Nobody has yet used it in order to implement support for BSD locks, but
> that is at least partly because it would break with the "standard"
> behaviour that BSD locks and POSIX locks should not see each other
> (remember those "sendmail" bugs a couple of years ago?).

Oh, the joys of netatalk using flock, and samba using fcntl... :(

> All NFSv4 adds of value when it comes to locking is support for
> mandatory locking, and a faster, more robust protocol than NLM ever was.
> Otherwise the combination NFSv3 w/ NLM has been keeping most people's
> mailboxes safe for several years now...

I'm very happy to hear this.

Thanks

Re: Does mandatory locking need to be set when the file is locked?

[Bryan Henderson]

>Do do the mandatory locking permissions have to be set before the lock is 

>set? Or will the kernel dynamically recognize the change on the fly so 
that 
>an active lock becomes mandatory?

It's the latter.  The permission bits (and mount option) matter when you 
read or write the file, not when you create the lock.

But dynamically changing the permission bits is not the model that the 
architects of this hack had in mind, so don't be surprised if it isn't a 
smooth road using it that way.