Proposal for a new lock type

Proposal for a new lock type

[Florian Weimer]

It seems that it might be useful to have a locally shared, globally
exclusive file lock.  "Local" is defined as "when the file is mapped
shared, memory accesses follow the architecture memory model across
threads and processes, and shared POSIX mutexes work".  This could be
used to make the new WAL code in SQLite 3.7 more foolproof, but there
are other applications which would benefit as well.

It seems to me that this would not need changes to network file
systems because you can implement it by acquring a traditional
fcntl()-style exclusive lock on the network side, without creating a
corresponding local lock.

Re: Proposal for a new lock type

[Matthew Wilcox]

On Sun, Aug 15, 2010 at 05:06:58PM +0200, Florian Weimer wrote:
> It seems that it might be useful to have a locally shared, globally
> exclusive file lock.  "Local" is defined as "when the file is mapped
> shared, memory accesses follow the architecture memory model across
> threads and processes, and shared POSIX mutexes work".  This could be
> used to make the new WAL code in SQLite 3.7 more foolproof, but there
> are other applications which would benefit as well.

This sounds like you're trying to fix a problem with NFS.  I think there
are network filesystems which practice mmap coherency.  We could define
a sideband protocol for NFS that would allow NFS to act the same way.

> It seems to me that this would not need changes to network file
> systems because you can implement it by acquring a traditional
> fcntl()-style exclusive lock on the network side, without creating a
> corresponding local lock.

fcntl locks are advisory ... I think you actually want leases / oplocks.
More than that, you probably want range oplocks rather than file scope
oplocks.

-- 
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: Proposal for a new lock type

[Florian Weimer]

* Matthew Wilcox:

> On Sun, Aug 15, 2010 at 05:06:58PM +0200, Florian Weimer wrote:
>> It seems that it might be useful to have a locally shared, globally
>> exclusive file lock.  "Local" is defined as "when the file is mapped
>> shared, memory accesses follow the architecture memory model across
>> threads and processes, and shared POSIX mutexes work".  This could be
>> used to make the new WAL code in SQLite 3.7 more foolproof, but there
>> are other applications which would benefit as well.
>
> This sounds like you're trying to fix a problem with NFS.  I think there
> are network filesystems which practice mmap coherency.

In the sense that I described?  This surprises me.

(I know that there is something called Distributed Shared Memory.  But
it's generally got different properties than a mapped file, AFAIK.)

>> It seems to me that this would not need changes to network file
>> systems because you can implement it by acquring a traditional
>> fcntl()-style exclusive lock on the network side, without creating a
>> corresponding local lock.
>
> fcntl locks are advisory ...

That's not a problem for this use case.  It's about making sure that
an SQLite-using application cannot accidentally access the same
database without proper locking because the mutexes are not visible
across a network file system.