locking a device for exclusive use by userspace process (or kernel thread)

[ydroneaud@opteya.com (Yann Droneaud)]

Hi,

I have a device and I have to write a driver that exposes the
following three operations to kernel modules AND to userspace
programs:

- LOCK

LOCK operation is used to get exclusive access to the device.
If exclusive access is already granted, the caller will wait
for it be released.

- PROCESS

PROCESS can be issued only after a successful LOCK operation.
PROCESS operations can be issued multiple times between LOCK
and UNLOCK.

- UNLOCK

UNLOCK operation release the exclusive access to the device.
This operation can be issued only after a successful LOCK
operation.

In kernelspace, the lock would be implemented using
a semaphore (with an initial count of 1). That's the easy
part.

In userspace, it's expected LOCK and UNLOCK can be issued in
different threads within one process, and, if, while holding
the lock, a process die, either being killed or by exit(),
an implicit UNLOCK should be done.

I've initially thought I could represent the lock by a file
descriptor, map LOCK to open() and  UNLOCK() to close() (with
the added benefit that opened file descriptor will be closed
on exit):

  operation    userspace     kernelspace

    LOCK        open()         ->open()
   PROCESS      ioctl()        ->ioctl()
   UNLOCK       close()        ->release()

open() would block if the lock is already held by some other
process (and would not block if O_NONBLOCK or O_NDELAY is used).

Seems a great scheme.

But that doesn't fit well with Unix / POSIX, as both fork()
and dup{,2,3}() or fcntl(, F_DUPFD*,) creates new references
to open()'ed device:

- A child will inherit the file descriptor for the device
after parent fork(). As the child has a file descriptor for
the device, the parent couldn't UNLOCK by calling close() since
the device won't be released until the child also close() its
copy of the file descriptor.

It should be noted that even if the child has the device opened
by the vertue of inheritance, it does not own the lock, so it
should not be allowed to issue PROCESS operation. But at the
same time, it prevents the real owner from releasing the lock
(... I'm seeing dead locks). So who really own the lock ?

It seems awkward to allow an exclusive lock to be shared
between two different process even if they are parents.

- In the very same process, multiple references to the device
could be created with dup(). So close()ing the file descriptor
returned by open() won't UNLOCK the device, as the device will
only be released once the last reference would be close()'d.

It seems awkward to allow an exclusive lock to be "duplicated"
within a process.


I've tried to (ab)use ->flush() to implement UNLOCK operation so
that a call to close() on the device's file descriptor would
UNLOCK, even if a duplicated file descriptor exists in the same
process or in a child. But this has the unfortunate consequence of
allowing to UNLOCK when closing a duplicated file descriptor
(and closing the file descriptor in the child could also mean
UNLOCK).

So in the end, I'm probably not going to use open() for LOCK and
close() for UNLOCK. I think I have to use dedicated LOCK and
UNLOCK ioctls.

But I failed to find a way to be able to release the lock when
the process owning the lock die / exit.

Perhaps I need a mechanism to be notified of the death of the
process owning the lock.

Do you have such mean in mind ?

Or any other reasonable solution to implement the three operations of 
such driver ?

Regards.

-- 
Yann Droneaud
OPTEYA