RE: [RFC] rdma/uverbs: Sketch for an ioctl framework

Hefty, Sean
2016-05-27T17:24:07Z

> > A lock associated with the idr is held while objects are acquired.
> > A simple reference count is bumped for each object when used. If
> > exclusive access is required, a flag is set. If the exclusive flag
> > is set, then any attempt to acquire the object is rejected and the
> > ioctl fails.
>
> How is that different from your basic idr+kref locking scheme common
> in the kernel?

I don't view it as significantly different. A kref is just an atomic anyway.

> What is a non-blocking exclusive useful for? Removal?

Removal and possibly modify calls. Note that I only apply the exclusive flag to the first object in the list, so some ordering of what the objects are is required. (I didn't expect clients to accept arbitrary ordering anyway.)

We can debate how the ioctl framework should serialize ioctl processing, versus pushing that functionality either into user space or the ioctl handlers. My proposal serialized against device removal only, otherwise it failed the call if exclusive access was required. This supports pushing the serialization either up or down, but it's outside the framework.

> > Coordination with the removal of the kernel client (i.e. device) is
> > done using a rw-lock. Ioctls acquire a read lock, while the removal
> > processing acquires the write lock.
>
> uverbs ended up using a more complex srcu scheme for this citing
> performance reasons.. I argued for the simpler rwlock at the time..

I guess I'm re-arguing for a simpler rwlock then. :) In any case, the actual locking mechanism can change.

> > synchronization seems sufficient. Any lower-level locking is the
> > responsibility of the kernel client.
>
> So there are still object locks, but the core doesn't help with them
> at all.

Moving this into a framework which doesn't understand what the objects actually are would be difficult. I can see this being done by a uverbs core, which is one of the users of the ioctl framework. The ucma is another user of this code.

- Sean