RE: [RFC] rdma/uverbs: Sketch for an ioctl framework

Hefty, Sean
2016-05-25T19:59:29Z

> It is time for the people who are hand waving about non-verbs, non-qp,
> 'new data transfers' to sit down and specify what they want to see out
> of the kernel uABI.

I want the core ioctl framework to do this:

1. Map user objects to/from the kernel to validate pointers.
2. Provide reference counting on such objects, in order to protect against the user attempting to destroy objects that are in use.
3. Provide protection that the state of an object is not driven in two different directions.
4. Safely process the removal of the underlying driver or device.
5. Support driver specific operations to the kernel.
6. Report events from multiple sources to a single file descriptor.
7. Minimize the kernel footprint for allocated kernel objects.
8. Minimize the code path needed to process any ioctl, without greatly affecting maintainability.
9. Copy ioctl input/output data to/from the kernel.

If designed correctly, I believe that the ioctl core will be suitably generic to support uverbs, ucm (if kept), urdma cm, umad (if there's a reason to add new ioctls), and any drivers. Honestly, there's very little that would stand in the way of future subsystems (e.g. accelerators like Xeon Phi, maybe nvm) adopting it, if they so choose, though I'm not suggesting that be a goal.

Even limiting the discussion to short term needs, the rdma cm requires the same update. The same ioctl core should work there as well. Otherwise, it's a very good indication that we will not meet the needs of whatever future drivers are thrown to linux-rdma, except to say use a driver specific channel and re-implement the above functionality. I would go as far as mixing rdma cm, verbs, and driver ioctls on the same fd (driven by shared event reporting), and that's a different file structure than what we have today.

IMO, the format of the ioctl data is a much lower concern and not something the framework should be concerned about. From the above requirements, the framework needs to understand specific types of operations: creation, close, and ones that require exclusive access. Event handling requires more thought.

I started re-working my patch to address the rdma cm. The original proposal wasn't generic enough and provided a poor fit, because the rdma cm has a single object and 25 versions of 'modify'.

- Sean