Re: [PATCH 1/2] target: Add transport_handle_cdb_direct optimization

[Christoph Hellwig <hch@lst.de>]

On Thu, Jun 16, 2011 at 03:29:30PM -0700, Andy Grover wrote:
> I don't think we should be making it easy on the core, I think we should
> be making it easy on the *fabrics*, if for no other reason that there
> are >1 of them, but only one core. Less code duplicated.

The userspace offload really is just a call to the workqueue code.  I
can't see how it can be made easier by moving it into the core, but if
there's a way to make it easier and I'm certainly not against it.

> Furthermore, many commands can be handled and generate a response
> without submitting a read/write request to the backstores, which is the
> only part that may need to sleep. If we decide all fabric calls to the
> core must be from a thread, then all fabrics that could've gotten a
> response to a command without context switching would then be the ones
> taking unneeded context switches.

There's actually a lot of those commands, but all of them are in the
slow path, that is discovery, error recovery, fail over, misc administration.
All the data path commands, and thas includes cache flushes and discards in
addition to reads and writes in various forms, actually need to do I/O and
allocations.

> It's not clear to me yet what the barriers to fabrics calling core APIs
> from interrupt are at this point. Do we just need to avoid GFP_KERNEL
> allocs? The architecture as-is may be pretty close to doing this, and
> then we'd be close to a reasonable framework.

It jut means a lot of irqsave locking, GFP_ATOMIC or conditional allocations
and generally a lot of pain.  If it would actually help with performance
that might be a worthwile tradeoff, but in the end we do require the process
context for all data path commands anyway.  Adding a new special case for
slow path commands doesn't seem like a worthwhile tradeoff to me.