Re: [RFC] A new SPI API for fast, low-latency regmap peripheral access

[David Jander <david@protonic.nl>]

On Tue, 17 May 2022 19:17:54 +0100
Mark Brown <broonie@kernel.org> wrote:

> On Tue, May 17, 2022 at 05:16:26PM +0200, David Jander wrote:
> > Mark Brown <broonie@kernel.org> wrote:  
> [...]
> > > OK, no - I'm proposing actually putting the message onto the hardware
> > > from interrupt context.  
> 
> > Nice! I like that idea. Do you want to somehow extend spi_async() to do this
> > transparently? So we just need to introduce a second function
> > ("spi_async_await()" ?) which would wait for completion and collect the RX
> > buffer?  
> 
> We shouldn't need a new API to wait for the async operation to complete,
> hopefully the existing one is fine.

Maybe there is something I am not seeing then. The client needs to make two
function calls. One to fill the FIFO and start the transfer, and a second one
to poll the FIFO and read out the RX data. With the existing API, I can only
think of these options:

 1. Call spi_async() twice for just one transfer. While that might work, IMHO
 it is confusing and would require to store the state of the transfer (i.e.
 "need_finishing=true") in the spi_message. But wouldn't that potentially
 break existing code that (for some reason) calls spi_async() twice on the
 same spi_message?

 2. Use a completion or callback. But I don't see how that will work without a
 context switch to some code that completes the completion or calls the
 callback, which is what we are trying to avoid having in the first place.

If you know something I am ignoring here, please advice.

> > To sum up all possible patches you would accept if I understood correctly:  
> 
> >  1. Make the stats/accounting code be NOP with a sysfs or similar toggle.  
> 
> Or otherwise make it unobtrusive (eg, with similar techniques to those
> used by the networking API).

I just tried this out by re-writing the statistics code using u64_stats_sync
and per-cpu statistics, which get totaled on sysfs read access as Andrew Lunn
suggested.
The results are truly amazing!

The overhead caused by statistics in my test dropped from 43us to just 1-2us.

This was tested on a 64-bit machine though, so I don't know how it will affect
32-bit systems. Nor do I have an easy means of testing this. Any ideas?

Also, I have converted all the struct spi_statistics members to u64_stats_t.
It was easier to test this way. Some of the original types were unsigned long,
which can have different sizes on 64bit or 32bit systems... is that
intentional?

> >  2. Enable the re-use of messages with once in lifetime prepare/map/validate.
> > 
> >  3. Introduce spi_async_await() (or similar), to wait for completion of an
> >  async message.
> > 
> >  4. Enable SPI drivers to tell the core (spi.c) under which conditions it can
> >  fire a message asynchronously without the need for the worker queue and
> >  implement support for those cases. Conditions involve max. transfer size, CS
> >  non-sleep access, etc... but it should probably be up to the SPI driver to
> >  decide I guess (ctlr->can_send_uninterruptible(msg)).
> > 
> > Do I miss something?  
> 
> That's roughly it, plus a general push to optimise the hot path.

Perfect.

> > Minor concern about 4. above: Hopefully the decision can be made very quickly
> > (i.e. without trying and failing). Maybe this decision result can be cached in
> > the struct spi_message, so it can be re-used (see point 2)? Maybe as part of
> > prepare or validate?  
> 
> Yes, we need to do this at validation time to play with the reuse I
> think.
> 
> > I feel confident that these 4 modifications will have enough of a performance
> > impact if fully exploited by the MCP2518FD driver, that overhead will no
> > longer be a concern.  
> 
> Just the small matter of implementing them then :/

Well, at least we can agree on a plan. I can't promise anything, but I will
try to continue working on this. Thanks again for the discussion.

Best regards,

-- 
David Jander