Re: [RFC] [PATCH 0/3] ioat: DMA engine support

[Andi Kleen <ak@suse.de>]

On Wed, Nov 23, 2005 at 07:17:01PM -0500, Benjamin LaHaise wrote:
> On Wed, Nov 23, 2005 at 11:30:08PM +0100, Andi Kleen wrote:
> > The main problem I see is that it'll likely only pay off when you can keep 
> > the queue of copies long (to amortize the cost of 
> > talking to an external chip). At least for the standard recvmsg 
> > skb->user space, user space-> skb cases these queues are 
> > likely short in most cases. That's because most applications
> > do relatively small recvmsg or sendmsgs. 
> 
> Don't forget that there are benefits of not polluting the cache with the 
> traffic for the incoming skbs.

Is that a general benefit outside benchmarks? I would expect
most real programs to actually do something with the data
- and that usually involves needing it in cache.

> > But it's not clear it's a good idea: a lot of these applications prefer to 
> > have the target in cache. And IOAT will force it out of cache.
> 
> In the I/O AT case it might make sense to do a few prefetch()es of the 
> userland data on the return-to-userspace code path.  

Some prefetches for user space might be a good idea yes

> Similarly, we should 
> make sure that network drivers prefetch the header at the earliest possible 
> time, too.

It's done kind of already but tricky to get right because
the prefetch distances upto use are not really long enough


-Andi