Re: [Qemu-devel] [RFC 1/2] pci-dma-api-v1

[Avi Kivity <avi@redhat.com>]

Andrea Arcangeli wrote:
> On Sat, Nov 29, 2008 at 09:48:22PM +0200, Avi Kivity wrote:
>   
>> Since Andrea's patches contain emulation for aio readv/writev, the 
>> performance degradation will not occur (though we will not see the benefit 
>> either).
>>     
>
> It will not occur especially if you #define DEBUG_BOUNCE. The way I
> emulated bdrv_aio_readv/writev didn't involve a bounce, but it's
> serially submitting the I/O so that it truly runs zerocopy when
> DEBUG_BOUNCE is not defined ;).
>
> IF you enable DEBUG_BOUNCE then the bounce layer will be forced on,
> and the iovec will be linearized and the DMA command executed on the
> hardware will be allowed to be as large as MAX_DMA_BOUNCE_BUFFER like
> before. So until we have a real bdrv_aio_readv/writev #defining
> DEBUG_BOUNCE is a must with cache=off.
>
>   

Oh okay.  In that case it should be committed with DEBUG_BOUNCE enabled, 
and that removed when we have proper aio *v.

>> I doubt you can get measure malloc overhead with anything less a thousand 
>> disks (even there, other overheads are likely to drown that malloc).
>>     
>
> I also eliminated any sign of malloc in the direct path with a small
> cache layer that caches as many pci dma sg params (with the max iovcnt
> seen so far embedded into it) as the max number of simultaneous
> in-flight I/O seen for the whole runtime. With a max param of 10 (so
> only if there are more than 10 simultaneous sg dma I/O operations in
> flight, malloc will have to run). Only params with iov arrays with a
> iovcnt < 2048 are cached. So worst case ram waste is limited, and it's
> auto-tuning at runtime to remain near zero for single disk setups etc..
>   

Overkill IMO (glibc likely caches mallocs), but can't hurt.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.