Re: [Qemu-devel] [PATCH] raw-posix: Linearize direct I/O on Linux NFS

[Badari Pulavarty <pbadari@us.ibm.com>]

On 4/15/2011 4:00 PM, Anthony Liguori wrote:
> On 04/15/2011 05:21 PM, pbadari@linux.vnet.ibm.com wrote:
>> On 4/15/2011 10:29 AM, Christoph Hellwig wrote:
>>> On Fri, Apr 15, 2011 at 09:23:54AM -0700, Badari Pulavarty wrote:
>>>> True. That brings up a different question - whether we are doing
>>>> enough testing on mainline QEMU :(
>>> It seems you're clearly not doing enough testing on any qemu.  Even
>>> the RHEL6 qemu has had preadv/pwritev since the first beta.
>>
>> Christoph,
>>
>> When you say "you're" - you really meant RH right ? RH should have 
>> caught this in their
>> regression year ago as part of their first beta. Correct ?
>>
>> Unfortunately, you are picking on person who spent time find & 
>> analyzing the regression,
>> narrowing the problem area and suggesting approaches to address the 
>> issue :(
>
> This is a pretty silly discussion to be having.
>
> The facts are:
>
> 1) NFS sucks with preadv/pwritev and O_DIRECT -- is anyone really 
> surprised?
>
> 2) We could work around this in QEMU by doing something ugly
>
> 3) We have no way to detect when we no longer need a work around which 
> makes (2) really unappealing.
>
> 4) That leaves us with:
>     a) waiting for NFS to get fixed properly and just living with 
> worse performance on older kernels
>
>     b) having a user-tunable switch to enable bouncing
>
> I really dislike the idea of (b) because we're stuck with it forever 
> and it's yet another switch for people to mistakenly depend on.
>
> I'm still waiting to see performance data without O_DIRECT.  I suspect 
> that using cache=writethrough will make most of this problem go away 
> in which case, we can just recommend that as a work around until NFS 
> is properly fixed.

We need to run through all cases and analyze the performance of 
cache=writethrough. Our initial (smaller setup) analysis
indicates that its better than unpatched O_DIRECT - but ~5% slower for 
sequential writes. But 30%+ slower for
random read/writes and mixed IO workloads. (In the past NFS O_SYNC is 
performance extremely poor compared to
O_DIRECT with no scaling - older kernels due to congestion control issues).

Khoa would collect the data over next few days.

To be honest with you, we should kill cache=none and just optimize only 
one case and live with it (like other commerical
hypervisor). :(

Thanks,
Badari