Re: sync guest calls made async on host - SQLite performance

[Matthew Tippett <tippettm@gmail.com>]

Thanks for your response.

Remember that I am not raising questions about the relative performance 
of KVM using guests.  The prevailing opinion would be that performance 
of a guest would range anywhere from considerably slower to around the 
same performance as native - depending on workload, tuning the guest and 
configuration.

I am looking further into a particular anomalous result.  The result is 
that SQLite experiences an _order of magnitude_ - 10x beneficial 
advantage when running under KVM.

My only rationalization of this would be as the subject suggests, is 
that somewhere between the host's HDD and the guests file layer 
something is making a synchronous call asynchronous and batching writes 
  together.

Intuitively this feels that running SQLite under at least a KVM 
virtualized environment will be putting the data at considerably higher 
risk than is present in a non-virtualized environment in case of system 
failure.  Performance is inconsequential in this case.

Focusing in particular on one response

 > Maybe it would be more appropriate to actually run the test in a tuned
 > environment and present some results rather than ask a developer to
 > prove KVM is working.

I am not asking for comparative performance results, I am looking for 
more data that indicates if the anomalous performance increase is a 
Ubuntu+KVM+2.6.31 thing or a KVM+2.6.31 thing or a KVM thing.

I am looking to the KVM developers to either confirm that the behaviour 
is safe and expected, or to provide other data points to indicate that 
it is a Ubuntu+2.6.31 or a 2.6.31 thing by showing that when KVM is 
properly configured KVM environment the performance sits in the expected 
"considerably slower to around the same speed".

Regards,

Matthew

-------- Original Message  --------
Subject: Re: sync guest calls made async on host - SQLite performance
From: Ian Woodstock <ian.woodstock@gmail.com>
To: kvm@vger.kernel.org
Date: 09/24/2009 10:11 PM

> The Phoronix Test Suite is designed to test a (client) operating
> system out of the box and it does a good job at that.
> It's certainly valid to run PTS inside a virtual machine but you
> you're going to need to tune the host, in this case Karmic.
> 
> The way you'd configure a client operating system to a server is
> obviously different, for example selecting the right I/O elevator, in
> the case of KVM you'll certainly see benefits there.
> You'd also want to make sure that the guest OS has been optimally
> installed - for exmaple in a VMware environment you'd install VMware
> tools - in KVM you'd ensure that you're using VirtIO in the guest for
> the same reason.
> They you'd also look at optimizations like cpu pinning, use of huge pages, etc.
> 
> Just taking an generic installation of Karmic out of the box and
> running VMs isn't going to give you real insight into the performance
> of KVM. When deploying Linux as a virtualization host you should be
> tuning it.
> It would certainly be appropriate to have a spin of Karmic that was
> designed to run as a virtualization host.
> 
> Maybe it would be more appropriate to actually run the test in a tuned
> environment and present some results rather than ask a developer to
> prove KVM is working.
> 
> 
> 
>> The test itself is a simple usage of SQLite.  It is stock KVM as
>> available in 2.6.31 on Ubuntu Karmic.  So it would be the environment,
>> not the test.
>>
>> So assuming that KVM upstream works as expected that would leave
>> either 2.6.31 having an issue, or Ubuntu having an issue.
>>
>> Care to make an assertion on the KVM in 2.6.31?  Leaving only Ubuntu's
>> installation.
>>
>> Can some KVM developers attempt to confirm that a 'correctly'
>> configured KVM will not demonstrate this behaviour?
>> http://www.phoronix-test-suite.com/ (or is already available in newer
>> distributions of Fedora, openSUSE and Ubuntu.
>>
>> Regards... Matthew
> 
> 
> On 9/24/09, Avi Kivity <avi@redhat.com> wrote:
>> On 09/24/2009 03:31 PM, Matthew Tippett wrote:
>>> Thanks Avi,
>>>
>>> I am still trying to reconcile the your statement with the potential
>>> data risks and the numbers observed.
>>>
>>> My read of your response is that the guest sees a consistent view -
>>> the data is commited to the virtual disk device.  Does a synchronous
>>> write within the guest trigger a synchronous write of the virtual
>>> device within the host?
>>>
>> Yes.
>>
>>> I don't think offering SQLite users a 10 fold increase in performance
>>> with no data integrity risks just by using KVM is a sane proposition.
>>>
>> It isn't, my guess is that the test setup is broken somehow.
>>
>> --
>> Do not meddle in the internals of kernels, for they are subtle and quick to
>> panic.
>>
>>
> 
> --
> Sent from my mobile device
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html