[Qemu-devel] linux aio and cache mode

[Qemu-devel] linux aio and cache mode

[ching]

Hi all,

My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"

I am trying to use unsafe cache mode to boost i/o performance.

After googling a while, there are some old web page telling that linux native aio do not use host cache. (http://stackoverflow.com/questions/5664105/buffered-asynchronous-file-i-o-on-linux)

is it still valid for recent kernel? do i need to change to aio="threads" to take advantage of unsafe cache mode?

Thanks in advance.

Regards,
ching

Re: [Qemu-devel] linux aio and cache mode

[Kevin Wolf]

Am 24.09.2012 13:32, schrieb ching:
> Hi all,
> 
> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
> 
> I am trying to use unsafe cache mode to boost i/o performance.

aio=native requires the image to be opened with O_DIRECT, i.e.
cache=none or cache=directsync. If you specify a different cache option,
it will silently fall back to aio=threads.

Kevin

Re: [Qemu-devel] linux aio and cache mode

[ching]

On 09/24/2012 08:30 PM, Kevin Wolf wrote:
> Am 24.09.2012 13:32, schrieb ching:
>> Hi all,
>>
>> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
>>
>> I am trying to use unsafe cache mode to boost i/o performance.
> aio=native requires the image to be opened with O_DIRECT, i.e.
> cache=none or cache=directsync. If you specify a different cache option,
> it will silently fall back to aio=threads.
>
> Kevin
>

will qemu log a entry for the silent fallback?


Reason:

I am testing sparse image on btrfs with mount option: rw,noatime,space_cache,autodefrag,inode_cache

i encounter a speed difference (around 2X-3X) between aio=threads,cache=unsafe and aio=native,cache=unsafe

aio=threads is much faster, i guest there is conflict between "autodefrag" and linux aio

ching

Re: [Qemu-devel] linux aio and cache mode

[Kevin Wolf]

Am 25.09.2012 00:40, schrieb ching:
> On 09/24/2012 08:30 PM, Kevin Wolf wrote:
>> Am 24.09.2012 13:32, schrieb ching:
>>> Hi all,
>>>
>>> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
>>>
>>> I am trying to use unsafe cache mode to boost i/o performance.
>> aio=native requires the image to be opened with O_DIRECT, i.e.
>> cache=none or cache=directsync. If you specify a different cache option,
>> it will silently fall back to aio=threads.
>>
>> Kevin
>>
> 
> will qemu log a entry for the silent fallback?

No, that's why it's silent. :-)

> Reason:
> 
> I am testing sparse image on btrfs with mount option: rw,noatime,space_cache,autodefrag,inode_cache
> 
> i encounter a speed difference (around 2X-3X) between aio=threads,cache=unsafe and aio=native,cache=unsafe
> 
> aio=threads is much faster, i guest there is conflict between "autodefrag" and linux aio

This is odd. The point is that with cache=unsafe it shouldn't even be
using Linux AIO in the first place. I can't see why there would be any
difference between aio=threads and aio=native with cache=unsafe.

Kevin

Re: [Qemu-devel] linux aio and cache mode

[ching]

On 09/25/2012 09:33 PM, Kevin Wolf wrote:
> Am 25.09.2012 00:40, schrieb ching:
>> On 09/24/2012 08:30 PM, Kevin Wolf wrote:
>>> Am 24.09.2012 13:32, schrieb ching:
>>>> Hi all,
>>>>
>>>> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
>>>>
>>>> I am trying to use unsafe cache mode to boost i/o performance.
>>> aio=native requires the image to be opened with O_DIRECT, i.e.
>>> cache=none or cache=directsync. If you specify a different cache option,
>>> it will silently fall back to aio=threads.
>>>
>>> Kevin
>>>
>> will qemu log a entry for the silent fallback?
> No, that's why it's silent. :-)
>
>> Reason:
>>
>> I am testing sparse image on btrfs with mount option: rw,noatime,space_cache,autodefrag,inode_cache
>>
>> i encounter a speed difference (around 2X-3X) between aio=threads,cache=unsafe and aio=native,cache=unsafe
>>
>> aio=threads is much faster, i guest there is conflict between "autodefrag" and linux aio
> This is odd. The point is that with cache=unsafe it shouldn't even be
> using Linux AIO in the first place. I can't see why there would be any
> difference between aio=threads and aio=native with cache=unsafe.
>
> Kevin
>

is it possible to check the open mode of file and whether it is using aio at runtime?

Re: [Qemu-devel] linux aio and cache mode

[Kevin Wolf]

Am 26.09.2012 01:22, schrieb ching:
> On 09/25/2012 09:33 PM, Kevin Wolf wrote:
>> Am 25.09.2012 00:40, schrieb ching:
>>> On 09/24/2012 08:30 PM, Kevin Wolf wrote:
>>>> Am 24.09.2012 13:32, schrieb ching:
>>>>> Hi all,
>>>>>
>>>>> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
>>>>>
>>>>> I am trying to use unsafe cache mode to boost i/o performance.
>>>> aio=native requires the image to be opened with O_DIRECT, i.e.
>>>> cache=none or cache=directsync. If you specify a different cache option,
>>>> it will silently fall back to aio=threads.
>>>>
>>>> Kevin
>>>>
>>> will qemu log a entry for the silent fallback?
>> No, that's why it's silent. :-)
>>
>>> Reason:
>>>
>>> I am testing sparse image on btrfs with mount option: rw,noatime,space_cache,autodefrag,inode_cache
>>>
>>> i encounter a speed difference (around 2X-3X) between aio=threads,cache=unsafe and aio=native,cache=unsafe
>>>
>>> aio=threads is much faster, i guest there is conflict between "autodefrag" and linux aio
>> This is odd. The point is that with cache=unsafe it shouldn't even be
>> using Linux AIO in the first place. I can't see why there would be any
>> difference between aio=threads and aio=native with cache=unsafe.
>>
>> Kevin
>>
> 
> is it possible to check the open mode of file and whether it is using aio at runtime?

You can attach strace and look for open/pwritev/iosubmit and friends. I
did that yesterday with qemu-io and didn't see any iosubmit for 'qemu-io
-k -t unsafe'.

Kevin

Re: [Qemu-devel] linux aio and cache mode

[ching]

On 09/26/2012 03:51 PM, Kevin Wolf wrote:
> Am 26.09.2012 01:22, schrieb ching:
>> On 09/25/2012 09:33 PM, Kevin Wolf wrote:
>>> Am 25.09.2012 00:40, schrieb ching:
>>>> On 09/24/2012 08:30 PM, Kevin Wolf wrote:
>>>>> Am 24.09.2012 13:32, schrieb ching:
>>>>>> Hi all,
>>>>>>
>>>>>> My host is qemu-1.1.1 and x64 kernel 3.5.4. The guest is using aio="native"
>>>>>>
>>>>>> I am trying to use unsafe cache mode to boost i/o performance.
>>>>> aio=native requires the image to be opened with O_DIRECT, i.e.
>>>>> cache=none or cache=directsync. If you specify a different cache option,
>>>>> it will silently fall back to aio=threads.
>>>>>
>>>>> Kevin
>>>>>
>>>> will qemu log a entry for the silent fallback?
>>> No, that's why it's silent. :-)
>>>
>>>> Reason:
>>>>
>>>> I am testing sparse image on btrfs with mount option: rw,noatime,space_cache,autodefrag,inode_cache
>>>>
>>>> i encounter a speed difference (around 2X-3X) between aio=threads,cache=unsafe and aio=native,cache=unsafe
>>>>
>>>> aio=threads is much faster, i guest there is conflict between "autodefrag" and linux aio
>>> This is odd. The point is that with cache=unsafe it shouldn't even be
>>> using Linux AIO in the first place. I can't see why there would be any
>>> difference between aio=threads and aio=native with cache=unsafe.
>>>
>>> Kevin
>>>
>> is it possible to check the open mode of file and whether it is using aio at runtime?
> You can attach strace and look for open/pwritev/iosubmit and friends. I
> did that yesterday with qemu-io and didn't see any iosubmit for 'qemu-io
> -k -t unsafe'.
>
> Kevin
>

The output of strace when using aio=native:

open("XXXXXX", O_RDWR|O_CLOEXEC) = 14


However, i can no longer reproduce the problem the VM image is defragged, it is weird.

Anyway, thank you very much for your help.

ching