[Qemu-devel] Avoiding nographic_timer exits

[Qemu-devel] Avoiding nographic_timer exits

[David Gibson]

With PowerKVM, exits from KVM to qemu are even more expensive than on
x86.  One significant source of these we're finding (since we usually
work in -nographic mode) is the nographic_timer.

At present, we're using a hack to disable it, but that's obviously not
a long term solution.  From examination, it looks like the only
purpose of this timer is to flush coalesced mmios.  So it seems like
the timer should only be activated when a coalesced mmio region
actually exists, but I'm not entirely sure how to go about that.

Thinking longer term, it seems very odd that a userspace periodic
timer handles this at all.  Surely it would make more sense to use a
kernel timer within KVM, which can be activated only when there are
actually pending coalesced MMIOs in the buffer.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [Qemu-devel] Avoiding nographic_timer exits

[Jan Kiszka]

On 2011-09-30 10:18, David Gibson wrote:
> With PowerKVM, exits from KVM to qemu are even more expensive than on
> x86.  One significant source of these we're finding (since we usually
> work in -nographic mode) is the nographic_timer.
> 
> At present, we're using a hack to disable it, but that's obviously not
> a long term solution.  From examination, it looks like the only
> purpose of this timer is to flush coalesced mmios.  So it seems like
> the timer should only be activated when a coalesced mmio region
> actually exists, but I'm not entirely sure how to go about that.
> 
> Thinking longer term, it seems very odd that a userspace periodic
> timer handles this at all.  Surely it would make more sense to use a
> kernel timer within KVM, which can be activated only when there are
> actually pending coalesced MMIOs in the buffer.

Coalesced MMIO should only be flushed when a device depending on it gets
accessed - either by a VCPU or by the iothread (to update the graphic
output e.g.). We are working on such a concept (to reduce latency for
VCPUs with real-time constraints).

And long-term, we need per-device MMIO buffers to avoid flushing
unrelated content.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] Avoiding nographic_timer exits

[Jan Kiszka]

On 2011-09-30 10:46, Jan Kiszka wrote:
> On 2011-09-30 10:18, David Gibson wrote:
>> With PowerKVM, exits from KVM to qemu are even more expensive than on
>> x86.  One significant source of these we're finding (since we usually
>> work in -nographic mode) is the nographic_timer.
>>
>> At present, we're using a hack to disable it, but that's obviously not
>> a long term solution.  From examination, it looks like the only
>> purpose of this timer is to flush coalesced mmios.  So it seems like
>> the timer should only be activated when a coalesced mmio region
>> actually exists, but I'm not entirely sure how to go about that.
>>
>> Thinking longer term, it seems very odd that a userspace periodic
>> timer handles this at all.  Surely it would make more sense to use a
>> kernel timer within KVM, which can be activated only when there are
>> actually pending coalesced MMIOs in the buffer.
> 
> Coalesced MMIO should only be flushed when a device depending on it gets
> accessed - either by a VCPU or by the iothread (to update the graphic
> output e.g.). We are working on such a concept (to reduce latency for
> VCPUs with real-time constraints).

The MMIO flush in nographic_update dates back to 62a2744ca0. Mmh, I do
understand the need for flushing with active graphic, but I do not see
the need without such output.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] Avoiding nographic_timer exits

[Jan Kiszka]

On 2011-09-30 10:55, Jan Kiszka wrote:
> On 2011-09-30 10:46, Jan Kiszka wrote:
>> On 2011-09-30 10:18, David Gibson wrote:
>>> With PowerKVM, exits from KVM to qemu are even more expensive than on
>>> x86.  One significant source of these we're finding (since we usually
>>> work in -nographic mode) is the nographic_timer.
>>>
>>> At present, we're using a hack to disable it, but that's obviously not
>>> a long term solution.  From examination, it looks like the only
>>> purpose of this timer is to flush coalesced mmios.  So it seems like
>>> the timer should only be activated when a coalesced mmio region
>>> actually exists, but I'm not entirely sure how to go about that.
>>>
>>> Thinking longer term, it seems very odd that a userspace periodic
>>> timer handles this at all.  Surely it would make more sense to use a
>>> kernel timer within KVM, which can be activated only when there are
>>> actually pending coalesced MMIOs in the buffer.
>>
>> Coalesced MMIO should only be flushed when a device depending on it gets
>> accessed - either by a VCPU or by the iothread (to update the graphic
>> output e.g.). We are working on such a concept (to reduce latency for
>> VCPUs with real-time constraints).
> 
> The MMIO flush in nographic_update dates back to 62a2744ca0. Mmh, I do
> understand the need for flushing with active graphic, but I do not see
> the need without such output.

We "need" it in nographic_update for VNC as that one registers its own
refresh timer.

However, the qemu_flush_coalesced_mmio_buffer calls in the update timers
are both misplaced. They belong into the update handlers of those
display device models that use coalesced MMIO. Will write a patch.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux