* About preemption timer
@ 2013-12-17 9:32 Arthur Chunqi Li
2013-12-17 11:21 ` Jan Kiszka
2013-12-17 11:22 ` Paolo Bonzini
0 siblings, 2 replies; 9+ messages in thread
From: Arthur Chunqi Li @ 2013-12-17 9:32 UTC (permalink / raw)
To: kvm; +Cc: Paolo Bonzini, Jan Kiszka
Hi Jan and Paolo,
I've tried to use preemption timer in KVM to trap vcpu regularly, but
there's something unexpected. I run a VM with 4 vcpus and give them
the same preemption timer value (e.g. 1000000) with all bits set
(activate/save bits), then reset the value in preemption time-out
handler.
Thus I expected these vcpus trap regularly in some special turns. But
I found that when the VM is not busy, some vcpus are trapped much less
frequently than others. In Intel SDM, I noticed that preemption timer
is only related to TSC, and I think all the vcpus should trap in a
similar frequency.
Could u help me explain this phenomenon?
Thanks,
Arthur
--
Arthur Chunqi Li
Department of Computer Science
School of EECS
Peking University
Beijing, China
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 9:32 About preemption timer Arthur Chunqi Li
@ 2013-12-17 11:21 ` Jan Kiszka
2013-12-17 11:31 ` Arthur Chunqi Li
2013-12-17 11:22 ` Paolo Bonzini
1 sibling, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2013-12-17 11:21 UTC (permalink / raw)
To: Arthur Chunqi Li, kvm; +Cc: Paolo Bonzini
On 2013-12-17 10:32, Arthur Chunqi Li wrote:
> Hi Jan and Paolo,
>
> I've tried to use preemption timer in KVM to trap vcpu regularly, but
> there's something unexpected. I run a VM with 4 vcpus and give them
> the same preemption timer value (e.g. 1000000) with all bits set
> (activate/save bits), then reset the value in preemption time-out
> handler.
>
> Thus I expected these vcpus trap regularly in some special turns. But
> I found that when the VM is not busy, some vcpus are trapped much less
> frequently than others. In Intel SDM, I noticed that preemption timer
> is only related to TSC, and I think all the vcpus should trap in a
> similar frequency.
>
> Could u help me explain this phenomenon?
Are you on a CPU that has non-broken preemption timer support? Anything
prior Haswell is known to tick with arbitrary frequencies.
BTW, we will have to re-implement preemption timer support with the help
of a regular host timer due to the breakage when halting L2 (see my test
case).
Jan
--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 9:32 About preemption timer Arthur Chunqi Li
2013-12-17 11:21 ` Jan Kiszka
@ 2013-12-17 11:22 ` Paolo Bonzini
1 sibling, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2013-12-17 11:22 UTC (permalink / raw)
To: Arthur Chunqi Li; +Cc: kvm, Jan Kiszka
Il 17/12/2013 10:32, Arthur Chunqi Li ha scritto:
> Hi Jan and Paolo,
>
> I've tried to use preemption timer in KVM to trap vcpu regularly, but
> there's something unexpected. I run a VM with 4 vcpus and give them
> the same preemption timer value (e.g. 1000000) with all bits set
> (activate/save bits), then reset the value in preemption time-out
> handler.
>
> Thus I expected these vcpus trap regularly in some special turns. But
> I found that when the VM is not busy, some vcpus are trapped much less
> frequently than others. In Intel SDM, I noticed that preemption timer
> is only related to TSC, and I think all the vcpus should trap in a
> similar frequency.
Does the preemption timer testcase pass on your machine? The preemption
timer is known to have bugs.
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 11:21 ` Jan Kiszka
@ 2013-12-17 11:31 ` Arthur Chunqi Li
2013-12-17 12:43 ` Jan Kiszka
0 siblings, 1 reply; 9+ messages in thread
From: Arthur Chunqi Li @ 2013-12-17 11:31 UTC (permalink / raw)
To: Jan Kiszka; +Cc: kvm, Paolo Bonzini
Hi Jan,
On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2013-12-17 10:32, Arthur Chunqi Li wrote:
>> Hi Jan and Paolo,
>>
>> I've tried to use preemption timer in KVM to trap vcpu regularly, but
>> there's something unexpected. I run a VM with 4 vcpus and give them
>> the same preemption timer value (e.g. 1000000) with all bits set
>> (activate/save bits), then reset the value in preemption time-out
>> handler.
>>
>> Thus I expected these vcpus trap regularly in some special turns. But
>> I found that when the VM is not busy, some vcpus are trapped much less
>> frequently than others. In Intel SDM, I noticed that preemption timer
>> is only related to TSC, and I think all the vcpus should trap in a
>> similar frequency.
>>
>> Could u help me explain this phenomenon?
>
> Are you on a CPU that has non-broken preemption timer support? Anything
> prior Haswell is known to tick with arbitrary frequencies.
My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.
Besides, what do you mean by arbitrary frequencies?
Arthur
>
> BTW, we will have to re-implement preemption timer support with the help
> of a regular host timer due to the breakage when halting L2 (see my test
> case).
>
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux
--
Arthur Chunqi Li
Department of Computer Science
School of EECS
Peking University
Beijing, China
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 11:31 ` Arthur Chunqi Li
@ 2013-12-17 12:43 ` Jan Kiszka
2013-12-17 12:59 ` Arthur Chunqi Li
0 siblings, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2013-12-17 12:43 UTC (permalink / raw)
To: Arthur Chunqi Li; +Cc: kvm, Paolo Bonzini
On 2013-12-17 12:31, Arthur Chunqi Li wrote:
> Hi Jan,
>
> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2013-12-17 10:32, Arthur Chunqi Li wrote:
>>> Hi Jan and Paolo,
>>>
>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but
>>> there's something unexpected. I run a VM with 4 vcpus and give them
>>> the same preemption timer value (e.g. 1000000) with all bits set
>>> (activate/save bits), then reset the value in preemption time-out
>>> handler.
>>>
>>> Thus I expected these vcpus trap regularly in some special turns. But
>>> I found that when the VM is not busy, some vcpus are trapped much less
>>> frequently than others. In Intel SDM, I noticed that preemption timer
>>> is only related to TSC, and I think all the vcpus should trap in a
>>> similar frequency.
>>>
>>> Could u help me explain this phenomenon?
>>
>> Are you on a CPU that has non-broken preemption timer support? Anything
>> prior Haswell is known to tick with arbitrary frequencies.
>
> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.
Hmm, this one seems unaffected. Didn't find a "specification update".
Just like Paolo asked: Your original test case passes?
>
> Besides, what do you mean by arbitrary frequencies?
On older CPUs, the tick rate of the preemption timer does not correlate
with the TSC, definitely not in the way the spec defined.
Back to your original question: Are we talking about native use of the
preemption timer via a patched KVM or nested use inside a KVM virtual
machine?
Jan
--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 12:43 ` Jan Kiszka
@ 2013-12-17 12:59 ` Arthur Chunqi Li
2013-12-17 15:17 ` Jan Kiszka
0 siblings, 1 reply; 9+ messages in thread
From: Arthur Chunqi Li @ 2013-12-17 12:59 UTC (permalink / raw)
To: Jan Kiszka; +Cc: kvm, Paolo Bonzini
On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2013-12-17 12:31, Arthur Chunqi Li wrote:
>> Hi Jan,
>>
>> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>> On 2013-12-17 10:32, Arthur Chunqi Li wrote:
>>>> Hi Jan and Paolo,
>>>>
>>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but
>>>> there's something unexpected. I run a VM with 4 vcpus and give them
>>>> the same preemption timer value (e.g. 1000000) with all bits set
>>>> (activate/save bits), then reset the value in preemption time-out
>>>> handler.
>>>>
>>>> Thus I expected these vcpus trap regularly in some special turns. But
>>>> I found that when the VM is not busy, some vcpus are trapped much less
>>>> frequently than others. In Intel SDM, I noticed that preemption timer
>>>> is only related to TSC, and I think all the vcpus should trap in a
>>>> similar frequency.
>>>>
>>>> Could u help me explain this phenomenon?
>>>
>>> Are you on a CPU that has non-broken preemption timer support? Anything
>>> prior Haswell is known to tick with arbitrary frequencies.
>>
>> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.
>
> Hmm, this one seems unaffected. Didn't find a "specification update".
> Just like Paolo asked: Your original test case passes?
>
>>
>> Besides, what do you mean by arbitrary frequencies?
>
> On older CPUs, the tick rate of the preemption timer does not correlate
> with the TSC, definitely not in the way the spec defined.
>
> Back to your original question: Are we talking about native use of the
> preemption timer via a patched KVM or nested use inside a KVM virtual
> machine?
It is about the native use. I think it may due to the scheduling. When
vcpu is scheduled out of pcpu, will the preemption timer work still?
Oh, another problem, I use the released kernel 3.11, not the latest
one. Does this matter?
Arthur
>
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux
--
Arthur Chunqi Li
Department of Computer Science
School of EECS
Peking University
Beijing, China
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 12:59 ` Arthur Chunqi Li
@ 2013-12-17 15:17 ` Jan Kiszka
2013-12-18 3:27 ` R
0 siblings, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2013-12-17 15:17 UTC (permalink / raw)
To: Arthur Chunqi Li; +Cc: kvm, Paolo Bonzini
On 2013-12-17 13:59, Arthur Chunqi Li wrote:
> On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2013-12-17 12:31, Arthur Chunqi Li wrote:
>>> Hi Jan,
>>>
>>> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>> On 2013-12-17 10:32, Arthur Chunqi Li wrote:
>>>>> Hi Jan and Paolo,
>>>>>
>>>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but
>>>>> there's something unexpected. I run a VM with 4 vcpus and give them
>>>>> the same preemption timer value (e.g. 1000000) with all bits set
>>>>> (activate/save bits), then reset the value in preemption time-out
>>>>> handler.
>>>>>
>>>>> Thus I expected these vcpus trap regularly in some special turns. But
>>>>> I found that when the VM is not busy, some vcpus are trapped much less
>>>>> frequently than others. In Intel SDM, I noticed that preemption timer
>>>>> is only related to TSC, and I think all the vcpus should trap in a
>>>>> similar frequency.
>>>>>
>>>>> Could u help me explain this phenomenon?
>>>>
>>>> Are you on a CPU that has non-broken preemption timer support? Anything
>>>> prior Haswell is known to tick with arbitrary frequencies.
>>>
>>> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.
>>
>> Hmm, this one seems unaffected. Didn't find a "specification update".
>> Just like Paolo asked: Your original test case passes?
>>
>>>
>>> Besides, what do you mean by arbitrary frequencies?
>>
>> On older CPUs, the tick rate of the preemption timer does not correlate
>> with the TSC, definitely not in the way the spec defined.
>>
>> Back to your original question: Are we talking about native use of the
>> preemption timer via a patched KVM or nested use inside a KVM virtual
>> machine?
>
> It is about the native use. I think it may due to the scheduling. When
> vcpu is scheduled out of pcpu, will the preemption timer work still?
The preemption timer ticks as long as the guest is running. Should be
specified like this as well. So your KVM patch needs to take care of
this when you want to expire it based on real-time, not based on guest
time. That's in fact similar to adjustments you implemented for the
emulation of the preemption timer.
Jan
>
> Oh, another problem, I use the released kernel 3.11, not the latest
> one. Does this matter?
>
> Arthur
>
>>
>> Jan
>>
>> --
>> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
>> Corporate Competence Center Embedded Linux
>
>
>
--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-17 15:17 ` Jan Kiszka
@ 2013-12-18 3:27 ` R
2013-12-18 6:58 ` Jan Kiszka
0 siblings, 1 reply; 9+ messages in thread
From: R @ 2013-12-18 3:27 UTC (permalink / raw)
To: Jan Kiszka; +Cc: Arthur Chunqi Li, kvm, Paolo Bonzini
Hi,
You must adjust the preemption timer according to the eclipsed time
when the guest runs.
And register a hrtimer triggered when a guest exit.
The timer should be alarmed when the remaining time runs out,
2013/12/17 Jan Kiszka <jan.kiszka@siemens.com>:
> On 2013-12-17 13:59, Arthur Chunqi Li wrote:
>> On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>> On 2013-12-17 12:31, Arthur Chunqi Li wrote:
>>>> Hi Jan,
>>>>
>>>> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>>> On 2013-12-17 10:32, Arthur Chunqi Li wrote:
>>>>>> Hi Jan and Paolo,
>>>>>>
>>>>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but
>>>>>> there's something unexpected. I run a VM with 4 vcpus and give them
>>>>>> the same preemption timer value (e.g. 1000000) with all bits set
>>>>>> (activate/save bits), then reset the value in preemption time-out
>>>>>> handler.
>>>>>>
>>>>>> Thus I expected these vcpus trap regularly in some special turns. But
>>>>>> I found that when the VM is not busy, some vcpus are trapped much less
>>>>>> frequently than others. In Intel SDM, I noticed that preemption timer
>>>>>> is only related to TSC, and I think all the vcpus should trap in a
>>>>>> similar frequency.
>>>>>>
>>>>>> Could u help me explain this phenomenon?
>>>>>
>>>>> Are you on a CPU that has non-broken preemption timer support? Anything
>>>>> prior Haswell is known to tick with arbitrary frequencies.
>>>>
>>>> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.
>>>
>>> Hmm, this one seems unaffected. Didn't find a "specification update".
>>> Just like Paolo asked: Your original test case passes?
>>>
>>>>
>>>> Besides, what do you mean by arbitrary frequencies?
>>>
>>> On older CPUs, the tick rate of the preemption timer does not correlate
>>> with the TSC, definitely not in the way the spec defined.
>>>
>>> Back to your original question: Are we talking about native use of the
>>> preemption timer via a patched KVM or nested use inside a KVM virtual
>>> machine?
>>
>> It is about the native use. I think it may due to the scheduling. When
>> vcpu is scheduled out of pcpu, will the preemption timer work still?
>
> The preemption timer ticks as long as the guest is running. Should be
> specified like this as well. So your KVM patch needs to take care of
> this when you want to expire it based on real-time, not based on guest
> time. That's in fact similar to adjustments you implemented for the
> emulation of the preemption timer.
>
> Jan
>
>>
>> Oh, another problem, I use the released kernel 3.11, not the latest
>> one. Does this matter?
>>
>> Arthur
>>
>>>
>>> Jan
>>>
>>> --
>>> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
>>> Corporate Competence Center Embedded Linux
>>
>>
>>
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SES-DE
> Corporate Competence Center Embedded Linux
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Thanks
Rui Wu
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer
2013-12-18 3:27 ` R
@ 2013-12-18 6:58 ` Jan Kiszka
0 siblings, 0 replies; 9+ messages in thread
From: Jan Kiszka @ 2013-12-18 6:58 UTC (permalink / raw)
To: R; +Cc: Arthur Chunqi Li, kvm, Paolo Bonzini
On 2013-12-18 04:27, R wrote:
> Hi,
>
> You must adjust the preemption timer according to the eclipsed time
> when the guest runs.
>
> And register a hrtimer triggered when a guest exit.
> The timer should be alarmed when the remaining time runs out,
In fact, as we need to register an hrtimer anyway when leaving the
guest, we can simply register it always and stop using the physical
preemption timer. This will also solve the breakage on older Intel CPU,
actually it will add preemption timer support unconditionally.
Jan
--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-12-18 6:58 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-17 9:32 About preemption timer Arthur Chunqi Li
2013-12-17 11:21 ` Jan Kiszka
2013-12-17 11:31 ` Arthur Chunqi Li
2013-12-17 12:43 ` Jan Kiszka
2013-12-17 12:59 ` Arthur Chunqi Li
2013-12-17 15:17 ` Jan Kiszka
2013-12-18 3:27 ` R
2013-12-18 6:58 ` Jan Kiszka
2013-12-17 11:22 ` Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).