* About preemption timer @ 2013-12-17 9:32 Arthur Chunqi Li 2013-12-17 11:21 ` Jan Kiszka 2013-12-17 11:22 ` Paolo Bonzini 0 siblings, 2 replies; 9+ messages in thread From: Arthur Chunqi Li @ 2013-12-17 9:32 UTC (permalink / raw) To: kvm; +Cc: Paolo Bonzini, Jan Kiszka Hi Jan and Paolo, I've tried to use preemption timer in KVM to trap vcpu regularly, but there's something unexpected. I run a VM with 4 vcpus and give them the same preemption timer value (e.g. 1000000) with all bits set (activate/save bits), then reset the value in preemption time-out handler. Thus I expected these vcpus trap regularly in some special turns. But I found that when the VM is not busy, some vcpus are trapped much less frequently than others. In Intel SDM, I noticed that preemption timer is only related to TSC, and I think all the vcpus should trap in a similar frequency. Could u help me explain this phenomenon? Thanks, Arthur -- Arthur Chunqi Li Department of Computer Science School of EECS Peking University Beijing, China ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 9:32 About preemption timer Arthur Chunqi Li @ 2013-12-17 11:21 ` Jan Kiszka 2013-12-17 11:31 ` Arthur Chunqi Li 2013-12-17 11:22 ` Paolo Bonzini 1 sibling, 1 reply; 9+ messages in thread From: Jan Kiszka @ 2013-12-17 11:21 UTC (permalink / raw) To: Arthur Chunqi Li, kvm; +Cc: Paolo Bonzini On 2013-12-17 10:32, Arthur Chunqi Li wrote: > Hi Jan and Paolo, > > I've tried to use preemption timer in KVM to trap vcpu regularly, but > there's something unexpected. I run a VM with 4 vcpus and give them > the same preemption timer value (e.g. 1000000) with all bits set > (activate/save bits), then reset the value in preemption time-out > handler. > > Thus I expected these vcpus trap regularly in some special turns. But > I found that when the VM is not busy, some vcpus are trapped much less > frequently than others. In Intel SDM, I noticed that preemption timer > is only related to TSC, and I think all the vcpus should trap in a > similar frequency. > > Could u help me explain this phenomenon? Are you on a CPU that has non-broken preemption timer support? Anything prior Haswell is known to tick with arbitrary frequencies. BTW, we will have to re-implement preemption timer support with the help of a regular host timer due to the breakage when halting L2 (see my test case). Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 11:21 ` Jan Kiszka @ 2013-12-17 11:31 ` Arthur Chunqi Li 2013-12-17 12:43 ` Jan Kiszka 0 siblings, 1 reply; 9+ messages in thread From: Arthur Chunqi Li @ 2013-12-17 11:31 UTC (permalink / raw) To: Jan Kiszka; +Cc: kvm, Paolo Bonzini Hi Jan, On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: > On 2013-12-17 10:32, Arthur Chunqi Li wrote: >> Hi Jan and Paolo, >> >> I've tried to use preemption timer in KVM to trap vcpu regularly, but >> there's something unexpected. I run a VM with 4 vcpus and give them >> the same preemption timer value (e.g. 1000000) with all bits set >> (activate/save bits), then reset the value in preemption time-out >> handler. >> >> Thus I expected these vcpus trap regularly in some special turns. But >> I found that when the VM is not busy, some vcpus are trapped much less >> frequently than others. In Intel SDM, I noticed that preemption timer >> is only related to TSC, and I think all the vcpus should trap in a >> similar frequency. >> >> Could u help me explain this phenomenon? > > Are you on a CPU that has non-broken preemption timer support? Anything > prior Haswell is known to tick with arbitrary frequencies. My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz. Besides, what do you mean by arbitrary frequencies? Arthur > > BTW, we will have to re-implement preemption timer support with the help > of a regular host timer due to the breakage when halting L2 (see my test > case). > > Jan > > -- > Siemens AG, Corporate Technology, CT RTC ITP SES-DE > Corporate Competence Center Embedded Linux -- Arthur Chunqi Li Department of Computer Science School of EECS Peking University Beijing, China ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 11:31 ` Arthur Chunqi Li @ 2013-12-17 12:43 ` Jan Kiszka 2013-12-17 12:59 ` Arthur Chunqi Li 0 siblings, 1 reply; 9+ messages in thread From: Jan Kiszka @ 2013-12-17 12:43 UTC (permalink / raw) To: Arthur Chunqi Li; +Cc: kvm, Paolo Bonzini On 2013-12-17 12:31, Arthur Chunqi Li wrote: > Hi Jan, > > On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >> On 2013-12-17 10:32, Arthur Chunqi Li wrote: >>> Hi Jan and Paolo, >>> >>> I've tried to use preemption timer in KVM to trap vcpu regularly, but >>> there's something unexpected. I run a VM with 4 vcpus and give them >>> the same preemption timer value (e.g. 1000000) with all bits set >>> (activate/save bits), then reset the value in preemption time-out >>> handler. >>> >>> Thus I expected these vcpus trap regularly in some special turns. But >>> I found that when the VM is not busy, some vcpus are trapped much less >>> frequently than others. In Intel SDM, I noticed that preemption timer >>> is only related to TSC, and I think all the vcpus should trap in a >>> similar frequency. >>> >>> Could u help me explain this phenomenon? >> >> Are you on a CPU that has non-broken preemption timer support? Anything >> prior Haswell is known to tick with arbitrary frequencies. > > My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz. Hmm, this one seems unaffected. Didn't find a "specification update". Just like Paolo asked: Your original test case passes? > > Besides, what do you mean by arbitrary frequencies? On older CPUs, the tick rate of the preemption timer does not correlate with the TSC, definitely not in the way the spec defined. Back to your original question: Are we talking about native use of the preemption timer via a patched KVM or nested use inside a KVM virtual machine? Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 12:43 ` Jan Kiszka @ 2013-12-17 12:59 ` Arthur Chunqi Li 2013-12-17 15:17 ` Jan Kiszka 0 siblings, 1 reply; 9+ messages in thread From: Arthur Chunqi Li @ 2013-12-17 12:59 UTC (permalink / raw) To: Jan Kiszka; +Cc: kvm, Paolo Bonzini On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: > On 2013-12-17 12:31, Arthur Chunqi Li wrote: >> Hi Jan, >> >> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>> On 2013-12-17 10:32, Arthur Chunqi Li wrote: >>>> Hi Jan and Paolo, >>>> >>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but >>>> there's something unexpected. I run a VM with 4 vcpus and give them >>>> the same preemption timer value (e.g. 1000000) with all bits set >>>> (activate/save bits), then reset the value in preemption time-out >>>> handler. >>>> >>>> Thus I expected these vcpus trap regularly in some special turns. But >>>> I found that when the VM is not busy, some vcpus are trapped much less >>>> frequently than others. In Intel SDM, I noticed that preemption timer >>>> is only related to TSC, and I think all the vcpus should trap in a >>>> similar frequency. >>>> >>>> Could u help me explain this phenomenon? >>> >>> Are you on a CPU that has non-broken preemption timer support? Anything >>> prior Haswell is known to tick with arbitrary frequencies. >> >> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz. > > Hmm, this one seems unaffected. Didn't find a "specification update". > Just like Paolo asked: Your original test case passes? > >> >> Besides, what do you mean by arbitrary frequencies? > > On older CPUs, the tick rate of the preemption timer does not correlate > with the TSC, definitely not in the way the spec defined. > > Back to your original question: Are we talking about native use of the > preemption timer via a patched KVM or nested use inside a KVM virtual > machine? It is about the native use. I think it may due to the scheduling. When vcpu is scheduled out of pcpu, will the preemption timer work still? Oh, another problem, I use the released kernel 3.11, not the latest one. Does this matter? Arthur > > Jan > > -- > Siemens AG, Corporate Technology, CT RTC ITP SES-DE > Corporate Competence Center Embedded Linux -- Arthur Chunqi Li Department of Computer Science School of EECS Peking University Beijing, China ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 12:59 ` Arthur Chunqi Li @ 2013-12-17 15:17 ` Jan Kiszka 2013-12-18 3:27 ` R 0 siblings, 1 reply; 9+ messages in thread From: Jan Kiszka @ 2013-12-17 15:17 UTC (permalink / raw) To: Arthur Chunqi Li; +Cc: kvm, Paolo Bonzini On 2013-12-17 13:59, Arthur Chunqi Li wrote: > On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >> On 2013-12-17 12:31, Arthur Chunqi Li wrote: >>> Hi Jan, >>> >>> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>>> On 2013-12-17 10:32, Arthur Chunqi Li wrote: >>>>> Hi Jan and Paolo, >>>>> >>>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but >>>>> there's something unexpected. I run a VM with 4 vcpus and give them >>>>> the same preemption timer value (e.g. 1000000) with all bits set >>>>> (activate/save bits), then reset the value in preemption time-out >>>>> handler. >>>>> >>>>> Thus I expected these vcpus trap regularly in some special turns. But >>>>> I found that when the VM is not busy, some vcpus are trapped much less >>>>> frequently than others. In Intel SDM, I noticed that preemption timer >>>>> is only related to TSC, and I think all the vcpus should trap in a >>>>> similar frequency. >>>>> >>>>> Could u help me explain this phenomenon? >>>> >>>> Are you on a CPU that has non-broken preemption timer support? Anything >>>> prior Haswell is known to tick with arbitrary frequencies. >>> >>> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz. >> >> Hmm, this one seems unaffected. Didn't find a "specification update". >> Just like Paolo asked: Your original test case passes? >> >>> >>> Besides, what do you mean by arbitrary frequencies? >> >> On older CPUs, the tick rate of the preemption timer does not correlate >> with the TSC, definitely not in the way the spec defined. >> >> Back to your original question: Are we talking about native use of the >> preemption timer via a patched KVM or nested use inside a KVM virtual >> machine? > > It is about the native use. I think it may due to the scheduling. When > vcpu is scheduled out of pcpu, will the preemption timer work still? The preemption timer ticks as long as the guest is running. Should be specified like this as well. So your KVM patch needs to take care of this when you want to expire it based on real-time, not based on guest time. That's in fact similar to adjustments you implemented for the emulation of the preemption timer. Jan > > Oh, another problem, I use the released kernel 3.11, not the latest > one. Does this matter? > > Arthur > >> >> Jan >> >> -- >> Siemens AG, Corporate Technology, CT RTC ITP SES-DE >> Corporate Competence Center Embedded Linux > > > -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 15:17 ` Jan Kiszka @ 2013-12-18 3:27 ` R 2013-12-18 6:58 ` Jan Kiszka 0 siblings, 1 reply; 9+ messages in thread From: R @ 2013-12-18 3:27 UTC (permalink / raw) To: Jan Kiszka; +Cc: Arthur Chunqi Li, kvm, Paolo Bonzini Hi, You must adjust the preemption timer according to the eclipsed time when the guest runs. And register a hrtimer triggered when a guest exit. The timer should be alarmed when the remaining time runs out, 2013/12/17 Jan Kiszka <jan.kiszka@siemens.com>: > On 2013-12-17 13:59, Arthur Chunqi Li wrote: >> On Tue, Dec 17, 2013 at 8:43 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>> On 2013-12-17 12:31, Arthur Chunqi Li wrote: >>>> Hi Jan, >>>> >>>> On Tue, Dec 17, 2013 at 7:21 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>>>> On 2013-12-17 10:32, Arthur Chunqi Li wrote: >>>>>> Hi Jan and Paolo, >>>>>> >>>>>> I've tried to use preemption timer in KVM to trap vcpu regularly, but >>>>>> there's something unexpected. I run a VM with 4 vcpus and give them >>>>>> the same preemption timer value (e.g. 1000000) with all bits set >>>>>> (activate/save bits), then reset the value in preemption time-out >>>>>> handler. >>>>>> >>>>>> Thus I expected these vcpus trap regularly in some special turns. But >>>>>> I found that when the VM is not busy, some vcpus are trapped much less >>>>>> frequently than others. In Intel SDM, I noticed that preemption timer >>>>>> is only related to TSC, and I think all the vcpus should trap in a >>>>>> similar frequency. >>>>>> >>>>>> Could u help me explain this phenomenon? >>>>> >>>>> Are you on a CPU that has non-broken preemption timer support? Anything >>>>> prior Haswell is known to tick with arbitrary frequencies. >>>> >>>> My CPU is Intel(R) Xeon(R) CPU E5620 @ 2.40GHz. >>> >>> Hmm, this one seems unaffected. Didn't find a "specification update". >>> Just like Paolo asked: Your original test case passes? >>> >>>> >>>> Besides, what do you mean by arbitrary frequencies? >>> >>> On older CPUs, the tick rate of the preemption timer does not correlate >>> with the TSC, definitely not in the way the spec defined. >>> >>> Back to your original question: Are we talking about native use of the >>> preemption timer via a patched KVM or nested use inside a KVM virtual >>> machine? >> >> It is about the native use. I think it may due to the scheduling. When >> vcpu is scheduled out of pcpu, will the preemption timer work still? > > The preemption timer ticks as long as the guest is running. Should be > specified like this as well. So your KVM patch needs to take care of > this when you want to expire it based on real-time, not based on guest > time. That's in fact similar to adjustments you implemented for the > emulation of the preemption timer. > > Jan > >> >> Oh, another problem, I use the released kernel 3.11, not the latest >> one. Does this matter? >> >> Arthur >> >>> >>> Jan >>> >>> -- >>> Siemens AG, Corporate Technology, CT RTC ITP SES-DE >>> Corporate Competence Center Embedded Linux >> >> >> > > -- > Siemens AG, Corporate Technology, CT RTC ITP SES-DE > Corporate Competence Center Embedded Linux > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Thanks Rui Wu ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-18 3:27 ` R @ 2013-12-18 6:58 ` Jan Kiszka 0 siblings, 0 replies; 9+ messages in thread From: Jan Kiszka @ 2013-12-18 6:58 UTC (permalink / raw) To: R; +Cc: Arthur Chunqi Li, kvm, Paolo Bonzini On 2013-12-18 04:27, R wrote: > Hi, > > You must adjust the preemption timer according to the eclipsed time > when the guest runs. > > And register a hrtimer triggered when a guest exit. > The timer should be alarmed when the remaining time runs out, In fact, as we need to register an hrtimer anyway when leaving the guest, we can simply register it always and stop using the physical preemption timer. This will also solve the breakage on older Intel CPU, actually it will add preemption timer support unconditionally. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: About preemption timer 2013-12-17 9:32 About preemption timer Arthur Chunqi Li 2013-12-17 11:21 ` Jan Kiszka @ 2013-12-17 11:22 ` Paolo Bonzini 1 sibling, 0 replies; 9+ messages in thread From: Paolo Bonzini @ 2013-12-17 11:22 UTC (permalink / raw) To: Arthur Chunqi Li; +Cc: kvm, Jan Kiszka Il 17/12/2013 10:32, Arthur Chunqi Li ha scritto: > Hi Jan and Paolo, > > I've tried to use preemption timer in KVM to trap vcpu regularly, but > there's something unexpected. I run a VM with 4 vcpus and give them > the same preemption timer value (e.g. 1000000) with all bits set > (activate/save bits), then reset the value in preemption time-out > handler. > > Thus I expected these vcpus trap regularly in some special turns. But > I found that when the VM is not busy, some vcpus are trapped much less > frequently than others. In Intel SDM, I noticed that preemption timer > is only related to TSC, and I think all the vcpus should trap in a > similar frequency. Does the preemption timer testcase pass on your machine? The preemption timer is known to have bugs. Paolo ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-12-18 6:58 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-12-17 9:32 About preemption timer Arthur Chunqi Li 2013-12-17 11:21 ` Jan Kiszka 2013-12-17 11:31 ` Arthur Chunqi Li 2013-12-17 12:43 ` Jan Kiszka 2013-12-17 12:59 ` Arthur Chunqi Li 2013-12-17 15:17 ` Jan Kiszka 2013-12-18 3:27 ` R 2013-12-18 6:58 ` Jan Kiszka 2013-12-17 11:22 ` Paolo Bonzini
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.