rdtsc() in kvm-unit-tests on x86

rdtsc() in kvm-unit-tests on x86

[Jintack Lim]

Hi all,

While I was looking at rdtsc() code in kvm-unit-tests (e.g. x86/vmexit.c),
I was getting curious that out-of-order execution on the processor
may make rdtsc() executed not in the place we expect.

Referring to this document from intel,
http://www.intel.com/content/www/us/en/embedded/training/ia-32-ia-64-benchmark-code-execution-paper.html
they suggested to use rdtscp instruction and other techniques to
serialize reading tsc register.

I wonder how the serialization is achieved when using rdtsc() in
kvm-unit-tests code.
Or, maybe the serialization is not necessary for some reason?

Thanks,
Jintack

Re: rdtsc() in kvm-unit-tests on x86

[Paolo Bonzini]

On 07/08/2015 21:19, Jintack Lim wrote:
> Hi all,
> 
> While I was looking at rdtsc() code in kvm-unit-tests (e.g. x86/vmexit.c),
> I was getting curious that out-of-order execution on the processor
> may make rdtsc() executed not in the place we expect.
> 
> Referring to this document from intel,
> http://www.intel.com/content/www/us/en/embedded/training/ia-32-ia-64-benchmark-code-execution-paper.html
> they suggested to use rdtscp instruction and other techniques to
> serialize reading tsc register.
> 
> I wonder how the serialization is achieved when using rdtsc() in
> kvm-unit-tests code.
> Or, maybe the serialization is not necessary for some reason?

kvm-unit-tests executes the instruction thousands of times, so any error
due to lack of serialization is lost in the noise.

Paolo

Re: rdtsc() in kvm-unit-tests on x86

[Paolo Bonzini]

On 10/08/2015 11:42, Christoffer Dall wrote:
> 
> Thanks for the reply.
> 
> On this note, is there an easy mechanism on x86 to obtain a TSC which is
> synchronized across PCPUs and between the host and the guest?
> 
> We can use the physical arch timer counter on arm64, but I'm not sure if
> there's something similar on x86?

Yes, you just use the TSC. :)  However, you first have to check that the
TSC is consistent across CPUs.  On older machines it's not, but the
kernel can detect it.

Paolo

> This is useful to trace the flow of events over an SMP system in detail.

Re: rdtsc() in kvm-unit-tests on x86

[Jintack Lim]

On Mon, Aug 10, 2015 at 9:58 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 10/08/2015 11:42, Christoffer Dall wrote:
>>
>> Thanks for the reply.
>>
>> On this note, is there an easy mechanism on x86 to obtain a TSC which is
>> synchronized across PCPUs and between the host and the guest?
>>
>> We can use the physical arch timer counter on arm64, but I'm not sure if
>> there's something similar on x86?
>
> Yes, you just use the TSC. :)  However, you first have to check that the
> TSC is consistent across CPUs.  On older machines it's not, but the
> kernel can detect it.

Thanks, Paolo.

What would be the best way to check if TSC is consistent across CPUs?
Is it synchronized in nano second (or even cpu clock cycle) resolution?

To get synchronized tsc across the host and the guest,
just calling rdtscll() in host and guest would be enough?

Thanks,
Jintack

>
> Paolo
>
>> This is useful to trace the flow of events over an SMP system in detail.
>

Re: rdtsc() in kvm-unit-tests on x86

[Paolo Bonzini]

On 10/08/2015 16:14, Jintack Lim wrote:
>> > Yes, you just use the TSC. :)  However, you first have to check that the
>> > TSC is consistent across CPUs.  On older machines it's not, but the
>> > kernel can detect it.
> Thanks, Paolo.
> 
> What would be the best way to check if TSC is consistent across CPUs?

You need to have boot_cpu_has(X86_FEATURE_CONSTANT_TSC) and
boot_cpu_has(X86_FEATURE_NONSTOP_TSC).

However, I would just use TAI (ktime_get_clocktai).  x86 KVM provides a
paravirtual interface that synchronizes CLOCK_TAI with the host, and
using it is the simplest way to get synchronized times between the host
and the guest.

Paolo

> Is it synchronized in nano second (or even cpu clock cycle) resolution?
> 
> To get synchronized tsc across the host and the guest,
> just calling rdtscll() in host and guest would be enough?

Re: rdtsc() in kvm-unit-tests on x86

[Jintack Lim]

On Mon, Aug 10, 2015 at 10:47 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 10/08/2015 16:14, Jintack Lim wrote:
>>> > Yes, you just use the TSC. :)  However, you first have to check that the
>>> > TSC is consistent across CPUs.  On older machines it's not, but the
>>> > kernel can detect it.
>> Thanks, Paolo.
>>
>> What would be the best way to check if TSC is consistent across CPUs?
>
> You need to have boot_cpu_has(X86_FEATURE_CONSTANT_TSC) and
> boot_cpu_has(X86_FEATURE_NONSTOP_TSC).
>
> However, I would just use TAI (ktime_get_clocktai).  x86 KVM provides a
> paravirtual interface that synchronizes CLOCK_TAI with the host, and
> using it is the simplest way to get synchronized times between the host
> and the guest.

Thank you so much. I'll try it.

Jintack
>
> Paolo
>
>> Is it synchronized in nano second (or even cpu clock cycle) resolution?
>>
>> To get synchronized tsc across the host and the guest,
>> just calling rdtscll() in host and guest would be enough?
>