Emulation of cmpxchg in Xen 4.2 (also possible bug)

Emulation of cmpxchg in Xen 4.2 (also possible bug)

[Ruslan Nikolaev]

Hi

While experimenting with a heavy testload which makes a lot of CMPXCHG in PVHVM domain (DomU) and Dom0, I noticed inconsistency (I am using official Xen 4.2 release):

Dom0's cmpxchg succeeds (i.e. set ZF flag appropriately) changing Val1 to Val2

PVHVM DomU cmpxchg also succeeds changing Val1 to Val3 even though it was supposedly replaced (Val1 to Val2, Val1 != Val2) by Dom0 already

I double checked my code and it seems to be fine. Moreover, the problem appears only on a high contention case (when both Dom0 and DomU are trying to change it).

While investigating the issue, I noticed that Xen emulates cmpxchg for PV and HVM cases:

1. Why is it needed to emulate CMPXCHG at all? (What is it used for and can it be disabled? I am using Linux guests.)

2. Is it emulated only in HVM DomU domain or in Dom0 as well?3. Is it possible that emulation has some bug? Is it possible to somehow avoid emulation at all and replace it with something? (I want to avoid performance overhead for this instruction) What if I use CMPXCHG16B, does it still use emulation?

Ruslan.

Re: Emulation of cmpxchg in Xen 4.2 (also possible bug)

[Ruslan Nikolaev]

Just one clarification to my message below:

I am using a shared page between DomU HVM and Dom0. (This is used as a shared buffer and cmpxchg changes atomic variable there.)

Ruslan



----- Original Message -----
From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
Cc: 
Sent: Wednesday, October 3, 2012 12:09 AM
Subject: Emulation of cmpxchg in Xen 4.2 (also possible bug)

Hi

While experimenting with a heavy testload which makes a lot of CMPXCHG in PVHVM domain (DomU) and Dom0, I noticed inconsistency (I am using official Xen 4.2 release):

Dom0's cmpxchg succeeds (i.e. set ZF flag appropriately) changing Val1 to Val2

PVHVM DomU cmpxchg also succeeds changing Val1 to Val3 even though it was supposedly replaced (Val1 to Val2, Val1 != Val2) by Dom0 already

I double checked my code and it seems to be fine. Moreover, the problem appears only on a high contention case (when both Dom0 and DomU are trying to change it).

While investigating the issue, I noticed that Xen emulates cmpxchg for PV and HVM cases:

1. Why is it needed to emulate CMPXCHG at all? (What is it used for and can it be disabled? I am using Linux guests.)

2. Is it emulated only in HVM DomU domain or in Dom0 as well?3. Is it possible that emulation has some bug? Is it possible to somehow avoid emulation at all and replace it with something? (I want to avoid performance overhead for this instruction) What if I use CMPXCHG16B, does it still use emulation?

Ruslan.

Re: Emulation of cmpxchg in Xen 4.2 (also possible bug)

[Keir Fraser]

Then CMPXCHG should not be being emulated. It is only emulated for things
like access to device memory and to pagetables. If the shared page is mapped
with write permission in both PV and HVM guests, no emulation should be
occurring.

 -- Keir

On 03/10/2012 05:14, "Ruslan Nikolaev" <nruslan_devel@yahoo.com> wrote:

> Just one clarification to my message below:
> 
> I am using a shared page between DomU HVM and Dom0. (This is used as a shared
> buffer and cmpxchg changes atomic variable there.)
> 
> Ruslan
> 
> 
> 
> ----- Original Message -----
> From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
> To: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
> Cc: 
> Sent: Wednesday, October 3, 2012 12:09 AM
> Subject: Emulation of cmpxchg in Xen 4.2 (also possible bug)
> 
> Hi
> 
> While experimenting with a heavy testload which makes a lot of CMPXCHG in
> PVHVM domain (DomU) and Dom0, I noticed inconsistency (I am using official Xen
> 4.2 release):
> 
> Dom0's cmpxchg succeeds (i.e. set ZF flag appropriately) changing Val1 to Val2
> 
> PVHVM DomU cmpxchg also succeeds changing Val1 to Val3 even though it was
> supposedly replaced (Val1 to Val2, Val1 != Val2) by Dom0 already
> 
> I double checked my code and it seems to be fine. Moreover, the problem
> appears only on a high contention case (when both Dom0 and DomU are trying to
> change it).
> 
> While investigating the issue, I noticed that Xen emulates cmpxchg for PV and
> HVM cases:
> 
> 1. Why is it needed to emulate CMPXCHG at all? (What is it used for and can it
> be disabled? I am using Linux guests.)
> 
> 2. Is it emulated only in HVM DomU domain or in Dom0 as well?3. Is it possible
> that emulation has some bug? Is it possible to somehow avoid emulation at all
> and replace it with something? (I want to avoid performance overhead for this
> instruction) What if I use CMPXCHG16B, does it still use emulation?
> 
> Ruslan.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel