XSA-60 solutions

XSA-60 solutions

[Liu, Jinsong]

[-- Attachment #1: Type: text/plain, Size: 2116 bytes --]

Hi, All

This email provides 2 solutions for XSA-60 issue found by Konrad (refer attached email for XSA-60 please).

Basically it involves how to emulate guest setting cr0.cd. For shadow, as Jan pointed out in earlier email Xen drop all shadows so that any new ones would be created with UC memory type, _not_ involving iteration over the whole address space. For EPT, currently Xen traverse all ept entries via problematic set_uc_mode, resulting in DOS-like behavior, so this email focus on Intel EPT case.

Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped, stop using normal EPT, switch to a temp EPT table and populate new EPT entries w/ UC type on demand at later EPT violation. When guest clearing cr0.cd, switch back to normal EPT. In this way, _no_ unbounded loop involved and hence security hole avoided.

Some concerns for Dual-EPT: the 1st concern comes from cachablity confliction between guest and Xen memory type point of view, though it also exists in current implementation. The 2nd concern comes from Dual EPT tables inconsistency/sync issue: things become complicated when p2m modifying, PoD populating, and super page spliting, etc.

Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest with VT-d but snooped, Xen need do nothing, just simply ignore guest setting cr0.cd, since hardware snoop mechanism has ensured cache coherency (under these cases currently Xen has set EPT iPAT bit, ignore guest's memory type opinion); For guest with VT-d but non-snooped, cache coherency can not be guaranteed by h/w snoop so guest's memory type opinion must be considered (under this case Xen set iPAT bit combining guest and host memory type opinion). Only under this case PAT emulation need set all IA32_PAT fields as UC so that guest memory type are all UC.

Concern for PAT solution still comes from cachablity confliction between guest and Xen.

Thoughts?
BTW, today is Chinese National day, I will have several days travel with no email access, but your comments/suggestions are highly appreciated and I will reply ASAP after I come back.

Thanks,
Jinsong

[-- Attachment #2: Type: message/rfc822, Size: 8333 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 2702 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

             Xen Security Advisory CVE-2013-2212 / XSA-60
                             version 4

   Excessive time to disable caching with HVM guests with PCI passthrough

UPDATES IN VERSION 4
====================

Public release.

ISSUE DESCRIPTION
=================

HVM guests are able to manipulate their physical address space such that
processing a subsequent request by that guest to disable caches takes an
extended amount of time changing the cachability of the memory pages assigned
to this guest. This applies only when the guest has been granted access to
some memory mapped I/O region (typically by way of assigning a passthrough
PCI device).

This can cause the CPU which processes the request to become unavailable,
possibly causing the hypervisor or a guest kernel (including the domain 0 one)
to halt itself ("panic").

For reference, as long as no patch implementing an approved alternative
solution is available (there's only a draft violating certain requirements
set by Intel's documentation), the problematic code is the function
vmx_set_uc_mode() (in that it calls ept_change_entry_emt_with_range() with
the full guest GFN range, which the guest has control over, but which also
would be a problem with sufficiently large but not malicious guests).

IMPACT
======

A malicious domain, given access to a device with memory mapped I/O
regions, can cause the host to become unresponsive for a period of
time, potentially leading to a DoS affecting the whole system.

VULNERABLE SYSTEMS
==================

Xen version 3.3 onwards is vulnerable.

Only systems using the Intel variant of Hardware Assisted Paging (aka EPT) are
vulnerable.

MITIGATION
==========

This issue can be avoided by not assigning PCI devices to untrusted guests, or
by running HVM guests with shadow mode paging (through adding "hap=0" to the
domain configuration file).

CREDITS
=======

Konrad Wilk found the issue as a bug, which on examination by the
Xenproject.org Security Team turned out to be a security problem.

RESOLUTION
==========

There is currently no resolution to this issue.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAEBAgAGBQJR77wrAAoJEIP+FMlX6CvZB5MH/ibfpjHuoGOIo7mWukld4NM5
UVIKC+rTrnkYhbF2f+xIM833+WAUjPuXZKZ6/EirDAPAAQCut2DouNvVdVnZ5cBx
rq0N8l9wy0/dq/7kCyI3kAGFlJ3VYz7aM5+TTPFGfO7Yq3ohUNu2EE4vv/t5KVjD
H4reh8UaA5QuRbdh3evCM9Vdt2syqi8JQwB5D2CJqrgAuFPwEVle8MLKSXWWb/+V
KUy+mRAb1tN3jbWIev0TZ7Hm3x61yO60/WFzsQzkmkd+qWvC5btkWDg05K5DHC+Q
yvFU3Y5u7J/ub00ZO4e9wjNDG5+ItQUK4xp8y5s65qx27P/eK9VLi8dvnHVMk04=
=HUbY
-----END PGP SIGNATURE-----

[-- Attachment #2.1.2: ATT00001.txt --]
[-- Type: text/plain, Size: 130 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: XSA-60 solutions

[Liu, Jinsong]

Liu, Jinsong wrote:
> Hi, All
> 
> This email provides 2 solutions for XSA-60 issue found by Konrad
> (refer attached email for XSA-60 please). 
> 
> Basically it involves how to emulate guest setting cr0.cd. For
> shadow, as Jan pointed out in earlier email Xen drop all shadows so
> that any new ones would be created with UC memory type, _not_
> involving iteration over the whole address space. For EPT, currently
> Xen traverse all ept entries via problematic set_uc_mode, resulting
> in DOS-like behavior, so this email focus on Intel EPT case.     
> 
> Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped,
> stop using normal EPT, switch to a temp EPT table and populate new
> EPT entries w/ UC type on demand at later EPT violation. When guest
> clearing cr0.cd, switch back to normal EPT. In this way, _no_
> unbounded loop involved and hence security hole avoided.    
> 
> Some concerns for Dual-EPT: the 1st concern comes from cachablity
> confliction between guest and Xen memory type point of view, though
> it also exists in current implementation. The 2nd concern comes from
> Dual EPT tables inconsistency/sync issue: things become complicated
> when p2m modifying, PoD populating, and super page spliting, etc.    
> 
> Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest
> with VT-d but snooped, Xen need do nothing, just simply ignore guest
> setting cr0.cd, since hardware snoop mechanism has ensured cache
> coherency (under these cases currently Xen has set EPT iPAT bit,
> ignore guest's memory type opinion); For guest with VT-d but
> non-snooped, cache coherency can not be guaranteed by h/w snoop so
> guest's memory type opinion must be considered (under this case Xen
> set iPAT bit combining guest and host memory type opinion). Only

Sorry, under this case Xen _clear_ iPAT, combining guest and host memory type opinion.

Thanks,
Jinsong

> under this case PAT emulation need set all IA32_PAT fields as UC so
> that guest memory type are all UC.         
> 
> Concern for PAT solution still comes from cachablity confliction
> between guest and Xen. 
> 
> Thoughts?
> BTW, today is Chinese National day, I will have several days travel
> with no email access, but your comments/suggestions are highly
> appreciated and I will reply ASAP after I come back.  
> 
> Thanks,
> Jinsong

Re: XSA-60 solutions

[Konrad Rzeszutek Wilk]

On Mon, Sep 30, 2013 at 08:28:53PM +0000, Liu, Jinsong wrote:
> Hi, All
> 
> This email provides 2 solutions for XSA-60 issue found by Konrad (refer attached email for XSA-60 please).

It is actually by Zhenzhong Duan <zhenzhong.duan@oracle.com> (CC-ed here).
> 
> Basically it involves how to emulate guest setting cr0.cd. For shadow, as Jan pointed out in earlier email Xen drop all shadows so that any new ones would be created with UC memory type, _not_ involving iteration over the whole address space. For EPT, currently Xen traverse all ept entries via problematic set_uc_mode, resulting in DOS-like behavior, so this email focus on Intel EPT case.
> 
> Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped, stop using normal EPT, switch to a temp EPT table and populate new EPT entries w/ UC type on demand at later EPT violation. When guest clearing cr0.cd, switch back to normal EPT. In this way, _no_ unbounded loop involved and hence security hole avoided.
> 
> Some concerns for Dual-EPT: the 1st concern comes from cachablity confliction between guest and Xen memory type point of view, though it also exists in current implementation. The 2nd concern comes from Dual EPT tables inconsistency/sync issue: things become complicated when p2m modifying, PoD populating, and super page spliting, etc.
> 
> Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest with VT-d but snooped, Xen need do nothing, just simply ignore guest setting cr0.cd, since hardware snoop mechanism has ensured cache coherency (under these cases currently Xen has set EPT iPAT bit, ignore guest's memory type opinion); For guest with VT-d but non-snooped, cache coherency can not be guaranteed by h/w snoop so guest's memory type opinion must be considered (under this case Xen set iPAT bit combining guest and host memory type opinion). Only under this case PAT emulation need set all IA32_PAT fields as UC so that guest memory type are all UC.
> 
> Concern for PAT solution still comes from cachablity confliction between guest and Xen.
> 
> Thoughts?
> BTW, today is Chinese National day, I will have several days travel with no email access, but your comments/suggestions are highly appreciated and I will reply ASAP after I come back.
> 
> Thanks,
> Jinsong

> Date: Wed, 24 Jul 2013 11:36:55 +0000
> From: "Xen.org security team" <security@xen.org>
> To: "xen-announce@lists.xen.org" <xen-announce@lists.xen.org>,
>  "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
>  "xen-users@lists.xen.org" <xen-users@lists.xen.org>,
>  "oss-security@lists.openwall.com" <oss-security@lists.openwall.com>
> CC: "Xen.org security team" <security@xen.org>
> Subject: [Xen-devel] Xen Security Advisory 60 (CVE-2013-2212) - Excessive
>  time to disable caching with HVM guests with PCI passthrough
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
>              Xen Security Advisory CVE-2013-2212 / XSA-60
>                              version 4
> 
>    Excessive time to disable caching with HVM guests with PCI passthrough
> 
> UPDATES IN VERSION 4
> ====================
> 
> Public release.
> 
> ISSUE DESCRIPTION
> =================
> 
> HVM guests are able to manipulate their physical address space such that
> processing a subsequent request by that guest to disable caches takes an
> extended amount of time changing the cachability of the memory pages assigned
> to this guest. This applies only when the guest has been granted access to
> some memory mapped I/O region (typically by way of assigning a passthrough
> PCI device).
> 
> This can cause the CPU which processes the request to become unavailable,
> possibly causing the hypervisor or a guest kernel (including the domain 0 one)
> to halt itself ("panic").
> 
> For reference, as long as no patch implementing an approved alternative
> solution is available (there's only a draft violating certain requirements
> set by Intel's documentation), the problematic code is the function
> vmx_set_uc_mode() (in that it calls ept_change_entry_emt_with_range() with
> the full guest GFN range, which the guest has control over, but which also
> would be a problem with sufficiently large but not malicious guests).
> 
> IMPACT
> ======
> 
> A malicious domain, given access to a device with memory mapped I/O
> regions, can cause the host to become unresponsive for a period of
> time, potentially leading to a DoS affecting the whole system.
> 
> VULNERABLE SYSTEMS
> ==================
> 
> Xen version 3.3 onwards is vulnerable.
> 
> Only systems using the Intel variant of Hardware Assisted Paging (aka EPT) are
> vulnerable.
> 
> MITIGATION
> ==========
> 
> This issue can be avoided by not assigning PCI devices to untrusted guests, or
> by running HVM guests with shadow mode paging (through adding "hap=0" to the
> domain configuration file).
> 
> CREDITS
> =======
> 
> Konrad Wilk found the issue as a bug, which on examination by the
> Xenproject.org Security Team turned out to be a security problem.
> 
> RESOLUTION
> ==========
> 
> There is currently no resolution to this issue.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iQEcBAEBAgAGBQJR77wrAAoJEIP+FMlX6CvZB5MH/ibfpjHuoGOIo7mWukld4NM5
> UVIKC+rTrnkYhbF2f+xIM833+WAUjPuXZKZ6/EirDAPAAQCut2DouNvVdVnZ5cBx
> rq0N8l9wy0/dq/7kCyI3kAGFlJ3VYz7aM5+TTPFGfO7Yq3ohUNu2EE4vv/t5KVjD
> H4reh8UaA5QuRbdh3evCM9Vdt2syqi8JQwB5D2CJqrgAuFPwEVle8MLKSXWWb/+V
> KUy+mRAb1tN3jbWIev0TZ7Hm3x61yO60/WFzsQzkmkd+qWvC5btkWDg05K5DHC+Q
> yvFU3Y5u7J/ub00ZO4e9wjNDG5+ItQUK4xp8y5s65qx27P/eK9VLi8dvnHVMk04=
> =HUbY
> -----END PGP SIGNATURE-----

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: XSA-60 solutions

[Liu, Jinsong]

Hi, All

Any comments/suggestions?

Thanks,
Jinsong


Liu, Jinsong wrote:
> Liu, Jinsong wrote:
>> Hi, All
>> 
>> This email provides 2 solutions for XSA-60 issue found by Konrad
>> (refer attached email for XSA-60 please).
>> 
>> Basically it involves how to emulate guest setting cr0.cd. For
>> shadow, as Jan pointed out in earlier email Xen drop all shadows so
>> that any new ones would be created with UC memory type, _not_
>> involving iteration over the whole address space. For EPT, currently
>> Xen traverse all ept entries via problematic set_uc_mode, resulting
>> in DOS-like behavior, so this email focus on Intel EPT case.
>> 
>> Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped,
>> stop using normal EPT, switch to a temp EPT table and populate new
>> EPT entries w/ UC type on demand at later EPT violation. When guest
>> clearing cr0.cd, switch back to normal EPT. In this way, _no_
>> unbounded loop involved and hence security hole avoided.
>> 
>> Some concerns for Dual-EPT: the 1st concern comes from cachablity
>> confliction between guest and Xen memory type point of view, though
>> it also exists in current implementation. The 2nd concern comes from
>> Dual EPT tables inconsistency/sync issue: things become complicated
>> when p2m modifying, PoD populating, and super page spliting, etc.
>> 
>> Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest
>> with VT-d but snooped, Xen need do nothing, just simply ignore guest
>> setting cr0.cd, since hardware snoop mechanism has ensured cache
>> coherency (under these cases currently Xen has set EPT iPAT bit,
>> ignore guest's memory type opinion); For guest with VT-d but
>> non-snooped, cache coherency can not be guaranteed by h/w snoop so
>> guest's memory type opinion must be considered (under this case Xen
>> set iPAT bit combining guest and host memory type opinion). Only
> 
> Sorry, under this case Xen _clear_ iPAT, combining guest and host
> memory type opinion. 
> 
> Thanks,
> Jinsong
> 
>> under this case PAT emulation need set all IA32_PAT fields as UC so
>> that guest memory type are all UC.
>> 
>> Concern for PAT solution still comes from cachablity confliction
>> between guest and Xen. 
>> 
>> Thoughts?
>> BTW, today is Chinese National day, I will have several days travel
>> with no email access, but your comments/suggestions are highly
>> appreciated and I will reply ASAP after I come back.
>> 
>> Thanks,
>> Jinsong

Re: XSA-60 solutions

[Jan Beulich]

>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
> Any comments/suggestions?

As pointed out in earlier private conversation, I think that the dual
table approach would be preferable if it can be made work.

I'm surprised no-one from Oracle responded so far, as it was them
originally having found the issue.

Jan

> Liu, Jinsong wrote:
>> Liu, Jinsong wrote:
>>> Hi, All
>>> 
>>> This email provides 2 solutions for XSA-60 issue found by Konrad
>>> (refer attached email for XSA-60 please).
>>> 
>>> Basically it involves how to emulate guest setting cr0.cd. For
>>> shadow, as Jan pointed out in earlier email Xen drop all shadows so
>>> that any new ones would be created with UC memory type, _not_
>>> involving iteration over the whole address space. For EPT, currently
>>> Xen traverse all ept entries via problematic set_uc_mode, resulting
>>> in DOS-like behavior, so this email focus on Intel EPT case.
>>> 
>>> Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped,
>>> stop using normal EPT, switch to a temp EPT table and populate new
>>> EPT entries w/ UC type on demand at later EPT violation. When guest
>>> clearing cr0.cd, switch back to normal EPT. In this way, _no_
>>> unbounded loop involved and hence security hole avoided.
>>> 
>>> Some concerns for Dual-EPT: the 1st concern comes from cachablity
>>> confliction between guest and Xen memory type point of view, though
>>> it also exists in current implementation. The 2nd concern comes from
>>> Dual EPT tables inconsistency/sync issue: things become complicated
>>> when p2m modifying, PoD populating, and super page spliting, etc.
>>> 
>>> Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest
>>> with VT-d but snooped, Xen need do nothing, just simply ignore guest
>>> setting cr0.cd, since hardware snoop mechanism has ensured cache
>>> coherency (under these cases currently Xen has set EPT iPAT bit,
>>> ignore guest's memory type opinion); For guest with VT-d but
>>> non-snooped, cache coherency can not be guaranteed by h/w snoop so
>>> guest's memory type opinion must be considered (under this case Xen
>>> set iPAT bit combining guest and host memory type opinion). Only
>> 
>> Sorry, under this case Xen _clear_ iPAT, combining guest and host
>> memory type opinion. 
>> 
>> Thanks,
>> Jinsong
>> 
>>> under this case PAT emulation need set all IA32_PAT fields as UC so
>>> that guest memory type are all UC.
>>> 
>>> Concern for PAT solution still comes from cachablity confliction
>>> between guest and Xen. 
>>> 
>>> Thoughts?
>>> BTW, today is Chinese National day, I will have several days travel
>>> with no email access, but your comments/suggestions are highly
>>> appreciated and I will reply ASAP after I come back.
>>> 
>>> Thanks,
>>> Jinsong

Re: XSA-60 solutions

[Andrew Cooper]

On 07/10/13 16:04, Jan Beulich wrote:
>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
>> Any comments/suggestions?
> As pointed out in earlier private conversation, I think that the dual
> table approach would be preferable if it can be made work.
>
> I'm surprised no-one from Oracle responded so far, as it was them
> originally having found the issue.
>
> Jan
>
>

I cant remember whether I asked this before or not, but is there a
reason why this cant be done in the same way as hypercall continuations?

~Andrew

Re: XSA-60 solutions

[Jan Beulich]

>>> On 07.10.13 at 17:08, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 07/10/13 16:04, Jan Beulich wrote:
>>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
>>> Any comments/suggestions?
>> As pointed out in earlier private conversation, I think that the dual
>> table approach would be preferable if it can be made work.
>>
>> I'm surprised no-one from Oracle responded so far, as it was them
>> originally having found the issue.
> 
> I cant remember whether I asked this before or not, but is there a
> reason why this cant be done in the same way as hypercall continuations?

I think you did; the reason is that we're in the middle of processing
a CR0 write here, and hence can't intermediately exit back to the
guest (which is what we'd do for hypercall continuations). However,
you mentioning this one makes me think whether we couldn't
leverage/extend/clone the MMIO retry logic, or at least deal with
this in ways similar to it (i.e. preventing VM re-entry until processing
is complete). But maybe that would result in even worse hackery...

Jan

Re: XSA-60 solutions

[Liu, Jinsong]

Jan Beulich wrote:
>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com>
>>>> wrote: Any comments/suggestions? 
> 
> As pointed out in earlier private conversation, I think that the dual
> table approach would be preferable if it can be made work.

No, solution 2 has an important update compared with our private conversation before:
The old solution 2 is, Xen do nothing for guest cr0.cd setting.
The new solution 2 in this email is, Xen set IA32_PAT fields as UC when needed (that means, only when guest work with vt-d but vt-d non-snooped, since under this case h/w can not guarantee cache coherency), so that all guest memory type are UC. For other cases (when guest work w/o vt-d, and when guest work with vt-d but vt-d snooped), Xen can still do nothing for guest cr0.cd setting (since h/w has guaranteed cache coherency, otherwise Xen doesn't dare to do what it does now -- it set iPAT bit as 1, totally ignoring guest memory type point of view).

Compared with Dual-EPT solution, it's much simpler (only need IA32_PAT MSR emulation under rare case) and avoid the inconsistency issue between 2 EPT tables.

Thanks,
Jinsong

> 
> I'm surprised no-one from Oracle responded so far, as it was them
> originally having found the issue.
> 
> Jan
> 
>> Liu, Jinsong wrote:
>>> Liu, Jinsong wrote:
>>>> Hi, All
>>>> 
>>>> This email provides 2 solutions for XSA-60 issue found by Konrad
>>>> (refer attached email for XSA-60 please).
>>>> 
>>>> Basically it involves how to emulate guest setting cr0.cd. For
>>>> shadow, as Jan pointed out in earlier email Xen drop all shadows so
>>>> that any new ones would be created with UC memory type, _not_
>>>> involving iteration over the whole address space. For EPT,
>>>> currently Xen traverse all ept entries via problematic
>>>> set_uc_mode, resulting in DOS-like behavior, so this email focus
>>>> on Intel EPT case. 
>>>> 
>>>> Solution 1 is Dual-EPT tables: When guest setting cr0.cd trapped,
>>>> stop using normal EPT, switch to a temp EPT table and populate new
>>>> EPT entries w/ UC type on demand at later EPT violation. When guest
>>>> clearing cr0.cd, switch back to normal EPT. In this way, _no_
>>>> unbounded loop involved and hence security hole avoided.
>>>> 
>>>> Some concerns for Dual-EPT: the 1st concern comes from cachablity
>>>> confliction between guest and Xen memory type point of view, though
>>>> it also exists in current implementation. The 2nd concern comes
>>>> from Dual EPT tables inconsistency/sync issue: things become
>>>> complicated when p2m modifying, PoD populating, and super page
>>>> spliting, etc. 
>>>> 
>>>> Solution 2 is via PAT emulation: For guest w/o VT-d, and for guest
>>>> with VT-d but snooped, Xen need do nothing, just simply ignore
>>>> guest setting cr0.cd, since hardware snoop mechanism has ensured
>>>> cache coherency (under these cases currently Xen has set EPT iPAT
>>>> bit, ignore guest's memory type opinion); For guest with VT-d but
>>>> non-snooped, cache coherency can not be guaranteed by h/w snoop so
>>>> guest's memory type opinion must be considered (under this case Xen
>>>> set iPAT bit combining guest and host memory type opinion). Only
>>> 
>>> Sorry, under this case Xen _clear_ iPAT, combining guest and host
>>> memory type opinion. 
>>> 
>>> Thanks,
>>> Jinsong
>>> 
>>>> under this case PAT emulation need set all IA32_PAT fields as UC so
>>>> that guest memory type are all UC.
>>>> 
>>>> Concern for PAT solution still comes from cachablity confliction
>>>> between guest and Xen. 
>>>> 
>>>> Thoughts?
>>>> BTW, today is Chinese National day, I will have several days travel
>>>> with no email access, but your comments/suggestions are highly
>>>> appreciated and I will reply ASAP after I come back.
>>>> 
>>>> Thanks,
>>>> Jinsong

Re: XSA-60 solutions

[DuanZhenzhong]

Jan Beulich wrote:
>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
>>>>         
>> Any comments/suggestions?
>>     
>
> As pointed out in earlier private conversation, I think that the dual
> table approach would be preferable if it can be made work.
>
> I'm surprised no-one from Oracle responded so far, as it was them
> originally having found the issue.
Just back from vacation, we could help test if there is any patch out.

zduan

Re: XSA-60 solutions

[Jan Beulich]

>>> On 08.10.13 at 04:25, DuanZhenzhong <zhenzhong.duan@oracle.com> wrote:
> Jan Beulich wrote:
>>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
>>>>>         
>>> Any comments/suggestions?
>>>     
>>
>> As pointed out in earlier private conversation, I think that the dual
>> table approach would be preferable if it can be made work.
>>
>> I'm surprised no-one from Oracle responded so far, as it was them
>> originally having found the issue.
> Just back from vacation, we could help test if there is any patch out.

Before getting to test anything, we first need to sort out how
we want to address the issue - that's where input would be
highly appreciated.

Jan

Re: XSA-60 solutions

[Jan Beulich]

>>> On 07.10.13 at 18:03, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
> Jan Beulich wrote:
>>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com>
>>>>> wrote: Any comments/suggestions? 
>> 
>> As pointed out in earlier private conversation, I think that the dual
>> table approach would be preferable if it can be made work.
> 
> No, solution 2 has an important update compared with our private 
> conversation before:
> The old solution 2 is, Xen do nothing for guest cr0.cd setting.
> The new solution 2 in this email is, Xen set IA32_PAT fields as UC when 
> needed (that means, only when guest work with vt-d but vt-d non-snooped, since 
> under this case h/w can not guarantee cache coherency), so that all guest 
> memory type are UC. For other cases (when guest work w/o vt-d, and when guest 
> work with vt-d but vt-d snooped), Xen can still do nothing for guest cr0.cd 
> setting (since h/w has guaranteed cache coherency, otherwise Xen doesn't dare 
> to do what it does now -- it set iPAT bit as 1, totally ignoring guest memory 
> type point of view).
> 
> Compared with Dual-EPT solution, it's much simpler (only need IA32_PAT MSR 
> emulation under rare case) and avoid the inconsistency issue between 2 EPT 
> tables.

Right - as long as PAT is guaranteed to only be used for guest
induced memory accesses, that sounds like a viable yet not too
intrusive solution. Short of anyone else having an opinion here,
why don't you start drafting a patch for this?

Jan

Re: XSA-60 solutions

[Liu, Jinsong]

Jan Beulich wrote:
>>>> On 07.10.13 at 18:03, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
>> Jan Beulich wrote:
>>>>>> On 07.10.13 at 16:29, "Liu, Jinsong" <jinsong.liu@intel.com>
>>>>>> wrote: Any comments/suggestions?
>>> 
>>> As pointed out in earlier private conversation, I think that the
>>> dual table approach would be preferable if it can be made work.
>> 
>> No, solution 2 has an important update compared with our private
>> conversation before:
>> The old solution 2 is, Xen do nothing for guest cr0.cd setting.
>> The new solution 2 in this email is, Xen set IA32_PAT fields as UC
>> when needed (that means, only when guest work with vt-d but vt-d
>> non-snooped, since under this case h/w can not guarantee cache
>> coherency), so that all guest memory type are UC. For other cases
>> (when guest work w/o vt-d, and when guest work with vt-d but vt-d
>> snooped), Xen can still do nothing for guest cr0.cd setting (since
>> h/w has guaranteed cache coherency, otherwise Xen doesn't dare to do
>> what it does now -- it set iPAT bit as 1, totally ignoring guest
>> memory type point of view).  
>> 
>> Compared with Dual-EPT solution, it's much simpler (only need
>> IA32_PAT MSR emulation under rare case) and avoid the inconsistency
>> issue between 2 EPT tables.
> 
> Right - as long as PAT is guaranteed to only be used for guest
> induced memory accesses, that sounds like a viable yet not too
> intrusive solution. Short of anyone else having an opinion here,
> why don't you start drafting a patch for this?
> 
> Jan

That's great! We will draft patch.

Another point is, for shadow, currently Xen drop all shadows so that any new ones would be created on demand with UC. It works safely for XSA-60, but seems not good enough. We can still use IA32_PAT to solve it, not touch shadow page table itself.

One thing I need more confirm from AMD side: iirc, AMD IOMMU always works with snooped so cache cohernecy has been guaranteed, so no need do anything for guest cr0.cd setting, no matter shadow or NPT, right? If yes, most code could be done w/ Intel specific.

Thanks,
Jinsong

Re: XSA-60 solutions

[Jan Beulich]

>>> On 09.10.13 at 12:41, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
> Another point is, for shadow, currently Xen drop all shadows so that any new 
> ones would be created on demand with UC. It works safely for XSA-60, but seems 
> not good enough. We can still use IA32_PAT to solve it, not touch shadow page 
> table itself.

Tim, any though on this?

> One thing I need more confirm from AMD side: iirc, AMD IOMMU always works 
> with snooped so cache cohernecy has been guaranteed, so no need do anything 
> for guest cr0.cd setting, no matter shadow or NPT, right? If yes, most code 
> could be done w/ Intel specific.

Iirc SVM allows the guest to run with CR0.CD set. Whether that's
entirely valid to do I'm not certain, but I suppose it's at least to some
degree due to the major inhibiting factor (hyperthreads) not existing
there.

Jan

Re: XSA-60 solutions

[Tim Deegan]

At 12:40 +0100 on 09 Oct (1381322425), Jan Beulich wrote:
> >>> On 09.10.13 at 12:41, "Liu, Jinsong" <jinsong.liu@intel.com> wrote:
> > Another point is, for shadow, currently Xen drop all shadows so that any new 
> > ones would be created on demand with UC. It works safely for XSA-60, but seems 
> > not good enough. We can still use IA32_PAT to solve it, not touch shadow page 
> > table itself.
> 
> Tim, any though on this?

The existing shadow solution is OK, I think -- but if the same solution
that's used for EPT can be applied there, so much the better.

Tim.

> > One thing I need more confirm from AMD side: iirc, AMD IOMMU always works 
> > with snooped so cache cohernecy has been guaranteed, so no need do anything 
> > for guest cr0.cd setting, no matter shadow or NPT, right? If yes, most code 
> > could be done w/ Intel specific.
> 
> Iirc SVM allows the guest to run with CR0.CD set. Whether that's
> entirely valid to do I'm not certain, but I suppose it's at least to some
> degree due to the major inhibiting factor (hyperthreads) not existing
> there.
> 
> Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel