On Fri, Apr 10, 2015 at 5:04 PM, Gareth Stockwell <= ;Gareth.Stock= well@arm.com> wrote:

My understanding of memaccess is that it provides th= e following functionality:

=C2=A0

1. Modify permission va= lues in the page table of the target domain.

2. When the domain gene= rates an illegal access, the exception handler delegates to memaccess, whic= h pauses the VCPU and records the event in a ring buffer.

3. Some permission valu= es are special in that they automatically mutate to a different value follo= wing the first access; others are persistently applied.

=C2=A0

We would like to use memaccess to perform (1) - but = rather than pausing the VCPU in (2), instead simply directly inject the exc= eption into the VCPU.=C2=A0 I can see two ways of doing this:=

=C2=A0

a) Implement an observe= r of the ring buffer, which triggers injection of the exception and unpausi= ng of the VCPU.

b) Define new xenmem_ac= cess_t values which cause the exception handler to reinject rather than add= ing a message to the ring buffer.

=C2=A0

(a) seems cumbersome, and requires multiple context = switches in order to handle the exception.

(b) therefore looks preferable, and I think should b= e fairly simple to implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.<= u>

=C2=A0

Does this sound reasonable?=C2=A0 Or is there a bett= er way of modifying access permissions for a specific pfn range of a target= domain?

=C2=A0

Gareth

=C2=A0

-- IMPORTANT NOTICE: The contents of t= his email and any attachments are confidential and may also be privileged. = If you are not the intended recipient, please notify the sender immediately= and do not disclose the contents to any other person, use it for any purpose, or store or copy the informat= ion in any medium. Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Regist= ered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, R= egistered in England & Wales, Company No: 2548782

Hi Gareth,

interesting use-case. It should be possible and= I agree that option b is your easiest route to go forward with. FYI my arm= _memaccess15 branch is still being worked on so it's not yet safe to br= anch off of it. I'm planning on submitting it in the very near future (= maybe later today) and hopefully it will be merged before the end of the mo= nth ;)

Cheers,

Tamas

--047d7b5dad56e6513705136069a4-- --===============1308682172943944889== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============1308682172943944889==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: memaccess: skipping mem_access_send_req Date: Fri, 10 Apr 2015 17:03:26 +0100 Message-ID: <5527F44E.6090302@citrix.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5410027389997848400==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org --===============5410027389997848400== Content-Type: multipart/alternative; boundary="------------050503020502020004040002" --------------050503020502020004040002 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit On 10/04/15 16:04, Gareth Stockwell wrote: > > My understanding of memaccess is that it provides the following > functionality: > > > > 1. Modify permission values in the page table of the target domain. > > 2. When the domain generates an illegal access, the exception handler > delegates to memaccess, which pauses the VCPU and records the event in > a ring buffer. > > 3. Some permission values are special in that they automatically > mutate to a different value following the first access; others are > persistently applied. > > > > We would like to use memaccess to perform (1) - but rather than > pausing the VCPU in (2), instead simply directly inject the exception > into the VCPU. I can see two ways of doing this: > > > > a) Implement an observer of the ring buffer, which triggers injection > of the exception and unpausing of the VCPU. > > b) Define new xenmem_access_t values which cause the exception handler > to reinject rather than adding a message to the ring buffer. > > > > (a) seems cumbersome, and requires multiple context switches in order > to handle the exception. > > (b) therefore looks preferable, and I think should be fairly simple to > implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15. > > > > Does this sound reasonable? Or is there a better way of modifying > access permissions for a specific pfn range of a target domain? > Are you perhaps looking for something similar to Intel #VE support? (although I guess you are looking at ARM rather than x86) It is not safe to always re-inject back into the vcpu, especially if the permission changes were around the guest kernel stack, but a restricted subset ought to be safe to bounce straight in. ~Andrew --------------050503020502020004040002 Content-Type: text/html; charset="windows-1252" Content-Length: 4646 Content-Transfer-Encoding: quoted-printable

On 10/04/15 16:04, Gareth Stockwell wrote:

My understanding of memaccess is that it provides the following functionality:

=A0

1. Modify permission values in the page table of the target domain.

2. When the domain generates an illegal access, the exception handler delegates to memaccess, which pauses the VCPU and records the event in a ring buffer.

3. Some permission values are special in that they automatically mutate to a different value following the first access; others are persistently applied.

=A0

We would like to use memaccess to perform (1) - but rather than pausing the VCPU in (2), instead simply directly inject the exception into the VCPU.=A0 I can see two ways of doing this:

=A0

a) Implement an observer of the ring buffer, which triggers injection of the exception and unpausing of the VCPU.

b) Define new xenmem_access_t values which cause the exception handler to reinject rather than adding a message to the ring buffer.

=A0

(a) seems cumbersome, and requires multiple context switches in order to handle the exception.

(b) therefore looks preferable, and I think should be fairly simple to implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.

=A0

Does this sound reasonable=3F=A0 Or is there a better way of modifying access permissions for a specific pfn range of a target domain=3F

Are you perhaps looking for something similar to Intel #VE support=3F=A0 (although I guess you are looking at ARM rather than x86)

It is not safe to always re-inject back into the vcpu, especially if the permission changes were around the guest kernel stack, but a restricted subset ought to be safe to bounce straight in.

~Andrew
--------------050503020502020004040002-- --===============5410027389997848400== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============5410027389997848400==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: memaccess: skipping mem_access_send_req Date: Wed, 15 Apr 2015 10:18:23 +0100 Message-ID: <1429089503.15516.149.camel@citrix.com> References: <5527E828.8010406@bitdefender.com> <5527EAD6.6040900@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5527EAD6.6040900@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Julien Grall Cc: "stefano.stabellini@citrix.com" , Gareth Stockwell , "tklengyel@sec.in.tum.de" , Razvan Cojocaru , "xen-devel (xen-devel@lists.xen.org)" List-Id: xen-devel@lists.xenproject.org On Fri, 2015-04-10 at 16:23 +0100, Julien Grall wrote: > Hi Razvan, > > On 10/04/15 16:11, Razvan Cojocaru wrote: > > It's not entirely clear to me what you're trying to do, but if I > > understand it correctly, there's always the third option of simply using > > the vm_event system as it is now, and when you get an EPT violation > > event simply use xc_hvm_inject_trap() from userspace to inject the > > exception. > > xc_hvm_inject_trap is not implemented on ARM. > > Also, looking to the function, most of the traps and arguments are x86 > specific. If we need similar behaviour then we should probably implement HVMOP_inject_trap for ARM, though, not invent another interface. Perhaps the arg struct needs to become per arch. FWIW I'd do that as * s/xen_hvm_inject_trap/xen_hvm_inject_trap_x86 * #define xen_hvm_inject_trap xen_hvm_inject_trap_x86 * introduce xen_hvm_inject_trap_arm, change #define to #ifdef tree picking the correct one. Ian. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: memaccess: skipping mem_access_send_req Date: Wed, 15 Apr 2015 10:26:52 +0100 Message-ID: <1429090012.15516.155.camel@citrix.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Gareth Stockwell Cc: "stefano.stabellini@citrix.com" , "tklengyel@sec.in.tum.de" , "xen-devel (xen-devel@lists.xen.org)" List-Id: xen-devel@lists.xenproject.org On Fri, 2015-04-10 at 16:04 +0100, Gareth Stockwell wrote: > We would like to use memaccess to perform (1) - but rather than > pausing the VCPU in (2), instead simply directly inject the exception > into the VCPU. That is, into the VCPUs whose permissions have been modified behind its back and not into some controlling domain? Is the guest expected to be aware of this, i.e. to be somewhat paravirtualised? I suppose it must have to be in order to accept seemingly spurious page faults. Which leads me to wonder whether an extra shared ring between the hypervisor and target VCPU would be desirable, i.e. to allow more fine grained semantics than just "computer says no". Specifically if you need to care about the reason for the fault being the actions of an external arbiter rather than some other guest-internal thing. If your application is just to allow the guest OS to kill a process which has tried to touch memory in a way which the external controller has disallowed then a page fault seems like a simple and effective way though. > b) Define new xenmem_access_t values which cause the exception handler > to reinject rather than adding a message to the ring buffer. I'm no xenaccess guru but that's the option I'd be inclined to take. My only concern would be limitations on the number of xenaccess types. On ARM we couldn't fit the type into the spare P2M PTE bits so we will already have a separate lookup and therefore not much limitation, but I don't know about on x86. Ian. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gareth Stockwell Subject: Re: memaccess: skipping mem_access_send_req Date: Fri, 17 Apr 2015 10:35:07 +0100 Message-ID: References: <5527F44E.6090302@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5527F44E.6090302@citrix.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Fri, Apr 10, 2015 at 17:03:26, Andrew Cooper wrote: > Are you perhaps looking for something similar to Intel #VE support? Yes, in that we want a way to notify a guest that it has made an access that violated a stage-2 / EPT permission. However, for our purposes a trap into the hypervisor followed by injection of a virtual interrupt into the guest is acceptable; we don't require the interrupt to be directly delivered to the guest. > (although I guess you are looking at ARM rather than x86) Yes, my email address is a giveaway ;) > It is not safe to always re-inject back into the vcpu, especially if > the permission changes were around the guest kernel stack, but a > restricted subset ought to be safe to bounce straight in. The permission changes will be applied only to DMA buffers, rather than to anything the guest kernel itself needs to touch - so should be safe to directly re-inject. -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gareth Stockwell Subject: Re: memaccess: skipping mem_access_send_req Date: Fri, 17 Apr 2015 10:35:44 +0100 Message-ID: References: <1429090012.15516.155.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1429090012.15516.155.camel@citrix.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Ian.Campbell@citrix.com" Cc: "stefano.stabellini@citrix.com" , "tklengyel@sec.in.tum.de" , "xen-devel (xen-devel@lists.xen.org)" List-Id: xen-devel@lists.xenproject.org On Wed, Apr 15, 2015 at 10:26:52, Ian Campbell wrote: > > We would like to use memaccess to perform (1) - but rather than > > pausing the VCPU in (2), instead simply directly inject the > > exception into the VCPU. > > That is, into the VCPUs whose permissions have been modified behind > its back and not into some controlling domain? Correct - the abort should be injected into the VCPU which initiated the invalid transaction. > Is the guest expected to be aware of this, i.e. to be somewhat > paravirtualised? I suppose it must have to be in order to accept > seemingly spurious page faults. > > Which leads me to wonder whether an extra shared ring between the > hypervisor and target VCPU would be desirable, i.e. to allow more fine > grained semantics than just "computer says no". Specifically if you > need to care about the reason for the fault being the actions of an > external arbiter rather than some other guest-internal thing. > > If your application is just to allow the guest OS to kill a process > which has tried to touch memory in a way which the external controller > has disallowed then a page fault seems like a simple and effective way though. The guest will be aware of the permission changes - in fact in our system permission changes are only enacted following a request from the guest itself. So, a data abort is sufficient - the guest should then be able to work that this was due to it violating its stage-2 permissions, and kill the appropriate process. > > b) Define new xenmem_access_t values which cause the exception > handler > > to reinject rather than adding a message to the ring buffer. > > I'm no xenaccess guru but that's the option I'd be inclined to take. > > My only concern would be limitations on the number of xenaccess types. > On ARM we couldn't fit the type into the spare P2M PTE bits so we will > already have a separate lookup and therefore not much limitation, but > I don't know about on x86. It seems to be the simplest approach, and based on some quick prototyping appears to work - at least on ARM. -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590 ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: memaccess: skipping mem_access_send_req Date: Fri, 17 Apr 2015 11:31:27 +0100 Message-ID: <1429266687.25195.257.camel@citrix.com> References: <1429090012.15516.155.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Gareth Stockwell Cc: "stefano.stabellini@citrix.com" , "tklengyel@sec.in.tum.de" , "xen-devel (xen-devel@lists.xen.org)" List-Id: xen-devel@lists.xenproject.org On Fri, 2015-04-17 at 10:35 +0100, Gareth Stockwell wrote: > On Wed, Apr 15, 2015 at 10:26:52, Ian Campbell wrote: > > > We would like to use memaccess to perform (1) - but rather than > > Is the guest expected to be aware of this, i.e. to be somewhat > > paravirtualised? I suppose it must have to be in order to accept > > seemingly spurious page faults. > > > > Which leads me to wonder whether an extra shared ring between the > > hypervisor and target VCPU would be desirable, i.e. to allow more fine > > grained semantics than just "computer says no". Specifically if you > > need to care about the reason for the fault being the actions of an > > external arbiter rather than some other guest-internal thing. > > > > If your application is just to allow the guest OS to kill a process > > which has tried to touch memory in a way which the external controller > > has disallowed then a page fault seems like a simple and effective way though. > > The guest will be aware of the permission changes - in fact in our > system permission changes are only enacted following a request from > the guest itself. So, a data abort is sufficient - the guest should > then be able to work that this was due to it violating its stage-2 > permissions, and kill the appropriate process. Great, in which case this does seem to be a good approach. [...] > It seems to be the simplest approach, and based on some quick prototyping appears to work - at least on ARM. Excellent! Ian