memaccess: skipping mem_access_send_req

memaccess: skipping mem_access_send_req

[Gareth Stockwell]

[-- Attachment #1.1: Type: text/plain, Size: 1869 bytes --]

My understanding of memaccess is that it provides the following functionality:

1. Modify permission values in the page table of the target domain.
2. When the domain generates an illegal access, the exception handler delegates to memaccess, which pauses the VCPU and records the event in a ring buffer.
3. Some permission values are special in that they automatically mutate to a different value following the first access; others are persistently applied.

We would like to use memaccess to perform (1) - but rather than pausing the VCPU in (2), instead simply directly inject the exception into the VCPU.  I can see two ways of doing this:

a) Implement an observer of the ring buffer, which triggers injection of the exception and unpausing of the VCPU.
b) Define new xenmem_access_t values which cause the exception handler to reinject rather than adding a message to the ring buffer.

(a) seems cumbersome, and requires multiple context switches in order to handle the exception.
(b) therefore looks preferable, and I think should be fairly simple to implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.

Does this sound reasonable?  Or is there a better way of modifying access permissions for a specific pfn range of a target domain?

Gareth


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782

[-- Attachment #1.2: Type: text/html, Size: 4457 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: memaccess: skipping mem_access_send_req

[Razvan Cojocaru]

On 04/10/2015 06:04 PM, Gareth Stockwell wrote:
> My understanding of memaccess is that it provides the following
> functionality:
> 
>  
> 
> 1. Modify permission values in the page table of the target domain.
> 
> 2. When the domain generates an illegal access, the exception handler
> delegates to memaccess, which pauses the VCPU and records the event in a
> ring buffer.
> 
> 3. Some permission values are special in that they automatically mutate
> to a different value following the first access; others are persistently
> applied.
> 
>  
> 
> We would like to use memaccess to perform (1) - but rather than pausing
> the VCPU in (2), instead simply directly inject the exception into the
> VCPU.  I can see two ways of doing this:
> 
>  
> 
> a) Implement an observer of the ring buffer, which triggers injection of
> the exception and unpausing of the VCPU.
> 
> b) Define new xenmem_access_t values which cause the exception handler
> to reinject rather than adding a message to the ring buffer.
> 
>  
> 
> (a) seems cumbersome, and requires multiple context switches in order to
> handle the exception.
> 
> (b) therefore looks preferable, and I think should be fairly simple to
> implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.
> 
>  
> 
> Does this sound reasonable?  Or is there a better way of modifying
> access permissions for a specific pfn range of a target domain?

It's not entirely clear to me what you're trying to do, but if I
understand it correctly, there's always the third option of simply using
the vm_event system as it is now, and when you get an EPT violation
event simply use xc_hvm_inject_trap() from userspace to inject the
exception.

If you don't want to remove the page restrictions after this (in order
to unpause the VCPU), you can simply emulate the instruction that caused
the event and continue that way. I've added support for that last year
(but I don't know right now if that part is still functional until
Tamas' final half of the series gets merged).


HTH,
Razvan

Re: memaccess: skipping mem_access_send_req

[Julien Grall]

Hi Razvan,

On 10/04/15 16:11, Razvan Cojocaru wrote:
> It's not entirely clear to me what you're trying to do, but if I
> understand it correctly, there's always the third option of simply using
> the vm_event system as it is now, and when you get an EPT violation
> event simply use xc_hvm_inject_trap() from userspace to inject the
> exception.

xc_hvm_inject_trap is not implemented on ARM.

Also, looking to the function, most of the traps and arguments are x86
specific.

Regards,

-- 
Julien Grall

Re: memaccess: skipping mem_access_send_req

[Ian Campbell]

On Fri, 2015-04-10 at 16:23 +0100, Julien Grall wrote:
> Hi Razvan,
> 
> On 10/04/15 16:11, Razvan Cojocaru wrote:
> > It's not entirely clear to me what you're trying to do, but if I
> > understand it correctly, there's always the third option of simply using
> > the vm_event system as it is now, and when you get an EPT violation
> > event simply use xc_hvm_inject_trap() from userspace to inject the
> > exception.
> 
> xc_hvm_inject_trap is not implemented on ARM.
> 
> Also, looking to the function, most of the traps and arguments are x86
> specific.

If we need similar behaviour then we should probably implement
HVMOP_inject_trap for ARM, though, not invent another interface.

Perhaps the arg struct needs to become per arch. FWIW I'd do that as
      * s/xen_hvm_inject_trap/xen_hvm_inject_trap_x86
      * #define xen_hvm_inject_trap xen_hvm_inject_trap_x86
      * introduce xen_hvm_inject_trap_arm, change #define to #ifdef tree
        picking the correct one.

Ian.

Re: memaccess: skipping mem_access_send_req

[Tamas K Lengyel]

[-- Attachment #1.1: Type: text/plain, Size: 2414 bytes --]

On Fri, Apr 10, 2015 at 5:04 PM, Gareth Stockwell <Gareth.Stockwell@arm.com>
wrote:

>  My understanding of memaccess is that it provides the following
> functionality:
>
>
>
> 1. Modify permission values in the page table of the target domain.
>
> 2. When the domain generates an illegal access, the exception handler
> delegates to memaccess, which pauses the VCPU and records the event in a
> ring buffer.
>
> 3. Some permission values are special in that they automatically mutate to
> a different value following the first access; others are persistently
> applied.
>
>
>
> We would like to use memaccess to perform (1) - but rather than pausing
> the VCPU in (2), instead simply directly inject the exception into the
> VCPU.  I can see two ways of doing this:
>
>
>
> a) Implement an observer of the ring buffer, which triggers injection of
> the exception and unpausing of the VCPU.
>
> b) Define new xenmem_access_t values which cause the exception handler to
> reinject rather than adding a message to the ring buffer.
>
>
>
> (a) seems cumbersome, and requires multiple context switches in order to
> handle the exception.
>
> (b) therefore looks preferable, and I think should be fairly simple to
> implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.
>
>
>
> Does this sound reasonable?  Or is there a better way of modifying access
> permissions for a specific pfn range of a target domain?
>
>
>
> Gareth
>
>
>
> -- IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sender immediately and do not disclose the
> contents to any other person, use it for any purpose, or store or copy the
> information in any medium. Thank you.
>
> ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
> Registered in England & Wales, Company No: 2557590
> ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
> Registered in England & Wales, Company No: 2548782
>

Hi Gareth,
interesting use-case. It should be possible and I agree that option b is
your easiest route to go forward with. FYI my arm_memaccess15 branch is
still being worked on so it's not yet safe to branch off of it. I'm
planning on submitting it in the very near future (maybe later today) and
hopefully it will be merged before the end of the month ;)

Cheers,
Tamas

[-- Attachment #1.2: Type: text/html, Size: 3872 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: memaccess: skipping mem_access_send_req

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 1728 bytes --]

On 10/04/15 16:04, Gareth Stockwell wrote:
>
> My understanding of memaccess is that it provides the following
> functionality:
>
>  
>
> 1. Modify permission values in the page table of the target domain.
>
> 2. When the domain generates an illegal access, the exception handler
> delegates to memaccess, which pauses the VCPU and records the event in
> a ring buffer.
>
> 3. Some permission values are special in that they automatically
> mutate to a different value following the first access; others are
> persistently applied.
>
>  
>
> We would like to use memaccess to perform (1) - but rather than
> pausing the VCPU in (2), instead simply directly inject the exception
> into the VCPU.  I can see two ways of doing this:
>
>  
>
> a) Implement an observer of the ring buffer, which triggers injection
> of the exception and unpausing of the VCPU.
>
> b) Define new xenmem_access_t values which cause the exception handler
> to reinject rather than adding a message to the ring buffer.
>
>  
>
> (a) seems cumbersome, and requires multiple context switches in order
> to handle the exception.
>
> (b) therefore looks preferable, and I think should be fairly simple to
> implement on top of https://github.com/tklengyel/xen/tree/arm_memaccess15.
>
>  
>
> Does this sound reasonable?  Or is there a better way of modifying
> access permissions for a specific pfn range of a target domain?
>

Are you perhaps looking for something similar to Intel #VE support? 
(although I guess you are looking at ARM rather than x86)

It is not safe to always re-inject back into the vcpu, especially if the
permission changes were around the guest kernel stack, but a restricted
subset ought to be safe to bounce straight in.

~Andrew

[-- Attachment #1.2: Type: text/html, Size: 4539 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: memaccess: skipping mem_access_send_req

[Gareth Stockwell]

On Fri, Apr 10, 2015 at 17:03:26, Andrew Cooper wrote:
> Are you perhaps looking for something similar to Intel #VE support?
Yes, in that we want a way to notify a guest that it has made an access that violated a stage-2 / EPT permission.  However, for our purposes a trap into the hypervisor followed by injection of a virtual interrupt into the guest is acceptable; we don't require the interrupt to be directly delivered to the guest.

> (although I guess you are looking at ARM rather than x86)
Yes, my email address is a giveaway ;)

> It is not safe to always re-inject back into the vcpu, especially if
> the permission changes were around the guest kernel stack, but a
> restricted subset ought to be safe to bounce straight in.
The permission changes will be applied only to DMA buffers, rather than to anything the guest kernel itself needs to touch - so should be safe to directly re-inject.


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2548782

Re: memaccess: skipping mem_access_send_req

[Ian Campbell]

On Fri, 2015-04-10 at 16:04 +0100, Gareth Stockwell wrote:

> We would like to use memaccess to perform (1) - but rather than
> pausing the VCPU in (2), instead simply directly inject the exception
> into the VCPU.

That is, into the VCPUs whose permissions have been modified behind its
back and not into some controlling domain?

Is the guest expected to be aware of this, i.e. to be somewhat
paravirtualised? I suppose it must have to be in order to accept
seemingly spurious page faults.

Which leads me to wonder whether an extra shared ring between the
hypervisor and target VCPU would be desirable, i.e. to allow more fine
grained semantics than just "computer says no". Specifically if you need
to care about the reason for the fault being the actions of an external
arbiter rather than some other guest-internal thing.

If your application is just to allow the guest OS to kill a process
which has tried to touch memory in a way which the external controller
has disallowed then a page fault seems like a simple and effective way
though.

> b) Define new xenmem_access_t values which cause the exception handler
> to reinject rather than adding a message to the ring buffer.

I'm no xenaccess guru but that's the option I'd be inclined to take.

My only concern would be limitations on the number of xenaccess types.
On ARM we couldn't fit the type into the spare P2M PTE bits so we will
already have a separate lookup and therefore not much limitation, but I
don't know about on x86.

Ian.

Re: memaccess: skipping mem_access_send_req

[Gareth Stockwell]

On Wed, Apr 15, 2015 at 10:26:52, Ian Campbell wrote:
> > We would like to use memaccess to perform (1) - but rather than
> > pausing the VCPU in (2), instead simply directly inject the
> > exception into the VCPU.
>
> That is, into the VCPUs whose permissions have been modified behind
> its back and not into some controlling domain?

Correct - the abort should be injected into the VCPU which initiated the invalid transaction.

> Is the guest expected to be aware of this, i.e. to be somewhat
> paravirtualised? I suppose it must have to be in order to accept
> seemingly spurious page faults.
>
> Which leads me to wonder whether an extra shared ring between the
> hypervisor and target VCPU would be desirable, i.e. to allow more fine
> grained semantics than just "computer says no". Specifically if you
> need to care about the reason for the fault being the actions of an
> external arbiter rather than some other guest-internal thing.
>
> If your application is just to allow the guest OS to kill a process
> which has tried to touch memory in a way which the external controller
> has disallowed then a page fault seems like a simple and effective way though.

The guest will be aware of the permission changes - in fact in our system permission changes are only enacted following a request from the guest itself.  So, a data abort is sufficient - the guest should then be able to work that this was due to it violating its stage-2 permissions, and kill the appropriate process.

> > b) Define new xenmem_access_t values which cause the exception
> handler
> > to reinject rather than adding a message to the ring buffer.
>
> I'm no xenaccess guru but that's the option I'd be inclined to take.
>
> My only concern would be limitations on the number of xenaccess types.
> On ARM we couldn't fit the type into the spare P2M PTE bits so we will
> already have a separate lookup and therefore not much limitation, but
> I don't know about on x86.

It seems to be the simplest approach, and based on some quick prototyping appears to work - at least on ARM.


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2548782

Re: memaccess: skipping mem_access_send_req

[Ian Campbell]

On Fri, 2015-04-17 at 10:35 +0100, Gareth Stockwell wrote:
> On Wed, Apr 15, 2015 at 10:26:52, Ian Campbell wrote:
> > > We would like to use memaccess to perform (1) - but rather than
> > Is the guest expected to be aware of this, i.e. to be somewhat
> > paravirtualised? I suppose it must have to be in order to accept
> > seemingly spurious page faults.
> >
> > Which leads me to wonder whether an extra shared ring between the
> > hypervisor and target VCPU would be desirable, i.e. to allow more fine
> > grained semantics than just "computer says no". Specifically if you
> > need to care about the reason for the fault being the actions of an
> > external arbiter rather than some other guest-internal thing.
> >
> > If your application is just to allow the guest OS to kill a process
> > which has tried to touch memory in a way which the external controller
> > has disallowed then a page fault seems like a simple and effective way though.
> 
> The guest will be aware of the permission changes - in fact in our
> system permission changes are only enacted following a request from
> the guest itself.  So, a data abort is sufficient - the guest should
> then be able to work that this was due to it violating its stage-2
> permissions, and kill the appropriate process.

Great, in which case this does seem to be a good approach.
[...]

> It seems to be the simplest approach, and based on some quick prototyping appears to work - at least on ARM.

Excellent!

Ian