run time memory trap question

run time memory trap question

[Andrii Tseglytskyi]

Hi,

Can anyone clarify - is it possible to make a run time memory trap in
Xen hypervisor?

To be more specific - I see the following function:
*int handle_mmio(mmio_info_t *info)*, which is called from *static
void do_trap_data_abort_guest(struct cpu_user_regs *regs, union hsr
hsr)*
Using these calls I can define memory region and create a trap for it.
But in current implementation I can do it only during compile time.
Is there any way to do similar in runtime - i.e. calculate memory
region value in code and add an entry to mmio_handlers[] ?
Is it a good idea - just to modify/extend existing code in
xen/arch/arm/io.c file to make it possible ?

regards,
Andrii

-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.comxen/arch/arm/io.c

Re: run time memory trap question

[Julien Grall]

On 02/17/2014 11:33 AM, Andrii Tseglytskyi wrote:
> Hi,

Hello Andrii,

> Can anyone clarify - is it possible to make a run time memory trap in
> Xen hypervisor?

I guess you are talking about ARM? If so, it's not possible right now.

> To be more specific - I see the following function:
> *int handle_mmio(mmio_info_t *info)*, which is called from *static
> void do_trap_data_abort_guest(struct cpu_user_regs *regs, union hsr
> hsr)*
> Using these calls I can define memory region and create a trap for it.
> But in current implementation I can do it only during compile time.
> Is there any way to do similar in runtime - i.e. calculate memory
> region value in code and add an entry to mmio_handlers[] ?
> Is it a good idea - just to modify/extend existing code in
> xen/arch/arm/io.c file to make it possible ?

I think yes, we might need that for a couple of new features in Xen such
as: clock drivers and multiple devices in a same page (see UARTs on
cubieboard 2).

Regards,

-- 
Julien Grall

Re: run time memory trap question

[Andrii Tseglytskyi]

[-- Attachment #1.1: Type: text/plain, Size: 659 bytes --]

Hi Julien,


>
> > Can anyone clarify - is it possible to make a run time memory trap in
> > Xen hypervisor?
>
> I guess you are talking about ARM? If so, it's not possible right now.
>
>
Does it mean, that it is possible on x86 ?


>
> I think yes, we might need that for a couple of new features in Xen such
> as: clock drivers and multiple devices in a same page (see UARTs on
> cubieboard 2).
>
>
This sounds good. For now I need this or similar solution for my
development. I'll try to make it generic and post it to this list as soon
as it is ready.

Thank you.

Regards,
Andrii



-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com

[-- Attachment #1.2: Type: text/html, Size: 2154 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: run time memory trap question

[Julien Grall]

On 02/17/2014 02:06 PM, Andrii Tseglytskyi wrote:
> Hi Julien,

Hi Andrii,

> 
>     > Can anyone clarify - is it possible to make a run time memory trap in
>     > Xen hypervisor?
> 
>     I guess you are talking about ARM? If so, it's not possible right now.
> 
> 
> Does it mean, that it is possible on x86 ?

Yes, you can look at register_io_handler in xen/arch/x86/hvm/intercept.c

It's used a static array, but I don't think this is the solution for
ARM. We don't know in advance the maximum number of MMIO region to handle.

> 
>     I think yes, we might need that for a couple of new features in Xen such
>     as: clock drivers and multiple devices in a same page (see UARTs on
>     cubieboard 2).
> 
> 
> This sounds good. For now I need this or similar solution for my
> development. I'll try to make it generic and post it to this list as
> soon as it is ready.

Great, thank you.

Regards,

-- 
Julien Grall

Re: run time memory trap question

[Andrii Tseglytskyi]

[-- Attachment #1.1: Type: text/plain, Size: 1654 bytes --]

Hi Julien,


> >
> >     > Can anyone clarify - is it possible to make a run time memory trap
> in
> >     > Xen hypervisor?
> >
> >     I guess you are talking about ARM? If so, it's not possible right
> now.
> >
> >
> > Does it mean, that it is possible on x86 ?
>
> Yes, you can look at register_io_handler in xen/arch/x86/hvm/intercept.c
>
> It's used a static array, but I don't think this is the solution for
> ARM. We don't know in advance the maximum number of MMIO region to handle.
>
>
What I'm thinking about for ARM - is to use linked list for MMIO handlers +
API to register / unregister handler.

xen/arch/arm/io.c:

 25 static const struct mmio_handler *const mmio_handlers[] =
 26 {
 27     &vgic_distr_mmio_handler,
 28     &vuart_mmio_handler,
 29 };

This can be changed to list. New API will add / remove entries. VGIC and
VUART will call something like
mmio_register_handler(&vgic_distr_mmio_handle) during corresponding
initcall.

Than the only change which is required for existing int
handle_mmio(mmio_info_t *info) function  - is to enumerate list, instead of
array
 32 int handle_mmio(mmio_info_t *info)
 33 {
 34     struct vcpu *v = current;
 35     int i;
 36
 37     for ( i = 0; i < MMIO_HANDLER_NR; i++ ) --> *list_for_each*
 38         if ( mmio_handlers[i]->check_handler(v, info->gpa) )
 39             return info->dabt.write ?
 40                 mmio_handlers[i]->write_handler(v, info) :
 41                 mmio_handlers[i]->read_handler(v, info);
 42
 43     return 0;
 44 }

Something like this.

Regards,
Andrii



> --
> Julien Grall
>



-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com

[-- Attachment #1.2: Type: text/html, Size: 3573 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: run time memory trap question

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1840 bytes --]

On Mon, 17 Feb 2014, Andrii Tseglytskyi wrote:
> Hi Julien,
> 
>       >
>       >     > Can anyone clarify - is it possible to make a run time memory trap in
>       >     > Xen hypervisor?
>       >
>       >     I guess you are talking about ARM? If so, it's not possible right now.
>       >
>       >
>       > Does it mean, that it is possible on x86 ?
> 
> Yes, you can look at register_io_handler in xen/arch/x86/hvm/intercept.c
> 
> It's used a static array, but I don't think this is the solution for
> ARM. We don't know in advance the maximum number of MMIO region to handle.
> 
> 
> What I'm thinking about for ARM - is to use linked list for MMIO handlers + API to register / unregister handler.
> 
> xen/arch/arm/io.c:
> 
>  25 static const struct mmio_handler *const mmio_handlers[] =
>  26 {   
>  27     &vgic_distr_mmio_handler,
>  28     &vuart_mmio_handler,
>  29 };  
> 
> This can be changed to list. New API will add / remove entries. VGIC and VUART will call something like
> mmio_register_handler(&vgic_distr_mmio_handle) during corresponding initcall.
> 
> Than the only change which is required for existing int handle_mmio(mmio_info_t *info) function  - is to enumerate list, instead of array
>  32 int handle_mmio(mmio_info_t *info)
>  33 {
>  34     struct vcpu *v = current;
>  35     int i;
>  36 
>  37     for ( i = 0; i < MMIO_HANDLER_NR; i++ ) --> *list_for_each*
>  38         if ( mmio_handlers[i]->check_handler(v, info->gpa) )
>  39             return info->dabt.write ?
>  40                 mmio_handlers[i]->write_handler(v, info) :
>  41                 mmio_handlers[i]->read_handler(v, info);
>  42 
>  43     return 0;
>  44 }
> 
> Something like this.

Sounds good in theory

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: run time memory trap question

[Julien Grall]

On 02/17/2014 03:01 PM, Andrii Tseglytskyi wrote:
> Hi Julien,
> 
> 
>     >
>     >     > Can anyone clarify - is it possible to make a run time
>     memory trap in
>     >     > Xen hypervisor?
>     >
>     >     I guess you are talking about ARM? If so, it's not possible
>     right now.
>     >
>     >
>     > Does it mean, that it is possible on x86 ?
> 
>     Yes, you can look at register_io_handler in xen/arch/x86/hvm/intercept.c
> 
>     It's used a static array, but I don't think this is the solution for
>     ARM. We don't know in advance the maximum number of MMIO region to
>     handle.
> 
> 
> What I'm thinking about for ARM - is to use linked list for MMIO
> handlers + API to register / unregister handler.
> 
> xen/arch/arm/io.c:
> 
>  25 static const struct mmio_handler *const mmio_handlers[] =
>  26 {   
>  27     &vgic_distr_mmio_handler,
>  28     &vuart_mmio_handler,
>  29 };  
> 
> This can be changed to list. New API will add / remove entries. VGIC and
> VUART will call something like
> mmio_register_handler(&vgic_distr_mmio_handle) during corresponding
> initcall.
> 
> Than the only change which is required for existing int
> handle_mmio(mmio_info_t *info) function  - is to enumerate list, instead
> of array
>  32 int handle_mmio(mmio_info_t *info)
>  33 {
>  34     struct vcpu *v = current;
>  35     int i;
>  36 
>  37     for ( i = 0; i < MMIO_HANDLER_NR; i++ ) --> *list_for_each*
>  38         if ( mmio_handlers[i]->check_handler(v, info->gpa) )
>  39             return info->dabt.write ?
>  40                 mmio_handlers[i]->write_handler(v, info) :
>  41                 mmio_handlers[i]->read_handler(v, info);
>  42 
>  43     return 0;
>  44 }
> 
> Something like this.

This solution sounds good.

If I remembered correctly, you are writing a driver for IPU/GPU MMU, right?
In this case, I think a per-domain MMU handlers would be better. Most of
handlers will be used for a specific guest (except the VGIC handler).

-- 
Julien Grall

Re: run time memory trap question

[Andrii Tseglytskyi]

Hi

>
> > Something like this.
>
> This solution sounds good.
>
> If I remembered correctly, you are writing a driver for IPU/GPU MMU, right?


Right. And for now I started developing something like shadow page
table algorithm. I need to create a trap for a real pagetable, which
is placed somewhere in domain heap memory. That's why I thought about
generic algorithm which can create a runtime trap for memory, which
address is not defined during compile time.


>
> In this case, I think a per-domain MMU handlers would be better. Most of
> handlers will be used for a specific guest (except the VGIC handler).
>

Do you mean that existing mmio_handlers[] are global and new runtime
memory handlers should be per domain? This sounds quite reasonable.

regards,
Andrii



-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com

Re: run time memory trap question

[Julien Grall]

Hi,

On 02/17/2014 03:38 PM, Andrii Tseglytskyi wrote:
> Do you mean that existing mmio_handlers[] are global and new runtime
> memory handlers should be per domain? This sounds quite reasonable.

Yes. For now, check_handler callback check if theses handlers can be
callback on the current domain.

Regards,

-- 
Julien Grall

Re: run time memory trap question

[Ian Campbell]

On Mon, 2014-02-17 at 17:38 +0200, Andrii Tseglytskyi wrote:
> Hi
> 
> >
> > > Something like this.
> >
> > This solution sounds good.
> >
> > If I remembered correctly, you are writing a driver for IPU/GPU MMU, right?
> 
> 
> Right. And for now I started developing something like shadow page
> table algorithm. I need to create a trap for a real pagetable, which
> is placed somewhere in domain heap memory. That's why I thought about
> generic algorithm which can create a runtime trap for memory, which
> address is not defined during compile time.

I think you should handle this case by making the page r/o in the p2m
and using the p2m fault handler rather than the MMIO trap
infrastructure.

See e.g. the live migration patches by Samsung.

Ian.

Re: run time memory trap question

[Andrii Tseglytskyi]

Hi Ian,


On Tue, Feb 18, 2014 at 1:05 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2014-02-17 at 17:38 +0200, Andrii Tseglytskyi wrote:
>> Hi
>>
>> >
>> > > Something like this.
>> >
>> > This solution sounds good.
>> >
>> > If I remembered correctly, you are writing a driver for IPU/GPU MMU, right?
>>
>>
>> Right. And for now I started developing something like shadow page
>> table algorithm. I need to create a trap for a real pagetable, which
>> is placed somewhere in domain heap memory. That's why I thought about
>> generic algorithm which can create a runtime trap for memory, which
>> address is not defined during compile time.
>
> I think you should handle this case by making the page r/o in the p2m
> and using the p2m fault handler rather than the MMIO trap
> infrastructure.
>
> See e.g. the live migration patches by Samsung.
>

I haven't thought in this direction. I will investigate this
possibility. Thank you for suggestion. Just for now I implemented
solution, discussed in previous mails and it works fine for me, but
I'll try what you suggesting - it may be a better way.

Thank you,

regards,
Andrii

-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com