help with understanding GPA/GVA issues

help with understanding GPA/GVA issues

[Gregory Haskins]

Hi All,
  I am working on some PV stuff and had some questions about the ability
to share memory across the Guest/Host boundary.

It seems that most examples of how to do this always involve starting
with a *page, converting it to a gfn via page_to_gfn(), and using that
as a gpa to pass across the boundary.

I understand that this method avoids a software traipse through the
page-walker, so it's nice.  What I can't quite figure out is what are
the other types of memory (if any) that can be passed across.

For instance, is a pointer from kmalloc() considered a gpa, a gva,
neither?  Or are gva's only pointers that come from guest-userspace,
etc.  Is it possible to pass something like a skb->data pointer (I
understand that I may have to run the page-walker for some of these)?
If so, how would I do this:  E.g. can I just pass the pointer, and then
do gva_to_hpa() on the host?  Or do I need to prep the pointer before
sending it?

TIA
-Greg


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Anthony Liguori]

Gregory Haskins wrote:
> Hi All,
>   I am working on some PV stuff and had some questions about the ability
> to share memory across the Guest/Host boundary.
>
> It seems that most examples of how to do this always involve starting
> with a *page, converting it to a gfn via page_to_gfn(), and using that
> as a gpa to pass across the boundary.
>   

Do you mean page_to_pfn?  I assume you're talking about a page within 
the guest right?

> I understand that this method avoids a software traipse through the
> page-walker, so it's nice.  What I can't quite figure out is what are
> the other types of memory (if any) that can be passed across.
>
> For instance, is a pointer from kmalloc() considered a gpa, a gva,
> neither?

gpa = guest physical address.  It's a pa or a pfn << PAGE_SHIFT.

gva = guest virtual address.  It's returned from pretty much anything 
that allocates memory (kmalloc for instance).  This is all within the 
guest of course.

>   Or are gva's only pointers that come from guest-userspace,
> etc.  Is it possible to pass something like a skb->data pointer (I
> understand that I may have to run the page-walker for some of these)?
>   

Yes, you can pass through any gva.  There are couple of things to be 
aware of though.  When passing a gva, you have to be sure that the gva 
is actually mapped in memory as KVM cannot cause Linux to fault in a 
page.  Also, for something like skb->data, you should probably just pass 
the gpa since they'll usually fall within a single page anyway.

> If so, how would I do this:  E.g. can I just pass the pointer, and then
> do gva_to_hpa() on the host?  Or do I need to prep the pointer before
> sending it?
>   

There's no need to prep provided that you know the va is mapped into 
memory in the guest.

Regards,

Anthony Liguori

> TIA
> -Greg
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Gregory Haskins]

On Fri, 2007-08-10 at 09:32 -0500, Anthony Liguori wrote:
> Gregory Haskins wrote:
> > Hi All,
> >   I am working on some PV stuff and had some questions about the ability
> > to share memory across the Guest/Host boundary.
> >
> > It seems that most examples of how to do this always involve starting
> > with a *page, converting it to a gfn via page_to_gfn(), and using that
> > as a gpa to pass across the boundary.
> >   
> 
> Do you mean page_to_pfn?  I assume you're talking about a page within 
> the guest right?

Er, ya.  Sorry...up too late last night ;)

> 
> > I understand that this method avoids a software traipse through the
> > page-walker, so it's nice.  What I can't quite figure out is what are
> > the other types of memory (if any) that can be passed across.
> >
> > For instance, is a pointer from kmalloc() considered a gpa, a gva,
> > neither?
> 
> gpa = guest physical address.  It's a pa or a pfn << PAGE_SHIFT.
> 
> gva = guest virtual address.  It's returned from pretty much anything 
> that allocates memory (kmalloc for instance).  This is all within the 
> guest of course.
> 
> >   Or are gva's only pointers that come from guest-userspace,
> > etc.  Is it possible to pass something like a skb->data pointer (I
> > understand that I may have to run the page-walker for some of these)?
> >   
> 
> Yes, you can pass through any gva.  There are couple of things to be 
> aware of though.  When passing a gva, you have to be sure that the gva 
> is actually mapped in memory as KVM cannot cause Linux to fault in a 
> page.

Cool!  Ya, I totally understand and agree that it has to be "DMA" class
memory that is mapped and pinned in guest context.  IIUC you basically
need to follow the same rules as you would for a DMA based device in a
bare-metal scenario.

>   Also, for something like skb->data, you should probably just pass 
> the gpa since they'll usually fall within a single page anyway.

Hmm...good point.

> 
> > If so, how would I do this:  E.g. can I just pass the pointer, and then
> > do gva_to_hpa() on the host?  Or do I need to prep the pointer before
> > sending it?
> >   
> 
> There's no need to prep provided that you know the va is mapped into 
> memory in the guest.
> 

Thanks Anthony!  Very helpful indeed and I appreciate the explanation.

On this topic:  I know there has been talk going on of giving each VM
its own linux va context.  IIUC, when that happens we wouldn't need the
gva_to_hpa type functions, right?  We could use things like
copy_to_user(), etc?  Out of curiosity, what's the status of that
project?


-Greg


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Avi Kivity]

Gregory Haskins wrote:
>>>   
>>>       
>> Yes, you can pass through any gva.  There are couple of things to be 
>> aware of though.  When passing a gva, you have to be sure that the gva 
>> is actually mapped in memory as KVM cannot cause Linux to fault in a 
>> page.
>>     
>
> Cool!  Ya, I totally understand and agree that it has to be "DMA" class
> memory that is mapped and pinned in guest context.  IIUC you basically
> need to follow the same rules as you would for a DMA based device in a
> bare-metal scenario.
>
>   

Note that passing a virtual address is highly discouraged as its meaning 
can change from vcpu to vcpu, it might not be mapped, translation is 
slow, etc.  Just let the guest do the translation.

> On this topic:  I know there has been talk going on of giving each VM
> its own linux va context.  IIUC, when that happens we wouldn't need the
> gva_to_hpa type functions, right?  We could use things like
> copy_to_user(), etc?  Out of curiosity, what's the status of that
> project?
>   

We have something running at qumranet, will be sent out soon.  I am 
somewhat discouraged in trying to get the thing to page -- Shaohua's 
approach is much simpler.



-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Gregory Haskins]

On Fri, 2007-08-10 at 19:59 +0300, Avi Kivity wrote:

> Note that passing a virtual address is highly discouraged as its meaning 
> can change from vcpu to vcpu, it might not be mapped, translation is 
> slow, etc.  Just let the guest do the translation.

Yeah, Hollis and Anthony straighted me out via IRC.  I will mostly be
dealing with kmalloc/skb buffers so we settled on the following as
optimal:

guest-side

gpa = __pa(ptr);

host-side:

gfn  = gpa >> PAGE_SHIFT
page = gfn_to_page(gfn);
ptr = kmap(page);

..

kunmap(ptr); 


> 
> We have something running at qumranet, will be sent out soon.  I am 
> somewhat discouraged in trying to get the thing to page -- Shaohua's 
> approach is much simpler.
> 

Cool!  Ill keep an eye out.

-Greg


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Anthony Liguori]

Gregory Haskins wrote:
> On Fri, 2007-08-10 at 19:59 +0300, Avi Kivity wrote:
>
>   
>> Note that passing a virtual address is highly discouraged as its meaning 
>> can change from vcpu to vcpu, it might not be mapped, translation is 
>> slow, etc.  Just let the guest do the translation.
>>     
>
> Yeah, Hollis and Anthony straighted me out via IRC.  I will mostly be
> dealing with kmalloc/skb buffers so we settled on the following as
> optimal:
>
> guest-side
>
> gpa = __pa(ptr);
>
> host-side:
>
> gfn  = gpa >> PAGE_SHIFT
> page = gfn_to_page(gfn);
> ptr = kmap(page);
>
> ..
>
> kunmap(ptr); 
>   

kmap can sleep so you often have to use kmap_atomic in KVM.  Although, 
I'm not sure if the pre-emption notification means that no longer is the 
case.

Regards,

Anthony Liguori

>   
>> We have something running at qumranet, will be sent out soon.  I am 
>> somewhat discouraged in trying to get the thing to page -- Shaohua's 
>> approach is much simpler.
>>
>>     
>
> Cool!  Ill keep an eye out.
>
> -Greg
>
>
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: help with understanding GPA/GVA issues

[Hollis Blanchard]

On Fri, 10 Aug 2007 00:09:18 -0400, Gregory Haskins wrote:
> 
> If so, how would I do this:  E.g. can I just pass the pointer, and then
> do gva_to_hpa() on the host?  Or do I need to prep the pointer before
> sending it?

We've talked about this a little in the
past: see the discussion around http://lkml.org/lkml/2007/1/7/60 (in
particular http://lkml.org/lkml/2007/1/7/126 ;) .

In short, please do *not* pass guest virtual addresses through a
paravirtual API.

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/