[Qemu-devel] About address mapping between host and guest in QEMU

[Qemu-devel] About address mapping between host and guest in QEMU

[Wenjie Liu]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

Hi all,
I have a little problem about the address mapping between host and guest in
QEMU.

I have got the guest physical address and the host virtual base address of
guest's memory space which by check the ramblocks list.

So, can I access to the guest's memory content by simply add the guest
physical address with the base?

For example, if the guest physical address of a guest memory request is
0x000000011f9ff000, and the host virtual base is 0x00007f37ef122000, I am
going to access the host virtual address 0x000000011f9ff000 +
0x00007f37ef122000.
Is this correct?


-- 

*Best regards*

[-- Attachment #2: Type: text/html, Size: 1142 bytes --]

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Wenjie Liu]

[-- Attachment #1: Type: text/plain, Size: 2083 bytes --]

In http://vm-kernel.org/blog/2009/07/10/qemu-internal-part-2-softmmu/, the
following equation is mentioned.
host_virtual_address = phys_ram_base(qemu variable) +
guest_physical_address – guest_physical_address_base(0 in MIPS)

I think I have got the phys_ram_base and guest_physical_address, but can
anybody tell me what is guest_physical_address_base?
I saw the memory allocation part in qemu/pc/pc.c:

    /* allocate RAM */
    ram_addr = qemu_ram_alloc(NULL, "pc.ram",
                              below_4g_mem_size + above_4g_mem_size);
    cpu_register_physical_memory(0, 0xa0000, ram_addr);
    cpu_register_physical_memory(0x100000,
                 below_4g_mem_size - 0x100000,
                 ram_addr + 0x100000);
#if TARGET_PHYS_ADDR_BITS > 32
    if (above_4g_mem_size > 0) {
        cpu_register_physical_memory(0x100000000ULL, above_4g_mem_size,
                                     ram_addr + below_4g_mem_size);
    }
#endif

I think the 0x100000, 0xa0000 and 0x100000000ULL have some relation with
the guest_physical_address_base.

Is that so?



2015-04-22 21:31 GMT+08:00 Wenjie Liu <lwj0012@gmail.com>:

> Hi all,
> I have a little problem about the address mapping between host and guest
> in QEMU.
>
> I have got the guest physical address and the host virtual base address of
> guest's memory space which by check the ramblocks list.
>
> So, can I access to the guest's memory content by simply add the guest
> physical address with the base?
>
> For example, if the guest physical address of a guest memory request is
> 0x000000011f9ff000, and the host virtual base is 0x00007f37ef122000, I am
> going to access the host virtual address 0x000000011f9ff000 +
> 0x00007f37ef122000.
> Is this correct?
>
>
> --
>
> *Best regards*
>



-- 

*Best regards **from*

Wenjie Liu

TEL: (+86)132 9723 0518

E-mail: *lwj0012@gmail.com <lwj0012@gmail.com>*

Wuhan National Laboratory for Optoelectronics,

Huazhong University of Science&Technology,

1037 Luoyu Road Wuhan, Hubei,P.R.C, 430074.

[-- Attachment #2: Type: text/html, Size: 4965 bytes --]

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Wenjie Liu]

[-- Attachment #1: Type: text/plain, Size: 2578 bytes --]

Can anybody take a few minutes to answer my question?

2015-04-22 23:26 GMT+08:00 Wenjie Liu <lwj0012@gmail.com>:

> In http://vm-kernel.org/blog/2009/07/10/qemu-internal-part-2-softmmu/,
> the following equation is mentioned.
> host_virtual_address = phys_ram_base(qemu variable) +
> guest_physical_address – guest_physical_address_base(0 in MIPS)
>
> I think I have got the phys_ram_base and guest_physical_address, but can
> anybody tell me what is guest_physical_address_base?
> I saw the memory allocation part in qemu/pc/pc.c:
>
>     /* allocate RAM */
>     ram_addr = qemu_ram_alloc(NULL, "pc.ram",
>                               below_4g_mem_size + above_4g_mem_size);
>     cpu_register_physical_memory(0, 0xa0000, ram_addr);
>     cpu_register_physical_memory(0x100000,
>                  below_4g_mem_size - 0x100000,
>                  ram_addr + 0x100000);
> #if TARGET_PHYS_ADDR_BITS > 32
>     if (above_4g_mem_size > 0) {
>         cpu_register_physical_memory(0x100000000ULL, above_4g_mem_size,
>                                      ram_addr + below_4g_mem_size);
>     }
> #endif
>
> I think the 0x100000, 0xa0000 and 0x100000000ULL have some relation with
> the guest_physical_address_base.
>
> Is that so?
>
>
>
> 2015-04-22 21:31 GMT+08:00 Wenjie Liu <lwj0012@gmail.com>:
>
>> Hi all,
>> I have a little problem about the address mapping between host and guest
>> in QEMU.
>>
>> I have got the guest physical address and the host virtual base address
>> of guest's memory space which by check the ramblocks list.
>>
>> So, can I access to the guest's memory content by simply add the guest
>> physical address with the base?
>>
>> For example, if the guest physical address of a guest memory request is
>> 0x000000011f9ff000, and the host virtual base is 0x00007f37ef122000, I am
>> going to access the host virtual address 0x000000011f9ff000 +
>> 0x00007f37ef122000.
>> Is this correct?
>>
>>
>> --
>>
>> *Best regards*
>>
>
>
>
> --
>
> *Best regards **from*
>
> Wenjie Liu
>
> TEL: (+86)132 9723 0518
>
> E-mail: *lwj0012@gmail.com <lwj0012@gmail.com>*
>
> Wuhan National Laboratory for Optoelectronics,
>
> Huazhong University of Science&Technology,
>
> 1037 Luoyu Road Wuhan, Hubei,P.R.C, 430074.
>



-- 

*Best regards **from*

Wenjie Liu

TEL: (+86)132 9723 0518

E-mail: *lwj0012@gmail.com <lwj0012@gmail.com>*

Wuhan National Laboratory for Optoelectronics,

Huazhong University of Science&Technology,

1037 Luoyu Road Wuhan, Hubei,P.R.C, 430074.

[-- Attachment #2: Type: text/html, Size: 7381 bytes --]

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Peter Maydell]

On 23 April 2015 at 14:56, Wenjie Liu <lwj0012@gmail.com> wrote:
> Can anybody take a few minutes to answer my question?

It might help if you explained at a higher level what you're
trying to achieve. Usually you don't need to manually try
to do this conversion from guest to host address because
there is an API which will do whatever memory operation
you wanted for you (including finding the host address,
dealing with the case where the guest physical address is
an MMIO device or has nothing mapped, and so on).

-- PMM

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Wenjie Liu]

[-- Attachment #1: Type: text/plain, Size: 1068 bytes --]

The thing I am trying to achieve is to get the data and guest physical
address of every guest memory access, so I need to known which API can be
used to do the address transform.

Thanks a lot.

2015-04-23 22:03 GMT+08:00 Peter Maydell <peter.maydell@linaro.org>:

> On 23 April 2015 at 14:56, Wenjie Liu <lwj0012@gmail.com> wrote:
> > Can anybody take a few minutes to answer my question?
>
> It might help if you explained at a higher level what you're
> trying to achieve. Usually you don't need to manually try
> to do this conversion from guest to host address because
> there is an API which will do whatever memory operation
> you wanted for you (including finding the host address,
> dealing with the case where the guest physical address is
> an MMIO device or has nothing mapped, and so on).
>
> -- PMM
>



-- 

*Best regards **from*

Wenjie Liu

TEL: (+86)132 9723 0518

E-mail: *lwj0012@gmail.com <lwj0012@gmail.com>*

Wuhan National Laboratory for Optoelectronics,

Huazhong University of Science&Technology,

1037 Luoyu Road Wuhan, Hubei,P.R.C, 430074.

[-- Attachment #2: Type: text/html, Size: 3165 bytes --]

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Paolo Bonzini]

On 24/04/2015 04:10, Wenjie Liu wrote:
> The thing I am trying to achieve is to get the data and guest physical
> address of every guest memory access, so I need to known which API can
> be used to do the address transform.

The short answer is that is difficult, because most guest memory
accesses do not call any C function.  QEMU has a virtual TLB; if you
have a TLB hit, the code generated by the JIT compiler does the conversion.

A C function is called only if you have a TLB miss, or if the guest is
accessing a device.  These C functions are defined in
softmmu_template.h.  The header is included multiple from cputlb.c.

Paolo

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Peter Maydell]

On 24 April 2015 at 12:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 24/04/2015 04:10, Wenjie Liu wrote:
>> The thing I am trying to achieve is to get the data and guest physical
>> address of every guest memory access, so I need to known which API can
>> be used to do the address transform.
>
> The short answer is that is difficult, because most guest memory
> accesses do not call any C function.  QEMU has a virtual TLB; if you
> have a TLB hit, the code generated by the JIT compiler does the conversion.

In fact on a TLB hit it's possible that it's not actually any
longer determinable what the guest physical address was. This
only really happens with buggy guests, but if the guest does
a load that brings an entry into the TLB and then rewrites the
page table but fails to do the TLB maintenance operation, then
at the point when a subsequent load hits in the TLB we know
the guest virtual address and the host virtual address but not
the guest physical address, and we can't find the guest physaddr
any more even if we walk the page tables, because those have
been changed...

Wanting to do this kind of memory access tracing is a pretty
common user request, though, and it would be nice if QEMU
had the facilities for it. It's just that our current design
is really not set up to make it easy to implement.

-- PMM

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Christopher Covington]

On 04/24/2015 08:46 AM, Peter Maydell wrote:
> On 24 April 2015 at 12:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> On 24/04/2015 04:10, Wenjie Liu wrote:
>>> The thing I am trying to achieve is to get the data and guest physical
>>> address of every guest memory access, so I need to known which API can
>>> be used to do the address transform.
>>
>> The short answer is that is difficult, because most guest memory
>> accesses do not call any C function.  QEMU has a virtual TLB; if you
>> have a TLB hit, the code generated by the JIT compiler does the conversion.
> 
> In fact on a TLB hit it's possible that it's not actually any
> longer determinable what the guest physical address was. This
> only really happens with buggy guests, but if the guest does
> a load that brings an entry into the TLB and then rewrites the
> page table but fails to do the TLB maintenance operation, then
> at the point when a subsequent load hits in the TLB we know
> the guest virtual address and the host virtual address but not
> the guest physical address, and we can't find the guest physaddr
> any more even if we walk the page tables, because those have
> been changed...
> 
> Wanting to do this kind of memory access tracing is a pretty
> common user request, though, and it would be nice if QEMU
> had the facilities for it. It's just that our current design
> is really not set up to make it easy to implement.

While probably very slow, could the kind of guest memory trace in question be
captured using the GDB stub and a breakpoint on every load an store
instruction (or alternatively, a watchpoint on every possible address)?

Chris

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Brendan Dolan-Gavitt]

One possible option that has come up in the past is disabling the TLB
for your testing. It is slower, though, and would only work on TCG,
not KVM.

The way I'm currently doing this is by changing the conditional jump
here to an absolute one inside of the TCG code generator for i386 (and
of course you can do something similar on other archs). The code here
has changed since I last looked at it, but this is roughly the place
to look:

https://github.com/qemu/qemu/blob/master/tcg/i386/tcg-target.c#L1222

Depending on your needs, you might want to also look at (shameless
plug) PANDA [1], which provides callbacks and a plugin architecture
for many things that happen in QEMU, including memory accesses. You
can see how we implement the memory callback here:

https://github.com/moyix/panda/blob/master/qemu/tcg/i386/tcg-target.c#L1079

Cheers,
Brendan

[1] https://github.com/moyix/panda

On Fri, Apr 24, 2015 at 3:09 PM, Christopher Covington
<cov@codeaurora.org> wrote:
> On 04/24/2015 08:46 AM, Peter Maydell wrote:
>> On 24 April 2015 at 12:26, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>> On 24/04/2015 04:10, Wenjie Liu wrote:
>>>> The thing I am trying to achieve is to get the data and guest physical
>>>> address of every guest memory access, so I need to known which API can
>>>> be used to do the address transform.
>>>
>>> The short answer is that is difficult, because most guest memory
>>> accesses do not call any C function.  QEMU has a virtual TLB; if you
>>> have a TLB hit, the code generated by the JIT compiler does the conversion.
>>
>> In fact on a TLB hit it's possible that it's not actually any
>> longer determinable what the guest physical address was. This
>> only really happens with buggy guests, but if the guest does
>> a load that brings an entry into the TLB and then rewrites the
>> page table but fails to do the TLB maintenance operation, then
>> at the point when a subsequent load hits in the TLB we know
>> the guest virtual address and the host virtual address but not
>> the guest physical address, and we can't find the guest physaddr
>> any more even if we walk the page tables, because those have
>> been changed...
>>
>> Wanting to do this kind of memory access tracing is a pretty
>> common user request, though, and it would be nice if QEMU
>> had the facilities for it. It's just that our current design
>> is really not set up to make it easy to implement.
>
> While probably very slow, could the kind of guest memory trace in question be
> captured using the GDB stub and a breakpoint on every load an store
> instruction (or alternatively, a watchpoint on every possible address)?
>
> Chris
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
>

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Christopher Covington]

Hi Brendan,

On 04/24/2015 03:50 PM, Brendan Dolan-Gavitt wrote:
> One possible option that has come up in the past is disabling the TLB
> for your testing. It is slower, though, and would only work on TCG,
> not KVM.
> 
> The way I'm currently doing this is by changing the conditional jump
> here to an absolute one inside of the TCG code generator for i386 (and
> of course you can do something similar on other archs). The code here
> has changed since I last looked at it, but this is roughly the place
> to look:
> 
> https://github.com/qemu/qemu/blob/master/tcg/i386/tcg-target.c#L1222
> 
> Depending on your needs, you might want to also look at (shameless
> plug) PANDA [1], which provides callbacks and a plugin architecture
> for many things that happen in QEMU, including memory accesses. You
> can see how we implement the memory callback here:
> 
> https://github.com/moyix/panda/blob/master/qemu/tcg/i386/tcg-target.c#L1079

This is neat. How do you see the relationship with upstream? Will you ever
upgrade QEMU versions?

Have you seen Pavel Dovgalyuk's upstream work on deterministic replay to
enable reverse execution? Does it differ fundamentally from PANDA record and
replay?

It looks like you're running K-means on your QEMU traces. I happen to be doing
that too. Do you happen to have any code examples of that included in PANDA or
other open source projects?

Thanks,
Chris

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Wenjie Liu]

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

Hi Peter,
Does the equation which is mentioned in
http://vm-kernel.org/blog/2009/07/10/qemu-internal-part-2-softmmu/ means
something?
host_virtual_address = phys_ram_base(qemu variable) +
guest_physical_address – guest_physical_address_base(0 in MIPS)
Currently I have the guest physical address and phys_ram_base, can I use
the equation  to get the host virtual address?

In fact I am using Marss which is based on QEMU, and in Marss the guest
physical address is provided.


2015-04-25 7:48 GMT+08:00 Christopher Covington <cov@codeaurora.org>:

> Hi Brendan,
>
> On 04/24/2015 03:50 PM, Brendan Dolan-Gavitt wrote:
> > One possible option that has come up in the past is disabling the TLB
> > for your testing. It is slower, though, and would only work on TCG,
> > not KVM.
> >
> > The way I'm currently doing this is by changing the conditional jump
> > here to an absolute one inside of the TCG code generator for i386 (and
> > of course you can do something similar on other archs). The code here
> > has changed since I last looked at it, but this is roughly the place
> > to look:
> >
> > https://github.com/qemu/qemu/blob/master/tcg/i386/tcg-target.c#L1222
> >
> > Depending on your needs, you might want to also look at (shameless
> > plug) PANDA [1], which provides callbacks and a plugin architecture
> > for many things that happen in QEMU, including memory accesses. You
> > can see how we implement the memory callback here:
> >
> >
> https://github.com/moyix/panda/blob/master/qemu/tcg/i386/tcg-target.c#L1079
>
> This is neat. How do you see the relationship with upstream? Will you ever
> upgrade QEMU versions?
>
> Have you seen Pavel Dovgalyuk's upstream work on deterministic replay to
> enable reverse execution? Does it differ fundamentally from PANDA record
> and
> replay?
>
> It looks like you're running K-means on your QEMU traces. I happen to be
> doing
> that too. Do you happen to have any code examples of that included in
> PANDA or
> other open source projects?
>
> Thanks,
> Chris
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
>



-- 

*Best regards **from*

Wenjie Liu

TEL: (+86)132 9723 0518

E-mail: *lwj0012@gmail.com <lwj0012@gmail.com>*

Wuhan National Laboratory for Optoelectronics,

Huazhong University of Science&Technology,

1037 Luoyu Road Wuhan, Hubei,P.R.C, 430074.

[-- Attachment #2: Type: text/html, Size: 5162 bytes --]

Re: [Qemu-devel] About address mapping between host and guest in QEMU

[Peter Maydell]

On 25 April 2015 at 02:51, Wenjie Liu <lwj0012@gmail.com> wrote:
> Hi Peter,
> Does the equation which is mentioned in
> http://vm-kernel.org/blog/2009/07/10/qemu-internal-part-2-softmmu/ means
> something?
> host_virtual_address = phys_ram_base(qemu variable) + guest_physical_address
> – guest_physical_address_base(0 in MIPS)
> Currently I have the guest physical address and phys_ram_base, can I use the
> equation  to get the host virtual address?

That blog post is now six years old, and the codebase has changed
a lot since then. You're better off looking at the actual code.

> In fact I am using Marss which is based on QEMU, and in Marss the guest
> physical address is provided.

In that case you should probably ask the Marss developers...

-- PMM