[Qemu-devel] tlb_update_dirty() question

[Qemu-devel] tlb_update_dirty() question

[Johannes Luber]

Hi,

Before I state my question I describe my assumptions how Qemu works internally. If I'm wrong there you'll notice it sooner.

The pointers of the emulation layer are transformed into physical addresses is a two-steps process. The emulated machine itself uses virtual addresses which are represented by the type ram_addr_t. These virtual addresses are different from the one the host OS, in fact they are a completely internal representation.

To actually work with ram_addr_t pointers, these have to be transformed into host virtual addresses. These are represented by target_phys_addr_t pointers. To access with the host virtual memory the physical memory, the host OS does its own magic which is no functionality of Qemu itself.

Taking all assumptions into account it is certainly possibly that ram_addr_t can be smaller than target_phys_addr_t. E.g., a 32-bit target system can access only 4 GB of memory while its 64-bit host put that memory anywhere in the whole address range.

But then I stumbled over these snippets:

static inline void tlb_update_dirty(CPUTLBEntry *tlb_entry)
{
    ram_addr_t ram_addr;
    void *p;

    if ((tlb_entry->addr_write & ~TARGET_PAGE_MASK) == IO_MEM_RAM) {
        p = (void *)(unsigned long)((tlb_entry->addr_write &
             TARGET_PAGE_MASK) + tlb_entry->addend);
        ram_addr = qemu_ram_addr_from_host(p);
        if (!cpu_physical_memory_is_dirty(ram_addr)) {
            tlb_entry->addr_write |= TLB_NOTDIRTY;
        }
    }
}

/* Some of the softmmu routines need to translate from a host pointer
   (typically a TLB entry) back to a ram offset.  */
ram_addr_t qemu_ram_addr_from_host(void *ptr)
{
...
}

The comment is particularly insightful. p is supposed to be a host pointer yet the initialization code uses "(unsigned long)" in a cast for an expression which has the type target_phys_addr_t because the struct variable "addend" has this type.

This cast assumes that unsigned long is at least as big as target_phys_addr_t. Under Unix this may be true, but Windows C compilers treat long == int and int remains a 32-bit type. Why isn't simply target_phys_addr_t used as cast? target_phys_addr_t does support max(target pointer size, host pointer size), doesn't it? Or is there another option?

Best regards,
Johannes
-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

Re: [Qemu-devel] tlb_update_dirty() question

[Blue Swirl]

On Mon, Sep 14, 2009 at 12:00 PM, Johannes Luber <JALuber@gmx.de> wrote:
> Hi,
>
> Before I state my question I describe my assumptions how Qemu works internally. If I'm wrong there you'll notice it sooner.
>
> The pointers of the emulation layer are transformed into physical addresses is a two-steps process. The emulated machine itself uses virtual addresses which are represented by the type ram_addr_t. These virtual addresses are different from the one the host OS, in fact they are a completely internal representation.
>
> To actually work with ram_addr_t pointers, these have to be transformed into host virtual addresses. These are represented by target_phys_addr_t pointers. To access with the host virtual memory the physical memory, the host OS does its own magic which is no functionality of Qemu itself.
>
> Taking all assumptions into account it is certainly possibly that ram_addr_t can be smaller than target_phys_addr_t. E.g., a 32-bit target system can access only 4 GB of memory while its 64-bit host put that memory anywhere in the whole address range.
>
> But then I stumbled over these snippets:
>
> static inline void tlb_update_dirty(CPUTLBEntry *tlb_entry)
> {
>    ram_addr_t ram_addr;
>    void *p;
>
>    if ((tlb_entry->addr_write & ~TARGET_PAGE_MASK) == IO_MEM_RAM) {
>        p = (void *)(unsigned long)((tlb_entry->addr_write &
>             TARGET_PAGE_MASK) + tlb_entry->addend);
>        ram_addr = qemu_ram_addr_from_host(p);
>        if (!cpu_physical_memory_is_dirty(ram_addr)) {
>            tlb_entry->addr_write |= TLB_NOTDIRTY;
>        }
>    }
> }
>
> /* Some of the softmmu routines need to translate from a host pointer
>   (typically a TLB entry) back to a ram offset.  */
> ram_addr_t qemu_ram_addr_from_host(void *ptr)
> {
> ...
> }
>
> The comment is particularly insightful. p is supposed to be a host pointer yet the initialization code uses "(unsigned long)" in a cast for an expression which has the type target_phys_addr_t because the struct variable "addend" has this type.

The addend is target_phys_addr_t type, because then we can get back to
host address ranges on 32 bit host. Consider for example guest address
at 8G backed by host memory at 1G: the addend is -7G.

> This cast assumes that unsigned long is at least as big as target_phys_addr_t. Under Unix this may be true, but Windows C compilers treat long == int and int remains a 32-bit type. Why isn't simply target_phys_addr_t used as cast? target_phys_addr_t does support max(target pointer size, host pointer size), doesn't it? Or is there another option?

No, the cast assumes that sum of guest addr and addend is a valid host
address, which should be true. For memory, the resulting address is
simply pointer to host memory. If any of the lowest bits of the sum
are set, the area is MMIO.

Re: [Qemu-devel] tlb_update_dirty() question

[Johannes Luber]

> On Mon, Sep 14, 2009 at 12:00 PM, Johannes Luber <JALuber@gmx.de> wrote:
...
> >
> > The comment is particularly insightful. p is supposed to be a host
> pointer yet the initialization code uses "(unsigned long)" in a cast for an
> expression which has the type target_phys_addr_t because the struct variable
> "addend" has this type.
> 
> The addend is target_phys_addr_t type, because then we can get back to
> host address ranges on 32 bit host. Consider for example guest address
> at 8G backed by host memory at 1G: the addend is -7G.

Looking at

int tlb_set_page_exec(CPUState *env, target_ulong vaddr,
                      target_phys_addr_t paddr, int prot,
                      int mmu_idx, int is_softmmu)
{
}

(I assume that the only place addend is set), I see these two lines:

    addend = (unsigned long)qemu_get_ram_ptr(pd & TARGET_PAGE_MASK);
    ...
    te->addend = addend - vaddr;

Assuming target_ulong and unsigned long as 32-bit values (despite being on 64-bit system) I don't see how your example can work. There is no way to make addend bigger than (+/-)4G.

> 
> > This cast assumes that unsigned long is at least as big as
> target_phys_addr_t. Under Unix this may be true, but Windows C compilers treat long ==
> int and int remains a 32-bit type. Why isn't simply target_phys_addr_t used
> as cast? target_phys_addr_t does support max(target pointer size, host
> pointer size), doesn't it? Or is there another option?
> 
> No, the cast assumes that sum of guest addr and addend is a valid host
> address, which should be true. For memory, the resulting address is
> simply pointer to host memory. If any of the lowest bits of the sum
> are set, the area is MMIO.
> 
All in all, I take it that Qemu basically targets only Unix (the link to the Windows source version is merely a patch set). At least I know that my assumptions have been right and so I can fix these "(unsigned long)" places for myself.

Thanks for your time!
Johannes
-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01