[Qemu-devel] PCI 64-bit BAR access with qemu

[Qemu-devel] PCI 64-bit BAR access with qemu

[Francois WELLENREITER]

Hi there,

     I've read a few days ago that it was possible to emulate PCI device 
with 64-bit BARs and have a real 64-bit memory access.
Thus, I've created a virtual device named toto accessible through a 
64-bit BAR

___

static const MemoryRegionOps bxi_common_mmio_ops = {
     .read = toto_mmio_read,
     .write = toto_mmio_write,
     .endianness = DEVICE_LITTLE_ENDIAN,
     .impl = {
             .min_access_size = 1,
             .max_access_size = 8,
     },
};

___

     memory_region_init_io(&d->mmio, &toto_mmio_ops, d, "toto-mmio",
         0x1000);

     pci_register_bar(&d->dev, BAR_0,
              PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64,
&d->mmio);

____

when I use in my driver pointers to "unsigned char", "unsigned short" or 
"unsigned int",
I can see that toto_mmio_read is called with increasing sizes.
But that does not work at all with unsigned long long.
In such a case, I always obtain two calls to toto_mmio_read with sizes 
limited to 4.
My understanding here is that a direct 64-bit memory access does not 
work (but is composed of 2 32-bit accesses).

Then, I've tried to understand why this happens and realized that 
cpu_physical_memory_rw was always called for this memory region (for in 
the old_mmio manner) :

___

(gdb) bt
#0  toto_mmio_read (opaque=0x2bec2c0, addr=0, size=4) at 
/home/workspace/qemu/hw/toto_hw.c:92
#1  0x00000000005f6d37 in memory_region_read_accessor (opaque=0x2bec738, 
addr=0, value=0x7f9997ffec78, size=
     4, shift=0, mask=4294967295) at /home/workspace/qemu/memory.c:239
#2  0x00000000005f6ee0 in access_with_adjusted_size (addr=0, 
value=0x7f9997ffec78, size=4, access_size_min=1,
     access_size_max=8, access=0x5f6cdf <memory_region_read_accessor>, 
opaque=0x2bec738)
     at /home/workspace/qemu/memory.c:284
#3  0x00000000005f8909 in memory_region_read_thunk_n (_mr=0x2bec738, 
addr=0, size=4)
     at /home/workspace/qemu/memory.c:824
#4  0x00000000005f8af1 in memory_region_read_thunk_l (mr=0x2bec738, addr=0)
     at /home/workspace/qemu/memory.c:867
#5  0x00000000005c8f69 in cpu_physical_memory_rw (addr=3892314112, 
buf=0x7f999c5d0028 "\320\b", len=8,
     is_write=0) at /home/workspace/qemu/exec.c:3965
#6  0x00000000005ecfa1 in kvm_cpu_exec (env=0x238ee10) at 
/home/workspace/qemu/kvm-all.c:985
#7  0x00000000005bf1fb in qemu_kvm_cpu_thread_fn (arg=0x238ee10) at 
/home/workspace/qemu/cpus.c:661
#8  0x00000034dfc077e1 in start_thread () from /lib64/libpthread.so.0
#9  0x00000034df4e68ed in clone () from /lib64/libc.so.6

____

Did I miss anything ? Is it a real defect ?
Are there developments still planned to allow a direct 64-bit access ?

Thanks for any help,

                              François

Re: [Qemu-devel] PCI 64-bit BAR access with qemu

[Max Filippov]

>    I've read a few days ago that it was possible to emulate PCI device with
> 64-bit BARs and have a real 64-bit memory access.
> Thus, I've created a virtual device named toto accessible through a 64-bit
> BAR

You've probably confused an ability to locate BAR anywhere in 64-bit
address space (such BAR actually spans 2 consecutive PCI BAR registers
and has 100 in its 3 least significant bits) and an ability to access
BAR-mapped memory in 64 bit items.

You obviously want the latter but currently it is not implemented, see e.g.

static inline DATA_TYPE glue(io_read, SUFFIX)(target_phys_addr_t physaddr,
                                              target_ulong addr,
                                              void *retaddr)

definition in the softmmu_template.h.

And it's quite simple to fix it, you only need to change
io_{read,write} in the softmmu_template.h and extend
io_mem_{read,write} loops in exec.c to 4 elements, taking care that
io_mem_{read,write}[3] can pass uint64_t.

-- 
Thanks.
-- Max

Re: [Qemu-devel] PCI 64-bit BAR access with qemu

[Francois WELLENREITER]

Hi Max,

thank you for your answer. Actually, I hadn't confused.
I already thought to your proposal but I found that it was a really ugly 
solution (essentially because of the uint32_t to uint64_t silent 
conversion).
Isn't there any other (current or future) development that may fix it ?

             François

Le 12/10/2011 13:00, Max Filippov a écrit :
>>     I've read a few days ago that it was possible to emulate PCI device with
>> 64-bit BARs and have a real 64-bit memory access.
>> Thus, I've created a virtual device named toto accessible through a 64-bit
>> BAR
> You've probably confused an ability to locate BAR anywhere in 64-bit
> address space (such BAR actually spans 2 consecutive PCI BAR registers
> and has 100 in its 3 least significant bits) and an ability to access
> BAR-mapped memory in 64 bit items.
>
> You obviously want the latter but currently it is not implemented, see e.g.
>
> static inline DATA_TYPE glue(io_read, SUFFIX)(target_phys_addr_t physaddr,
>                                                target_ulong addr,
>                                                void *retaddr)
>
> definition in the softmmu_template.h.
>
> And it's quite simple to fix it, you only need to change
> io_{read,write} in the softmmu_template.h and extend
> io_mem_{read,write} loops in exec.c to 4 elements, taking care that
> io_mem_{read,write}[3] can pass uint64_t.
>