Re: [Qemu-devel] BUG: Re: [PATCH v3 11/14] ioport: Switch dispatching to memory core layer

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 07/20/2013 10:55 AM, Alexey Kardashevskiy wrote:
> On 07/20/2013 01:48 AM, Alexey Kardashevskiy wrote:
>> Ok. So.
>>
>> What broke is...
>> I could try explaining but backtraces are lot better :)
>>
>> Shortly - virtio_pci_config_ops.endianness was ignored before (was bad but
>> we had a workaround in spapr_io_ops), now it works so double swap happens
>> and everything gets broken.
>>
>> If we talk about VGA (in powerpc, it is all about powerpc), I guess
>> memory_region_iorange_write() will go through mr->ops->old_portio branch
>> and won't do any byte swapping (so spapr_io_ops will do the job), so we are
>> fine here. I do not understand yet why it works on mac99 though, too late
>> here :)
> 
> 
> I understood. VGA does not work for mac99 either with this command line:
> ./qemu-system-ppc64 -m "1024" -M "mac99" -vga "std"
> So it works for pseries only because of parity bug in spapr_io_ops.


oops. I am wrong and VGA works on mac99 in upstream because isa_mmio_ops
does the swapping in this case and portio_ops does not swap (in upstream).

Oh. Ah. Uh.

Adding cc:Benh...


> So the right fix is to get rid of spapr_io_ops and every other hack like
> that and to add byte swapping to every "if (mr->ops->old_portio)" branch
> (should fix VGA and all other old_portio users). Current byte swapping in
> memory regions seems to be right.
> 
> I would try fixing it but since all my patches were terrible shit so far, I
> won't risk :)
> 
> 
> 
>> h_logical_store is a hypercall for system firmware to do cache inhibited
>> read/write.
>>
>>
>> This is with the patch applied (git checkout  b40acf9):
>>
>>
>> #0  virtqueue_init (vq=0x11014ac0) at
>> /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:90
>> #1  0x0000000010371f28 in virtio_queue_set_addr (vdev=0x11019dd0, n=0x0,
>> addr=0xd0fb0000000)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:662
>> #2  0x00000000102027f0 in virtio_ioport_write (opaque=0x11019580, addr=0x8,
>> val=0xd0fb0000)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:278
>> #3  0x0000000010202f08 in virtio_pci_config_write (opaque=0x11019580,
>> addr=0x8, val=0xd0fb0000, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:416
>> #4  0x000000001037e220 in memory_region_write_accessor (opaque=0x11019c78,
>> addr=0x8, value=0x1fffff0edc00,
>>     size=0x4, shift=0x0, mask=0xffffffff) at
>> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
>> #5  0x000000001037e36c in access_with_adjusted_size (addr=0x8,
>> value=0x1fffff0edc00, size=0x4,
>>     access_size_min=0x1, access_size_max=0x4, access=
>>     @0x1069df40: 0x1037e164 <memory_region_write_accessor>, opaque=0x11019c78)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
>> #6  0x0000000010380b5c in memory_region_dispatch_write (mr=0x11019c78,
>> addr=0x8, data=0xd0fb0000, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:905
>> #7  0x0000000010383fa4 in io_mem_write (mr=0x11019c78, addr=0x8,
>> val=0xfbd0, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1608
>> #8  0x00000000102e2fdc in address_space_rw (as=0x10ef4350
>> <address_space_io>, addr=0x48,
>>     buf=0x1fffff0edde0 "", len=0x4, is_write=0x1) at
>> /home/alexey/pcipassthru/qemu-impreza/exec.c:1918
>> #9  0x00000000102e33c8 in address_space_write (as=0x10ef4350
>> <address_space_io>, addr=0x48,
>>     buf=0x1fffff0edde0 "", len=0x4) at
>> /home/alexey/pcipassthru/qemu-impreza/exec.c:1969
>> #10 0x0000000010375754 in cpu_outl (addr=0x48, val=0xfbd0)
>>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:309
>> #11 0x0000000010358240 in spapr_io_write (opaque=0x11016a00, addr=0x48,
>> data=0xfbd0, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_pci.c:468
>> #12 0x000000001037e220 in memory_region_write_accessor (opaque=0x110191f8,
>> addr=0x48, value=0x1fffff0ee060,
>>     size=0x4, shift=0x0, mask=0xffffffff) at
>> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
>> #13 0x000000001037e36c in access_with_adjusted_size (addr=0x48,
>> value=0x1fffff0ee060, size=0x4,
>>     access_size_min=0x1, access_size_max=0x4, access=
>>     @0x1069df40: 0x1037e164 <memory_region_write_accessor>, opaque=0x110191f8)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
>> #14 0x0000000010380b5c in memory_region_dispatch_write (mr=0x110191f8,
>> addr=0x48, data=0xfbd0, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:905
>> #15 0x0000000010383fa4 in io_mem_write (mr=0x110191f8, addr=0x48,
>> val=0xd0fb0000, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1608
>> #16 0x00000000102e47ac in stl_phys_internal (addr=0x10080000048,
>> val=0xd0fb0000, endian=
>>     DEVICE_NATIVE_ENDIAN) at /home/alexey/pcipassthru/qemu-impreza/exec.c:2420
>> #17 0x00000000102e48a8 in stl_phys (addr=0x10080000048, val=0xd0fb0000)
>>     at /home/alexey/pcipassthru/qemu-impreza/exec.c:2442
>> #18 0x0000000010354f1c in h_logical_store (cpu=0x1fffff0f0010,
>> spapr=0x10fe9510, opcode=0x40,
>>     args=0x1ffffffd0030) at
>> /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:570
>>
>>
>>
>> This is without this patch (i.e. git checkout  b40acf9^ ):
>>
>> #0  virtqueue_init (vq=0x11014ac0) at
>> /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:90
>> #1  0x00000000103720e4 in virtio_queue_set_addr (vdev=0x11019dd0, n=0x0,
>> addr=0xffe2000)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:662
>> #2  0x00000000102027f0 in virtio_ioport_write (opaque=0x11019580, addr=0x8,
>> val=0xffe2)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:278
>> #3  0x0000000010202f08 in virtio_pci_config_write (opaque=0x11019580,
>> addr=0x8, val=0xffe2, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:416
>> #4  0x000000001037dca8 in memory_region_write_accessor (opaque=0x11019c78,
>> addr=0x8, value=0x1fffff0edca8,
>>     size=0x4, shift=0x0, mask=0xffffffff) at
>> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
>> #5  0x000000001037ddf4 in access_with_adjusted_size (addr=0x8,
>> value=0x1fffff0edca8, size=0x4,
>>     access_size_min=0x1, access_size_max=0x4, access=
>>     @0x1069def8: 0x1037dbec <memory_region_write_accessor>, opaque=0x11019c78)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
>> #6  0x000000001037e474 in memory_region_iorange_write
>> (iorange=0x1ffff0005430, offset=0x8, width=0x4,
>>     data=0xffe2) at /home/alexey/pcipassthru/qemu-impreza/memory.c:475
>> #7  0x00000000103750d4 in ioport_writel_thunk (opaque=0x1ffff0005430,
>> addr=0x48, data=0xffe2)
>>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:226
>> #8  0x0000000010374728 in ioport_write (index=0x2, address=0x48, data=0xffe2)
>>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:83
>> #9  0x0000000010375688 in cpu_outl (addr=0x48, val=0xffe2)
>>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:296
>> #10 0x00000000103583fc in spapr_io_write (opaque=0x11016a00, addr=0x48,
>> data=0xffe2, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_pci.c:468
>> #11 0x000000001037dca8 in memory_region_write_accessor (opaque=0x110191f8,
>> addr=0x48, value=0x1fffff0ee060,
>>     size=0x4, shift=0x0, mask=0xffffffff) at
>> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
>> #12 0x000000001037ddf4 in access_with_adjusted_size (addr=0x48,
>> value=0x1fffff0ee060, size=0x4,
>>     access_size_min=0x1, access_size_max=0x4, access=
>>     @0x1069def8: 0x1037dbec <memory_region_write_accessor>, opaque=0x110191f8)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
>> #13 0x0000000010380c90 in memory_region_dispatch_write (mr=0x110191f8,
>> addr=0x48, data=0xffe2, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:993
>> #14 0x00000000103840d8 in io_mem_write (mr=0x110191f8, addr=0x48,
>> val=0xe2ff0000, size=0x4)
>>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1696
>> #15 0x00000000102e4968 in stl_phys_internal (addr=0x10080000048,
>> val=0xe2ff0000, endian=
>>     DEVICE_NATIVE_ENDIAN) at /home/alexey/pcipassthru/qemu-impreza/exec.c:2447
>> #16 0x00000000102e4a64 in stl_phys (addr=0x10080000048, val=0xe2ff0000)
>>     at /home/alexey/pcipassthru/qemu-impreza/exec.c:2469
>> #17 0x00000000103550d8 in h_logical_store (cpu=0x1fffff0f0010,
>> spapr=0x10fe9510, opcode=0x40,
>>     args=0x1ffffffd0030) at
>> /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:570
>> #18 0x0000000010355698 in spapr_hypercall (cpu=0x1fffff0f0010, opcode=0x40,
>> args=0x1ffffffd0030)
>>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:689
>>
>>
>>
>>
>>
>>
>> On 07/19/2013 10:49 PM, Paolo Bonzini wrote:
>>> Il 19/07/2013 13:09, Alexey Kardashevskiy ha scritto:
>>>> Hi!
>>>>
>>>> This patch also breaks virtio on powerpc. I thought it was fixed
>>>> (reverted?) in the master branch from qemu.org but it is still there. What
>>>> did I miss?
>>>
>>> It was not reverted, only the "DEVICE_LITTLE_ENDIAN" marking was.
>>>
>>> Let me check if I can reproduce this, it looks like a endianness
>>> problems reading virtio-blk config space.
>>>
>>> Paolo
>>>
>>>> Trying to load:  from: disk ... virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> virtioblk_read: Access beyond end of device!
>>>> [many of those]
>>>>
>>>>
>>>>
>>>> On 07/11/2013 10:29 PM, Alexander Graf wrote:
>>>>>
>>>>> On 24.06.2013, at 08:07, Jan Kiszka wrote:
>>>>>
>>>>>> On 2013-06-23 22:50, Hervé Poussineau wrote:
>>>>>>> Jan Kiszka a écrit :
>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>
>>>>>>>> The current ioport dispatcher is a complex beast, mostly due to the
>>>>>>>> need to deal with old portio interface users. But we can overcome it
>>>>>>>> without converting all portio users by embedding the required base
>>>>>>>> address of a MemoryRegionPortio access into that data structure. That
>>>>>>>> removes the need to have the additional MemoryRegionIORange structure
>>>>>>>> in the loop on every access.
>>>>>>>>
>>>>>>>> To handle old portio memory ops, we simply install dispatching handlers
>>>>>>>> for portio memory regions when registering them with the memory core.
>>>>>>>> This removes the need for the old_portio field.
>>>>>>>>
>>>>>>>> We can drop the additional aliasing of ioport regions and also the
>>>>>>>> special address space listener. cpu_in and cpu_out now simply call
>>>>>>>> address_space_read/write. And we can concentrate portio handling in a
>>>>>>>> single source file.
>>>>>>>>
>>>>>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>> ---
>>>>>>>
>>>>>>> ...
>>>>>>>
>>>>>>>> +
>>>>>>>> +static void portio_write(void *opaque, hwaddr addr, uint64_t data,
>>>>>>>> +                         unsigned size)
>>>>>>>> +{
>>>>>>>> +    MemoryRegionPortioList *mrpio = opaque;
>>>>>>>> +    const MemoryRegionPortio *mrp = find_portio(mrpio, addr, size,
>>>>>>>> true);
>>>>>>>> +
>>>>>>>> +    if (mrp) {
>>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr, data);
>>>>>>>> +    } else if (size == 2) {
>>>>>>>> +        mrp = find_portio(mrpio, addr, 1, true);
>>>>>>>> +        assert(mrp);
>>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr, data & 0xff);
>>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr + 1, data
>>>>>>>>>> 8);
>>>>>>>> +    }
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static const MemoryRegionOps portio_ops = {
>>>>>>>> +    .read = portio_read,
>>>>>>>> +    .write = portio_write,
>>>>>>>> +    .valid.unaligned = true,
>>>>>>>> +    .impl.unaligned = true,
>>>>>>>> +};
>>>>>>>> +
>>>>>>>
>>>>>>> You need to mark these operations as DEVICE_LITTLE_ENDIAN.
>>>>>>> In portio_write above, you clearly assume that data is in LE format.
>>>>>>
>>>>>> Anything behind PIO is little endian, of course. Will add this.
>>>>>
>>>>> This patch breaks VGA on PPC as it is in master today.
>>>>>
>>>>>
>>>>> Alex
>>>>>
>>>>>>
>>>>>>>
>>>>>>> This fixes PPC PReP emulation, which would otherwise be broken with this
>>>>>>> patchset.
>>>>>>
>>>>>> Thanks,
>>>>>> Jan
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
> 
> 


-- 
Alexey