Re: [Qemu-devel] [PATCH] spapr-pci: remove io ports workaround

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 12/17/2013 06:52 PM, Greg Kurz wrote:
> On Wed, 11 Dec 2013 18:07:58 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>
>> Hm. Nack. This fails:
>>
>> ./qemu-system-ppc64 \
>>  -trace "events=qemu_trace_events" \
>>  -L "qemu-ppc64-bios/" \
>>  -nographic \
>>  -vga "none" \
>>  -device \
>>  virtio-blk-pci,id=virtioiblk0,drive=drive0,bootindex=20,ioeventfd=on \
>>  -drive \
>> file=virtimg/fc19_16GB.qcow2,if=none,id=drive0,readonly=off,format=qcow2,media=disk,werror=stop,rerror=stop,discard=on
>> \
>>  -S \
>>  -m "2048" \
>>  -machine "pseries" \
>>  -enable-kvm
>>
>>
>>
>>
>> command line: BOOT_IMAGE=/vmlinux-3.10.0-rc7-aik-guest+
>> root=UUID=27cde746-128e-4528-b4de-44a00d807ea0 ro rd.md=0 rd.lvm=0 rd.dm=0
>> vconsole.font=latarcyrheb-sun16 vconsole.keymap=us rd.luks=0 console=hvc0
>> debug memory layout at init:
>>   memory_limit : 0000000000000000 (16 MB aligned)
>>   alloc_bottom : 0000000003400000
>>   alloc_top    : 0000000030000000
>>   alloc_top_hi : 0000000080000000
>>   rmo_top      : 0000000030000000
>>   ram_top      : 0000000080000000
>> instantiating rtas at 0x000000002fff0000... done
>> boot cpu hw idx 0
>> copying OF device tree...
>> Building dt strings...
>> Building dt structure...
>> Device tree strings 0x0000000003410000 -> 0x0000000003410817
>> Device tree struct  0x0000000003420000 -> 0x0000000003430000
>> [Switching to Thread 0x3fff8ca4eee0 (LWP 10370)]
>>
>> Breakpoint 8, unassigned_mem_accepts (opaque=0x0, addr=0x10080000052,
>> size=0x1, is_write=0x1)
>>     at /home/alexey/p/qemu/memory.c:892
>> 892	    return false;
>> Missing separate debuginfos, use: debuginfo-install
>> glusterfs-api-3.4.0-8.fc19.ppc64 glusterfs-libs-3.4.0-8.fc19.ppc64
>> gnutls-3.1.16-1.fc19.ppc64 keyutils-libs-1.5.6-1.fc19.ppc64
>> libgcc-4.8.2-1.fc19.ppc64 libgcrypt-1.5.3-2.fc19.ppc64
>> libibverbs-1.1.7-3.fc19.ppc64 libiscsi-1.7.0-5.fc19.ppc64
>> libpng-1.5.13-2.fc19.ppc64 librdmacm-1.0.17-1.fc19.ppc64
>> libusbx-1.0.16-3.fc19.ppc64 systemd-libs-204-17.fc19.ppc64
>> usbredir-0.6-2.fc19.ppc64
>> (gdb) bt
>> #0  unassigned_mem_accepts (opaque=0x0, addr=0x10080000052, size=0x1,
>> is_write=0x1)
>>     at /home/alexey/p/qemu/memory.c:892
>> #1  0x00000000103cb238 in memory_region_access_valid (mr=0x10b76ec8
>> <io_mem_unassigned>,
>>     addr=0x10080000052, size=0x1, is_write=0x1) at
>> /home/alexey/p/qemu/memory.c:928
>> #2  0x00000000103cb4bc in memory_region_dispatch_write (mr=0x10b76ec8
>> <io_mem_unassigned>,
>>     addr=0x10080000052, data=0x80, size=0x1) at
>> /home/alexey/p/qemu/memory.c:976
>> #3  0x00000000103cebec in io_mem_write (mr=0x10b76ec8 <io_mem_unassigned>,
>> addr=0x10080000052,
>>     val=0x80, size=0x1) at /home/alexey/p/qemu/memory.c:1748
>> #4  0x0000000010329b54 in address_space_rw (as=0x10b80cf0
>> <address_space_memory>,
>>     addr=0x10080000052, buf=0x3fff8ad9e190 "\200", len=0x1, is_write=0x1)
>>     at /home/alexey/p/qemu/exec.c:1941
>> #5  0x000000001032a000 in cpu_physical_memory_rw (addr=0x10080000052,
>>     buf=0x3fff8ad9e190 "\200", len=0x1, is_write=0x1) at
>> /home/alexey/p/qemu/exec.c:2010
>> #6  0x00000000103231c4 in cpu_physical_memory_write (addr=0x10080000052,
>> buf=0x3fff8ad9e190,
>>     len=0x1) at /home/alexey/p/qemu/include/exec/cpu-common.h:68
>> #7  0x000000001032b5b0 in stb_phys (addr=0x10080000052, val=0x80)
>>     at /home/alexey/p/qemu/exec.c:2506
>> #8  0x00000000103a0c10 in h_logical_store (cpu=0x3fff8ada0010,
>> spapr=0x100118ca410,
>>     opcode=0x40, args=0x3fff8bfc0030) at
>> /home/alexey/p/qemu/hw/ppc/spapr_hcall.c:564
>> #9  0x00000000103a140c in spapr_hypercall (cpu=0x3fff8ada0010,
>> opcode=0x40, args=0x3fff8bfc0030)
>>     at /home/alexey/p/qemu/hw/ppc/spapr_hcall.c:737
>> #10 0x000000001041b080 in kvm_arch_handle_exit (cs=0x3fff8ada0010,
>> run=0x3fff8bfc0000)
>>     at /home/alexey/p/qemu/target-ppc/kvm.c:1223
>> #11 0x00000000103c5cbc in kvm_cpu_exec (cpu=0x3fff8ada0010)
>>     at /home/alexey/p/qemu/kvm-all.c:1726
>> #12 0x000000001031902c in qemu_kvm_cpu_thread_fn (arg=0x3fff8ada0010)
>>     at /home/alexey/p/qemu/cpus.c:874
>> #13 0x00000080bcd0c29c in start_thread (arg=0x3fff8ad9eee0) at
>> pthread_create.c:310
>> #14 0x00000080bcb1ddb0 in .__clone ()
>>     at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:111
>>
>>
>>
>> Without your patch, unassigned_mem_accepts() is not called so it tells us
>> that IO stopped working after your patch.
>>
> 
> Alex,
> 
> Can you elaborate ? Does the kernel fail to boot ?
> 
>> Unfortunately I do not have good 3D imagination, do not understand this
>> memory region well enough to give advice and cannot tell quickly what
>> exactly is wrong here :)
>>
> 
> Heh no problem. Let me share my findings then ! :)
> 
> First, if I pass kernel/initrd/cmdline directly to the qemu command
> line, I don't get this weird access to 0x10080000052 at all (with or
> without my patch).
> 
> Second, if I run the very same test WITHOUT my patch and set a breakpoint
> in unassigned_io_write(), it pops with the same 0x10080000052 address.


It does not stop there when I try it "WITHOUT my patch", that's my entire
point.

I retried right now, with the upstream QEMU + KVM breakpoint stubs patch.

With your patch - it still 100% reproducible - gdb stops at
unassigned_io_write().

Without your patch there is no stopping in unassigned_io_write().

The exact command line is:
./qemu-system-ppc64 \
	-enable-kvm \
	-m 2048 \
	-L qemu-ppc64-bios/ \
	-machine pseries \
	-trace events=qemu_trace_events \
	-nographic \
	-vga none \
	-drive
id=id0,if=none,readonly=off,werror=stop,rerror=stop,discard=on,file=virtimg/fc19_16GB.qcow2,format=qcow2
\
	-device virtio-blk-pci,id=id1,drive=id0


There is always a chance that you have fixed one bug and make another show
up, this happens sometime, but we do not know for sure that this is the case :)


> Third but not least, I have not hit a single issue so far... I mean, when
> the kernel is booted, virtio is functional with my patch, as far as I could
> test (having / mounted on virtio, multiple 9p shares).


Yes, it is functional in this particular example. However something else
might get broken, something what makes use of IO ports, I do not even know
what it could be (need to test every emulated PCI device with all supported
distros to make sure OR understand what the difference is and fix it).



> It proves the patch is not responsible for the "unassigned" thing, IMHO.
> 
> Thanks for your time anyway.

Welcome :)


-- 
Alexey