Re: [Qemu-devel] [PATCH v4 for-2.3 00/25] hw/pc: implement multiple primary busses for pc machines

[Marcel Apfelbaum <marcel@redhat.com>]

On 03/09/2015 06:55 PM, Gerd Hoffmann wrote:
> On Mo, 2015-03-09 at 18:26 +0200, Marcel Apfelbaum wrote:
>> On 03/09/2015 04:19 PM, Gerd Hoffmann wrote:
>>>     Hi,
>>>
>>>> My series is based on commit 09d219a. Try please on top of this commit.
>>>
>>> Ok, that works.  Going to play with that now ;)
>> Good luck! ... and tell me what you think :)
>> If you need any help with the command line of the pxb device, let me know,.
>
> First thing I've noticed:  You need to define a numa node so you can
> pass a valid numa node to the pxb-device.  Guess that is ok as the whole
> point of this is to assign pci devices to numa nodes.  More complete
> test instructions would be nice though.
Exactly, this is by design. But you can also use it without specifying the NUMA node...

A detailed command line would be:

[qemu-bin + storage options]
-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
-m 2G
-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1

Here you have:
  - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
  - a pxb host bridge attached to NUMA 1 with an e1000 behind it
  - a pxb host bridge attached to NUMA 0 with an e1000 behind it
  - a pxb host bridge not attached to any NUMA with a hard drive behind it.

As you can see, since you already "decide" NUMA mapping at command line, it is "natural" also to attach the pxbs to the NUMA nodes.

>
> Second thing:  Booting with an unpatched seabios has bad effects:
>
> [root@localhost ~]# cat /proc/iomem
> 00000000-000fffff : PCI Bus 0000:10
>    00000000-00000fff : reserved
>    00001000-0009fbff : System RAM
>    0009fc00-0009ffff : reserved
>    000c0000-000c91ff : Video ROM
>    000c9800-000ca1ff : Adapter ROM
>    000ca800-000ccbff : Adapter ROM
>    000f0000-000fffff : reserved
>      000f0000-000fffff : System ROM
> 00100000-3ffdffff : System RAM
>    01000000-0174bde4 : Kernel code
>    0174bde5-01d30cff : Kernel data
>    01eaa000-0202afff : Kernel bss
> 3ffe0000-3fffffff : reserved
> fd000000-fdffffff : 0000:00:02.0
>    fd000000-fdffffff : bochs-drm
> febc0000-febdffff : 0000:00:03.0
>    febc0000-febdffff : e1000
> febf0000-febf0fff : 0000:00:02.0
>    febf0000-febf0fff : bochs-drm
> fec00000-fec003ff : IOAPIC 0
> fed00000-fed003ff : HPET 0
>    fed00000-fed003ff : PNP0103:00
> fee00000-fee00fff : Local APIC
> feffc000-feffffff : reserved
> fffc0000-ffffffff : reserved
>
> "PCI Bus 0000:10" is bogus and "PCI Bus 0000:00" isn't there at all.
Yes, you shouldn't use pxb if you are not using the corresponding SeaBIOS.
However, as I understand we always attach a SeaBIOS binary with a QEMU release,
so we should be OK with this.

And this is the reason I wanted bios support *before* the PXB device implementation,
but anyway, even if we have them in the same time, as long as the release
has both pxb and BIOS with pxb support, is OK. (I think...)

I appreciate you looking into this and if you need further assistance
don't hesitate to mail me! :)

Thanks,
Marcel

>
> cheers,
>    Gerd
>
>