Re: [Qemu-devel] [PATCH v5 for-2.3 28/28] docs: Add PXB documentation

[Marcel Apfelbaum <marcel@redhat.com>]

On 03/10/2015 07:42 PM, Michael S. Tsirkin wrote:
> On Tue, Mar 10, 2015 at 06:21:14PM +0200, Marcel Apfelbaum wrote:
>> On 03/10/2015 05:47 PM, Michael S. Tsirkin wrote:
>>> On Tue, Mar 10, 2015 at 05:32:14PM +0200, Marcel Apfelbaum wrote:
>>>> Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
>>>> ---
>>>>   docs/pci_expander_bridge.txt | 52 ++++++++++++++++++++++++++++++++++++++++++++
>>>>   1 file changed, 52 insertions(+)
>>>>   create mode 100644 docs/pci_expander_bridge.txt
>>>>
>>>> diff --git a/docs/pci_expander_bridge.txt b/docs/pci_expander_bridge.txt
>>>> new file mode 100644
>>>> index 0000000..58bf7a8
>>>> --- /dev/null
>>>> +++ b/docs/pci_expander_bridge.txt
>>>> @@ -0,0 +1,52 @@
>>>> +PCI EXPANDER BRIDGE (PXB)
>>>> +=========================
>>>> +
>>>> +Description
>>>> +===========
>>>> +PXB is a "light-weight" host bridge in the same PCI domain
>>>> +as the main host bridge whose purpose is to enable
>>>> +the main host bridge to support multiple PCI root buses.
>>>> +It is implemented only for i440fx.
>>>
>>> BTW what makes it i440fx specific?
>>> Also, what happens if you try to use it
>>> with a different machine type?
>> Is is i440fx specific, please look at patch 22/28.
>> Also we have a specific check for i440fx, so CRS
>> will not be emitted for other machine types.
>>
>> Thanks,
>> Marcel
>
> In fact it won't work at all. Need to think about it,
> maybe we can make it work more generally.
> For CRS, should be possible to emit for q35 too?
We can make it work, but not on the scope of this series.
However, I'll add a IHostBridgeSnoop interface that will
make the device work only with associated bus and
this will make it less general.

Thanks,
Marcel

>
>>>
>>>> +
>>>> +As opposed to PCI-2-PCI bridge's secondary bus, PXB's bus
>>>> +is a primary bus and can be associated with a NUMA node
>>>> +(different from the main host bridge) allowing the guest OS
>>>> +to recognize the proximity of a pass-through device to
>>>> +other resources as RAM and CPUs.
>>>> +
>>>> +Usage
>>>> +=====
>>>> +A detailed command line would be:
>>>> +
>>>> +[qemu-bin + storage options]
>>>> +-bios [seabios-dir]/out/bios.bin -L [seabios-dir]/out/
>>>> +-m 2G
>>>> +-object memory-backend-ram,size=1024M,policy=bind,host-nodes=0,id=ram-node0 -numa node,nodeid=0,cpus=0,memdev=ram-node0
>>>> +-object memory-backend-ram,size=1024M,policy=interleave,host-nodes=0,id=ram-node1 -numa node,nodeid=1,cpus=1,memdev=ram-node1
>>>> +-device pxb-device,id=bridge1,bus=pci.0,numa_node=1,bus_nr=4 -netdev user,id=nd-device e1000,bus=bridge1,addr=0x4,netdev=nd
>>>> +-device pxb-device,id=bridge2,bus=pci.0,numa_node=0,bus_nr=8 -device e1000,bus=bridge2,addr=0x3
>>>> +-device pxb-device,id=bridge3,bus=pci.0,bus_nr=40 -drive if=none,id=drive0,file=[img] -device virtio-blk-pci,drive=drive0,scsi=off,bus=bridge3,addr=1
>>>> +
>>>> +Here you have:
>>>> + - 2 NUMA nodes for the guest, 0 and 1. (both mapped to the same NUMA node in host, but you can and should put it in different host NUMA nodes)
>>>> + - a pxb host bridge attached to NUMA 1 with an e1000 behind it
>>>> + - a pxb host bridge attached to NUMA 0 with an e1000 behind it
>>>> + - a pxb host bridge not attached to any NUMA with a hard drive behind it.
>>>> +
>>>> +Implementation
>>>> +==============
>>>> +The PXB is composed by:
>>>> +- HostBridge (TYPE_PXB_HOST)
>>>> +  The host bridge allows to register and query the PXB's rPCI root bus in QEMU.
>>>> +- PXBDev(TYPE_PXB_DEVICE)
>>>> +  It is a regular PCI Device that resides on the piix host-bridge bus and its bus uses the same PCI domain.
>>>> +  However, the bus behind is exposed through ACPI as a primary PCI bus and starts a new PCI hierarchy.
>>>> +  The interrupts from devices behind the PXB are routed through this device the same as if it were a
>>>> +  PCI-2-PCI bridge. The _PRT follows the i440fx model.
>>>> +- PCIBridgeDev(TYPE_PCI_BRIDGE_DEV)
>>>> +  Created automatically as part of init sequence.
>>>> +  When adding a device to PXB it is attached to the bridge for two reasons:
>>>> +  - Using the bridge will enable hotplug support
>>>> +  - All the devices behind the bridge will use bridge's IO/MEM windows compacting
>>>> +    the PCI address space.
>>>> +
>>>> --
>>>> 2.1.0