Re: [RFC] Add support for Xen ARM guest on FreeBSD

[Nathan Whitehorn <nwhitehorn@freebsd.org>]

On 01/16/14 18:36, Julien Grall wrote:
>
>
> On 01/16/2014 01:56 AM, Nathan Whitehorn wrote:
>>
>> Thanks for the CC. Could you explain what you mean by "grant-parent"
>> etc? "interrupt-parent" is a fundamental part of the way PAPR (and
>> ePAPR) work, so I'm very very hesitant to start guessing. I think things
>> have broken for you because some (a lot, actually) of OF code does not
>> expect #interrupt-cells to be more than 2. Some APIs might need
>> updating, which I'll try to take care of. Could you tell me what the
>> difference between SPI and PPI is, by the way?
>
> Sorry, I also made some typoes in my explanation so it was not clear.
>
> interrupt-parent is a property in a device node which links this node 
> to an interrupt controller (in our case the GIC controller).
>
> The way to handle it on Linux and the ePAR is different:
>    - ePAR (chapter 2.4) says:
> The physical wiring of an interrupt source to an interrupt controller 
> is represented in the device tree with the interrupt-parent property. 
> Nodes that represent interrupt-generating devices contain an
> interrupt-parent property which has a phandle value that points to the 
> device to which the device's interrupts are routed, typically an 
> interrupt controller. If an interrupt-generating device does not have
> an interrupt-parent property, its interrupt parent is assumed to be 
> its device tree parent.
> From my understanding, it's mandatory to have an interrupt-parent 
> property on each device node which is using interrupts. If it doesn't 
> exist it will assume that the parent is interrupt controller.
> If I'm mistaken, at least FreeBSD handle the interrupt-parent property 
> in this way.
>    - Linux implementation will look at to the node, if the property 
> doesn't exists, it will check if the ancestor has this property ...



> So the device tree below is valid on Linux, but not on FreeBSD:
>
> / {
>   interrupt-parent = &gic
>
>   gic: gic@10
>   {
>     ...
>   }
>
>   timer@1
>   {
>     interrupts = <...>
>   }
> }
>
> Most of shipped device tree use this trick.
>
> IanC: I was reading the linux binding documentation 
> (devicetree/booting-without-of.txt VII.2) and it seems that the 
> explanation differs from the implementation, right?
>
> For the #interrupt-cells property, the problem starts in 
> fdt_intr_to_rl (sys/dev/fdt/fdt_common.c:476). ofw_bus_map_intr is 
> called always with the first cells of the interrupt no matter the 
> number of cells specified by #interrupt-cells.

The specification is actually a little unclear on this point, but 
FreeBSD follows the same rules as Linux in any case. Most, if not all, 
FreeBSD code should check any ancestor at this point as well. In 
particular fdt_intr_to_rl does this. What it *doesn't* do is allow 
#interrupt-cells to be larger than 2. I'll fix this this weekend.

>> On the subject of simple-bus, they usually aren't necessary. For
>> example, all hypervisor devices on IBM hardware live under /vdevice,
>> which is attached to the device tree root. They don't use MMIO, so
>> simple-bus doesn't really make sense. How does Xen communicate with the
>> OS in these devices?
>> -Nathan
>
> As I understand, only the simple bus code (see simplebus_attach) is 
> translating the interrupts in the device on a resource.
> So if you have a node directly attached to the root node with 
> interrupts and MMIO, the driver won't be able to retrieve and 
> translate the interrupts via bus_alloc_resources.

Why not? nexus on ARM, MIPS, PowerPC, and sparc64 can do this.

> In the Xen device tree, we have an hypervisor node directly attach to 
> the root which contains both MMIO and interrupt used by Xen to 
> communicate with the guest.
>

OK. This should be fine, though simplebus would also work if you use MMIO.
-Nathan