Pass-through on Arndale 5250

Pass-through on Arndale 5250

[xen sandberg]

[-- Attachment #1.1: Type: text/plain, Size: 5636 bytes --]

I followed this guide:

  http://ssup2.iptime.org/wiki/Xen_4.5.0_on_Arndale

to get xen installed on the arndale 5250 with an ubuntu Dom0 and I am
able to create guests without problems.  Now I am would like to pass a
device to a guest, as a proof of concept.  I choose the ethernet port
because that will be useful down the road but right now I would be happy
passing anything through.  I then followed the example from this
guide:

  https://events.linuxfoundation.org/sites/events/files/slides/talk_5.pdf

but so far I have not been able to start a guest with the device
passed to it.  I tried updating the Flat Device Tree, u-boot and
dtc.  I am unsure what values to use for irqs and iomem as the
example did not give details on where they got their numbers.

Thoughts?

======================
Details of what I tried:

I get the error:

libxl: error: libxl_arm.c:604:check_partial_fdt: Partial FDT is not a
valid Flat Device Tree
libxl: error: libxl_dom.c:710:libxl__build_pv:
libxl__arch_domain_init_hw_description failed: No such file or directory


I found that there was not a separate entry for the ethernet port under
/proc/device-tree as there was in the example.  Instead the port seems
to be viewed as just a usb device and is given a random new mac address
at each boot time.  I set it for pass-through by adding:

fdt set /usb@12110000 status disabled
fdt set /usb@12110000 xen,passthrough

to my u-boot startup image.  This disabled all of the usb ports and the
ethernet port on the board but that is acceptable to me as long as it is
a workable option to pass them all through to a DomU.

Then using the example as a template it looked like the values for
'compatible' and 'interrupts' would be copied to the partial device tree
so I ran:

# hexdump -C interrupts
00000000  00 00 00 00 00 00 00 47  00 00 00 00              |.......G....|
0000000c
# hexdump -C compatible
00000000  73 61 6d 73 75 6e 67 2c  65 78 79 6e 6f 73 34 32
|samsung,exynos42|
00000010  31 30 2d 65 68 63 69 00                           |10-ehci.|
00000018

Where the value of 'reg' came from was unclear to me so I used the same
value as the example, and I wrote the following:

Here are the contents of my partial device tree:

/dts-v1/;

/ {
    /* #*cells are here to keep DTC happy */
    #address-cells = <2>;
    #size-cells = <2>;

    aliases {
        net = &usb;
    };

    passthrough {
        compatible = "simple-bus";
        ranges;
        #address-cells = <2>;
        #size-cells = <2>;
    usb: usb@10000000 {
        compatible = "samsung,exynos42";
        reg = <0 0x10000000 0 0x1000>;
                interrupts = <0 47 0>;
    };
    };
};

and compiled it with:

dtc guest-midway.dts > guest-midway.dtb

BTW: The pdf said to get the value for reg I would need to find a hole
in the guest layout defined in xen/include/public/arch-arm.h. I checked
out RELEASE-4.6.0 for my build and I found that file but couldn't figure
out how to use the file to find a hole, an example would be much
appreciated.

I then added the following to the DomU config file:

device_tree = "/root/guest-midway.dtb"
dtdev = [ "/usb@12110000" ]
irqs = [ 112, 113, 114 ]
iomem = [ "0x12110000,1@0x10000" ]

I was unsure what values to use for irqs and iomem as the example did
not give details on where they got their numbers.  I looked at
/proc/interrupts and saw this:

root@dom0:~# cat /proc/interrupts
            CPU0
16:         84       GIC  16  events
27:       6698       GIC  27  arch_timer
65:          0       GIC  65  10800000.mdma
66:          0       GIC  66  121a0000.pdma
67:          0       GIC  67  121b0000.pdma
74:          0       GIC  74  101d0000.watchdog
75:          0       GIC  75  s3c2410-rtc alarm
76:          0       GIC  76  s3c2410-rtc tick
77:          0       GIC  77  13400000.pinctrl
78:          0       GIC  78  11400000.pinctrl
79:          0       GIC  79  3860000.pinctrl
82:          0       GIC  82  10d10000.pinctrl
88:        462       GIC  88  12c60000.i2c
90:          0       GIC  90  12c80000.i2c
91:          0       GIC  91  12c90000.i2c
96:          0       GIC  96  12ce0000.i2c
97:          0       GIC  97  10060000.tmu
103:       1537       GIC 103  ehci_hcd:usb3, ohci_hcd:usb4
104:          0       GIC 104  xhci-hcd:usb1
107:       5107       GIC 107  dw-mci
109:      21989       GIC 109  dw-mci
156:          0       GIC 156  11c10000.mdma
160:         51   xen-dyn-event     xenbus
181:          1  exynos_wkup_irq_chip   2  s5m8767
182:          0  xen-percpu-virq      hvc_console
183:          0   s5m8767  12  rtc-alarm0
184:          0  exynos_wkup_irq_chip   4  SW-TACT2
185:          0  exynos_wkup_irq_chip   5  SW-TACT3
186:          0  exynos_wkup_irq_chip   6  SW-TACT4
187:          0  exynos_wkup_irq_chip   7  SW-TACT5
188:          0  exynos_wkup_irq_chip   0  SW-TACT6
189:          0  exynos_wkup_irq_chip   1  SW-TACT7
190:         32   xen-dyn-event     evtchn:oxenstored
191:          0   xen-dyn-event     evtchn:oxenstored
IPI0:          0  CPU wakeup interrupts
IPI1:          0  Timer broadcast interrupts
IPI2:          0  Rescheduling interrupts
IPI3:          0  Function call interrupts
IPI4:          0  Single function call interrupts
IPI5:          0  CPU stop interrupts
IPI6:          0  IRQ work interrupts
IPI7:          0  completion interrupts
Err:          0


and thought perhaps I would need to use numbers 103 and 104 as they are
the only usb devices in the list but I didn't know if I was on the right
track. So I again just naively copied the irq values from the example
and put the address of the usb as the first value under iomem.

[-- Attachment #1.2: Type: text/html, Size: 7166 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Pass-through on Arndale 5250

[Ian Campbell]

On Tue, 2016-02-02 at 08:37 -0600, xen sandberg wrote:
> I followed this guide:
> 
>   http://ssup2.iptime.org/wiki/Xen_4.5.0_on_Arndale
> 
> to get xen installed on the arndale 5250 with an ubuntu Dom0 and I am
> able to create guests without problems.

Did you install something newer than 4.5.0? I'd highly recommend at least
4.6 for ARM, especially with passthrough.

I should also say upfront that platform device passthrough for ARM is not
for the faint hearted and you will need a pretty good familiarity with the
h/w in question as well as topics such as device tree syntax, semantics and
the bindings for the devices you are using as well as the ARM GIC
architecture (SPIs vs PPIs etc).

I should also further point out that the Arndale lacks SMMU hardware (I/O
virtualisation support). Therefore you cannot passthrough any device which
does DMA and there may be caveats even for devices which do not. I have not
looked to see if the USB controller on Arndale does DMA -- but I expect it
probably does. You should certainly confirm this before going any further
because if the controller does DMA then what you are trying to do is
essentially impossible with this hardware.

> ======================
> Details of what I tried:
> 
> I get the error:
> 
> libxl: error: libxl_arm.c:604:check_partial_fdt: Partial FDT is not a
> valid Flat Device Tree

I'll cover this below.

> libxl: error: libxl_dom.c:710:libxl__build_pv:
> libxl__arch_domain_init_hw_description failed: No such file or directory

This seems to correspond to the file named as your ramdisk in the guest cfg
not existing. Full logs with "xl -vvv create ..." would be useful for
diagnosing this sort of thing.

> I found that there was not a separate entry for the ethernet port under
> /proc/device-tree as there was in the example.  Instead the port seems
> to be viewed as just a usb device and is given a random new mac address
> at each boot time.

Yes, Ethernet on Arndale is provided by a USB device. Since USB is a
"discoverable bus" only the root controller is described in DT and thus the
root controller is the only thing you can try to passthrough.

>   I set it for pass-through by adding:
> 
> fdt set /usb@12110000 status disabled
> fdt set /usb@12110000 xen,passthrough
> 
> to my u-boot startup image.  This disabled all of the usb ports and the
> ethernet port on the board but that is acceptable to me as long as it is
> a workable option to pass them all through to a DomU.
> 
> Then using the example as a template it looked like the values for
> 'compatible' and 'interrupts' would be copied to the partial device tree
> so I ran:

You can use "dtc -I fd /proc/device-tree" to get the DTS from the running
system. Of course you also ought to have the source in your kernel build
tree somewhere.

>     passthrough {
>         compatible = "simple-bus";
>         ranges;
>         #address-cells = <2>;
>         #size-cells = <2>;
>     usb: usb@10000000 {
>         compatible = "samsung,exynos42";

This looks truncated and/or wrong, it needs to be the compatible string
associated with the particular USB controller, which looks to be one of
"samsung,exynos5250-dwusb3", "samsung,exynos4210-ehci" or
"samsung,exynos4210-ohci" depending on which controller this is.

>         reg = <0 0x10000000 0 0x1000>;
>                 interrupts = <0 47 0>;
>     };
>     };
> };
> 
> and compiled it with:
> 
> dtc guest-midway.dts > guest-midway.dtb
> 
> BTW: The pdf said to get the value for reg I would need to find a hole
> in the guest layout defined in xen/include/public/arch-arm.h. I checked
> out RELEASE-4.6.0 for my build and I found that file but couldn't figure
> out how to use the file to find a hole, an example would be much
> appreciated.

Towards the end of arch-arm.h you will find a section "Virtual machine
platform (memory layout, interrupts)" which defines the memory layout seen
by guests, in the form of various GUEST_FOO_{BASE,SIZE} defines.

You need to find a suitable hole not covered by any of those which is large
enough to contain the device which you are passing through.

> I then added the following to the DomU config file:
> 
> device_tree = "/root/guest-midway.dtb"

Does this path correspond to where the dtc command above wrote to?

What does "file /root/guest-midway.dtb" say?

> dtdev = [ "/usb@12110000" ]
> irqs = [ 112, 113, 114 ]

These need to match the interrupts property in the partial device tree. You
will probably need to read up on device tree semantics in order to make
progress here. In particular the interrupt property for a device connected
to a  GIC interrupt controller is documented in
linux/Documentation/devicetree/bindings/interrupt-controller/arm and the DT
property must follow the correct syntax.

> iomem = [ "0x12110000,1@0x10000" ]

And the @0x10000 needs to correspond with the region defined in the partial
device tree's reg property. You need the size to match also.

> and thought perhaps I would need to use numbers 103 and 104 as they are
> the only usb devices in the list but I didn't know if I was on the right
> track.

Those are likely the correct IRQ numbers (so what is used in the guest
cfg). However the DT (as described in the binding doc referenced above)
needs the SPI number, which means it is offset by -32.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel