All of lore.kernel.org
 help / color / mirror / Atom feed
From: Robin Murphy <robin.murphy@arm.com>
To: Peter Geis <pgwipeout@gmail.com>, Heiko Stuebner <heiko@sntech.de>
Cc: "open list:ARM/Rockchip SoC..."
	<linux-rockchip@lists.infradead.org>,
	linux-pci@vger.kernel.org
Subject: Re: [Question] rk3399 vfio-pci/sr-iov support
Date: Mon, 9 Dec 2019 14:07:02 +0000	[thread overview]
Message-ID: <b597b9a6-870a-8fbd-6490-59734c04367f@arm.com> (raw)
In-Reply-To: <CAMdYzYoPXWbv4zXet6c9JQEMbqcJi6ZEOui_n82NVmrqNLy_pw@mail.gmail.com>

On 09/12/2019 1:28 pm, Peter Geis wrote:
> Good Morning,
> 
> I'm back with more pcie fun on the rk3399.
> I'm trying to get pcie passthrough working for a vm on the rk3399, and
> have encountered some roadblocks.
> 
> First, vfio-pci doesn't work on the rk3399, as the pcie controller
> doesn't bind explicitly to a iommu.
> [37528.138212] vfio-pci 0000:01:00.0: assign IRQ: got 226
> [37528.138254] vfio-pci: probe of 0000:01:00.0 failed with error -22
> 
> # find /sys/kernel/iommu_groups/ -type l
> /sys/kernel/iommu_groups/1/devices/ff8f0000.vop
> /sys/kernel/iommu_groups/2/devices/ff900000.vop
> 
> # virsh start openwrt
> error: Failed to start domain openwrt
> error: internal error: Process exited prior to exec: libvirt:  error :
> internal error: Invalid device 0000:01:00.0 iommu_group file
> /sys/bus/pci/devices/0000:01:00.0/iommu_group is not a symlink

That much I can help with somewhat: the major impediment is that RK3399 
doesn't have an IOMMU in front of PCIe. As far as I'm aware your only 
option is to resort to the "here be dragons" CONFIG_VFIO_NOIOMMU mode 
(which I don't know an awful lot about beyond that it's a thing).

Robin.

> Second, sr-iov support is broken.
> root@rockpro64:/sys/bus/pci/devices/0000:01:00.0# echo 1 > sriov_numvfs
> bash: echo: write error: Input/output error
> [37352.907558] pci 0000:01:10.0: [8086:1520] type 7f class 0xffffff
> [37352.907578] pci 0000:01:10.0: unknown header type 7f, ignoring device
> 
> Do any of y'all have some insight into these issues?
> 
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip
> 

WARNING: multiple messages have this Message-ID (diff)
From: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
To: Peter Geis <pgwipeout-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Heiko Stuebner <heiko-4mtYJXux2i+zQB+pC5nmwQ@public.gmane.org>
Cc: "open list:ARM/Rockchip SoC..."
	<linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>,
	linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [Question] rk3399 vfio-pci/sr-iov support
Date: Mon, 9 Dec 2019 14:07:02 +0000	[thread overview]
Message-ID: <b597b9a6-870a-8fbd-6490-59734c04367f@arm.com> (raw)
In-Reply-To: <CAMdYzYoPXWbv4zXet6c9JQEMbqcJi6ZEOui_n82NVmrqNLy_pw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On 09/12/2019 1:28 pm, Peter Geis wrote:
> Good Morning,
> 
> I'm back with more pcie fun on the rk3399.
> I'm trying to get pcie passthrough working for a vm on the rk3399, and
> have encountered some roadblocks.
> 
> First, vfio-pci doesn't work on the rk3399, as the pcie controller
> doesn't bind explicitly to a iommu.
> [37528.138212] vfio-pci 0000:01:00.0: assign IRQ: got 226
> [37528.138254] vfio-pci: probe of 0000:01:00.0 failed with error -22
> 
> # find /sys/kernel/iommu_groups/ -type l
> /sys/kernel/iommu_groups/1/devices/ff8f0000.vop
> /sys/kernel/iommu_groups/2/devices/ff900000.vop
> 
> # virsh start openwrt
> error: Failed to start domain openwrt
> error: internal error: Process exited prior to exec: libvirt:  error :
> internal error: Invalid device 0000:01:00.0 iommu_group file
> /sys/bus/pci/devices/0000:01:00.0/iommu_group is not a symlink

That much I can help with somewhat: the major impediment is that RK3399 
doesn't have an IOMMU in front of PCIe. As far as I'm aware your only 
option is to resort to the "here be dragons" CONFIG_VFIO_NOIOMMU mode 
(which I don't know an awful lot about beyond that it's a thing).

Robin.

> Second, sr-iov support is broken.
> root@rockpro64:/sys/bus/pci/devices/0000:01:00.0# echo 1 > sriov_numvfs
> bash: echo: write error: Input/output error
> [37352.907558] pci 0000:01:10.0: [8086:1520] type 7f class 0xffffff
> [37352.907578] pci 0000:01:10.0: unknown header type 7f, ignoring device
> 
> Do any of y'all have some insight into these issues?
> 
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip
> 

  reply	other threads:[~2019-12-09 14:07 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-09 13:28 [Question] rk3399 vfio-pci/sr-iov support Peter Geis
2019-12-09 13:28 ` Peter Geis
2019-12-09 14:07 ` Robin Murphy [this message]
2019-12-09 14:07   ` Robin Murphy
2019-12-09 17:05   ` Alex Williamson
2019-12-09 17:05     ` Alex Williamson
2019-12-10  1:23     ` Peter Geis
2019-12-10  1:23       ` Peter Geis
2021-11-15 19:50   ` Adam Joseph
2021-11-15 19:50     ` Adam Joseph
2021-11-16 10:28     ` Robin Murphy
2021-11-16 10:28       ` Robin Murphy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b597b9a6-870a-8fbd-6490-59734c04367f@arm.com \
    --to=robin.murphy@arm.com \
    --cc=heiko@sntech.de \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-rockchip@lists.infradead.org \
    --cc=pgwipeout@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.