SMMU 2-stage support

SMMU 2-stage support

[Baptiste Reynal]

[-- Attachment #1.1: Type: text/plain, Size: 681 bytes --]

We are currently working on the ARM SMMU 2-stage support in order to expose
SMMU capabilities to a guest OS.
By doing some research, we noticed some work has already be done by Will
Deacon on exposing virtual IOMMU interface to KVM guest. After going
through slides and code (on git://
git.kernel.org/pub/scm/linux/kernel/git/will/linux.git, branch:
iommu/vsmmu), we have some questions :

- What is the status of this work ? Is Alex still working on it ?
- Are there any test case about the current vsmmu implementation ?

- What is exposed to the guest in the current implementation ?
- How do the guest program the stage-1 translation ?

Thanks for your consideration,
Baptiste

[-- Attachment #1.2: Type: text/html, Size: 843 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: SMMU 2-stage support

[Will Deacon]

On Wed, Feb 18, 2015 at 05:04:21PM +0000, Baptiste Reynal wrote:
> We are currently working on the ARM SMMU 2-stage support in order to
> expose SMMU capabilities to a guest OS.
> By doing some research, we noticed some work has already be done by Will
> Deacon on exposing virtual IOMMU interface to KVM guest. After going
> through slides and code (on
> git://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git<http://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git>,
> branch: iommu/vsmmu), we have some questions :
> 
> - What is the status of this work ? Is Alex still working on it ?

I got a reasonable way through implementing something, but ran into a
wall for a few reasons:

  (1) I don't have anything where I can test this (yet)

  (2) Nobody's actually asking for it...

> - Are there any test case about the current vsmmu implementation ?

No.

> - What is exposed to the guest in the current implementation ?

An emulation of a stage-1 only SMMU.

> - How do the guest program the stage-1 translation ?

Just like it would for a physical SMMU.
It's probably easiest if you look at the (incomplete, WIP) code:

  https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/commit/?h=iommu/vsmmu&id=539dbcb964e5c730791a6a31a4f58ceff4b945c2

Anyway, until somebody actually wants this feature I've put it on ice as
it adds a whole bunch of complication to the ARM SMMU driver, as well as
new user ABI extensions that I don't really want to maintain for fun.

The only murmurs I've heard about IOMMU usage from a guest relate to a
paravirtualised interface, which abstracts the details of the host IOMMU
altogether and allows the translation to be collapsed (which can be
better for TLB utilisation).

Will

Re: SMMU 2-stage support

[Baptiste Reynal]

[-- Attachment #1.1: Type: text/plain, Size: 2438 bytes --]

Hello,

We are eventually working on the vSMMU implementation. Relying on the talk
Will Deacon gave at the Linux Plumbers IOMMU Microconference on October
2014 (http://linuxplumbersconf.org/2014/ocw/proposals/2019), I tried the
vSMMU initialization.

I have a question about the KVM_DEV_ARM_SMMU_V2_CFG_INIT call. What is
exactly expected as parameters ?
My guess is, as it is the base address of the vSMMU, it should be the
address expected to be the "physical address" of the vSMMU on the guest
side. Am I right ?

Thanks,
Baptiste

On Wed, Feb 18, 2015 at 7:56 PM, Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org> wrote:

> On Wed, Feb 18, 2015 at 05:04:21PM +0000, Baptiste Reynal wrote:
> > We are currently working on the ARM SMMU 2-stage support in order to
> > expose SMMU capabilities to a guest OS.
> > By doing some research, we noticed some work has already be done by Will
> > Deacon on exposing virtual IOMMU interface to KVM guest. After going
> > through slides and code (on
> > git://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git<
> http://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git>,
> > branch: iommu/vsmmu), we have some questions :
> >
> > - What is the status of this work ? Is Alex still working on it ?
>
> I got a reasonable way through implementing something, but ran into a
> wall for a few reasons:
>
>   (1) I don't have anything where I can test this (yet)
>
>   (2) Nobody's actually asking for it...
>
> > - Are there any test case about the current vsmmu implementation ?
>
> No.
>
> > - What is exposed to the guest in the current implementation ?
>
> An emulation of a stage-1 only SMMU.
>
> > - How do the guest program the stage-1 translation ?
>
> Just like it would for a physical SMMU.
> It's probably easiest if you look at the (incomplete, WIP) code:
>
>
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/commit/?h=iommu/vsmmu&id=539dbcb964e5c730791a6a31a4f58ceff4b945c2
>
> Anyway, until somebody actually wants this feature I've put it on ice as
> it adds a whole bunch of complication to the ARM SMMU driver, as well as
> new user ABI extensions that I don't really want to maintain for fun.
>
> The only murmurs I've heard about IOMMU usage from a guest relate to a
> paravirtualised interface, which abstracts the details of the host IOMMU
> altogether and allows the translation to be collapsed (which can be
> better for TLB utilisation).
>
> Will
>

[-- Attachment #1.2: Type: text/html, Size: 3634 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: SMMU 2-stage support

[Will Deacon]

On Fri, Apr 03, 2015 at 10:55:02AM +0100, Baptiste Reynal wrote:
> We are eventually working on the vSMMU implementation. Relying on the talk
> Will Deacon gave at the Linux Plumbers IOMMU Microconference on October
> 2014 (http://linuxplumbersconf.org/2014/ocw/proposals/2019), I tried the
> vSMMU initialization.

My position on the vSMMU still hasn't changed:

> > Anyway, until somebody actually wants this feature I've put it on ice as
> > it adds a whole bunch of complication to the ARM SMMU driver, as well as
> > new user ABI extensions that I don't really want to maintain for fun.

So, whilst it's great that you're looking at the code, I'm not very keen
on merging anything until we have people committed to using it. Right now,
the only feedback I've had has been going in the para-virt direction and I
don't think we should do this "for fun".

> I have a question about the KVM_DEV_ARM_SMMU_V2_CFG_INIT call. What is
> exactly expected as parameters ?

Yes, it's the IPA of the vSMMU device. I hacked something into kvmtool
when I was playing with this before:

  https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/commit/?h=kvmtool/vfio

Will

RE: SMMU 2-stage support

[Varun Sethi]

Hi Will,

> -----Original Message-----
> From: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org [mailto:iommu-
> bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org] On Behalf Of Will Deacon
> Sent: Monday, April 13, 2015 4:11 PM
> To: Baptiste Reynal
> Cc: Linux IOMMU
> Subject: Re: SMMU 2-stage support
> 
> On Fri, Apr 03, 2015 at 10:55:02AM +0100, Baptiste Reynal wrote:
> > We are eventually working on the vSMMU implementation. Relying on the
> > talk Will Deacon gave at the Linux Plumbers IOMMU Microconference on
> > October
> > 2014 (http://linuxplumbersconf.org/2014/ocw/proposals/2019), I tried
> > the vSMMU initialization.
> 
> My position on the vSMMU still hasn't changed:
> 
> > > Anyway, until somebody actually wants this feature I've put it on
> > > ice as it adds a whole bunch of complication to the ARM SMMU driver,
> > > as well as new user ABI extensions that I don't really want to maintain for
> fun.
> 
> So, whilst it's great that you're looking at the code, I'm not very keen on
> merging anything until we have people committed to using it. Right now, the
> only feedback I've had has been going in the para-virt direction and I don't
> think we should do this "for fun".
Freescale would be interested in using the vSMMU implementation. We have use cases for assigning devices to guest user space. Are you suggesting that you are more inclined to using the para virtualized approach?

Thanks,
Varun

Re: SMMU 2-stage support

[Will Deacon]

On Tue, Apr 14, 2015 at 02:32:26PM +0100, Varun Sethi wrote:
> Hi Will,

Hey Varun,

> > -----Original Message-----
> > From: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org [mailto:iommu-
> > bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org] On Behalf Of Will Deacon
> > Sent: Monday, April 13, 2015 4:11 PM
> > To: Baptiste Reynal
> > Cc: Linux IOMMU
> > Subject: Re: SMMU 2-stage support
> > 
> > On Fri, Apr 03, 2015 at 10:55:02AM +0100, Baptiste Reynal wrote:
> > > We are eventually working on the vSMMU implementation. Relying on the
> > > talk Will Deacon gave at the Linux Plumbers IOMMU Microconference on
> > > October
> > > 2014 (http://linuxplumbersconf.org/2014/ocw/proposals/2019), I tried
> > > the vSMMU initialization.
> > 
> > My position on the vSMMU still hasn't changed:
> > 
> > > > Anyway, until somebody actually wants this feature I've put it on
> > > > ice as it adds a whole bunch of complication to the ARM SMMU driver,
> > > > as well as new user ABI extensions that I don't really want to maintain for
> > fun.
> > 
> > So, whilst it's great that you're looking at the code, I'm not very keen on
> > merging anything until we have people committed to using it. Right now, the
> > only feedback I've had has been going in the para-virt direction and I don't
> > think we should do this "for fun".
> Freescale would be interested in using the vSMMU implementation. We have
> use cases for assigning devices to guest user space. Are you suggesting
> that you are more inclined to using the para virtualized approach?

I can see arguments either way; the vSMMU means that the guest can use the
same SMMU driver as the host but a para-virtualised approach could
theoretically be used across multiple SMMU implementations as well as
potentially being kinder on the TLB.

Will

RE: SMMU 2-stage support

[Stuart Yoder]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon-5wv7dgnIgG8@public.gmane.org]
> Sent: Wednesday, April 15, 2015 11:18 AM
> To: Sethi Varun-B16395
> Cc: Baptiste Reynal; Linux IOMMU; Yoder Stuart-B08248
> Subject: Re: SMMU 2-stage support
> 
> On Tue, Apr 14, 2015 at 02:32:26PM +0100, Varun Sethi wrote:
> > Hi Will,
> 
> Hey Varun,
> 
> > > -----Original Message-----
> > > From: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org [mailto:iommu-
> > > bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org] On Behalf Of Will Deacon
> > > Sent: Monday, April 13, 2015 4:11 PM
> > > To: Baptiste Reynal
> > > Cc: Linux IOMMU
> > > Subject: Re: SMMU 2-stage support
> > >
> > > On Fri, Apr 03, 2015 at 10:55:02AM +0100, Baptiste Reynal wrote:
> > > > We are eventually working on the vSMMU implementation. Relying on the
> > > > talk Will Deacon gave at the Linux Plumbers IOMMU Microconference on
> > > > October
> > > > 2014 (http://linuxplumbersconf.org/2014/ocw/proposals/2019), I tried
> > > > the vSMMU initialization.
> > >
> > > My position on the vSMMU still hasn't changed:
> > >
> > > > > Anyway, until somebody actually wants this feature I've put it on
> > > > > ice as it adds a whole bunch of complication to the ARM SMMU driver,
> > > > > as well as new user ABI extensions that I don't really want to maintain for
> > > fun.
> > >
> > > So, whilst it's great that you're looking at the code, I'm not very keen on
> > > merging anything until we have people committed to using it. Right now, the
> > > only feedback I've had has been going in the para-virt direction and I don't
> > > think we should do this "for fun".
> > Freescale would be interested in using the vSMMU implementation. We have
> > use cases for assigning devices to guest user space. Are you suggesting
> > that you are more inclined to using the para virtualized approach?
> 
> I can see arguments either way; the vSMMU means that the guest can use the
> same SMMU driver as the host but a para-virtualised approach could
> theoretically be used across multiple SMMU implementations as well as
> potentially being kinder on the TLB.

In the paravirt approach is the guest managing its own stage 1 
page tables (read directly by the hardware SMMU)?

Is there anything written up anywhere sketching out how the paravirt
approach would work?   Are there any limitations vs the vSMMU?

Thanks,
Stuart

Re: SMMU 2-stage support

[Will Deacon]

On Thu, Apr 16, 2015 at 04:46:59PM +0100, Stuart Yoder wrote:
> > > > So, whilst it's great that you're looking at the code, I'm not very keen on
> > > > merging anything until we have people committed to using it. Right now, the
> > > > only feedback I've had has been going in the para-virt direction and I don't
> > > > think we should do this "for fun".
> > > Freescale would be interested in using the vSMMU implementation. We have
> > > use cases for assigning devices to guest user space. Are you suggesting
> > > that you are more inclined to using the para virtualized approach?
> > 
> > I can see arguments either way; the vSMMU means that the guest can use the
> > same SMMU driver as the host but a para-virtualised approach could
> > theoretically be used across multiple SMMU implementations as well as
> > potentially being kinder on the TLB.
> 
> In the paravirt approach is the guest managing its own stage 1 
> page tables (read directly by the hardware SMMU)?

No; ideally the paravirtualised interface would abstract any details of
the hardware, including page table format. So you might have a "map" hvc
call that takes iova, (guest) phys, size, for example. Then the hypervisor
would manually collapse that into the current page tables using iommu_map.

> Is there anything written up anywhere sketching out how the paravirt
> approach would work?   Are there any limitations vs the vSMMU?

Not yet, and that's exactly the kind of research/investigation I'd like to
see happening before we commit ourselves one way or the other.

Will