From: David Gibson <david@gibson.dropbear.id.au>
To: Halil Pasic <pasic@linux.ibm.com>
Cc: linux-s390@vger.kernel.org, Janosch Frank <frankja@linux.ibm.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Cornelia Huck <cohuck@redhat.com>, Ram Pai <linuxram@us.ibm.com>,
linux-kernel@vger.kernel.org,
virtualization@lists.linux-foundation.org,
Christian Borntraeger <borntraeger@de.ibm.com>,
iommu@lists.linux-foundation.org,
Michael Mueller <mimu@linux.ibm.com>,
"Lendacky, Thomas" <Thomas.Lendacky@amd.com>,
Viktor Mihajlovski <mihajlov@linux.ibm.com>,
Robin Murphy <robin.murphy@arm.com>,
Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 0/2] virtio: decouple protected guest RAM form VIRTIO_F_IOMMU_PLATFORM
Date: Mon, 24 Feb 2020 14:38:53 +1100 [thread overview]
Message-ID: <20200224033853.GE1751@umbus.fritz.box> (raw)
In-Reply-To: <20200221155602.4de41fa7.pasic@linux.ibm.com>
[-- Attachment #1.1: Type: text/plain, Size: 4045 bytes --]
On Fri, Feb 21, 2020 at 03:56:02PM +0100, Halil Pasic wrote:
> On Fri, 21 Feb 2020 14:22:26 +0800
> Jason Wang <jasowang@redhat.com> wrote:
>
> >
> > On 2020/2/21 上午12:06, Halil Pasic wrote:
> > > Currently if one intends to run a memory protection enabled VM with
> > > virtio devices and linux as the guest OS, one needs to specify the
> > > VIRTIO_F_IOMMU_PLATFORM flag for each virtio device to make the guest
> > > linux use the DMA API, which in turn handles the memory
> > > encryption/protection stuff if the guest decides to turn itself into
> > > a protected one. This however makes no sense due to multiple reasons:
> > > * The device is not changed by the fact that the guest RAM is
> > > protected. The so called IOMMU bypass quirk is not affected.
> > > * This usage is not congruent with standardised semantics of
> > > VIRTIO_F_IOMMU_PLATFORM. Guest memory protected is an orthogonal reason
> > > for using DMA API in virtio (orthogonal with respect to what is
> > > expressed by VIRTIO_F_IOMMU_PLATFORM).
> > >
> > > This series aims to decouple 'have to use DMA API because my (guest) RAM
> > > is protected' and 'have to use DMA API because the device told me
> > > VIRTIO_F_IOMMU_PLATFORM'.
> > >
> > > Please find more detailed explanations about the conceptual aspects in
> > > the individual patches. There is however also a very practical problem
> > > that is addressed by this series.
> > >
> > > For vhost-net the feature VIRTIO_F_IOMMU_PLATFORM has the following side
> > > effect The vhost code assumes it the addresses on the virtio descriptor
> > > ring are not guest physical addresses but iova's, and insists on doing a
> > > translation of these regardless of what transport is used (e.g. whether
> > > we emulate a PCI or a CCW device). (For details see commit 6b1e6cc7855b
> > > "vhost: new device IOTLB API".) On s390 this results in severe
> > > performance degradation (c.a. factor 10).
> >
> >
> > Do you see a consistent degradation on the performance, or it only
> > happen when for during the beginning of the test?
> >
>
> AFAIK the degradation is consistent.
>
> >
> > > BTW with ccw I/O there is
> > > (architecturally) no IOMMU, so the whole address translation makes no
> > > sense in the context of virtio-ccw.
> >
> >
> > I suspect we can do optimization in qemu side.
> >
> > E.g send memtable entry via IOTLB API when vIOMMU is not enabled.
> >
> > If this makes sense, I can draft patch to see if there's any difference.
>
> Frankly I would prefer to avoid IOVAs on the descriptor ring (and the
> then necessary translation) for virtio-ccw altogether. But Michael
> voiced his opinion that we should mandate F_IOMMU_PLATFORM for devices
> that could be used with guests running in protected mode. I don't share
> his opinion, but that's an ongoing discussion.
I'm a bit confused by this. For the ccw specific case,
F_ACCESS_PLATFORM shouldn't have any impact: for you, IOVA == GPA so
everything is easy.
> Should we end up having to do translation from IOVA in vhost, we are
> very interested in that translation being fast and efficient.
>
> In that sense we would be very happy to test any optimization that aim
> into that direction.
>
> Thank you very much for your input!
>
> Regards,
> Halil
>
> >
> > Thanks
> >
> >
> > >
> > > Halil Pasic (2):
> > > mm: move force_dma_unencrypted() to mem_encrypt.h
> > > virtio: let virtio use DMA API when guest RAM is protected
> > >
> > > drivers/virtio/virtio_ring.c | 3 +++
> > > include/linux/dma-direct.h | 9 ---------
> > > include/linux/mem_encrypt.h | 10 ++++++++++
> > > 3 files changed, 13 insertions(+), 9 deletions(-)
> > >
> > >
> > > base-commit: ca7e1fd1026c5af6a533b4b5447e1d2f153e28f2
> >
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 156 bytes --]
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
WARNING: multiple messages have this Message-ID (diff)
From: David Gibson <david@gibson.dropbear.id.au>
To: Halil Pasic <pasic@linux.ibm.com>
Cc: Jason Wang <jasowang@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Robin Murphy <robin.murphy@arm.com>,
Christoph Hellwig <hch@lst.de>,
linux-s390@vger.kernel.org,
virtualization@lists.linux-foundation.org,
linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
Christian Borntraeger <borntraeger@de.ibm.com>,
Janosch Frank <frankja@linux.ibm.com>,
Viktor Mihajlovski <mihajlov@linux.ibm.com>,
Cornelia Huck <cohuck@redhat.com>, Ram Pai <linuxram@us.ibm.com>,
Thiago Jung Bauermann <bauerman@linux.ibm.com>,
"Lendacky, Thomas" <Thomas.Lendacky@amd.com>,
Michael Mueller <mimu@linux.ibm.com>
Subject: Re: [PATCH 0/2] virtio: decouple protected guest RAM form VIRTIO_F_IOMMU_PLATFORM
Date: Mon, 24 Feb 2020 14:38:53 +1100 [thread overview]
Message-ID: <20200224033853.GE1751@umbus.fritz.box> (raw)
In-Reply-To: <20200221155602.4de41fa7.pasic@linux.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 4045 bytes --]
On Fri, Feb 21, 2020 at 03:56:02PM +0100, Halil Pasic wrote:
> On Fri, 21 Feb 2020 14:22:26 +0800
> Jason Wang <jasowang@redhat.com> wrote:
>
> >
> > On 2020/2/21 上午12:06, Halil Pasic wrote:
> > > Currently if one intends to run a memory protection enabled VM with
> > > virtio devices and linux as the guest OS, one needs to specify the
> > > VIRTIO_F_IOMMU_PLATFORM flag for each virtio device to make the guest
> > > linux use the DMA API, which in turn handles the memory
> > > encryption/protection stuff if the guest decides to turn itself into
> > > a protected one. This however makes no sense due to multiple reasons:
> > > * The device is not changed by the fact that the guest RAM is
> > > protected. The so called IOMMU bypass quirk is not affected.
> > > * This usage is not congruent with standardised semantics of
> > > VIRTIO_F_IOMMU_PLATFORM. Guest memory protected is an orthogonal reason
> > > for using DMA API in virtio (orthogonal with respect to what is
> > > expressed by VIRTIO_F_IOMMU_PLATFORM).
> > >
> > > This series aims to decouple 'have to use DMA API because my (guest) RAM
> > > is protected' and 'have to use DMA API because the device told me
> > > VIRTIO_F_IOMMU_PLATFORM'.
> > >
> > > Please find more detailed explanations about the conceptual aspects in
> > > the individual patches. There is however also a very practical problem
> > > that is addressed by this series.
> > >
> > > For vhost-net the feature VIRTIO_F_IOMMU_PLATFORM has the following side
> > > effect The vhost code assumes it the addresses on the virtio descriptor
> > > ring are not guest physical addresses but iova's, and insists on doing a
> > > translation of these regardless of what transport is used (e.g. whether
> > > we emulate a PCI or a CCW device). (For details see commit 6b1e6cc7855b
> > > "vhost: new device IOTLB API".) On s390 this results in severe
> > > performance degradation (c.a. factor 10).
> >
> >
> > Do you see a consistent degradation on the performance, or it only
> > happen when for during the beginning of the test?
> >
>
> AFAIK the degradation is consistent.
>
> >
> > > BTW with ccw I/O there is
> > > (architecturally) no IOMMU, so the whole address translation makes no
> > > sense in the context of virtio-ccw.
> >
> >
> > I suspect we can do optimization in qemu side.
> >
> > E.g send memtable entry via IOTLB API when vIOMMU is not enabled.
> >
> > If this makes sense, I can draft patch to see if there's any difference.
>
> Frankly I would prefer to avoid IOVAs on the descriptor ring (and the
> then necessary translation) for virtio-ccw altogether. But Michael
> voiced his opinion that we should mandate F_IOMMU_PLATFORM for devices
> that could be used with guests running in protected mode. I don't share
> his opinion, but that's an ongoing discussion.
I'm a bit confused by this. For the ccw specific case,
F_ACCESS_PLATFORM shouldn't have any impact: for you, IOVA == GPA so
everything is easy.
> Should we end up having to do translation from IOVA in vhost, we are
> very interested in that translation being fast and efficient.
>
> In that sense we would be very happy to test any optimization that aim
> into that direction.
>
> Thank you very much for your input!
>
> Regards,
> Halil
>
> >
> > Thanks
> >
> >
> > >
> > > Halil Pasic (2):
> > > mm: move force_dma_unencrypted() to mem_encrypt.h
> > > virtio: let virtio use DMA API when guest RAM is protected
> > >
> > > drivers/virtio/virtio_ring.c | 3 +++
> > > include/linux/dma-direct.h | 9 ---------
> > > include/linux/mem_encrypt.h | 10 ++++++++++
> > > 3 files changed, 13 insertions(+), 9 deletions(-)
> > >
> > >
> > > base-commit: ca7e1fd1026c5af6a533b4b5447e1d2f153e28f2
> >
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2020-02-24 6:44 UTC|newest]
Thread overview: 124+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-20 16:06 [PATCH 0/2] virtio: decouple protected guest RAM form VIRTIO_F_IOMMU_PLATFORM Halil Pasic
2020-02-20 16:06 ` Halil Pasic
2020-02-20 16:06 ` [PATCH 1/2] mm: move force_dma_unencrypted() to mem_encrypt.h Halil Pasic
2020-02-20 16:06 ` Halil Pasic
2020-02-20 16:11 ` Christoph Hellwig
2020-02-20 16:11 ` Christoph Hellwig
2020-02-20 16:23 ` Christian Borntraeger
2020-02-20 16:23 ` Christian Borntraeger
2020-02-20 16:31 ` Christoph Hellwig
2020-02-20 16:31 ` Christoph Hellwig
2020-02-20 16:31 ` Christoph Hellwig
2020-02-20 17:00 ` Christian Borntraeger
2020-02-20 17:00 ` Christian Borntraeger
2020-02-21 3:27 ` David Gibson
2020-02-21 3:27 ` David Gibson
2020-02-21 13:06 ` Halil Pasic
2020-02-21 13:06 ` Halil Pasic
2020-02-21 15:48 ` Michael S. Tsirkin
2020-02-21 15:48 ` Michael S. Tsirkin
2020-02-21 18:07 ` Halil Pasic
2020-02-21 18:07 ` Halil Pasic
2020-02-24 3:33 ` David Gibson
2020-02-24 3:33 ` David Gibson
2020-02-24 18:49 ` Halil Pasic
2020-02-24 18:49 ` Halil Pasic
2020-02-25 18:08 ` Cornelia Huck
2020-02-25 18:08 ` Cornelia Huck
2020-02-28 0:23 ` David Gibson
2020-02-28 0:23 ` David Gibson
2020-02-20 16:06 ` [PATCH 2/2] virtio: let virtio use DMA API when guest RAM is protected Halil Pasic
2020-02-20 16:06 ` Halil Pasic
2020-02-20 16:13 ` Christoph Hellwig
2020-02-20 16:13 ` Christoph Hellwig
2020-02-21 2:59 ` David Gibson
2020-02-21 2:59 ` David Gibson
2020-02-21 3:41 ` Jason Wang
2020-02-21 3:41 ` Jason Wang
2020-02-21 13:31 ` Halil Pasic
2020-02-21 13:31 ` Halil Pasic
2020-02-21 13:27 ` Halil Pasic
2020-02-21 13:27 ` Halil Pasic
2020-02-21 16:36 ` Christoph Hellwig
2020-02-21 16:36 ` Christoph Hellwig
2020-02-24 6:50 ` David Gibson
2020-02-24 6:50 ` David Gibson
2020-02-24 18:59 ` Halil Pasic
2020-02-24 18:59 ` Halil Pasic
2020-02-24 18:59 ` Halil Pasic
2020-02-21 14:33 ` Halil Pasic
2020-02-21 14:33 ` Halil Pasic
2020-02-21 16:39 ` Christoph Hellwig
2020-02-21 16:39 ` Christoph Hellwig
2020-02-21 18:16 ` Halil Pasic
2020-02-21 18:16 ` Halil Pasic
2020-02-21 18:16 ` Halil Pasic
2020-02-22 19:07 ` Michael S. Tsirkin
2020-02-22 19:07 ` Michael S. Tsirkin
2020-02-24 17:16 ` Christoph Hellwig
2020-02-24 17:16 ` Christoph Hellwig
2020-10-28 14:24 ` Alexander Graf via iommu
2020-10-28 18:01 ` Michael S. Tsirkin
2020-10-28 18:01 ` Michael S. Tsirkin
2020-10-28 18:01 ` Michael S. Tsirkin
2020-02-20 20:55 ` Michael S. Tsirkin
2020-02-20 20:55 ` Michael S. Tsirkin
2020-02-21 1:17 ` Ram Pai
2020-02-21 1:17 ` Ram Pai
2020-02-21 1:17 ` Ram Pai
2020-02-21 3:29 ` David Gibson
2020-02-21 3:29 ` David Gibson
2020-02-21 13:12 ` Halil Pasic
2020-02-21 13:12 ` Halil Pasic
2020-02-21 15:39 ` Tom Lendacky
2020-02-21 15:39 ` Tom Lendacky
2020-02-24 6:40 ` David Gibson
2020-02-24 6:40 ` David Gibson
2020-02-24 6:40 ` David Gibson
2020-02-21 15:56 ` Michael S. Tsirkin
2020-02-21 15:56 ` Michael S. Tsirkin
2020-02-21 16:35 ` Christoph Hellwig
2020-02-21 16:35 ` Christoph Hellwig
2020-02-21 18:03 ` Halil Pasic
2020-02-21 18:03 ` Halil Pasic
2020-02-20 20:48 ` [PATCH 0/2] virtio: decouple protected guest RAM form VIRTIO_F_IOMMU_PLATFORM Michael S. Tsirkin
2020-02-20 20:48 ` Michael S. Tsirkin
2020-02-20 21:29 ` Michael S. Tsirkin
2020-02-20 21:29 ` Michael S. Tsirkin
2020-02-21 13:37 ` Halil Pasic
2020-02-21 13:37 ` Halil Pasic
2020-02-20 21:33 ` Michael S. Tsirkin
2020-02-20 21:33 ` Michael S. Tsirkin
2020-02-21 13:49 ` Halil Pasic
2020-02-21 13:49 ` Halil Pasic
2020-02-21 16:41 ` Christoph Hellwig
2020-02-21 16:41 ` Christoph Hellwig
2020-02-24 5:44 ` David Gibson
2020-02-24 5:44 ` David Gibson
2020-02-24 5:44 ` David Gibson
2020-02-21 6:22 ` Jason Wang
2020-02-21 6:22 ` Jason Wang
2020-02-21 14:56 ` Halil Pasic
2020-02-21 14:56 ` Halil Pasic
2020-02-24 3:38 ` David Gibson [this message]
2020-02-24 3:38 ` David Gibson
2020-02-24 4:01 ` Jason Wang
2020-02-24 4:01 ` Jason Wang
2020-02-24 4:01 ` Jason Wang
2020-02-24 6:06 ` Michael S. Tsirkin
2020-02-24 6:06 ` Michael S. Tsirkin
2020-02-24 6:45 ` Jason Wang
2020-02-24 6:45 ` Jason Wang
2020-02-24 7:48 ` Michael S. Tsirkin
2020-02-24 7:48 ` Michael S. Tsirkin
2020-02-24 9:26 ` Jason Wang
2020-02-24 9:26 ` Jason Wang
2020-02-24 13:40 ` Michael S. Tsirkin
2020-02-24 13:40 ` Michael S. Tsirkin
2020-02-25 3:38 ` Jason Wang
2020-02-25 3:38 ` Jason Wang
2020-02-24 13:56 ` Halil Pasic
2020-02-24 13:56 ` Halil Pasic
2020-02-25 3:30 ` Jason Wang
2020-02-25 3:30 ` Jason Wang
2020-02-25 3:30 ` Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200224033853.GE1751@umbus.fritz.box \
--to=david@gibson.dropbear.id.au \
--cc=Thomas.Lendacky@amd.com \
--cc=borntraeger@de.ibm.com \
--cc=cohuck@redhat.com \
--cc=frankja@linux.ibm.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linuxram@us.ibm.com \
--cc=mihajlov@linux.ibm.com \
--cc=mimu@linux.ibm.com \
--cc=mst@redhat.com \
--cc=pasic@linux.ibm.com \
--cc=robin.murphy@arm.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.