* Re: virtio-gpu dedicated heap [not found] <CAAfnVBmDf1fA1ZAufPsPhyOWj0=ynGCVfzX_Cx=pgVmkXe8Tog@mail.gmail.com> @ 2022-03-04 4:07 ` Gurchetan Singh via iommu 2022-03-04 4:56 ` Michael S. Tsirkin [not found] ` <eecdc1a2-ea5b-9d4f-9d58-ba87ffa5044d@arm.com> 1 sibling, 1 reply; 3+ messages in thread From: Gurchetan Singh via iommu @ 2022-03-04 4:07 UTC (permalink / raw) To: virtio-dev, peterz, m.szyprowski, hch, Michael S. Tsirkin, robin.murphy, will, Claire Chang, Tomasz Figa, iommu [-- Attachment #1.1: Type: text/plain, Size: 2462 bytes --] +iommu@lists.linux-foundation.org not iommu-request On Thu, Mar 3, 2022 at 8:05 PM Gurchetan Singh <gurchetansingh@chromium.org> wrote: > Hi everyone, > > With the current virtio setup, all of guest memory is shared with host > devices. There has been interest in changing this, to improve isolation of > guest memory and increase confidentiality. > > The recently introduced restricted DMA mechanism makes excellent progress > in this area: > > > https://patchwork.kernel.org/project/xen-devel/cover/20210624155526.2775863-1-tientzu@chromium.org/ > > > Devices without an IOMMU (traditional virtio devices for example) would > allocate from a specially designated region. Swiotlb bouncing is done for > all DMA transfers. This is controlled by the VIRTIO_F_ACCESS_PLATFORM > feature bit. > > > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3064198 > > This mechanism works great for the devices it was designed for, such as > virtio-net. However, when trying to adapt to it for other devices, there > are some limitations. > > It would be great to have a dedicated heap for virtio-gpu rather than > allocating from guest memory. > > We would like to use dma_alloc_noncontiguous on the restricted dma pool, > ideally with page-level granularity somehow. Continuous buffers are > definitely going out of fashion. > > There are two considerations when using it with the restricted DMA > approach: > > 1) No bouncing (aka memcpy) > > Expensive with graphics buffers, since guest user space would designate > shareable graphics buffers with the host. We plan to use > DMA_ATTR_SKIP_CPU_SYNC when doing any DMA transactions with GPU buffers. > > Bounce buffering will be utilized with virtio-cmds, like the other virtio > devices that use the restricted DMA mechanism. > > 2) IO_TLB_SEGSIZE is too small for graphics buffers > > This issue was hit before here too: > > https://www.spinics.net/lists/kernel/msg4154086.html > > The suggestion was to use shared-dma-pool rather than restricted DMA. But > we're not sure a single device can have restricted DMA (for > VIRTIO_F_ACCESS_PLATFORM) and shared-dma-pool (for larger buffers) at the > same time. Does anyone know? > > If not, it sounds like "splitting the allocation into > dma_max_mapping_size() chunks" for restricted-dma is also possible. What > is the preferred method? > > More generally, we would love more feedback on the proposed design or > consider alternatives! > [-- Attachment #1.2: Type: text/html, Size: 4356 bytes --] [-- Attachment #2: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: virtio-gpu dedicated heap 2022-03-04 4:07 ` virtio-gpu dedicated heap Gurchetan Singh via iommu @ 2022-03-04 4:56 ` Michael S. Tsirkin 0 siblings, 0 replies; 3+ messages in thread From: Michael S. Tsirkin @ 2022-03-04 4:56 UTC (permalink / raw) To: Gurchetan Singh Cc: virtio-dev, Claire Chang, peterz, will, Tomasz Figa, iommu, robin.murphy, hch $ ./scripts/get_maintainer.pl -f ./drivers/gpu/drm/virtio/ David Airlie <airlied@linux.ie> (maintainer:VIRTIO GPU DRIVER) Gerd Hoffmann <kraxel@redhat.com> (maintainer:VIRTIO GPU DRIVER) Daniel Vetter <daniel@ffwll.ch> (maintainer:DRM DRIVERS) dri-devel@lists.freedesktop.org (open list:VIRTIO GPU DRIVER) virtualization@lists.linux-foundation.org (open list:VIRTIO GPU DRIVER) linux-kernel@vger.kernel.org (open list) You might want to CC these people. On Thu, Mar 03, 2022 at 08:07:03PM -0800, Gurchetan Singh wrote: > +iommu@lists.linux-foundation.org not iommu-request > > On Thu, Mar 3, 2022 at 8:05 PM Gurchetan Singh <gurchetansingh@chromium.org> > wrote: > > Hi everyone, > > With the current virtio setup, all of guest memory is shared with host > devices. There has been interest in changing this, to improve isolation of > guest memory and increase confidentiality. > > The recently introduced restricted DMA mechanism makes excellent progress > in this area: > > https://patchwork.kernel.org/project/xen-devel/cover/ > 20210624155526.2775863-1-tientzu@chromium.org/ > > Devices without an IOMMU (traditional virtio devices for example) would > allocate from a specially designated region. Swiotlb bouncing is done for > all DMA transfers. This is controlled by the VIRTIO_F_ACCESS_PLATFORM > feature bit. > > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/ > 3064198 > > This mechanism works great for the devices it was designed for, such as > virtio-net. However, when trying to adapt to it for other devices, there > are some limitations. > > It would be great to have a dedicated heap for virtio-gpu rather than > allocating from guest memory. > > We would like to use dma_alloc_noncontiguous on the restricted dma pool, > ideally with page-level granularity somehow. Continuous buffers are > definitely going out of fashion. > > There are two considerations when using it with the restricted DMA > approach: > > 1) No bouncing (aka memcpy) > > Expensive with graphics buffers, since guest user space would designate > shareable graphics buffers with the host. We plan to use > DMA_ATTR_SKIP_CPU_SYNC when doing any DMA transactions with GPU buffers. > > Bounce buffering will be utilized with virtio-cmds, like the other virtio > devices that use the restricted DMA mechanism. > > 2) IO_TLB_SEGSIZE is too small for graphics buffers > > This issue was hit before here too: > > https://www.spinics.net/lists/kernel/msg4154086.html > > The suggestion was to use shared-dma-pool rather than restricted DMA. But > we're not sure a single device can have restricted DMA (for > VIRTIO_F_ACCESS_PLATFORM) and shared-dma-pool (for larger buffers) at the > same time. Does anyone know? > > If not, it sounds like "splitting the allocation into dma_max_mapping_size > () chunks" for restricted-dma is also possible. What is the preferred > method? > > More generally, we would love more feedback on the proposed design or > consider alternatives! > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <eecdc1a2-ea5b-9d4f-9d58-ba87ffa5044d@arm.com>]
* Re: virtio-gpu dedicated heap [not found] ` <eecdc1a2-ea5b-9d4f-9d58-ba87ffa5044d@arm.com> @ 2022-03-04 18:36 ` Gurchetan Singh 0 siblings, 0 replies; 3+ messages in thread From: Gurchetan Singh @ 2022-03-04 18:36 UTC (permalink / raw) To: Robin Murphy Cc: virtio-dev, Claire Chang, Michael S. Tsirkin, peterz, Tomasz Figa, iommu, will, hch [-- Attachment #1.1: Type: text/plain, Size: 4214 bytes --] On Fri, Mar 4, 2022 at 4:53 AM Robin Murphy <robin.murphy@arm.com> wrote: > On 2022-03-04 04:05, Gurchetan Singh wrote: > > Hi everyone, > > > > With the current virtio setup, all of guest memory is shared with host > > devices. There has been interest in changing this, to improve isolation > of > > guest memory and increase confidentiality. > > > > The recently introduced restricted DMA mechanism makes excellent progress > > in this area: > > > > > https://patchwork.kernel.org/project/xen-devel/cover/20210624155526.2775863-1-tientzu@chromium.org/ > > > > > > Devices without an IOMMU (traditional virtio devices for example) would > > allocate from a specially designated region. Swiotlb bouncing is done > for > > all DMA transfers. This is controlled by the VIRTIO_F_ACCESS_PLATFORM > > feature bit. > > > > > https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3064198 > > > > This mechanism works great for the devices it was designed for, such as > > virtio-net. However, when trying to adapt to it for other devices, there > > are some limitations. > > > > It would be great to have a dedicated heap for virtio-gpu rather than > > allocating from guest memory. > > > > We would like to use dma_alloc_noncontiguous on the restricted dma pool, > > ideally with page-level granularity somehow. Continuous buffers are > > definitely going out of fashion. > > > > There are two considerations when using it with the restricted DMA > approach: > > > > 1) No bouncing (aka memcpy) > > > > Expensive with graphics buffers, since guest user space would designate > > shareable graphics buffers with the host. We plan to use > > DMA_ATTR_SKIP_CPU_SYNC when doing any DMA transactions with GPU buffers. > > > > Bounce buffering will be utilized with virtio-cmds, like the other virtio > > devices that use the restricted DMA mechanism. > > > > 2) IO_TLB_SEGSIZE is too small for graphics buffers > > > > This issue was hit before here too: > > > > https://www.spinics.net/lists/kernel/msg4154086.html > > > > The suggestion was to use shared-dma-pool rather than restricted DMA. > But > > we're not sure a single device can have restricted DMA (for > > VIRTIO_F_ACCESS_PLATFORM) and shared-dma-pool (for larger buffers) at the > > same time. Does anyone know? > > Yes, it is absolutely intended that a device can have both a > "restricted-dma-pool" for bouncing data from e.g. user pages, and a > "shared-dma-pool" from which to allocate dedicated buffers. The > "restricted-dma-pool" binding even explicitly calls this case out. > > As long as the "shared-dma-pool" is suitably sized, it shouldn't make > much difference to Linux whether it's a simple system memory carveout or > a special hardware/hypervisor-isolated region. The only real > considerations for the latter case are firstly that you probably want to > make sure it is used as a coherent pool rather than a CMA pool (i.e. > omit the "reusable" property), since if the guest exposes private data > in shared pages that aren't currently in use for DMA then it rather > defeats the point, and secondly that if allocating from the pool fails, > Linux will currently fall back to allocating from regular protected > memory, which is liable to end badly. There's certainly potential to > improve on the latter point, but the short-term easy dodge is just to > make the pool big enough that normal operation won't exhaust it. > Thanks for the feedback! Will experiment with the mixed "shared-dma-pool" + "restricted-dma" approach for virtio-gpu. > > > If not, it sounds like "splitting the allocation into > > dma_max_mapping_size() chunks" for restricted-dma is also possible. What > > is the preferred method? > > > > More generally, we would love more feedback on the proposed design or > > consider alternatives! > > Another alternative, if the protection mechanism allows, is to hook into > the set_memory_(de,en)crypted() APIs to dynamically share buffers as > they are allocated instead of using a static pool. This should mostly > work as-is - I think the only impediment is pgprot_decrypted, which > would need some kind of hook in vmap() to detect it and make the > corresponding hypervisor call. > > Robin. > [-- Attachment #1.2: Type: text/html, Size: 5635 bytes --] [-- Attachment #2: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-03-04 18:36 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAAfnVBmDf1fA1ZAufPsPhyOWj0=ynGCVfzX_Cx=pgVmkXe8Tog@mail.gmail.com>
2022-03-04 4:07 ` virtio-gpu dedicated heap Gurchetan Singh via iommu
2022-03-04 4:56 ` Michael S. Tsirkin
[not found] ` <eecdc1a2-ea5b-9d4f-9d58-ba87ffa5044d@arm.com>
2022-03-04 18:36 ` Gurchetan Singh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox