From: Jason Gunthorpe via iommu <iommu@lists.linux-foundation.org>
To: Barry Song <21cnbao@gmail.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
Dave Jiang <dave.jiang@intel.com>,
Raj Ashok <ashok.raj@intel.com>,
"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
LKML <linux-kernel@vger.kernel.org>,
Christoph Hellwig <hch@infradead.org>,
iommu@lists.linux-foundation.org, Tony Luck <tony.luck@intel.com>,
mike.campin@intel.com
Subject: Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA
Date: Thu, 7 Oct 2021 08:59:18 -0300 [thread overview]
Message-ID: <20211007115918.GH2744544@nvidia.com> (raw)
In-Reply-To: <CAGsJ_4x2UEmNXCVhJAVRyB8568VMrTkOLeVCNp8CyP6xZJwCig@mail.gmail.com>
On Fri, Oct 08, 2021 at 12:54:52AM +1300, Barry Song wrote:
> On Fri, Oct 8, 2021 at 12:32 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Thu, Oct 07, 2021 at 06:43:33PM +1300, Barry Song wrote:
> >
> > > So do we have a case where devices can directly access the kernel's data
> > > structure such as a list/graph/tree with pointers to a kernel virtual address?
> > > then devices don't need to translate the address of pointers in a structure.
> > > I assume this is one of the most useful features userspace SVA can provide.
> >
> > AFIACT that is the only good case for KVA, but it is also completely
> > against the endianess, word size and DMA portability design of the
> > kernel.
> >
> > Going there requires some new set of portable APIs for gobally
> > coherent KVA dma.
>
> yep. I agree. it would be very weird if accelerators/gpu are sharing
> kernel' data struct, but for each "DMA" operation - reading or writing
> the data struct, we have to call dma_map_single/sg or
> dma_sync_single_for_cpu/device etc. It seems once devices and cpus
> are sharing virtual address(SVA), code doesn't need to do explicit
> map/sync each time.
No, it still need to do something to manage visibility from the
current CPU to the DMA - it might not be flushing a cache, but it is
probably a arch specific CPU barrier instruction.
Jason
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@nvidia.com>
To: Barry Song <21cnbao@gmail.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>,
iommu@lists.linux-foundation.org,
LKML <linux-kernel@vger.kernel.org>,
Joerg Roedel <joro@8bytes.org>,
Christoph Hellwig <hch@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Raj Ashok <ashok.raj@intel.com>,
"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
Dave Jiang <dave.jiang@intel.com>,
Tony Luck <tony.luck@intel.com>,
mike.campin@intel.com, Yi Liu <yi.l.liu@intel.com>,
"Tian, Kevin" <kevin.tian@intel.com>
Subject: Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA
Date: Thu, 7 Oct 2021 08:59:18 -0300 [thread overview]
Message-ID: <20211007115918.GH2744544@nvidia.com> (raw)
In-Reply-To: <CAGsJ_4x2UEmNXCVhJAVRyB8568VMrTkOLeVCNp8CyP6xZJwCig@mail.gmail.com>
On Fri, Oct 08, 2021 at 12:54:52AM +1300, Barry Song wrote:
> On Fri, Oct 8, 2021 at 12:32 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Thu, Oct 07, 2021 at 06:43:33PM +1300, Barry Song wrote:
> >
> > > So do we have a case where devices can directly access the kernel's data
> > > structure such as a list/graph/tree with pointers to a kernel virtual address?
> > > then devices don't need to translate the address of pointers in a structure.
> > > I assume this is one of the most useful features userspace SVA can provide.
> >
> > AFIACT that is the only good case for KVA, but it is also completely
> > against the endianess, word size and DMA portability design of the
> > kernel.
> >
> > Going there requires some new set of portable APIs for gobally
> > coherent KVA dma.
>
> yep. I agree. it would be very weird if accelerators/gpu are sharing
> kernel' data struct, but for each "DMA" operation - reading or writing
> the data struct, we have to call dma_map_single/sg or
> dma_sync_single_for_cpu/device etc. It seems once devices and cpus
> are sharing virtual address(SVA), code doesn't need to do explicit
> map/sync each time.
No, it still need to do something to manage visibility from the
current CPU to the DMA - it might not be flushing a cache, but it is
probably a arch specific CPU barrier instruction.
Jason
next prev parent reply other threads:[~2021-10-07 11:59 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-21 20:29 [RFC 0/7] Support in-kernel DMA with PASID and SVA Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 1/7] ioasid: reserve special PASID for in-kernel DMA Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 2/7] dma-iommu: Add API for DMA request with PASID Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 3/7] iommu/vt-d: Add DMA w/ PASID support for PA and IOVA Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 4/7] dma-iommu: Add support for DMA w/ PASID in KVA Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 5/7] iommu/vt-d: Add support for KVA PASID mode Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 6/7] iommu: Add KVA map API Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-21 20:29 ` [RFC 7/7] dma/idxd: Use dma-iommu PASID API instead of SVA lib Jacob Pan
2021-09-21 20:29 ` Jacob Pan
2021-09-22 17:04 ` [RFC 0/7] Support in-kernel DMA with PASID and SVA Jason Gunthorpe via iommu
2021-09-22 17:04 ` Jason Gunthorpe
2021-09-29 19:37 ` Jacob Pan
2021-09-29 19:37 ` Jacob Pan
2021-09-29 19:39 ` Jason Gunthorpe via iommu
2021-09-29 19:39 ` Jason Gunthorpe
2021-09-29 22:57 ` Jacob Pan
2021-09-29 22:57 ` Jacob Pan
2021-09-29 23:43 ` Jason Gunthorpe via iommu
2021-09-29 23:43 ` Jason Gunthorpe
2021-09-30 14:22 ` Campin, Mike
2021-09-30 14:22 ` Campin, Mike
2021-09-30 15:21 ` Jacob Pan
2021-09-30 15:21 ` Jacob Pan
2021-10-01 12:24 ` Barry Song
2021-10-01 12:24 ` Barry Song
2021-10-01 12:36 ` Jason Gunthorpe via iommu
2021-10-01 12:36 ` Jason Gunthorpe
2021-10-01 12:45 ` Barry Song
2021-10-01 12:45 ` Barry Song
2021-10-04 16:40 ` Jacob Pan
2021-10-04 16:40 ` Jacob Pan
2021-10-04 18:21 ` Jason Gunthorpe via iommu
2021-10-04 18:21 ` Jason Gunthorpe
2021-10-07 5:43 ` Barry Song
2021-10-07 5:43 ` Barry Song
2021-10-07 11:32 ` Jason Gunthorpe via iommu
2021-10-07 11:32 ` Jason Gunthorpe
2021-10-07 11:54 ` Barry Song
2021-10-07 11:54 ` Barry Song
2021-10-07 11:59 ` Jason Gunthorpe via iommu [this message]
2021-10-07 11:59 ` Jason Gunthorpe
2021-10-07 17:50 ` Jacob Pan
2021-10-07 17:50 ` Jacob Pan
2021-10-07 17:48 ` Jason Gunthorpe via iommu
2021-10-07 17:48 ` Jason Gunthorpe
2021-10-07 18:08 ` Jacob Pan
2021-10-07 18:08 ` Jacob Pan
2021-10-07 19:11 ` Jacob Pan
2021-10-07 19:11 ` Jacob Pan
2021-10-07 19:10 ` Jason Gunthorpe via iommu
2021-10-07 19:10 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211007115918.GH2744544@nvidia.com \
--to=iommu@lists.linux-foundation.org \
--cc=21cnbao@gmail.com \
--cc=ashok.raj@intel.com \
--cc=dave.jiang@intel.com \
--cc=hch@infradead.org \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mike.campin@intel.com \
--cc=sanjay.k.kumar@intel.com \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.