From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Hellwig Subject: Re: [PATCH 22/28] sgiseeq: convert from dma_cache_sync to dma_sync_single_for_device Date: Tue, 1 Sep 2020 19:16:27 +0200 Message-ID: <20200901171627.GA8255@lst.de> References: <20200819065555.1802761-1-hch@lst.de> <20200819065555.1802761-23-hch@lst.de> <20200901152209.GA14288@alpha.franken.de> <20200901171241.GA20685@alpha.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20200901171241.GA20685-I1c7kopa9pxLokYuJOExCg@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Sender: "iommu" To: Thomas Bogendoerfer Cc: alsa-devel-K7yf7f+aM1XWsZ/bQMPhNw@public.gmane.org, linux-ia64-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-mips-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "James E.J. Bottomley" , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Christoph Hellwig , linux-samsung-soc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Joonyoung Shim , linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Ben Skeggs , Matt Porter , linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Tom Lendacky , Pawel Osciak , Mauro Carvalho Chehab , linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-parisc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Seung-Woo Kim , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Kyungmin Park List-Id: nouveau.vger.kernel.org On Tue, Sep 01, 2020 at 07:12:41PM +0200, Thomas Bogendoerfer wrote: > On Tue, Sep 01, 2020 at 05:22:09PM +0200, Thomas Bogendoerfer wrote: > > On Wed, Aug 19, 2020 at 08:55:49AM +0200, Christoph Hellwig wrote: > > > Use the proper modern API to transfer cache ownership for incoherent DMA. > > > > > > Signed-off-by: Christoph Hellwig > > > --- > > > drivers/net/ethernet/seeq/sgiseeq.c | 12 ++++++++---- > > > 1 file changed, 8 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/net/ethernet/seeq/sgiseeq.c b/drivers/net/ethernet/seeq/sgiseeq.c > > > index 39599bbb5d45b6..f91dae16d69a19 100644 > > > --- a/drivers/net/ethernet/seeq/sgiseeq.c > > > +++ b/drivers/net/ethernet/seeq/sgiseeq.c > > > @@ -112,14 +112,18 @@ struct sgiseeq_private { > > > > > > static inline void dma_sync_desc_cpu(struct net_device *dev, void *addr) > > > { > > > - dma_cache_sync(dev->dev.parent, addr, sizeof(struct sgiseeq_rx_desc), > > > - DMA_FROM_DEVICE); > > > + struct sgiseeq_private *sp = netdev_priv(dev); > > > + > > > + dma_sync_single_for_cpu(dev->dev.parent, VIRT_TO_DMA(sp, addr), > > > + sizeof(struct sgiseeq_rx_desc), DMA_BIDIRECTIONAL); > > > } > > > > > > static inline void dma_sync_desc_dev(struct net_device *dev, void *addr) > > > { > > > - dma_cache_sync(dev->dev.parent, addr, sizeof(struct sgiseeq_rx_desc), > > > - DMA_TO_DEVICE); > > > + struct sgiseeq_private *sp = netdev_priv(dev); > > > + > > > + dma_sync_single_for_device(dev->dev.parent, VIRT_TO_DMA(sp, addr), > > > + sizeof(struct sgiseeq_rx_desc), DMA_BIDIRECTIONAL); > > > } > > > > this breaks ethernet on IP22 completely, but I haven't figured out why, yet. > > the problem is that dma_sync_single_for_cpu() doesn't flush anything > for IP22, because it only flushes for CPUs which do speculation. So > either MIPS arch_sync_dma_for_cpu() should always flush or sgiseeq > needs to use a different sync funktion, when it wants to re-read descriptors > from memory. Well, if IP22 doesn't speculate (which I'm pretty sure is the case), dma_sync_single_for_cpu should indeeed be a no-op. But then there also shouldn't be anything in the cache, as the previous dma_sync_single_for_device should have invalidated it. So it seems like we are missing one (or more) ownership transfers to the device. I'll try to look at the the ownership management in a little more detail tomorrow.