Re: [PATCH v4 07/17] octeontx2-pf: Add packet transmission support

[Jakub Kicinski <kuba@kernel.org>]

On Thu, 23 Jan 2020 01:20:51 +0530, Sunil Kovvuri wrote:
> > Why do you care about big endian bitfields tho, if you don't care about
> > endianness of the data itself?  
> 
> At this point of time we are not addressing big endian functionality,
> so few things
> might be broken in that aspect. If it's preferred to remove, i will remove it.

Yes, I think removing it would be cleaner than having it partially
working. The driver depends on ARM64, anyway.

> > > +};
> > > +
> > >  #endif /* OTX2_STRUCT_H */
> > > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> > > index e6be18d..f416603 100644
> > > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> > > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> > > @@ -32,6 +32,78 @@ static struct nix_cqe_hdr_s *otx2_get_next_cqe(struct otx2_cq_queue *cq)
> > >       return cqe_hdr;
> > >  }
> > >  
> >> > +static void otx2_sqe_flush(struct otx2_snd_queue *sq, int size)  
> > > +{
> > > +     u64 status;
> > > +
> > > +     /* Packet data stores should finish before SQE is flushed to HW */  
> >
> > Packet data is synced by the dma operations the barrier shouldn't be
> > needed AFAIK (and if it would be, dma_wmb() would not be the one, as it
> > only works for iomem AFAIU).
> >  
> > > +     dma_wmb();  
> 
> Due to out of order execution by CPU, HW folks have suggested add a barrier
> to avoid scenarios where packet is transmitted before all stores from
> CPU are committed.
> On arm64 a dmb() is less costlier than a dsb() barrier and as per HW
> folks a dmb(st)
> is sufficient to ensure all stores from CPU are committed. And
> dma_wmb() uses dmb(st)
> hence it is used. It's more of choice of architecture specific
> instruction rather than the API.

Hmm.. I'm not a DMA API expert but AFAIU it'd be a serious bug if
dma_map..() APIs didn't guarantee that packet data is written out.
Not only out of the CPU store buffer but also the caches to DRAM.