Re: [PATCH v2 3/6] net: ocelot: pre-compute injection frame header content

[Vladimir Oltean <vladimir.oltean@nxp.com>]

On Mon, Nov 15, 2021 at 11:13:44AM +0100, Clément Léger wrote:
> Le Wed, 3 Nov 2021 14:53:51 +0100,
> Clément Léger <clement.leger@bootlin.com> a écrit :
> 
> > Le Wed, 3 Nov 2021 12:38:12 +0000,
> > Vladimir Oltean <vladimir.oltean@nxp.com> a écrit :
> > 
> > > On Wed, Nov 03, 2021 at 10:19:40AM +0100, Clément Léger wrote:  
> > > > IFH preparation can take quite some time on slow processors (up to
> > > > 5% in a iperf3 test for instance). In order to reduce the cost of
> > > > this preparation, pre-compute IFH since most of the parameters are
> > > > fixed per port. Only rew_op and vlan tag will be set when sending
> > > > if different than 0. This allows to remove entirely the calls to
> > > > packing() with basic usage. In the same time, export this function
> > > > that will be used by FDMA.
> > > > 
> > > > Signed-off-by: Clément Léger <clement.leger@bootlin.com>
> > > > ---    
> > > 
> > > Honestly, this feels a bit cheap/gimmicky, and not really the
> > > fundamental thing to address. In my testing of a similar idea (see
> > > commits 67c2404922c2 ("net: dsa: felix: create a template for the DSA
> > > tags on xmit") and then 7c4bb540e917 ("net: dsa: tag_ocelot: create
> > > separate tagger for Seville"), the net difference is not that stark,
> > > considering that now you need to access one more memory region which
> > > you did not need before, do a memcpy, and then patch the IFH anyway
> > > for the non-constant stuff.  
> > 
> > The memcpy is neglectable and the patching happens only in a few
> > cases (at least vs the packing function call). The VSC7514 CPU is really
> > slow and lead to 2.5% up to 5% time spent in packing() when using iperf3
> > and depending on the use case (according to ftrace).
> > 
> > > 
> > > Certainly, for the calls to ocelot_port_inject_frame() from DSA, I
> > > would prefer not having this pre-computed IFH.
> > > 
> > > Could you provide some before/after performance numbers and perf
> > > counters?  
> > 
> > I will make another round of measure to confirm my previous number and
> > check the impact on the injection rate on ocelot.
> 
> I checked again my bandwith numbers (obtained with iperf3) with and
> without the pre-computed header:
> 
> Test on standard packets with UDP (iperf3 -t 100 -l 1460 -u -b 0 -c *)
> - With pre-computed header: UDP TX: 	33Mbit/s
> - Without UDP TX: 			31Mbit/s
> -> 6.5% improvement
> 
> Test on small packets with UDP (iperf3 -t 100 -l 700 -u -b 0 -c *)
> - With pre-computed header: UDP TX: 	15.8Mbit/s
> - Without UDP TX: 			16.4Mbit/s
> -> 4.3% improvement
> 
> The improvement might not be huge but also not negligible at all.
> Please tell me if you want me to drop it or not based on those numbers.

Is this with manual injection or with FDMA? Do you have before/after
numbers with FDMA as well? At 31 vs 33 Mbps, this isn't going to compete
for any races anyway :)