Re: [PATCH] examples/l3fwd: add option to set refetch offset

[Stephen Hemminger <stephen@networkplumber.org>]

On Tue, 14 Jan 2025 17:22:08 +0800
huangdengdui <huangdengdui@huawei.com> wrote:

> On 2025/1/11 1:20, Stephen Hemminger wrote:
> > This will make it slower for many platforms.
> > GCC will unroll a loop of fixed small size, which is what we want.  
> 
> Do you mean to replace option with a macro?
> But most of prefetch_offset are used with the nb_rx, So using macros is the same as using options.
> 
> const int32_t k = RTE_ALIGN_FLOOR(nb_rx, FWDSTEP);
> for (j = 0; j != k; j += FWDSTEP) {
> 	for (i = 0, pos = j + prefetch_offset;
> 	     i < FWDSTEP && pos < k; i++, pos++)
> 		rte_prefetch0(rte_pktmbuf_mtod(pkts_burst[pos], void *));
> 	processx4_step1(&pkts_burst[j], &dip, &ipv4_flag);
> 	processx4_step2(qconf, dip, ipv4_flag, portid,
> 			&pkts_burst[j], &dst_port[j]);
> 	if (do_step3)
> 		processx4_step3(&pkts_burst[j], &dst_port[j]);
> }
> 
> The option can dynamically adjust the prefetch window, which makes it easier to find the prefetch window for a HW platform.
> So I think it's better to use option.

The tradeoff is that loop unrolling most often is only done on small fix sized loops.
And the cost of a loop with variable small values (branch prediction) is high enough that it 
could make things slower.

Prefetching is a balancing act, and more is not better especially on real workloads.