Re: [PATCH v4 net-next 2/2] net: introduce budget_squeeze to help us tune rx behavior

[Jakub Kicinski <kuba@kernel.org>]

On Wed, 15 Mar 2023 17:20:41 +0800 Jason Xing wrote:
> In our production environment, there're hundreds of machines hitting the
> old time_squeeze limit often from which we cannot tell what exactly causes
> such issues. Hitting limits aranged from 400 to 2000 times per second,
> Especially, when users are running on the guest OS with veth policy
> configured, it is relatively easier to hit the limit. After several tries
> without this patch, I found it is only real time_squeeze not including
> budget_squeeze that hinders the receive process.

That is the common case, and can be understood from the napi trace
point and probing the kernel with bpftrace. We should only add
uAPI for statistics which must be maintained contiguously. For
investigations tracing will always be orders of magnitude more
powerful :(

On the time squeeze BTW, have you found out what the problem was?
In workloads I've seen the time problems are often because of noise 
in how jiffies are accounted (cgroup code disables interrupts
for long periods of time, for example, making jiffies increment 
by 2, 3 or 4 rather than by 1).

> So when we encounter some related performance issue and then get lost on
> how to tune the budget limit and time limit in net_rx_action() function,
> we can separately counting both of them to avoid the confusion.