[RFC] ethtool bandwidth per-queue commands

[RFC] ethtool bandwidth per-queue commands

[ivan]

Hi everyone.

The ethtool interface has ETHTOOL_PERQUEUE command that is used to
read/set per-queue parameters for multiqueue Ethernet devices.

I want to apply additional per-queue commands like per-queue bandwidth
and per-queue number of descriptors (like number of rings).

Currently only ETHTOOL_G/SCOALESCE sub commands are added for
per-queue interface (but I haven't seen counterpart for ethtool util).

Using ETHTOOL_G/SCOALESCE per-queue command as an example,
I'm going to add 2 additional and 2 known commands for ethtool
per-queue interface:

ETHTOOL_G/SBANDWIDTH     /* Get/Set ethtool per queue bandwidth.
Bandwidth can be set with % or absolute in Mb/s */

ETHTOOL_G/SRINGPARAM     /* is going to be used to control number of
descriptors per queue in my driver */

It's not clear how PERQUEUE command splits rx and tx queues, as
they can have different enumeration and different parameters.

Actually, there are several questions:
- how to split enumeration of rx / tx queues in per queue command,
   as it uses linear bitmap to mirror affected channels.
- where is ethtool util counterpart for per queue command?
- is it correct way to use ethtool to configure h/w shaper?

In my case, by default, multi queue h/w shaper is working in
priority mode. The setting of bandwidth for queues can be used
to switch shaper to throughput aware mode when every queue can be
bandwidth limited.

-- 
Regards,
Ivan Khoronzhuk

RE: [RFC] ethtool bandwidth per-queue commands

[Liang, Kan]

> 
> Hi everyone.
> 
> The ethtool interface has ETHTOOL_PERQUEUE command that is used to
> read/set per-queue parameters for multiqueue Ethernet devices.
> 
> I want to apply additional per-queue commands like per-queue bandwidth
> and per-queue number of descriptors (like number of rings).
> 
> Currently only ETHTOOL_G/SCOALESCE sub commands are added for per-
> queue interface (but I haven't seen counterpart for ethtool util).
> 
> Using ETHTOOL_G/SCOALESCE per-queue command as an example, I'm
> going to add 2 additional and 2 known commands for ethtool per-queue
> interface:
> 
> ETHTOOL_G/SBANDWIDTH     /* Get/Set ethtool per queue bandwidth.
> Bandwidth can be set with % or absolute in Mb/s */
> 
> ETHTOOL_G/SRINGPARAM     /* is going to be used to control number of
> descriptors per queue in my driver */
> 
> It's not clear how PERQUEUE command splits rx and tx queues, as they can
> have different enumeration and different parameters.
> 
> Actually, there are several questions:
> - how to split enumeration of rx / tx queues in per queue command,
>    as it uses linear bitmap to mirror affected channels.
> - where is ethtool util counterpart for per queue command?
> - is it correct way to use ethtool to configure h/w shaper?
> 

I once submitted a patch series for ethtool to do per-queue setting.
https://www.mail-archive.com/netdev%40vger.kernel.org/msg110727.html
But it's not merged. Because I didn't have a chance to address Ben's
comments yet.
https://www.mail-archive.com/netdev%40vger.kernel.org/msg116630.html

I guess you may extend the ethtool patch series, and use it to verify your
kernel changes.

Kan


> In my case, by default, multi queue h/w shaper is working in priority mode.
> The setting of bandwidth for queues can be used to switch shaper to
> throughput aware mode when every queue can be bandwidth limited.
> 
> --
> Regards,
> Ivan Khoronzhuk

Re: [RFC] ethtool bandwidth per-queue commands

[John Fastabend]

On 16-07-20 01:00 PM, ivan wrote:
> Hi everyone.
> 
> The ethtool interface has ETHTOOL_PERQUEUE command that is used to
> read/set per-queue parameters for multiqueue Ethernet devices.
> 
> I want to apply additional per-queue commands like per-queue bandwidth
> and per-queue number of descriptors (like number of rings).
> 
> Currently only ETHTOOL_G/SCOALESCE sub commands are added for
> per-queue interface (but I haven't seen counterpart for ethtool util).
> 
> Using ETHTOOL_G/SCOALESCE per-queue command as an example,
> I'm going to add 2 additional and 2 known commands for ethtool
> per-queue interface:
> 
> ETHTOOL_G/SBANDWIDTH     /* Get/Set ethtool per queue bandwidth.
> Bandwidth can be set with % or absolute in Mb/s */
> 

For tx rate limiting we have an ndo op for this already,

   ndo_set_tx_maxrate

which can be tuned via sysfs at the moment. It would logically make
sense to add an rx rate limit if you are looking for something more
like a policer. Or do this via setup_tc() interface as we do with
mqprio qdisc today.

If you are interested in max and min rates per queue groups one such
possibility being a queue group of one you may find the ./net/dcb
stack interesting as well. Although the DCB spec has lots of bits
about protocols and such the kernel interfaces do not care if the
protocol is running at all. Unfortunately when this was done originally
we didn't make this as clear as we should have.

.John