From: Jakub Kicinski <kuba@kernel.org>
To: Joe Damato <jdamato@fastly.com>
Cc: netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org,
davem@davemloft.net
Subject: Re: [Intel-wired-lan] [net-queue bugfix RFC] i40e: Clear IFF_RXFH_CONFIGURED when RSS is reset
Date: Mon, 17 Oct 2022 12:45:55 -0700 [thread overview]
Message-ID: <20221017124555.5d79d3f7@kernel.org> (raw)
In-Reply-To: <1665701671-6353-1-git-send-email-jdamato@fastly.com>
On Thu, 13 Oct 2022 15:54:31 -0700 Joe Damato wrote:
> Before this change, reconfiguring the queue count using ethtool doesn't
> always work, even for queue counts that were previously accepted because
> the IFF_RXFH_CONFIGURED bit was not cleared when the flow indirection hash
> is cleared by the driver.
It's not cleared but when was it set? Could you describe the flow that
gets us to this set a bit more?
Normally clearing the IFF_RXFH_CONFIGURED in the driver is _only_
acceptable on error recovery paths, and should come with a "this should
never happen" warning.
> For example:
>
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 34 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 20 21 22 23
> 24: 24 25 26 27 28 29 30 31
> 32: 32 33 0 1 2 3 4 5
> [...snip...]
>
> As you can see, the flow indirection hash distributes flows to 34 queues.
>
> Increasing the number of queues from 34 to 64 works, and the flow
> indirection hash is reset automatically:
>
> $ sudo ethtool -L eth0 combined 64
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 64 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 20 21 22 23
> 24: 24 25 26 27 28 29 30 31
> 32: 32 33 34 35 36 37 38 39
> 40: 40 41 42 43 44 45 46 47
> 48: 48 49 50 51 52 53 54 55
> 56: 56 57 58 59 60 61 62 63
This is odd, if IFF_RXFH_CONFIGURED is set driver should not
re-initialize the indirection table. Which I believe is what
you describe at the end of your message:
> But, I can increase the queue count and the flow hash is preserved:
>
> $ sudo ethtool -L eth0 combined 64
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 64 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 0 1 2 3
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
WARNING: multiple messages have this Message-ID (diff)
From: Jakub Kicinski <kuba@kernel.org>
To: Joe Damato <jdamato@fastly.com>
Cc: intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org,
davem@davemloft.net, anthony.l.nguyen@intel.com,
jesse.brandeburg@intel.com
Subject: Re: [net-queue bugfix RFC] i40e: Clear IFF_RXFH_CONFIGURED when RSS is reset
Date: Mon, 17 Oct 2022 12:45:55 -0700 [thread overview]
Message-ID: <20221017124555.5d79d3f7@kernel.org> (raw)
In-Reply-To: <1665701671-6353-1-git-send-email-jdamato@fastly.com>
On Thu, 13 Oct 2022 15:54:31 -0700 Joe Damato wrote:
> Before this change, reconfiguring the queue count using ethtool doesn't
> always work, even for queue counts that were previously accepted because
> the IFF_RXFH_CONFIGURED bit was not cleared when the flow indirection hash
> is cleared by the driver.
It's not cleared but when was it set? Could you describe the flow that
gets us to this set a bit more?
Normally clearing the IFF_RXFH_CONFIGURED in the driver is _only_
acceptable on error recovery paths, and should come with a "this should
never happen" warning.
> For example:
>
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 34 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 20 21 22 23
> 24: 24 25 26 27 28 29 30 31
> 32: 32 33 0 1 2 3 4 5
> [...snip...]
>
> As you can see, the flow indirection hash distributes flows to 34 queues.
>
> Increasing the number of queues from 34 to 64 works, and the flow
> indirection hash is reset automatically:
>
> $ sudo ethtool -L eth0 combined 64
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 64 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 20 21 22 23
> 24: 24 25 26 27 28 29 30 31
> 32: 32 33 34 35 36 37 38 39
> 40: 40 41 42 43 44 45 46 47
> 48: 48 49 50 51 52 53 54 55
> 56: 56 57 58 59 60 61 62 63
This is odd, if IFF_RXFH_CONFIGURED is set driver should not
re-initialize the indirection table. Which I believe is what
you describe at the end of your message:
> But, I can increase the queue count and the flow hash is preserved:
>
> $ sudo ethtool -L eth0 combined 64
> $ sudo ethtool -x eth0
> RX flow hash indirection table for eth0 with 64 RX ring(s):
> 0: 0 1 2 3 4 5 6 7
> 8: 8 9 10 11 12 13 14 15
> 16: 16 17 18 19 0 1 2 3
next prev parent reply other threads:[~2022-10-17 19:46 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-13 22:54 [Intel-wired-lan] [net-queue bugfix RFC] i40e: Clear IFF_RXFH_CONFIGURED when RSS is reset Joe Damato
2022-10-13 22:54 ` Joe Damato
2022-10-17 19:45 ` Jakub Kicinski [this message]
2022-10-17 19:45 ` Jakub Kicinski
2022-10-17 20:25 ` [Intel-wired-lan] " Jacob Keller
2022-10-17 20:25 ` Jacob Keller
2022-10-17 20:36 ` [Intel-wired-lan] " Joe Damato
2022-10-17 20:36 ` Joe Damato
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221017124555.5d79d3f7@kernel.org \
--to=kuba@kernel.org \
--cc=davem@davemloft.net \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jdamato@fastly.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.