* [PATCH] net: atlantic: make RX page order tunable via module param
@ 2025-09-06 13:54 Yangyu Chen
2025-09-08 17:12 ` Andrew Lunn
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Yangyu Chen @ 2025-09-06 13:54 UTC (permalink / raw)
To: netdev
Cc: Igor Russkikh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, linux-kernel, Yangyu Chen
On systems like AMD Strix Halo with Thunderbolt, RX map/unmap operations
with IOMMU introduce significant performance overhead, making it difficult
to achieve line rate with 10G NICs even with TCP over MTU 1500. Using
higher order pages reduces this overhead, so this parameter is now
configurable.
After applying this patch and setting `rxpageorder=3`, testing with QNAP
QNA-T310G1S on 10G Ethernet (MTU 1500) using `iperf3 -R` on IPv6 achieved
9.28Gbps compared to only 2.26Gbps previously.
Signed-off-by: Yangyu Chen <cyy@cyyself.name>
---
Should we also consider make default AQ_CFG_RX_PAGEORDER to 3?
Test result showing performance improvement:
$ sudo insmod drivers/net/ethernet/aquantia/atlantic/atlantic.ko
$ sudo ip link set enp99s0 up
$ iperf3 -c fe80::3a63:bbff:fe2e:1a68%enp99s0 -R
Connecting to host fe80::3a63:bbff:fe2e:1a68%enp99s0, port 5201
Reverse mode, remote host fe80::3a63:bbff:fe2e:1a68%enp99s0 is sending
[ 5] local fe80::265e:beff:fe6a:4da1 port 39588 connected to fe80::3a63:bbff:fe2e:1a68 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 271 MBytes 2.27 Gbits/sec
[ 5] 1.00-2.00 sec 270 MBytes 2.27 Gbits/sec
[ 5] 2.00-3.00 sec 268 MBytes 2.25 Gbits/sec
[ 5] 3.00-4.00 sec 270 MBytes 2.26 Gbits/sec
[ 5] 4.00-5.00 sec 268 MBytes 2.25 Gbits/sec
[ 5] 5.00-6.00 sec 269 MBytes 2.26 Gbits/sec
[ 5] 6.00-7.00 sec 268 MBytes 2.25 Gbits/sec
[ 5] 7.00-8.00 sec 268 MBytes 2.25 Gbits/sec
[ 5] 8.00-9.00 sec 268 MBytes 2.25 Gbits/sec
[ 5] 9.00-10.00 sec 268 MBytes 2.25 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec 1 sender
[ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec receiver
iperf Done.
$ sudo rmmod atlantic
$ sudo insmod drivers/net/ethernet/aquantia/atlantic/atlantic.ko rxpageorder=3
$ sudo ip link set enp99s0 up
$ iperf3 -c fe80::3a63:bbff:fe2e:1a68%enp99s0 -R
Connecting to host fe80::3a63:bbff:fe2e:1a68%enp99s0, port 5201
Reverse mode, remote host fe80::3a63:bbff:fe2e:1a68%enp99s0 is sending
[ 5] local fe80::265e:beff:fe6a:4da1 port 43356 connected to fe80::3a63:bbff:fe2e:1a68 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 1.00-2.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 2.00-3.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 3.00-4.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 4.00-5.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 5.00-6.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 6.00-7.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 7.00-8.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 8.00-9.00 sec 1.08 GBytes 9.28 Gbits/sec
[ 5] 9.00-10.00 sec 1.08 GBytes 9.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 10.8 GBytes 9.28 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 10.8 GBytes 9.28 Gbits/sec receiver
iperf Done.
---
drivers/net/ethernet/aquantia/atlantic/aq_nic.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
index b24eaa5283fa..48f35fbf9a70 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
@@ -40,6 +40,10 @@ static unsigned int aq_itr_rx;
module_param_named(aq_itr_rx, aq_itr_rx, uint, 0644);
MODULE_PARM_DESC(aq_itr_rx, "RX interrupt throttle rate");
+static unsigned int rxpageorder = AQ_CFG_RX_PAGEORDER;
+module_param_named(rxpageorder, rxpageorder, uint, 0644);
+MODULE_PARM_DESC(rxpageorder, "RX page order");
+
static void aq_nic_update_ndev_stats(struct aq_nic_s *self);
static void aq_nic_rss_init(struct aq_nic_s *self, unsigned int num_rss_queues)
@@ -106,7 +110,7 @@ void aq_nic_cfg_start(struct aq_nic_s *self)
cfg->tx_itr = aq_itr_tx;
cfg->rx_itr = aq_itr_rx;
- cfg->rxpageorder = AQ_CFG_RX_PAGEORDER;
+ cfg->rxpageorder = rxpageorder;
cfg->is_rss = AQ_CFG_IS_RSS_DEF;
cfg->aq_rss.base_cpu_number = AQ_CFG_RSS_BASE_CPU_NUM_DEF;
cfg->fc.req = AQ_CFG_FC_MODE;
--
2.47.2
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH] net: atlantic: make RX page order tunable via module param
2025-09-06 13:54 [PATCH] net: atlantic: make RX page order tunable via module param Yangyu Chen
@ 2025-09-08 17:12 ` Andrew Lunn
2025-09-09 0:45 ` Jakub Kicinski
2025-09-09 9:13 ` Simon Horman
2 siblings, 0 replies; 5+ messages in thread
From: Andrew Lunn @ 2025-09-08 17:12 UTC (permalink / raw)
To: Yangyu Chen
Cc: netdev, Igor Russkikh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, linux-kernel
> +static unsigned int rxpageorder = AQ_CFG_RX_PAGEORDER;
> +module_param_named(rxpageorder, rxpageorder, uint, 0644);
> +MODULE_PARM_DESC(rxpageorder, "RX page order");
Sorry, but netdev does not like module params. They are a bad API to
use. Please find a different way to do this.
Also, there is a comment:
/* Only order-2 is allowed if XDP is enabled */
if (READ_ONCE(self->xdp_prog)) {
You need to ensure this assumption is not broken.
Andrew
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH] net: atlantic: make RX page order tunable via module param
2025-09-06 13:54 [PATCH] net: atlantic: make RX page order tunable via module param Yangyu Chen
2025-09-08 17:12 ` Andrew Lunn
@ 2025-09-09 0:45 ` Jakub Kicinski
2025-09-09 9:13 ` Simon Horman
2 siblings, 0 replies; 5+ messages in thread
From: Jakub Kicinski @ 2025-09-09 0:45 UTC (permalink / raw)
To: Yangyu Chen
Cc: netdev, Igor Russkikh, Andrew Lunn, David S. Miller, Eric Dumazet,
Paolo Abeni, linux-kernel
On Sat, 6 Sep 2025 21:54:34 +0800 Yangyu Chen wrote:
> On systems like AMD Strix Halo with Thunderbolt, RX map/unmap operations
> with IOMMU introduce significant performance overhead, making it difficult
> to achieve line rate with 10G NICs even with TCP over MTU 1500. Using
> higher order pages reduces this overhead, so this parameter is now
> configurable.
Please convert this driver to use page_pool instead
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] net: atlantic: make RX page order tunable via module param
2025-09-06 13:54 [PATCH] net: atlantic: make RX page order tunable via module param Yangyu Chen
2025-09-08 17:12 ` Andrew Lunn
2025-09-09 0:45 ` Jakub Kicinski
@ 2025-09-09 9:13 ` Simon Horman
2025-09-09 9:15 ` Simon Horman
2 siblings, 1 reply; 5+ messages in thread
From: Simon Horman @ 2025-09-09 9:13 UTC (permalink / raw)
To: Yangyu Chen
Cc: netdev, Igor Russkikh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, linux-kernel
On Sat, Sep 06, 2025 at 09:54:34PM +0800, Yangyu Chen wrote:
> On systems like AMD Strix Halo with Thunderbolt, RX map/unmap operations
> with IOMMU introduce significant performance overhead, making it difficult
> to achieve line rate with 10G NICs even with TCP over MTU 1500. Using
> higher order pages reduces this overhead, so this parameter is now
> configurable.
>
> After applying this patch and setting `rxpageorder=3`, testing with QNAP
> QNA-T310G1S on 10G Ethernet (MTU 1500) using `iperf3 -R` on IPv6 achieved
> 9.28Gbps compared to only 2.26Gbps previously.
VerU nice.
> Signed-off-by: Yangyu Chen <cyy@cyyself.name>
> ---
> Should we also consider make default AQ_CFG_RX_PAGEORDER to 3?
I have the same question.
...
> diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> index b24eaa5283fa..48f35fbf9a70 100644
> --- a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> +++ b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> @@ -40,6 +40,10 @@ static unsigned int aq_itr_rx;
> module_param_named(aq_itr_rx, aq_itr_rx, uint, 0644);
> MODULE_PARM_DESC(aq_itr_rx, "RX interrupt throttle rate");
>
> +static unsigned int rxpageorder = AQ_CFG_RX_PAGEORDER;
> +module_param_named(rxpageorder, rxpageorder, uint, 0644);
> +MODULE_PARM_DESC(rxpageorder, "RX page order");
> +
Unfortunately adding new module parameters to networking drivers
is strongly discouraged. Can we find another way to address the problem
described in your cover: e.g.
1. Changing the fixed value
2. Somehow making the value auto detected
3. Some other mechanism to allow the user to configure the value, e.g. devlink
...
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] net: atlantic: make RX page order tunable via module param
2025-09-09 9:13 ` Simon Horman
@ 2025-09-09 9:15 ` Simon Horman
0 siblings, 0 replies; 5+ messages in thread
From: Simon Horman @ 2025-09-09 9:15 UTC (permalink / raw)
To: Yangyu Chen
Cc: netdev, Igor Russkikh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, linux-kernel
On Tue, Sep 09, 2025 at 10:13:17AM +0100, Simon Horman wrote:
> On Sat, Sep 06, 2025 at 09:54:34PM +0800, Yangyu Chen wrote:
...
> > diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> > index b24eaa5283fa..48f35fbf9a70 100644
> > --- a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> > +++ b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
> > @@ -40,6 +40,10 @@ static unsigned int aq_itr_rx;
> > module_param_named(aq_itr_rx, aq_itr_rx, uint, 0644);
> > MODULE_PARM_DESC(aq_itr_rx, "RX interrupt throttle rate");
> >
> > +static unsigned int rxpageorder = AQ_CFG_RX_PAGEORDER;
> > +module_param_named(rxpageorder, rxpageorder, uint, 0644);
> > +MODULE_PARM_DESC(rxpageorder, "RX page order");
> > +
>
> Unfortunately adding new module parameters to networking drivers
> is strongly discouraged. Can we find another way to address the problem
> described in your cover: e.g.
>
> 1. Changing the fixed value
> 2. Somehow making the value auto detected
> 3. Some other mechanism to allow the user to configure the value, e.g. devlink
>
> ...
Oops. I now see that Andrew and Jakub already responded.
And my comment doesn't add much. Sorry about that.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2025-09-09 9:15 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-06 13:54 [PATCH] net: atlantic: make RX page order tunable via module param Yangyu Chen
2025-09-08 17:12 ` Andrew Lunn
2025-09-09 0:45 ` Jakub Kicinski
2025-09-09 9:13 ` Simon Horman
2025-09-09 9:15 ` Simon Horman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).