* [PATCH net-next] r8169: use unlimited DMA burst for TX
@ 2012-09-09 23:55 Michal Schmidt
2012-09-10 19:50 ` David Miller
2012-09-14 5:19 ` Francois Romieu
0 siblings, 2 replies; 7+ messages in thread
From: Michal Schmidt @ 2012-09-09 23:55 UTC (permalink / raw)
To: netdev
Cc: Francois Romieu, Hayes Wang, Realtek linux nic maintainers,
Ivan Vecera
The r8169 driver currently limits the DMA burst for TX to 1024 bytes. I have
a box where this prevents the interface from using the gigabit line to its full
potential. This patch solves the problem by setting TX_DMA_BURST to unlimited.
The box has an ASRock B75M motherboard with on-board RTL8168evl/8111evl
(XID 0c900880). TSO is enabled.
I used netperf (TCP_STREAM test) to measure the dependency of TX throughput
on MTU. I did it for three different values of TX_DMA_BURST ('5'=512, '6'=1024,
'7'=unlimited). This chart shows the results:
http://michich.fedorapeople.org/r8169/r8169-effects-of-TX_DMA_BURST.png
Interesting points:
- With the current DMA burst limit (1024):
- at the default MTU=1500 I get only 842 Mbit/s.
- when going from small MTU, the performance rises monotonically with
increasing MTU only up to a peak at MTU=1076 (908 MBit/s). Then there's
a sudden drop to 762 MBit/s from which the throughput rises monotonically
again with further MTU increases.
- With a smaller DMA burst limit (512):
- there's a similar peak at MTU=1076 and another one at MTU=564.
- With unlimited DMA burst:
- at the default MTU=1500 I get nice 940 Mbit/s.
- the throughput rises monotonically with increasing MTU with no strange
peaks.
Notice that the peaks occur at MTU sizes that are multiples of the DMA burst
limit plus 52. Why 52? Because:
20 (IP header) + 20 (TCP header) + 12 (TCP options) = 52
The Realtek-provided r8168 driver (v8.032.00) uses unlimited TX DMA burst too,
except for CFG_METHOD_1 where the TX DMA burst is set to 512 bytes.
CFG_METHOD_1 appears to be the oldest MAC version of "RTL8168B/8111B",
i.e. RTL_GIGA_MAC_VER_11 in r8169. Not sure if this MAC version really needs
the smaller burst limit, or if any other versions have similar requirements.
Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
---
drivers/net/ethernet/realtek/r8169.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index b47d5b3..549314f 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -77,7 +77,7 @@
static const int multicast_filter_limit = 32;
#define MAX_READ_REQUEST_SHIFT 12
-#define TX_DMA_BURST 6 /* Maximum PCI burst, '6' is 1024 */
+#define TX_DMA_BURST 7 /* Maximum PCI burst, '7' is unlimited */
#define SafeMtu 0x1c20 /* ... actually life sucks beyond ~7k */
#define InterFrameGap 0x03 /* 3 means InterFrameGap = the shortest one */
--
1.7.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH net-next] r8169: use unlimited DMA burst for TX
2012-09-09 23:55 [PATCH net-next] r8169: use unlimited DMA burst for TX Michal Schmidt
@ 2012-09-10 19:50 ` David Miller
2012-09-10 21:55 ` Francois Romieu
2012-09-14 5:19 ` Francois Romieu
1 sibling, 1 reply; 7+ messages in thread
From: David Miller @ 2012-09-10 19:50 UTC (permalink / raw)
To: mschmidt; +Cc: netdev, romieu, hayeswang, nic_swsd, ivecera
From: Michal Schmidt <mschmidt@redhat.com>
Date: Mon, 10 Sep 2012 01:55:26 +0200
> The r8169 driver currently limits the DMA burst for TX to 1024 bytes. I have
> a box where this prevents the interface from using the gigabit line to its full
> potential. This patch solves the problem by setting TX_DMA_BURST to unlimited.
>
> The box has an ASRock B75M motherboard with on-board RTL8168evl/8111evl
> (XID 0c900880). TSO is enabled.
>
> I used netperf (TCP_STREAM test) to measure the dependency of TX throughput
> on MTU. I did it for three different values of TX_DMA_BURST ('5'=512, '6'=1024,
> '7'=unlimited). This chart shows the results:
> http://michich.fedorapeople.org/r8169/r8169-effects-of-TX_DMA_BURST.png
>
> Interesting points:
> - With the current DMA burst limit (1024):
> - at the default MTU=1500 I get only 842 Mbit/s.
> - when going from small MTU, the performance rises monotonically with
> increasing MTU only up to a peak at MTU=1076 (908 MBit/s). Then there's
> a sudden drop to 762 MBit/s from which the throughput rises monotonically
> again with further MTU increases.
> - With a smaller DMA burst limit (512):
> - there's a similar peak at MTU=1076 and another one at MTU=564.
> - With unlimited DMA burst:
> - at the default MTU=1500 I get nice 940 Mbit/s.
> - the throughput rises monotonically with increasing MTU with no strange
> peaks.
>
> Notice that the peaks occur at MTU sizes that are multiples of the DMA burst
> limit plus 52. Why 52? Because:
> 20 (IP header) + 20 (TCP header) + 12 (TCP options) = 52
>
> The Realtek-provided r8168 driver (v8.032.00) uses unlimited TX DMA burst too,
> except for CFG_METHOD_1 where the TX DMA burst is set to 512 bytes.
> CFG_METHOD_1 appears to be the oldest MAC version of "RTL8168B/8111B",
> i.e. RTL_GIGA_MAC_VER_11 in r8169. Not sure if this MAC version really needs
> the smaller burst limit, or if any other versions have similar requirements.
>
> Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
I really need Francois et al. to take a look at this before even thinking
about applying it.
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH net-next] r8169: use unlimited DMA burst for TX
2012-09-10 19:50 ` David Miller
@ 2012-09-10 21:55 ` Francois Romieu
2012-09-11 8:09 ` hayeswang
0 siblings, 1 reply; 7+ messages in thread
From: Francois Romieu @ 2012-09-10 21:55 UTC (permalink / raw)
To: David Miller; +Cc: mschmidt, netdev, hayeswang, nic_swsd, ivecera
David Miller <davem@davemloft.net> :
[...]
> I really need Francois et al. to take a look at this before even thinking
> about applying it.
Hayes tweaked RX_DMA_BURST to an unlimited value one year ago (see
4f6b00e5f139d7be3ca8371b769778f94fa549dd). Other than that, both
RX_DMA_BURST and TX_DMA_BURST have been set at the same (6 i.e. 1024 bytes)
value since the driver was added to the kernel tree.
Realtek's own 8169 driver uses unlimited bursts as well. However their 810x
driver doesn't (whence differing from the kernel driver after 4f6b00).
The current situation is a bit messy to say the least but it does not seem
to match any problem report. I don't have any special "don't".
Hayes, should we:
- mimic Realtek's 8168, 8169 and 810x drivers ?
- always set TX_DMA_BURST at the max value ?
- do something different (per chipset) ?
Thanks.
--
Ueimor
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [PATCH net-next] r8169: use unlimited DMA burst for TX
2012-09-10 21:55 ` Francois Romieu
@ 2012-09-11 8:09 ` hayeswang
2012-09-13 23:27 ` Michal Schmidt
0 siblings, 1 reply; 7+ messages in thread
From: hayeswang @ 2012-09-11 8:09 UTC (permalink / raw)
To: 'Francois Romieu', 'David Miller'
Cc: mschmidt, netdev, ivecera
Hi,
Francois Romieu [mailto:romieu@fr.zoreil.com]
> Sent: Tuesday, September 11, 2012 5:55 AM
> To: David Miller
> Cc: mschmidt@redhat.com; netdev@vger.kernel.org; Hayeswang;
> nic_swsd; ivecera@redhat.com
> Subject: Re: [PATCH net-next] r8169: use unlimited DMA burst for TX
>
> Hayes, should we:
> - mimic Realtek's 8168, 8169 and 810x drivers ?
> - always set TX_DMA_BURST at the max value ?
> - do something different (per chipset) ?
Our hw engineer suggets to set unlimited for both TX_DMA_BURST and RX_DMA_BURST
for all chipsets.
Best Regards,
Hayes
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net-next] r8169: use unlimited DMA burst for TX
2012-09-11 8:09 ` hayeswang
@ 2012-09-13 23:27 ` Michal Schmidt
0 siblings, 0 replies; 7+ messages in thread
From: Michal Schmidt @ 2012-09-13 23:27 UTC (permalink / raw)
To: 'Francois Romieu'
Cc: hayeswang, 'David Miller', netdev, ivecera
On 09/11/2012 10:09 AM, hayeswang wrote:
> [Francois Romieu wrote:]
>> Hayes, should we:
>> - mimic Realtek's 8168, 8169 and 810x drivers ?
>> - always set TX_DMA_BURST at the max value ?
>> - do something different (per chipset) ?
>
> Our hw engineer suggets to set unlimited for both TX_DMA_BURST and RX_DMA_BURST
> for all chipsets.
Francois,
as this is exactly what the patch does, would you give an ACK?
Michal
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net-next] r8169: use unlimited DMA burst for TX
2012-09-09 23:55 [PATCH net-next] r8169: use unlimited DMA burst for TX Michal Schmidt
2012-09-10 19:50 ` David Miller
@ 2012-09-14 5:19 ` Francois Romieu
2012-09-19 23:19 ` David Miller
1 sibling, 1 reply; 7+ messages in thread
From: Francois Romieu @ 2012-09-14 5:19 UTC (permalink / raw)
To: Michal Schmidt; +Cc: netdev, Hayes Wang, Ivan Vecera
Michal Schmidt <mschmidt@redhat.com> :
[...]
> Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
Acked-by: Francois Romieu <romieu@fr.zoreil.com>
--
Ueimor
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2012-09-19 23:24 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-09-09 23:55 [PATCH net-next] r8169: use unlimited DMA burst for TX Michal Schmidt
2012-09-10 19:50 ` David Miller
2012-09-10 21:55 ` Francois Romieu
2012-09-11 8:09 ` hayeswang
2012-09-13 23:27 ` Michal Schmidt
2012-09-14 5:19 ` Francois Romieu
2012-09-19 23:19 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).
