* [PATCH v2 net-next] net: introduce SO_MAX_PACING_RATE
2013-09-23 15:10 [PATCH net-next] net: introduce SO_MAX_PACING_RATE Eric Dumazet
2013-09-24 15:14 ` Eric Dumazet
@ 2013-09-24 15:20 ` Eric Dumazet
2013-09-28 22:36 ` David Miller
2013-09-27 21:16 ` [PATCH net-next] net: introduce SO_MAX_PACING_RATE Steinar H. Gunderson
2 siblings, 1 reply; 8+ messages in thread
From: Eric Dumazet @ 2013-09-24 15:20 UTC (permalink / raw)
To: David Miller; +Cc: netdev, Steinar H. Gunderson, Michael Kerrisk
From: Eric Dumazet <edumazet@google.com>
As mentioned in commit afe4fd062416b ("pkt_sched: fq: Fair Queue packet
scheduler"), this patch adds a new socket option.
SO_MAX_PACING_RATE offers the application the ability to cap the
rate computed by transport layer. Value is in bytes per second.
u32 val = 1000000;
setsockopt(sockfd, SOL_SOCKET, SO_MAX_PACING_RATE, &val, sizeof(val));
To be effectively paced, a flow must use FQ packet scheduler.
Note that a packet scheduler takes into account the headers for its
computations. The effective payload rate depends on MSS and retransmits
if any.
I chose to make this pacing rate a SOL_SOCKET option instead of a
TCP one because this can be used by other protocols.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Steinar H. Gunderson <sesse@google.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
---
v2: cap sk->sk_pacing_rate in sock_setsockopt()
arch/alpha/include/uapi/asm/socket.h | 4 +++-
arch/avr32/include/uapi/asm/socket.h | 2 ++
arch/cris/include/uapi/asm/socket.h | 2 ++
arch/frv/include/uapi/asm/socket.h | 2 ++
arch/h8300/include/uapi/asm/socket.h | 2 ++
arch/ia64/include/uapi/asm/socket.h | 2 ++
arch/m32r/include/uapi/asm/socket.h | 2 ++
arch/mips/include/uapi/asm/socket.h | 2 ++
arch/mn10300/include/uapi/asm/socket.h | 2 ++
arch/parisc/include/uapi/asm/socket.h | 2 ++
arch/powerpc/include/uapi/asm/socket.h | 2 ++
arch/s390/include/uapi/asm/socket.h | 2 ++
arch/sparc/include/uapi/asm/socket.h | 2 ++
arch/xtensa/include/uapi/asm/socket.h | 2 ++
include/net/sock.h | 1 +
include/uapi/asm-generic/socket.h | 2 ++
net/core/sock.c | 12 ++++++++++++
net/ipv4/tcp_input.c | 2 +-
18 files changed, 45 insertions(+), 2 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h
index 467de01..e3a1491 100644
--- a/arch/alpha/include/uapi/asm/socket.h
+++ b/arch/alpha/include/uapi/asm/socket.h
@@ -81,6 +81,8 @@
#define SO_SELECT_ERR_QUEUE 45
-#define SO_BUSY_POLL 46
+#define SO_BUSY_POLL 46
+
+#define SO_MAX_PACING_RATE 47
#endif /* _UAPI_ASM_SOCKET_H */
diff --git a/arch/avr32/include/uapi/asm/socket.h b/arch/avr32/include/uapi/asm/socket.h
index 11c4259..4399364 100644
--- a/arch/avr32/include/uapi/asm/socket.h
+++ b/arch/avr32/include/uapi/asm/socket.h
@@ -76,4 +76,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* __ASM_AVR32_SOCKET_H */
diff --git a/arch/cris/include/uapi/asm/socket.h b/arch/cris/include/uapi/asm/socket.h
index eb723e5..13829aa 100644
--- a/arch/cris/include/uapi/asm/socket.h
+++ b/arch/cris/include/uapi/asm/socket.h
@@ -78,6 +78,8 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/frv/include/uapi/asm/socket.h b/arch/frv/include/uapi/asm/socket.h
index f0cb1c3..5d42997 100644
--- a/arch/frv/include/uapi/asm/socket.h
+++ b/arch/frv/include/uapi/asm/socket.h
@@ -76,5 +76,7 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/h8300/include/uapi/asm/socket.h b/arch/h8300/include/uapi/asm/socket.h
index 9490758..214ccaf 100644
--- a/arch/h8300/include/uapi/asm/socket.h
+++ b/arch/h8300/include/uapi/asm/socket.h
@@ -76,4 +76,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/ia64/include/uapi/asm/socket.h b/arch/ia64/include/uapi/asm/socket.h
index 556d070..c25302f 100644
--- a/arch/ia64/include/uapi/asm/socket.h
+++ b/arch/ia64/include/uapi/asm/socket.h
@@ -85,4 +85,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_IA64_SOCKET_H */
diff --git a/arch/m32r/include/uapi/asm/socket.h b/arch/m32r/include/uapi/asm/socket.h
index 24be7c8..5296665 100644
--- a/arch/m32r/include/uapi/asm/socket.h
+++ b/arch/m32r/include/uapi/asm/socket.h
@@ -76,4 +76,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_M32R_SOCKET_H */
diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h
index 61c01f0..0df9787 100644
--- a/arch/mips/include/uapi/asm/socket.h
+++ b/arch/mips/include/uapi/asm/socket.h
@@ -94,4 +94,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _UAPI_ASM_SOCKET_H */
diff --git a/arch/mn10300/include/uapi/asm/socket.h b/arch/mn10300/include/uapi/asm/socket.h
index e2a2b203..71dedca 100644
--- a/arch/mn10300/include/uapi/asm/socket.h
+++ b/arch/mn10300/include/uapi/asm/socket.h
@@ -76,4 +76,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h
index 71700e6..7c614d0 100644
--- a/arch/parisc/include/uapi/asm/socket.h
+++ b/arch/parisc/include/uapi/asm/socket.h
@@ -75,6 +75,8 @@
#define SO_BUSY_POLL 0x4027
+#define SO_MAX_PACING_RATE 0x4048
+
/* O_NONBLOCK clashes with the bits used for socket types. Therefore we
* have to define SOCK_NONBLOCK to a different value here.
*/
diff --git a/arch/powerpc/include/uapi/asm/socket.h b/arch/powerpc/include/uapi/asm/socket.h
index a6d7446..fa69832 100644
--- a/arch/powerpc/include/uapi/asm/socket.h
+++ b/arch/powerpc/include/uapi/asm/socket.h
@@ -83,4 +83,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_POWERPC_SOCKET_H */
diff --git a/arch/s390/include/uapi/asm/socket.h b/arch/s390/include/uapi/asm/socket.h
index 9249449..c286c2e 100644
--- a/arch/s390/include/uapi/asm/socket.h
+++ b/arch/s390/include/uapi/asm/socket.h
@@ -82,4 +82,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h
index 4e1d66c..0f21e9a 100644
--- a/arch/sparc/include/uapi/asm/socket.h
+++ b/arch/sparc/include/uapi/asm/socket.h
@@ -72,6 +72,8 @@
#define SO_BUSY_POLL 0x0030
+#define SO_MAX_PACING_RATE 0x0031
+
/* Security levels - as per NRL IPv6 - don't actually do anything */
#define SO_SECURITY_AUTHENTICATION 0x5001
#define SO_SECURITY_ENCRYPTION_TRANSPORT 0x5002
diff --git a/arch/xtensa/include/uapi/asm/socket.h b/arch/xtensa/include/uapi/asm/socket.h
index c114483..7db5c22 100644
--- a/arch/xtensa/include/uapi/asm/socket.h
+++ b/arch/xtensa/include/uapi/asm/socket.h
@@ -87,4 +87,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* _XTENSA_SOCKET_H */
diff --git a/include/net/sock.h b/include/net/sock.h
index 4625d2e..240aa3f 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -363,6 +363,7 @@ struct sock {
int sk_wmem_queued;
gfp_t sk_allocation;
u32 sk_pacing_rate; /* bytes per second */
+ u32 sk_max_pacing_rate;
netdev_features_t sk_route_caps;
netdev_features_t sk_route_nocaps;
int sk_gso_type;
diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h
index f04b69b..38f14d0 100644
--- a/include/uapi/asm-generic/socket.h
+++ b/include/uapi/asm-generic/socket.h
@@ -78,4 +78,6 @@
#define SO_BUSY_POLL 46
+#define SO_MAX_PACING_RATE 47
+
#endif /* __ASM_GENERIC_SOCKET_H */
diff --git a/net/core/sock.c b/net/core/sock.c
index 5b6beba..2bd9b3f 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -914,6 +914,13 @@ set_rcvbuf:
}
break;
#endif
+
+ case SO_MAX_PACING_RATE:
+ sk->sk_max_pacing_rate = val;
+ sk->sk_pacing_rate = min(sk->sk_pacing_rate,
+ sk->sk_max_pacing_rate);
+ break;
+
default:
ret = -ENOPROTOOPT;
break;
@@ -1177,6 +1184,10 @@ int sock_getsockopt(struct socket *sock, int level, int optname,
break;
#endif
+ case SO_MAX_PACING_RATE:
+ v.val = sk->sk_max_pacing_rate;
+ break;
+
default:
return -ENOPROTOOPT;
}
@@ -2319,6 +2330,7 @@ void sock_init_data(struct socket *sock, struct sock *sk)
sk->sk_ll_usec = sysctl_net_busy_read;
#endif
+ sk->sk_max_pacing_rate = ~0U;
/*
* Before updating sk_refcnt, we must commit prior changes to memory
* (Documentation/RCU/rculist_nulls.txt for details)
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 25a89ea..75372c0 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -713,7 +713,7 @@ static void tcp_update_pacing_rate(struct sock *sk)
if (tp->srtt > 8 + 2)
do_div(rate, tp->srtt);
- sk->sk_pacing_rate = min_t(u64, rate, ~0U);
+ sk->sk_pacing_rate = min_t(u64, rate, sk->sk_max_pacing_rate);
}
/* Calculate rto without backoff. This is the second half of Van Jacobson's
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH net-next] net: introduce SO_MAX_PACING_RATE
2013-09-23 15:10 [PATCH net-next] net: introduce SO_MAX_PACING_RATE Eric Dumazet
2013-09-24 15:14 ` Eric Dumazet
2013-09-24 15:20 ` [PATCH v2 " Eric Dumazet
@ 2013-09-27 21:16 ` Steinar H. Gunderson
2013-09-28 0:28 ` Eric Dumazet
2 siblings, 1 reply; 8+ messages in thread
From: Steinar H. Gunderson @ 2013-09-27 21:16 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, Michael Kerrisk
On Mon, Sep 23, 2013 at 08:10:14AM -0700, Eric Dumazet wrote:
> SO_MAX_PACING_RATE offers the application the ability to cap the
> rate computed by transport layer. Value is in bytes per second.
I tried this (the first version) on top of 3.12-rc2, but it doesn't actually
appear to work.
The setsockopt goes through:
[pid 30501] setsockopt(11, SOL_SOCKET, 0x2f /* SO_??? */, [1920000], 4) = 0
And the fq scheduler is active on the machine:
pannekake:~> sudo tc qdisc
qdisc fq 8001: dev eth0 root refcnt 2 limit 10000p flow_limit 100p buckets 1024 quantum 3028 initial_quantum 15140
Yet the packets come out way too close on the wire, as measured by tcpdump
(I have tso/gso off):
23:11:13.687773 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 706316:707764, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687787 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 707764:709212, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687793 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 709212:710660, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687799 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 710660:712108, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687804 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 712108:713556, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687809 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 713556:715004, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687814 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 715004:716452, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687819 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 716452:717900, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687825 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 717900:719348, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687830 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 719348:720796, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687835 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 720796:722244, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687840 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 722244:723692, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687845 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 723692:725140, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687850 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 725140:726588, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687855 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 726588:728036, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687860 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 728036:729484, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687864 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 729484:730932, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687869 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 730932:732380, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687874 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 732380:733828, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687878 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 733828:735276, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687883 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 735276:736724, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687888 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 736724:738172, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687893 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 738172:739620, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687902 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 739620:741068, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.687921 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 741068:742516, ack 1, win 114, options [nop,nop,TS val 5666415 ecr 46581981], length 1448
23:11:13.688818 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 742516:743964, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.688833 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 743964:745412, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.689780 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 745412:746860, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.689794 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 746860:748308, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.690647 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 748308:749756, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.690660 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 749756:751204, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.691493 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 751204:752652, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.691505 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 752652:754100, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
23:11:13.692299 IP 193.35.52.50.9094 > 178.83.234.79.51249: Flags [.], seq 754100:755548, ack 1, win 114, options [nop,nop,TS val 5666416 ecr 46581981], length 1448
/* Steinar */
--
Software Engineer, Google Switzerland
^ permalink raw reply [flat|nested] 8+ messages in thread