* Re: [PATCH net-next 2/2] virtio-net: get rid of unnecessary container of rq stats
From: Toshiaki Makita @ 2018-08-01 1:46 UTC (permalink / raw)
To: Jason Wang; +Cc: netdev, virtualization, linux-kernel, mst
In-Reply-To: <d11c65d8-20be-1a85-c687-c6f182264e41@redhat.com>
On 2018/08/01 10:39, Jason Wang wrote:
> On 2018年07月31日 18:02, Toshiaki Makita wrote:
>> On 2018/07/31 18:43, Jason Wang wrote:
>>> We don't maintain tx counters in rx stats any more. There's no need
>>> for an extra container of rq stats.
>>>
>>> Cc: Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> ---
>>> drivers/net/virtio_net.c | 80
>>> ++++++++++++++++++++++--------------------------
>>> 1 file changed, 36 insertions(+), 44 deletions(-)
>>>
>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>> index 72d3f68..14f661c 100644
>>> --- a/drivers/net/virtio_net.c
>>> +++ b/drivers/net/virtio_net.c
>>> @@ -87,7 +87,8 @@ struct virtnet_sq_stats {
>>> u64 kicks;
>>> };
>>> -struct virtnet_rq_stat_items {
>>> +struct virtnet_rq_stats {
>>> + struct u64_stats_sync syncp;
>>> u64 packets;
>>> u64 bytes;
>>> u64 drops;
>>> @@ -98,17 +99,8 @@ struct virtnet_rq_stat_items {
>>> u64 kicks;
>>> };
>>> -struct virtnet_rq_stats {
>>> - struct u64_stats_sync syncp;
>>> - struct virtnet_rq_stat_items items;
>>> -};
>> I'm not thinking removing sq stat is needed but even if it is I want to
>> keep virtnet_rq_stats to avoid allocating unnecessary u64_stats_syncp on
>> stack in virtnet_receive. I would just remove virtnet_rx_stats if
>> necessary.
>
> It's a nop on 64bit machines. And an unsigned on 32bit. So it's overhead
> could be ignored I think.
It's not a big problem so that's OK. I just felt like you reverted
unnecessarily too much. Anyway it is already applied and I'm not
thinking of changing this any more.
--
Toshiaki Makita
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* [PATCH net-next] ip_gre: remove redundant variables t_hlen
From: YueHaibing @ 2018-08-01 2:04 UTC (permalink / raw)
To: davem, kuznet, yoshfuji; +Cc: linux-kernel, netdev, YueHaibing
After commit ffc2b6ee4174 ("ip_gre: fix IFLA_MTU ignored on NEWLINK")
variable t_hlen is assigned values that are never read,
hence they are redundant and can be removed.
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
---
net/ipv4/ip_gre.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index c8ca5d8..51a5d06 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -985,7 +985,6 @@ static void ipgre_tunnel_setup(struct net_device *dev)
static void __gre_tunnel_init(struct net_device *dev)
{
struct ip_tunnel *tunnel;
- int t_hlen;
tunnel = netdev_priv(dev);
tunnel->tun_hlen = gre_calc_hlen(tunnel->parms.o_flags);
@@ -993,8 +992,6 @@ static void __gre_tunnel_init(struct net_device *dev)
tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen;
- t_hlen = tunnel->hlen + sizeof(struct iphdr);
-
dev->features |= GRE_FEATURES;
dev->hw_features |= GRE_FEATURES;
@@ -1304,13 +1301,11 @@ static const struct net_device_ops gre_tap_netdev_ops = {
static int erspan_tunnel_init(struct net_device *dev)
{
struct ip_tunnel *tunnel = netdev_priv(dev);
- int t_hlen;
tunnel->tun_hlen = 8;
tunnel->parms.iph.protocol = IPPROTO_GRE;
tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen +
erspan_hdr_len(tunnel->erspan_ver);
- t_hlen = tunnel->hlen + sizeof(struct iphdr);
dev->features |= GRE_FEATURES;
dev->hw_features |= GRE_FEATURES;
--
2.7.0
^ permalink raw reply related
* [PATCH] netfilter: nf_tables: remove redundant variables 'create' and 'ext'
From: YueHaibing @ 2018-08-01 2:14 UTC (permalink / raw)
To: davem, pablo, kadlec, fw
Cc: linux-kernel, netdev, netfilter-devel, coreteam, YueHaibing
Variables 'create' and 'ext' are being assigned but are never used hence
they are redundant and can be removed.
Cleans up clang warnings:
net/netfilter/nf_tables_api.c:3364:7: warning: variable ‘create’ set but not used [-Wunused-but-set-variable]
net/netfilter/nf_tables_api.c:4032:28: warning: variable ‘ext’ set but not used [-Wunused-but-set-variable]
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
---
net/netfilter/nf_tables_api.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index f180856..4a3085f 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -3361,7 +3361,6 @@ static int nf_tables_newset(struct net *net, struct sock *nlsk,
struct nft_ctx ctx;
char *name;
unsigned int size;
- bool create;
u64 timeout;
u32 ktype, dtype, flags, policy, gc_int, objtype;
struct nft_set_desc desc;
@@ -3462,8 +3461,6 @@ static int nf_tables_newset(struct net *net, struct sock *nlsk,
return err;
}
- create = nlh->nlmsg_flags & NLM_F_CREATE ? true : false;
-
table = nft_table_lookup(net, nla[NFTA_SET_TABLE], family, genmask);
if (IS_ERR(table)) {
NL_SET_BAD_ATTR(extack, nla[NFTA_SET_TABLE]);
@@ -4029,7 +4026,6 @@ static int nft_get_set_elem(struct nft_ctx *ctx, struct nft_set *set,
const struct nlattr *attr)
{
struct nlattr *nla[NFTA_SET_ELEM_MAX + 1];
- const struct nft_set_ext *ext;
struct nft_data_desc desc;
struct nft_set_elem elem;
struct sk_buff *skb;
@@ -4063,7 +4059,6 @@ static int nft_get_set_elem(struct nft_ctx *ctx, struct nft_set *set,
return PTR_ERR(priv);
elem.priv = priv;
- ext = nft_set_elem_ext(set, &elem);
err = -ENOMEM;
skb = nlmsg_new(NLMSG_GOODSIZE, GFP_ATOMIC);
--
2.7.0
^ permalink raw reply related
* [PATCH v2 net-next 0/5] tcp: add 4 new stats
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
From: Wei Wang <weiwan@google.com>
This patch series adds 3 RFC4898 stats:
1. tcpEStatsPerfHCDataOctetsOut
2. tcpEStatsPerfOctetsRetrans
3. tcpEStatsStackDSACKDups
and an addtional stat to record the number of data packet reordering
events seen:
4. tcp_reord_seen
Together with the existing stats, application can use them to measure
the retransmission rate in bytes, exclude spurious retransmissions
reflected by DSACK, and keep track of the reordering events on live
connections.
In particular the networks with different MTUs make bytes-based loss stats
more useful. Google servers have been using these stats for many years to
instrument transport and network performance.
Note: The first patch is a refactor to add a helper to calculate
opt_stats size in order to make later changes cleaner.
Wei Wang (5):
tcp: add a helper to calculate size of opt_stats
tcp: add data bytes sent stats
tcp: add data bytes retransmitted stats
tcp: add dsack blocks received stats
tcp: add stat of data packet reordering events
include/linux/tcp.h | 13 ++++++++++--
include/uapi/linux/tcp.h | 10 ++++++++-
net/ipv4/tcp.c | 46 +++++++++++++++++++++++++++++++++++++---
net/ipv4/tcp_input.c | 4 +++-
net/ipv4/tcp_output.c | 2 ++
net/ipv4/tcp_recovery.c | 2 +-
6 files changed, 69 insertions(+), 8 deletions(-)
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply
* [PATCH v2 net-next 1/5] tcp: add a helper to calculate size of opt_stats
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
In-Reply-To: <20180801004624.154346-1-tracywwnj@gmail.com>
From: Wei Wang <weiwan@google.com>
This is to refactor the calculation of the size of opt_stats to a helper
function to make the code cleaner and easier for later changes.
Suggested-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Yuchung Cheng <ycheng@google.com>
---
net/ipv4/tcp.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index f3bfb9f29520..27bbe6a792b7 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -3205,6 +3205,29 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
}
EXPORT_SYMBOL_GPL(tcp_get_info);
+static size_t tcp_opt_stats_get_size(void)
+{
+ return
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BUSY */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_RWND_LIMITED */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_SNDBUF_LIMITED */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_DATA_SEGS_OUT */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_TOTAL_RETRANS */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_PACING_RATE */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_DELIVERY_RATE */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_SND_CWND */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_REORDERING */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_MIN_RTT */
+ nla_total_size(sizeof(u8)) + /* TCP_NLA_RECUR_RETRANS */
+ nla_total_size(sizeof(u8)) + /* TCP_NLA_DELIVERY_RATE_APP_LMT */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_SNDQ_SIZE */
+ nla_total_size(sizeof(u8)) + /* TCP_NLA_CA_STATE */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_SND_SSTHRESH */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED_CE */
+ 0;
+}
+
struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
{
const struct tcp_sock *tp = tcp_sk(sk);
@@ -3213,9 +3236,7 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
u64 rate64;
u32 rate;
- stats = alloc_skb(7 * nla_total_size_64bit(sizeof(u64)) +
- 7 * nla_total_size(sizeof(u32)) +
- 3 * nla_total_size(sizeof(u8)), GFP_ATOMIC);
+ stats = alloc_skb(tcp_opt_stats_get_size(), GFP_ATOMIC);
if (!stats)
return NULL;
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply related
* [PATCH v2 net-next 2/5] tcp: add data bytes sent stats
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
In-Reply-To: <20180801004624.154346-2-tracywwnj@gmail.com>
From: Wei Wang <weiwan@google.com>
Introduce a new TCP stat to record the number of bytes sent
(RFC4898 tcpEStatsPerfHCDataOctetsOut) and expose it in both tcp_info
(TCP_INFO) and opt_stats (SOF_TIMESTAMPING_OPT_STATS).
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Yuchung Cheng <ycheng@google.com>
---
include/linux/tcp.h | 3 +++
include/uapi/linux/tcp.h | 4 +++-
net/ipv4/tcp.c | 6 ++++++
net/ipv4/tcp_output.c | 1 +
4 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index 58a8d7d71354..d0798dcd2cab 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -181,6 +181,9 @@ struct tcp_sock {
u32 data_segs_out; /* RFC4898 tcpEStatsPerfDataSegsOut
* total number of data segments sent.
*/
+ u64 bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut
+ * total number of data bytes sent.
+ */
u64 bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked
* sum(delta(snd_una)), or how many bytes
* were acked.
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index e3f6ed8a7064..1c70ed287c3b 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -235,6 +235,8 @@ struct tcp_info {
__u32 tcpi_delivered;
__u32 tcpi_delivered_ce;
+
+ __u64 tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
@@ -257,7 +259,7 @@ enum {
TCP_NLA_SND_SSTHRESH, /* Slow start size threshold */
TCP_NLA_DELIVERED, /* Data pkts delivered incl. out-of-order */
TCP_NLA_DELIVERED_CE, /* Like above but only ones w/ CE marks */
-
+ TCP_NLA_BYTES_SENT, /* Data bytes sent including retransmission */
};
/* for TCP_MD5SIG socket option */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 27bbe6a792b7..873cb9968ff5 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2594,6 +2594,7 @@ int tcp_disconnect(struct sock *sk, int flags)
sk->sk_rx_dst = NULL;
tcp_saved_syn_free(tp);
tp->compressed_ack = 0;
+ tp->bytes_sent = 0;
/* Clean up fastopen related fields */
tcp_free_fastopen_req(tp);
@@ -3201,6 +3202,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_delivery_rate = rate64;
info->tcpi_delivered = tp->delivered;
info->tcpi_delivered_ce = tp->delivered_ce;
+ info->tcpi_bytes_sent = tp->bytes_sent;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
@@ -3225,6 +3227,7 @@ static size_t tcp_opt_stats_get_size(void)
nla_total_size(sizeof(u32)) + /* TCP_NLA_SND_SSTHRESH */
nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED */
nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED_CE */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_SENT */
0;
}
@@ -3272,6 +3275,9 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
nla_put_u32(stats, TCP_NLA_SNDQ_SIZE, tp->write_seq - tp->snd_una);
nla_put_u8(stats, TCP_NLA_CA_STATE, inet_csk(sk)->icsk_ca_state);
+ nla_put_u64_64bit(stats, TCP_NLA_BYTES_SENT, tp->bytes_sent,
+ TCP_NLA_PAD);
+
return stats;
}
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 490df62f26d4..861531fe0e97 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1136,6 +1136,7 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb,
if (skb->len != tcp_header_size) {
tcp_event_data_sent(tp, sk);
tp->data_segs_out += tcp_skb_pcount(skb);
+ tp->bytes_sent += skb->len - tcp_header_size;
tcp_internal_pacing(sk, skb);
}
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply related
* [PATCH v2 net-next 3/5] tcp: add data bytes retransmitted stats
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
In-Reply-To: <20180801004624.154346-3-tracywwnj@gmail.com>
From: Wei Wang <weiwan@google.com>
Introduce a new TCP stat to record the number of bytes retransmitted
(RFC4898 tcpEStatsPerfOctetsRetrans) and expose it in both tcp_info
(TCP_INFO) and opt_stats (SOF_TIMESTAMPING_OPT_STATS).
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Yuchung Cheng <ycheng@google.com>
---
include/linux/tcp.h | 3 +++
include/uapi/linux/tcp.h | 2 ++
net/ipv4/tcp.c | 5 +++++
net/ipv4/tcp_output.c | 1 +
4 files changed, 11 insertions(+)
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index d0798dcd2cab..fb67f9a51b95 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -333,6 +333,9 @@ struct tcp_sock {
* the first SYN. */
u32 undo_marker; /* snd_una upon a new recovery episode. */
int undo_retrans; /* number of undoable retransmissions. */
+ u64 bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans
+ * Total data bytes retransmitted
+ */
u32 total_retrans; /* Total retransmits for entire connection */
u32 urg_seq; /* Seq of received urgent pointer */
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 1c70ed287c3b..c31f5100b744 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -237,6 +237,7 @@ struct tcp_info {
__u32 tcpi_delivered_ce;
__u64 tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
+ __u64 tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
@@ -260,6 +261,7 @@ enum {
TCP_NLA_DELIVERED, /* Data pkts delivered incl. out-of-order */
TCP_NLA_DELIVERED_CE, /* Like above but only ones w/ CE marks */
TCP_NLA_BYTES_SENT, /* Data bytes sent including retransmission */
+ TCP_NLA_BYTES_RETRANS, /* Data bytes retransmitted */
};
/* for TCP_MD5SIG socket option */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 873cb9968ff5..5ed1be88e922 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2595,6 +2595,7 @@ int tcp_disconnect(struct sock *sk, int flags)
tcp_saved_syn_free(tp);
tp->compressed_ack = 0;
tp->bytes_sent = 0;
+ tp->bytes_retrans = 0;
/* Clean up fastopen related fields */
tcp_free_fastopen_req(tp);
@@ -3203,6 +3204,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_delivered = tp->delivered;
info->tcpi_delivered_ce = tp->delivered_ce;
info->tcpi_bytes_sent = tp->bytes_sent;
+ info->tcpi_bytes_retrans = tp->bytes_retrans;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
@@ -3228,6 +3230,7 @@ static size_t tcp_opt_stats_get_size(void)
nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED */
nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED_CE */
nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_SENT */
+ nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_RETRANS */
0;
}
@@ -3277,6 +3280,8 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
nla_put_u64_64bit(stats, TCP_NLA_BYTES_SENT, tp->bytes_sent,
TCP_NLA_PAD);
+ nla_put_u64_64bit(stats, TCP_NLA_BYTES_RETRANS, tp->bytes_retrans,
+ TCP_NLA_PAD);
return stats;
}
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 861531fe0e97..50cabf7656f3 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -2871,6 +2871,7 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)
if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_SYN)
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPSYNRETRANS);
tp->total_retrans += segs;
+ tp->bytes_retrans += skb->len;
/* make sure skb->data is aligned on arches that require it
* and check if ack-trimming & collapsing extended the headroom
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply related
* [PATCH v2 net-next 4/5] tcp: add dsack blocks received stats
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
In-Reply-To: <20180801004624.154346-4-tracywwnj@gmail.com>
From: Wei Wang <weiwan@google.com>
Introduce a new TCP stat to record the number of DSACK blocks received
(RFC4989 tcpEStatsStackDSACKDups) and expose it in both tcp_info
(TCP_INFO) and opt_stats (SOF_TIMESTAMPING_OPT_STATS).
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Yuchung Cheng <ycheng@google.com>
---
include/linux/tcp.h | 3 +++
include/uapi/linux/tcp.h | 2 ++
net/ipv4/tcp.c | 4 ++++
net/ipv4/tcp_input.c | 1 +
4 files changed, 10 insertions(+)
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index fb67f9a51b95..da6281c549a5 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -188,6 +188,9 @@ struct tcp_sock {
* sum(delta(snd_una)), or how many bytes
* were acked.
*/
+ u32 dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups
+ * total number of DSACK blocks received
+ */
u32 snd_una; /* First byte we want an ack for */
u32 snd_sml; /* Last byte of the most recently transmitted small packet */
u32 rcv_tstamp; /* timestamp of last received ACK (for keepalives) */
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index c31f5100b744..0e1c0aec0153 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -238,6 +238,7 @@ struct tcp_info {
__u64 tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
__u64 tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
+ __u32 tcpi_dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
@@ -262,6 +263,7 @@ enum {
TCP_NLA_DELIVERED_CE, /* Like above but only ones w/ CE marks */
TCP_NLA_BYTES_SENT, /* Data bytes sent including retransmission */
TCP_NLA_BYTES_RETRANS, /* Data bytes retransmitted */
+ TCP_NLA_DSACK_DUPS, /* DSACK blocks received */
};
/* for TCP_MD5SIG socket option */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 5ed1be88e922..d6232b598cae 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2596,6 +2596,7 @@ int tcp_disconnect(struct sock *sk, int flags)
tp->compressed_ack = 0;
tp->bytes_sent = 0;
tp->bytes_retrans = 0;
+ tp->dsack_dups = 0;
/* Clean up fastopen related fields */
tcp_free_fastopen_req(tp);
@@ -3205,6 +3206,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_delivered_ce = tp->delivered_ce;
info->tcpi_bytes_sent = tp->bytes_sent;
info->tcpi_bytes_retrans = tp->bytes_retrans;
+ info->tcpi_dsack_dups = tp->dsack_dups;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
@@ -3231,6 +3233,7 @@ static size_t tcp_opt_stats_get_size(void)
nla_total_size(sizeof(u32)) + /* TCP_NLA_DELIVERED_CE */
nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_SENT */
nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_RETRANS */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_DSACK_DUPS */
0;
}
@@ -3282,6 +3285,7 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
TCP_NLA_PAD);
nla_put_u64_64bit(stats, TCP_NLA_BYTES_RETRANS, tp->bytes_retrans,
TCP_NLA_PAD);
+ nla_put_u32(stats, TCP_NLA_DSACK_DUPS, tp->dsack_dups);
return stats;
}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index d51fa358b2b1..fbc85ff7d71d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -874,6 +874,7 @@ static void tcp_dsack_seen(struct tcp_sock *tp)
{
tp->rx_opt.sack_ok |= TCP_DSACK_SEEN;
tp->rack.dsack_seen = 1;
+ tp->dsack_dups++;
}
/* It's reordering when higher sequence was delivered (i.e. sacked) before
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply related
* [PATCH v2 net-next 5/5] tcp: add stat of data packet reordering events
From: Wei Wang @ 2018-08-01 0:46 UTC (permalink / raw)
To: David Miller, netdev
Cc: Eric Dumazet, Neal Cardwell, Soheil Hassas Yeganeh, Yuchung Cheng,
Wei Wang
In-Reply-To: <20180801004624.154346-5-tracywwnj@gmail.com>
From: Wei Wang <weiwan@google.com>
Introduce a new TCP stats to record the number of reordering events seen
and expose it in both tcp_info (TCP_INFO) and opt_stats
(SOF_TIMESTAMPING_OPT_STATS).
Application can use this stats to track the frequency of the reordering
events in addition to the existing reordering stats which tracks the
magnitude of the latest reordering event.
Note: this new stats tracks reordering events triggered by ACKs, which
could often be fewer than the actual number of packets being delivered
out-of-order.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Yuchung Cheng <ycheng@google.com>
---
include/linux/tcp.h | 4 ++--
include/uapi/linux/tcp.h | 2 ++
net/ipv4/tcp.c | 4 ++++
net/ipv4/tcp_input.c | 3 ++-
net/ipv4/tcp_recovery.c | 2 +-
5 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index da6281c549a5..263e37271afd 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -220,8 +220,7 @@ struct tcp_sock {
#define TCP_RACK_RECOVERY_THRESH 16
u8 reo_wnd_persist:5, /* No. of recovery since last adj */
dsack_seen:1, /* Whether DSACK seen after last adj */
- advanced:1, /* mstamp advanced since last lost marking */
- reord:1; /* reordering detected */
+ advanced:1; /* mstamp advanced since last lost marking */
} rack;
u16 advmss; /* Advertised MSS */
u8 compressed_ack;
@@ -267,6 +266,7 @@ struct tcp_sock {
u8 ecn_flags; /* ECN status bits. */
u8 keepalive_probes; /* num of allowed keep alive probes */
u32 reordering; /* Packet reordering metric. */
+ u32 reord_seen; /* number of data packet reordering events */
u32 snd_up; /* Urgent pointer */
/*
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 0e1c0aec0153..e02d31986ff9 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -239,6 +239,7 @@ struct tcp_info {
__u64 tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
__u64 tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
__u32 tcpi_dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups */
+ __u32 tcpi_reord_seen; /* reordering events seen */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
@@ -264,6 +265,7 @@ enum {
TCP_NLA_BYTES_SENT, /* Data bytes sent including retransmission */
TCP_NLA_BYTES_RETRANS, /* Data bytes retransmitted */
TCP_NLA_DSACK_DUPS, /* DSACK blocks received */
+ TCP_NLA_REORD_SEEN, /* reordering events seen */
};
/* for TCP_MD5SIG socket option */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index d6232b598cae..31fa1c080f28 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2597,6 +2597,7 @@ int tcp_disconnect(struct sock *sk, int flags)
tp->bytes_sent = 0;
tp->bytes_retrans = 0;
tp->dsack_dups = 0;
+ tp->reord_seen = 0;
/* Clean up fastopen related fields */
tcp_free_fastopen_req(tp);
@@ -3207,6 +3208,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_bytes_sent = tp->bytes_sent;
info->tcpi_bytes_retrans = tp->bytes_retrans;
info->tcpi_dsack_dups = tp->dsack_dups;
+ info->tcpi_reord_seen = tp->reord_seen;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
@@ -3234,6 +3236,7 @@ static size_t tcp_opt_stats_get_size(void)
nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_SENT */
nla_total_size_64bit(sizeof(u64)) + /* TCP_NLA_BYTES_RETRANS */
nla_total_size(sizeof(u32)) + /* TCP_NLA_DSACK_DUPS */
+ nla_total_size(sizeof(u32)) + /* TCP_NLA_REORD_SEEN */
0;
}
@@ -3286,6 +3289,7 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk)
nla_put_u64_64bit(stats, TCP_NLA_BYTES_RETRANS, tp->bytes_retrans,
TCP_NLA_PAD);
nla_put_u32(stats, TCP_NLA_DSACK_DUPS, tp->dsack_dups);
+ nla_put_u32(stats, TCP_NLA_REORD_SEEN, tp->reord_seen);
return stats;
}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index fbc85ff7d71d..3d6156f07a8d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -906,8 +906,8 @@ static void tcp_check_sack_reordering(struct sock *sk, const u32 low_seq,
sock_net(sk)->ipv4.sysctl_tcp_max_reordering);
}
- tp->rack.reord = 1;
/* This exciting event is worth to be remembered. 8) */
+ tp->reord_seen++;
NET_INC_STATS(sock_net(sk),
ts ? LINUX_MIB_TCPTSREORDER : LINUX_MIB_TCPSACKREORDER);
}
@@ -1871,6 +1871,7 @@ static void tcp_check_reno_reordering(struct sock *sk, const int addend)
tp->reordering = min_t(u32, tp->packets_out + addend,
sock_net(sk)->ipv4.sysctl_tcp_max_reordering);
+ tp->reord_seen++;
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPRENOREORDER);
}
diff --git a/net/ipv4/tcp_recovery.c b/net/ipv4/tcp_recovery.c
index 71593e4400ab..c81aadff769b 100644
--- a/net/ipv4/tcp_recovery.c
+++ b/net/ipv4/tcp_recovery.c
@@ -25,7 +25,7 @@ static u32 tcp_rack_reo_wnd(const struct sock *sk)
{
struct tcp_sock *tp = tcp_sk(sk);
- if (!tp->rack.reord) {
+ if (!tp->reord_seen) {
/* If reordering has not been observed, be aggressive during
* the recovery or starting the recovery by DUPACK threshold.
*/
--
2.18.0.345.g5c9ce644c3-goog
^ permalink raw reply related
* [PATCH net-next v7 0/4] net: vhost: improve performance when enable busyloop
From: xiangxia.m.yue @ 2018-08-01 3:00 UTC (permalink / raw)
To: jasowang; +Cc: netdev, virtualization, mst
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
This patches improve the guest receive performance.
On the handle_tx side, we poll the sock receive queue
at the same time. handle_rx do that in the same way.
For more performance report, see patch 4.
v6->v7:
fix issue and rebase codes:
1. on tx, busypoll will vhost_net_disable/enable_vq rx vq.
[This is suggested by Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>]
2. introduce common helper vhost_net_busy_poll_try_queue().
v5->v6:
rebase the codes.
Tonghao Zhang (4):
net: vhost: lock the vqs one by one
net: vhost: replace magic number of lock annotation
net: vhost: factor out busy polling logic to vhost_net_busy_poll()
net: vhost: add rx busy polling in tx path
drivers/vhost/net.c | 168 +++++++++++++++++++++++++++++++-------------------
drivers/vhost/vhost.c | 24 +++-----
2 files changed, 113 insertions(+), 79 deletions(-)
--
1.8.3.1
^ permalink raw reply
* [PATCH net-next v7 1/4] net: vhost: lock the vqs one by one
From: xiangxia.m.yue @ 2018-08-01 3:00 UTC (permalink / raw)
To: jasowang; +Cc: netdev, virtualization, mst
In-Reply-To: <1533092454-37196-1-git-send-email-xiangxia.m.yue@gmail.com>
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
This patch changes the way that lock all vqs
at the same, to lock them one by one. It will
be used for next patch to avoid the deadlock.
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/vhost.c | 24 +++++++-----------------
1 file changed, 7 insertions(+), 17 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index a502f1a..a1c06e7 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -294,8 +294,11 @@ static void vhost_vq_meta_reset(struct vhost_dev *d)
{
int i;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->nvqs; ++i) {
+ mutex_lock(&d->vqs[i]->mutex);
__vhost_vq_meta_reset(d->vqs[i]);
+ mutex_unlock(&d->vqs[i]->mutex);
+ }
}
static void vhost_vq_reset(struct vhost_dev *dev,
@@ -890,20 +893,6 @@ static inline void __user *__vhost_get_user(struct vhost_virtqueue *vq,
#define vhost_get_used(vq, x, ptr) \
vhost_get_user(vq, x, ptr, VHOST_ADDR_USED)
-static void vhost_dev_lock_vqs(struct vhost_dev *d)
-{
- int i = 0;
- for (i = 0; i < d->nvqs; ++i)
- mutex_lock_nested(&d->vqs[i]->mutex, i);
-}
-
-static void vhost_dev_unlock_vqs(struct vhost_dev *d)
-{
- int i = 0;
- for (i = 0; i < d->nvqs; ++i)
- mutex_unlock(&d->vqs[i]->mutex);
-}
-
static int vhost_new_umem_range(struct vhost_umem *umem,
u64 start, u64 size, u64 end,
u64 userspace_addr, int perm)
@@ -953,7 +942,10 @@ static void vhost_iotlb_notify_vq(struct vhost_dev *d,
if (msg->iova <= vq_msg->iova &&
msg->iova + msg->size - 1 > vq_msg->iova &&
vq_msg->type == VHOST_IOTLB_MISS) {
+ mutex_lock(&node->vq->mutex);
vhost_poll_queue(&node->vq->poll);
+ mutex_unlock(&node->vq->mutex);
+
list_del(&node->node);
kfree(node);
}
@@ -985,7 +977,6 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev,
int ret = 0;
mutex_lock(&dev->mutex);
- vhost_dev_lock_vqs(dev);
switch (msg->type) {
case VHOST_IOTLB_UPDATE:
if (!dev->iotlb) {
@@ -1019,7 +1010,6 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev,
break;
}
- vhost_dev_unlock_vqs(dev);
mutex_unlock(&dev->mutex);
return ret;
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next] tcp: remove set but not used variable 'skb_size'
From: Wei Yongjun @ 2018-08-01 1:59 UTC (permalink / raw)
To: Eric Dumazet, Alexey Kuznetsov, Hideaki YOSHIFUJI
Cc: Wei Yongjun, netdev, kernel-janitors
In-Reply-To: <1533048144-39731-1-git-send-email-weiyongjun1@huawei.com>
Fixes gcc '-Wunused-but-set-variable' warning:
net/ipv4/tcp_output.c: In function 'tcp_collapse_retrans':
net/ipv4/tcp_output.c:2700:6: warning:
variable 'skb_size' set but not used [-Wunused-but-set-variable]
int skb_size, next_skb_size;
^
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
---
net/ipv4/tcp_output.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 490df62..731e6d4 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -2697,9 +2697,8 @@ static bool tcp_collapse_retrans(struct sock *sk, struct sk_buff *skb)
{
struct tcp_sock *tp = tcp_sk(sk);
struct sk_buff *next_skb = skb_rb_next(skb);
- int skb_size, next_skb_size;
+ int next_skb_size;
- skb_size = skb->len;
next_skb_size = next_skb->len;
BUG_ON(tcp_skb_pcount(skb) != 1 || tcp_skb_pcount(next_skb) != 1);
^ permalink raw reply related
* Re: [PATCH net-next] tcp: remove set but not used variable 'skb_size'
From: Eric Dumazet @ 2018-08-01 2:36 UTC (permalink / raw)
To: Wei Yongjun, Eric Dumazet, Alexey Kuznetsov, Hideaki YOSHIFUJI
Cc: netdev, kernel-janitors
In-Reply-To: <1533088796-149451-1-git-send-email-weiyongjun1@huawei.com>
On 07/31/2018 06:59 PM, Wei Yongjun wrote:
> Fixes gcc '-Wunused-but-set-variable' warning:
>
> net/ipv4/tcp_output.c: In function 'tcp_collapse_retrans':
> net/ipv4/tcp_output.c:2700:6: warning:
> variable 'skb_size' set but not used [-Wunused-but-set-variable]
> int skb_size, next_skb_size;
> ^
>
> Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Thanks.
^ permalink raw reply
* Re: linux-next: manual merge of the net-next tree with the bpf tree
From: Yonghong Song @ 2018-08-01 4:23 UTC (permalink / raw)
To: Stephen Rothwell, David Miller, Networking, Daniel Borkmann,
Alexei Starovoitov
Cc: Linux-Next Mailing List, Linux Kernel Mailing List, Okash Khawaja
In-Reply-To: <20180801113508.54323706@canb.auug.org.au>
On 7/31/18 6:35 PM, Stephen Rothwell wrote:
> Hi all,
>
> Today's linux-next merge of the net-next tree got a conflict in:
>
> tools/bpf/bpftool/map.c
>
> between commit:
>
> 573b3aa69406 ("tools/bpftool: fix a percpu_array map dump problem")
>
> from the bpf tree and commit:
>
> 2d3feca8c44f ("bpf: btf: print map dump and lookup with btf info")
>
> from the net-next tree.
The merge looks good to me. Thanks!
>
> I fixed it up (see below) and can carry the fix as necessary. This
> is now fixed as far as linux-next is concerned, but any non trivial
> conflicts should be mentioned to your upstream maintainer when your tree
> is submitted for merging. You may also want to consider cooperating
> with the maintainer of the conflicting tree to minimise any particularly
> complex conflicts.
>
^ permalink raw reply
* RE: [PATCH 3/3] ptp_qoriq: convert to use module parameters for initialization
From: Y.b. Lu @ 2018-08-01 4:36 UTC (permalink / raw)
To: Richard Cochran
Cc: netdev@vger.kernel.org, Madalin-cristian Bucur, Rob Herring,
Shawn Guo, David S . Miller, devicetree@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
In-Reply-To: <20180730143043.qu66ttzwyjbw6si7@localhost>
Hi Richard,
> -----Original Message-----
> From: Richard Cochran [mailto:richardcochran@gmail.com]
> Sent: Monday, July 30, 2018 10:31 PM
> To: Y.b. Lu <yangbo.lu@nxp.com>
> Cc: netdev@vger.kernel.org; Madalin-cristian Bucur
> <madalin.bucur@nxp.com>; Rob Herring <robh+dt@kernel.org>; Shawn Guo
> <shawnguo@kernel.org>; David S . Miller <davem@davemloft.net>;
> devicetree@vger.kernel.org; linuxppc-dev@lists.ozlabs.org;
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 3/3] ptp_qoriq: convert to use module parameters for
> initialization
>
> On Mon, Jul 30, 2018 at 06:01:54PM +0800, Yangbo Lu wrote:
> > The ptp_qoriq driver initialized the 1588 timer with the
> > configurations provided by the properties of device tree node. For
> > example,
> >
> > fsl,tclk-period = <5>;
> > fsl,tmr-prsc = <2>;
> > fsl,tmr-add = <0xaaaaaaab>;
> > fsl,tmr-fiper1 = <999999995>;
> > fsl,tmr-fiper2 = <99990>;
> > fsl,max-adj = <499999999>;
> >
> > These things actually were runtime configurations which were not
> > proper to be put into dts.
>
> That is debatable. While I agree that the dts isn't ideal for these, still it is the
> lesser of two or more evils.
[Y.b. Lu] Ok. You're right indeed :)
>
> > This patch is to convert
> > to use module parameters for 1588 timer initialization, and to support
> > initial register values calculation.
>
> It is hard for me to understand how using module parameters improves the
> situation.
[Y.b. Lu] Actually I'm not sure whether module_param will be accepted to replace dts.
I thought the most possibility would be rejection before sending them out.
Just want suggestion and confirmation whether there is better idea than dts from your comments.
Since we should keep the dts, I will drop the module_param.
Could I add a function to calculate a set of default register values to initialize ptp timer when dts method failed to get required properties in driver?
I think this will be useful. The ptp timer on new platforms (you may see two dts patches in this patchset. Many platforms will be affected.) will work without these dts properties. If user want specific setting, they can set dts properties.
>
> > If the parameters are not provided, the driver will calculate register
> > values with a set of default parameters. With this patch, those dts
> > properties are no longer needed for new platform to support 1588
> > timer, and many QorIQ DPAA platforms (some P series and T series
> > platforms of PowerPC, and some LS series platforms of ARM64) could use
> > this driver for their fman ptp timer with default module parameters.
> > However, this patch didn't remove the dts method. Because there were
> > still many old platforms using the dts method. We need to clean up
> > their dts files, verify module parameters on them, and convert them to
> > the new method gradually in case of breaking any function.
>
> In addition, like it or not, because the dts is an ABI, you must continue support
> of the dts values as a legacy option.
[Y.b. Lu] I get your point now. The dts should be kept :)
>
> Thanks,
> Richard
^ permalink raw reply
* [PATCH net-next v7 2/4] net: vhost: replace magic number of lock annotation
From: xiangxia.m.yue @ 2018-08-01 3:00 UTC (permalink / raw)
To: jasowang; +Cc: mst, makita.toshiaki, virtualization, netdev, Tonghao Zhang
In-Reply-To: <1533092454-37196-1-git-send-email-xiangxia.m.yue@gmail.com>
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Use the VHOST_NET_VQ_XXX as a subclass for mutex_lock_nested.
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/net.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 367d802..32c1b52 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -712,7 +712,7 @@ static void handle_tx(struct vhost_net *net)
struct vhost_virtqueue *vq = &nvq->vq;
struct socket *sock;
- mutex_lock(&vq->mutex);
+ mutex_lock_nested(&vq->mutex, VHOST_NET_VQ_TX);
sock = vq->private_data;
if (!sock)
goto out;
@@ -777,7 +777,7 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk,
/* Flush batched heads first */
vhost_net_signal_used(rnvq);
/* Both tx vq and rx socket were polled here */
- mutex_lock_nested(&tvq->mutex, 1);
+ mutex_lock_nested(&tvq->mutex, VHOST_NET_VQ_TX);
vhost_disable_notify(&net->dev, tvq);
preempt_disable();
@@ -919,7 +919,7 @@ static void handle_rx(struct vhost_net *net)
__virtio16 num_buffers;
int recv_pkts = 0;
- mutex_lock_nested(&vq->mutex, 0);
+ mutex_lock_nested(&vq->mutex, VHOST_NET_VQ_RX);
sock = vq->private_data;
if (!sock)
goto out;
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next v7 3/4] net: vhost: factor out busy polling logic to vhost_net_busy_poll()
From: xiangxia.m.yue @ 2018-08-01 3:00 UTC (permalink / raw)
To: jasowang; +Cc: mst, makita.toshiaki, virtualization, netdev, Tonghao Zhang
In-Reply-To: <1533092454-37196-1-git-send-email-xiangxia.m.yue@gmail.com>
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Factor out generic busy polling logic and will be
used for in tx path in the next patch. And with the patch,
qemu can set differently the busyloop_timeout for rx queue.
In the handle_tx, the busypoll will vhost_net_disable/enable_vq
because we have poll the sock. This can improve performance.
[This is suggested by Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>]
And when the sock receive skb, we should queue the poll if necessary.
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
---
drivers/vhost/net.c | 131 ++++++++++++++++++++++++++++++++++++----------------
1 file changed, 91 insertions(+), 40 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 32c1b52..5b45463 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -440,6 +440,95 @@ static void vhost_net_signal_used(struct vhost_net_virtqueue *nvq)
nvq->done_idx = 0;
}
+static int sk_has_rx_data(struct sock *sk)
+{
+ struct socket *sock = sk->sk_socket;
+
+ if (sock->ops->peek_len)
+ return sock->ops->peek_len(sock);
+
+ return skb_queue_empty(&sk->sk_receive_queue);
+}
+
+static void vhost_net_busy_poll_try_queue(struct vhost_net *net,
+ struct vhost_virtqueue *vq)
+{
+ if (!vhost_vq_avail_empty(&net->dev, vq)) {
+ vhost_poll_queue(&vq->poll);
+ } else if (unlikely(vhost_enable_notify(&net->dev, vq))) {
+ vhost_disable_notify(&net->dev, vq);
+ vhost_poll_queue(&vq->poll);
+ }
+}
+
+static void vhost_net_busy_poll_check(struct vhost_net *net,
+ struct vhost_virtqueue *rvq,
+ struct vhost_virtqueue *tvq,
+ bool rx)
+{
+ struct socket *sock = rvq->private_data;
+
+ if (rx)
+ vhost_net_busy_poll_try_queue(net, tvq);
+ else if (sock && sk_has_rx_data(sock->sk))
+ vhost_net_busy_poll_try_queue(net, rvq);
+ else {
+ /* On tx here, sock has no rx data, so we
+ * will wait for sock wakeup for rx, and
+ * vhost_enable_notify() is not needed. */
+ }
+}
+
+static void vhost_net_busy_poll(struct vhost_net *net,
+ struct vhost_virtqueue *rvq,
+ struct vhost_virtqueue *tvq,
+ bool *busyloop_intr,
+ bool rx)
+{
+ unsigned long busyloop_timeout;
+ unsigned long endtime;
+ struct socket *sock;
+ struct vhost_virtqueue *vq = rx ? tvq : rvq;
+
+ mutex_lock_nested(&vq->mutex, rx ? VHOST_NET_VQ_TX: VHOST_NET_VQ_RX);
+ vhost_disable_notify(&net->dev, vq);
+ sock = rvq->private_data;
+
+ busyloop_timeout = rx ? rvq->busyloop_timeout:
+ tvq->busyloop_timeout;
+
+
+ /* Busypoll the sock, so don't need rx wakeups during it. */
+ if (!rx)
+ vhost_net_disable_vq(net, vq);
+
+ preempt_disable();
+ endtime = busy_clock() + busyloop_timeout;
+
+ while (vhost_can_busy_poll(endtime)) {
+ if (vhost_has_work(&net->dev)) {
+ *busyloop_intr = true;
+ break;
+ }
+
+ if ((sock && sk_has_rx_data(sock->sk) &&
+ !vhost_vq_avail_empty(&net->dev, rvq)) ||
+ !vhost_vq_avail_empty(&net->dev, tvq))
+ break;
+
+ cpu_relax();
+ }
+
+ preempt_enable();
+
+ if (!rx)
+ vhost_net_enable_vq(net, vq);
+
+ vhost_net_busy_poll_check(net, rvq, tvq, rx);
+
+ mutex_unlock(&vq->mutex);
+}
+
static int vhost_net_tx_get_vq_desc(struct vhost_net *net,
struct vhost_net_virtqueue *nvq,
unsigned int *out_num, unsigned int *in_num,
@@ -753,16 +842,6 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk)
return len;
}
-static int sk_has_rx_data(struct sock *sk)
-{
- struct socket *sock = sk->sk_socket;
-
- if (sock->ops->peek_len)
- return sock->ops->peek_len(sock);
-
- return skb_queue_empty(&sk->sk_receive_queue);
-}
-
static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk,
bool *busyloop_intr)
{
@@ -770,41 +849,13 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk,
struct vhost_net_virtqueue *tnvq = &net->vqs[VHOST_NET_VQ_TX];
struct vhost_virtqueue *rvq = &rnvq->vq;
struct vhost_virtqueue *tvq = &tnvq->vq;
- unsigned long uninitialized_var(endtime);
int len = peek_head_len(rnvq, sk);
- if (!len && tvq->busyloop_timeout) {
+ if (!len && rvq->busyloop_timeout) {
/* Flush batched heads first */
vhost_net_signal_used(rnvq);
/* Both tx vq and rx socket were polled here */
- mutex_lock_nested(&tvq->mutex, VHOST_NET_VQ_TX);
- vhost_disable_notify(&net->dev, tvq);
-
- preempt_disable();
- endtime = busy_clock() + tvq->busyloop_timeout;
-
- while (vhost_can_busy_poll(endtime)) {
- if (vhost_has_work(&net->dev)) {
- *busyloop_intr = true;
- break;
- }
- if ((sk_has_rx_data(sk) &&
- !vhost_vq_avail_empty(&net->dev, rvq)) ||
- !vhost_vq_avail_empty(&net->dev, tvq))
- break;
- cpu_relax();
- }
-
- preempt_enable();
-
- if (!vhost_vq_avail_empty(&net->dev, tvq)) {
- vhost_poll_queue(&tvq->poll);
- } else if (unlikely(vhost_enable_notify(&net->dev, tvq))) {
- vhost_disable_notify(&net->dev, tvq);
- vhost_poll_queue(&tvq->poll);
- }
-
- mutex_unlock(&tvq->mutex);
+ vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, true);
len = peek_head_len(rnvq, sk);
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next v7 4/4] net: vhost: add rx busy polling in tx path
From: xiangxia.m.yue @ 2018-08-01 3:00 UTC (permalink / raw)
To: jasowang; +Cc: mst, makita.toshiaki, virtualization, netdev, Tonghao Zhang
In-Reply-To: <1533092454-37196-1-git-send-email-xiangxia.m.yue@gmail.com>
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
This patch improves the guest receive performance.
On the handle_tx side, we poll the sock receive queue at the
same time. handle_rx do that in the same way.
We set the poll-us=100us and use the netperf to test throughput
and mean latency. When running the tests, the vhost-net kthread
of that VM, is alway 100% CPU. The commands are shown as below.
Rx performance is greatly improved by this patch. There is not
notable performance change on tx with this series though. This
patch is useful for bi-directional traffic.
netperf -H IP -t TCP_STREAM -l 20 -- -O "THROUGHPUT, THROUGHPUT_UNITS, MEAN_LATENCY"
Topology:
[Host] ->linux bridge -> tap vhost-net ->[Guest]
TCP_STREAM:
* Without the patch: 19842.95 Mbps, 6.50 us mean latency
* With the patch: 38570.00 Mbps, 3.43 us mean latency
Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
---
drivers/vhost/net.c | 33 +++++++++++++--------------------
1 file changed, 13 insertions(+), 20 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 5b45463..b56db65 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -530,31 +530,24 @@ static void vhost_net_busy_poll(struct vhost_net *net,
}
static int vhost_net_tx_get_vq_desc(struct vhost_net *net,
- struct vhost_net_virtqueue *nvq,
+ struct vhost_net_virtqueue *tnvq,
unsigned int *out_num, unsigned int *in_num,
bool *busyloop_intr)
{
- struct vhost_virtqueue *vq = &nvq->vq;
- unsigned long uninitialized_var(endtime);
- int r = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov),
+ struct vhost_net_virtqueue *rnvq = &net->vqs[VHOST_NET_VQ_RX];
+ struct vhost_virtqueue *rvq = &rnvq->vq;
+ struct vhost_virtqueue *tvq = &tnvq->vq;
+
+ int r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov),
out_num, in_num, NULL, NULL);
- if (r == vq->num && vq->busyloop_timeout) {
- if (!vhost_sock_zcopy(vq->private_data))
- vhost_net_signal_used(nvq);
- preempt_disable();
- endtime = busy_clock() + vq->busyloop_timeout;
- while (vhost_can_busy_poll(endtime)) {
- if (vhost_has_work(vq->dev)) {
- *busyloop_intr = true;
- break;
- }
- if (!vhost_vq_avail_empty(vq->dev, vq))
- break;
- cpu_relax();
- }
- preempt_enable();
- r = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov),
+ if (r == tvq->num && tvq->busyloop_timeout) {
+ if (!vhost_sock_zcopy(tvq->private_data))
+ vhost_net_signal_used(tnvq);
+
+ vhost_net_busy_poll(net, rvq, tvq, busyloop_intr, false);
+
+ r = vhost_get_vq_desc(tvq, tvq->iov, ARRAY_SIZE(tvq->iov),
out_num, in_num, NULL, NULL);
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH bpf-next 00/13] docs: Convert BPF filter.txt to RST
From: Tobin C. Harding @ 2018-08-01 5:08 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
Hi,
Sending this to BPF maintainers. Recent patch set to Documentation/bpf
went through bpf tree so I'm guessing this one will too. I rekon there
will be less merge conflicts through your tree. I do not know exactly
the path for docs into the mainline or who fixes up merge conflicts
though so if this is better done another way please say so.
Patch set applies on top of bpf-next commit: ("2e96187b6967 Merge branch
'bpf-docs-rst-improvements'")
Use of --subject-prefix='PATCH bpf-next' was a guess.
While doing the final read over of this set I realised that perhaps we
should have just moved this file to Documentation/bpf?
Anyways, here is the conversion of filter.txt to filter.rst split up
into a bunch of patches to ease review. I failed to fully grok the
contents of this file so there may be mistakes in the 'Fix various
minor tyos' patch.
Also the diff looks like it contains spurious whitespace changes in
places, I think this is when spaces have been replaced by tabs. I'm
guessing you guys are experienced at reading patches so can tell this
already - it took me a bit of thinking :)
Some of the formatting for code snippets is subjective, if you'd like me
to change it please say so. In particular I added a line of whitespace
before error output in a bunch of the snippets. Also indented slightly
differently to the original so it looks okay in HTML. I did not check
any other rendered format.
thanks,
Tobin.
Tobin C. Harding (13):
docs: net: Rename filter.txt to filter.rst
docs: Update references to filter.rst
docs: net: Add filter to index toctree
docs: net: Use correct heading adornments
docs: net: Fix indentation issues for code snippets
docs: net: Remove non-standard identifiers
docs: net: Use double ticks instead of single quote
docs: net: Use double ticks instead of single tick
docs: net: Use lowercase 'k' for kernel
docs: net: Embed reference to seccomp_filter
docs: net: Use correct RST list construct
docs: net: Fix various minor typos
docs: net: Fix authors list to render as a list
Documentation/bpf/index.rst | 9 +-
Documentation/networking/00-INDEX | 2 -
.../networking/{filter.txt => filter.rst} | 1007 +++++++++--------
Documentation/networking/index.rst | 1 +
Documentation/networking/packet_mmap.txt | 2 +-
.../userspace-api/seccomp_filter.rst | 2 +
MAINTAINERS | 2 +-
tools/bpf/bpf_asm.c | 2 +-
tools/bpf/bpf_dbg.c | 2 +-
9 files changed, 545 insertions(+), 484 deletions(-)
rename Documentation/networking/{filter.txt => filter.rst} (69%)
--
2.17.1
^ permalink raw reply
* [PATCH bpf-next 01/13] docs: net: Rename filter.txt to filter.rst
From: Tobin C. Harding @ 2018-08-01 5:08 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
In preparation for conversion to RST format. Rename file in a single
commit to ease review.
Rename Documentation/networking/filter.txt to
Documentation/networking/filter.rst
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/00-INDEX | 2 --
Documentation/networking/{filter.txt => filter.rst} | 0
MAINTAINERS | 2 +-
3 files changed, 1 insertion(+), 3 deletions(-)
rename Documentation/networking/{filter.txt => filter.rst} (100%)
diff --git a/Documentation/networking/00-INDEX b/Documentation/networking/00-INDEX
index 1e5153ed8990..37c65b3a90be 100644
--- a/Documentation/networking/00-INDEX
+++ b/Documentation/networking/00-INDEX
@@ -78,8 +78,6 @@ eql.txt
- serial IP load balancing
fib_trie.txt
- Level Compressed Trie (LC-trie) notes: a structure for routing.
-filter.txt
- - Linux Socket Filtering
fore200e.txt
- FORE Systems PCA-200E/SBA-200E ATM NIC driver info.
framerelay.txt
diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.rst
similarity index 100%
rename from Documentation/networking/filter.txt
rename to Documentation/networking/filter.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index fce04d747a1a..a95431e90ab9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2746,7 +2746,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
Q: https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147
S: Supported
F: arch/x86/net/bpf_jit*
-F: Documentation/networking/filter.txt
+F: Documentation/networking/filter.rst
F: Documentation/bpf/
F: include/linux/bpf*
F: include/linux/filter.h
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next 03/13] docs: net: Add filter to index toctree
From: Tobin C. Harding @ 2018-08-01 5:08 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
filter.rst was just created (from filter.txt). We should include it in
the networking docs toctree.
Add filter to Documentation/networking/index.rst toctree.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/index.rst | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index f0ae9b65dfba..1f926bb347b1 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -19,6 +19,7 @@ Contents:
net_failover
alias
bridge
+ filter
.. only:: subproject
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next 04/13] docs: net: Use correct heading adornments
From: Tobin C. Harding @ 2018-08-01 5:08 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
Conversion to RST format requires the use of correct RST heading
adornments. While we are at it we can make sure spacing around headings
is uniform. Choose to use one empty line after any heading.
Use correct levels of heading adornment.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/filter.rst | 36 +++++++++++++++++------------
1 file changed, 21 insertions(+), 15 deletions(-)
diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
index 14ce8901c245..19325286780b 100644
--- a/Documentation/networking/filter.rst
+++ b/Documentation/networking/filter.rst
@@ -1,10 +1,11 @@
+=======================================================
Linux Socket Filtering aka Berkeley Packet Filter (BPF)
=======================================================
.. _bpf_filter:
Introduction
-------------
+============
Linux Socket Filtering (LSF) is derived from the Berkeley Packet Filter.
Though there are some distinct differences between the BSD and Linux
@@ -58,7 +59,7 @@ Conference Proceedings (USENIX'93). USENIX Association, Berkeley,
CA, USA, 2-2. [http://www.tcpdump.org/papers/bpf-usenix93.pdf]
Structure
----------
+=========
User space applications include <linux/filter.h> which contains the
following relevant structures:
@@ -169,7 +170,7 @@ implementors may wish to manually write test cases and thus need low-level
access to BPF code as well.
BPF engine and instruction set
-------------------------------
+==============================
Under tools/bpf/ there's a small helper tool called bpf_asm which can
be used to write low-level filters for example scenarios mentioned in the
@@ -462,7 +463,7 @@ breakpoints: 0 1
Exits bpf_dbg.
JIT compiler
-------------
+============
The Linux kernel has a built-in BPF JIT compiler for x86_64, SPARC, PowerPC,
ARM, ARM64, MIPS and s390 and can be enabled through CONFIG_BPF_JIT. The JIT
@@ -569,7 +570,8 @@ For BPF JIT developers, bpf_jit_disasm, bpf_asm and bpf_dbg provides a useful
toolchain for developing and testing the kernel's JIT compiler.
BPF kernel internals
---------------------
+====================
+
Internally, for the kernel interpreter, a different instruction set
format with similar underlying principles from BPF described in previous
paragraphs is being used. However, the instruction set format is modelled
@@ -843,7 +845,7 @@ descends all possible paths. It simulates execution of every insn and observes
the state change of registers and stack.
eBPF opcode encoding
---------------------
+====================
eBPF is reusing most of the opcode encoding from classic to simplify conversion
of classic BPF to eBPF. For arithmetic and jump instructions the 8-bit 'code'
@@ -1014,7 +1016,8 @@ Classic BPF has similar instruction: BPF_LD | BPF_W | BPF_IMM which loads
32-bit immediate value into a register.
eBPF verifier
--------------
+=============
+
The safety of the eBPF program is determined in two steps.
First step does DAG check to disallow loops and other CFG validation.
@@ -1107,7 +1110,8 @@ all use cases.
See details of eBPF verifier in kernel/bpf/verifier.c
Register value tracking
------------------------
+=======================
+
In order to determine the safety of an eBPF program, the verifier must track
the range of possible values in each register and also in each stack slot.
This is done with 'struct bpf_reg_state', defined in include/linux/
@@ -1175,7 +1179,7 @@ bytes (NET_IP_ALIGN) gives a 4-byte alignment and so word-sized accesses through
that pointer are safe.
Direct packet access
---------------------
+====================
In cls_bpf and act_bpf programs the verifier allows direct access to the packet
data via skb->data and skb->data_end pointers.
Ex:
@@ -1261,7 +1265,8 @@ which makes such programs easier to write comparing to LD_ABS insn
and significantly faster.
eBPF maps
----------
+=========
+
'maps' is a generic storage of different types for sharing data between kernel
and userspace.
@@ -1300,7 +1305,8 @@ The map is defined by:
. value size in bytes
Pruning
--------
+=======
+
The verifier does not actually walk all possible paths through the program. For
each new branch to analyse, the verifier looks at all the states it's previously
been in when at this instruction. If any of them contain the current state as a
@@ -1316,7 +1322,7 @@ registers it may hold). They must all be safe for the branch to be pruned.
This is implemented in states_equal().
Understanding eBPF verifier messages
-------------------------------------
+====================================
The following are few examples of invalid eBPF programs and verifier error
messages as seen in the log:
@@ -1447,7 +1453,7 @@ Error:
R0 invalid mem access 'imm'
Testing
--------
+=======
Next to the BPF toolchain, the kernel also ships a test module that contains
various test cases for classic and internal BPF that can be executed against
@@ -1461,13 +1467,13 @@ via insmod or modprobe against 'test_bpf' module. Results of the test cases
including timings in nsec can be found in the kernel log (dmesg).
Misc
-----
+====
Also trinity, the Linux syscall fuzzer, has built-in support for BPF and
SECCOMP-BPF kernel fuzzing.
Written by
-----------
+==========
The document was written in the hope that it is found useful and in order
to give potential BPF hackers or security auditors a better overview of
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next 06/13] docs: net: Remove non-standard identifiers
From: Tobin C. Harding @ 2018-08-01 5:09 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
File uses a custom identifier '**'. Documentation build is cleaner if
we use standard identifiers. Usage is not _really_ a list and its not
_really_ a section header. Lets just remove the double stars since
we just added code snippet format for following text so the identifiers
are redundant now.
Remove non-standard identifiers '**'.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/filter.rst | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
index 916e7ac01576..62630e8f023a 100644
--- a/Documentation/networking/filter.rst
+++ b/Documentation/networking/filter.rst
@@ -292,14 +292,14 @@ Possible BPF extensions are shown in the following table::
These extensions can also be prefixed with '#'.
Examples for low-level BPF:
-** ARP packets::
+ARP packets::
ldh [12]
jne #0x806, drop
ret #-1
drop: ret #0
-** IPv4 TCP packets::
+IPv4 TCP packets::
ldh [12]
jne #0x800, drop
@@ -308,14 +308,14 @@ Examples for low-level BPF:
ret #-1
drop: ret #0
-** (Accelerated) VLAN w/ id 10::
+(Accelerated) VLAN w/ id 10::
ld vlan_tci
jneq #10, drop
ret #-1
drop: ret #0
-** icmp random packet sampling, 1 in 4::
+icmp random packet sampling, 1 in 4::
ldh [12]
jne #0x800, drop
@@ -328,7 +328,7 @@ Examples for low-level BPF:
ret #-1
drop: ret #0
-** SECCOMP filter example::
+SECCOMP filter example::
ld [4] /* offsetof(struct seccomp_data, arch) */
jne #0xc000003e, bad /* AUDIT_ARCH_X86_64 */
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next 07/13] docs: net: Use double ticks instead of single quote
From: Tobin C. Harding @ 2018-08-01 5:09 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
Currently there are many places that use single quote to guard text.
RST has the double tick for code. Also the document is full of
identifiers. Using too many double ticks hinders reading the file in
text format. For this reason we only change a few select cases to use
the double tick.
Sphinx also emits a bunch of cryptic warnings due to not correctly
wrapping lines.
These two fixes touch the same lines so its all in one patch.
Use double ticks instead of single quote to guard inline code.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/filter.rst | 100 ++++++++++++++--------------
1 file changed, 49 insertions(+), 51 deletions(-)
diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
index 62630e8f023a..f9ec58144ed3 100644
--- a/Documentation/networking/filter.rst
+++ b/Documentation/networking/filter.rst
@@ -289,7 +289,7 @@ Possible BPF extensions are shown in the following table::
vlan_tpid skb->vlan_proto
rand prandom_u32()
-These extensions can also be prefixed with '#'.
+These extensions can also be prefixed with ``#``.
Examples for low-level BPF:
ARP packets::
@@ -382,7 +382,7 @@ file "~/.bpf_dbg_init" and the command history is stored in the file
"~/.bpf_dbg_history".
Interaction in bpf_dbg happens through a shell that also has auto-completion
-support (follow-up example commands starting with '>' denote bpf_dbg shell).
+support (follow-up example commands starting with ``>`` denote bpf_dbg shell).
The usual workflow would be to ...
::
@@ -689,7 +689,7 @@ Some core changes of the new internal format:
to in-kernel function. If R1 - R5 registers are mapped to CPU registers
that are used for argument passing on given architecture, the JIT compiler
doesn't need to emit extra moves. Function arguments will be in the correct
- registers and BPF_CALL instruction will be JITed as single 'call' HW
+ registers and BPF_CALL instruction will be JITed as single ``call`` HW
instruction. This calling convention was picked to cover common call
situations without performance penalty.
@@ -803,7 +803,7 @@ Some core changes of the new internal format:
In-kernel functions foo() and bar() with prototype: u64 (*)(u64 arg1, u64
arg2, u64 arg3, u64 arg4, u64 arg5); will receive arguments in proper
- registers and place their return value into '%rax' which is R0 in eBPF.
+ registers and place their return value into ``%rax`` which is R0 in eBPF.
Prologue and epilogue are emitted by JIT and are implicit in the
interpreter. R0-R5 are scratch registers, so eBPF program needs to preserve
them across the calls as defined by calling convention.
@@ -831,7 +831,7 @@ A program, that is translated internally consists of the following elements::
op:16, jt:8, jf:8, k:32 ==> op:8, dst_reg:4, src_reg:4, off:16, imm:32
-So far 87 internal BPF instructions were implemented. 8-bit 'op' opcode field
+So far 87 internal BPF instructions were implemented. 8-bit ``op`` opcode field
has room for new instructions. Some of them may use 16/24/32 byte encoding. New
instructions must be multiple of 8 bytes to preserve backward compatibility.
@@ -992,7 +992,7 @@ eBPF has two non-generic instructions: (BPF_ABS | <size> | BPF_LD) and
They had to be carried over from classic to have strong performance of
socket filters running in eBPF interpreter. These instructions can only
-be used when interpreter context is a pointer to 'struct sk_buff' and
+be used when interpreter context is a pointer to ``struct sk_buff`` and
have seven implicit operands. Register R6 is an implicit input that must
contain pointer to sk_buff. Register R0 is an implicit output which contains
the data fetched from the packet. Registers R1-R5 are scratch registers
@@ -1024,7 +1024,7 @@ Where size is one of: BPF_B or BPF_H or BPF_W or BPF_DW. Note that 1 and
2 byte atomic increments are not supported.
eBPF has one 16-byte instruction: BPF_LD | BPF_DW | BPF_IMM which consists
-of two consecutive 'struct bpf_insn' 8-byte blocks and interpreted as single
+of two consecutive ``struct bpf_insn`` 8-byte blocks and interpreted as single
instruction that loads 64-bit immediate value into a dst_reg.
Classic BPF has similar instruction: BPF_LD | BPF_W | BPF_IMM which loads
32-bit immediate value into a register.
@@ -1083,7 +1083,7 @@ For example::
will be rejected, since R1 doesn't have a valid pointer type at the time of
execution of instruction bpf_xadd.
-At the start R1 type is PTR_TO_CTX (a pointer to generic 'struct bpf_context')
+At the start R1 type is PTR_TO_CTX (a pointer to generic ``struct bpf_context``)
A callback is used to customize verifier to restrict eBPF program access to only
certain fields within ctx structure with specified size and alignment.
@@ -1138,7 +1138,7 @@ Register value tracking
In order to determine the safety of an eBPF program, the verifier must track
the range of possible values in each register and also in each stack slot.
-This is done with 'struct bpf_reg_state', defined in include/linux/
+This is done with ``struct bpf_reg_state``, defined in include/linux/
bpf_verifier.h, which unifies tracking of scalar and pointer values. Each
register state has a type, which is either NOT_INIT (the register has not been
written to), SCALAR_VALUE (some value which is not usable as a pointer), or a
@@ -1219,7 +1219,7 @@ Ex::
6: r0 = *(u16 *)(r3 +12) /* access 12 and 13 bytes of the packet */
this 2byte load from the packet is safe to do, since the program author
-did check 'if (skb->data + 14 > skb->data_end) goto err' at insn #5 which
+did check ``if (skb->data + 14 > skb->data_end) goto err`` at insn #5 which
means that in the fall-through case the register R3 (which points to skb->data)
has at least 14 directly accessible bytes. The verifier marks it
as R3=pkt(id=0,off=0,r=14).
@@ -1228,7 +1228,7 @@ off=0 means that no additional constants were added.
r=14 is the range of safe access which means that bytes [R3, R3 + 14) are ok.
Note that R5 is marked as R5=pkt(id=0,off=14,r=14). It also points
to the packet data, but constant 14 was added to the register, so
-it now points to 'skb->data + 14' and accessible range is [R5, R5 + 14 - 14)
+it now points to ``skb->data + 14`` and accessible range is [R5, R5 + 14 - 14)
which is zero bytes.
More complex packet access may look like::
@@ -1250,29 +1250,30 @@ More complex packet access may look like::
R0=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R1=pkt_end R2=pkt(id=2,off=8,r=8) R3=pkt(id=2,off=0,r=8) R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)) R5=pkt(id=0,off=14,r=14) R10=fp
19: r1 = *(u8 *)(r3 +4)
-The state of the register R3 is R3=pkt(id=2,off=0,r=8)
-id=2 means that two 'r3 += rX' instructions were seen, so r3 points to some
-offset within a packet and since the program author did
-'if (r3 + 8 > r1) goto err' at insn #18, the safe range is [R3, R3 + 8).
-The verifier only allows 'add'/'sub' operations on packet registers. Any other
-operation will set the register state to 'SCALAR_VALUE' and it won't be
-available for direct packet access.
-Operation 'r3 += rX' may overflow and become less than original skb->data,
-therefore the verifier has to prevent that. So when it sees 'r3 += rX'
-instruction and rX is more than 16-bit value, any subsequent bounds-check of r3
-against skb->data_end will not give us 'range' information, so attempts to read
-through the pointer will give "invalid access to packet" error.
-Ex. after insn 'r4 = *(u8 *)(r3 +12)' (insn #7 above) the state of r4 is
-R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) which means that upper 56 bits
-of the register are guaranteed to be zero, and nothing is known about the lower
-8 bits. After insn 'r4 *= 14' the state becomes
-R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)), since multiplying an 8-bit
-value by constant 14 will keep upper 52 bits as zero, also the least significant
-bit will be zero as 14 is even. Similarly 'r2 >>= 48' will make
-R2=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)), since the shift is not sign
-extending. This logic is implemented in adjust_reg_min_max_vals() function,
-which calls adjust_ptr_min_max_vals() for adding pointer to scalar (or vice
-versa) and adjust_scalar_min_max_vals() for operations on two scalars.
+The state of the register R3 is R3=pkt(id=2,off=0,r=8) id=2 means that
+two ``r3 += rX`` instructions were seen, so r3 points to some offset within
+a packet and since the program author did ``if (r3 + 8 > r1) goto err`` at
+insn #18, the safe range is [R3, R3 + 8). The verifier only allows
+'add'/'sub' operations on packet registers. Any other operation will set
+the register state to 'SCALAR_VALUE' and it won't be available for direct
+packet access. Operation ``r3 += rX`` may overflow and become less than
+original skb->data, therefore the verifier has to prevent that. So when it
+sees ``r3 += rX`` instruction and rX is more than 16-bit value, any
+subsequent bounds-check of r3 against skb->data_end will not give us
+'range' information, so attempts to read through the pointer will give
+"invalid access to packet" error. Ex. after insn ``r4 = *(u8 *)(r3 +12)``
+(insn #7 above) the state of r4 is
+R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) which means that upper
+56 bits of the register are guaranteed to be zero, and nothing is known
+about the lower 8 bits. After insn ``r4 *= 14`` the state becomes
+R4=inv(id=0,umax_value=3570,var_off=(0x0; 0xfffe)), since multiplying
+an 8-bit value by constant 14 will keep upper 52 bits as zero, also the
+least significant bit will be zero as 14 is even. Similarly ``r2 >>= 48``
+will make R2=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)), since
+the shift is not sign extending. This logic is implemented in
+adjust_reg_min_max_vals() function, which calls adjust_ptr_min_max_vals()
+for adding pointer to scalar (or vice versa) and
+adjust_scalar_min_max_vals() for operations on two scalars.
The end result is that bpf program author can access packet directly
using normal C code as::
@@ -1303,27 +1304,24 @@ and userspace.
The maps are accessed from user space via BPF syscall, which has commands:
-- create a map with given type and attributes
- map_fd = bpf(BPF_MAP_CREATE, union bpf_attr *attr, u32 size)
- using attr->map_type, attr->key_size, attr->value_size, attr->max_entries
- returns process-local file descriptor or negative error
+- create a map with given type and attributes ``map_fd =
+ bpf(BPF_MAP_CREATE, union bpf_attr *attr, u32 size)`` using
+ attr->map_type, attr->key_size, attr->value_size, attr->max_entries
+ returns process-local file descriptor or negative error.
-- lookup key in a given map
- err = bpf(BPF_MAP_LOOKUP_ELEM, union bpf_attr *attr, u32 size)
- using attr->map_fd, attr->key, attr->value
- returns zero and stores found elem into value or negative error
+- lookup key in a given map ``err = bpf(BPF_MAP_LOOKUP_ELEM, union bpf_attr
+ *attr, u32 size)`` using attr->map_fd, attr->key, attr->value returns
+ zero and stores found elem into value or negative error.
-- create or update key/value pair in a given map
- err = bpf(BPF_MAP_UPDATE_ELEM, union bpf_attr *attr, u32 size)
- using attr->map_fd, attr->key, attr->value
- returns zero or negative error
+- create or update key/value pair in a given map ``err =
+ bpf(BPF_MAP_UPDATE_ELEM, union bpf_attr *attr, u32 size)`` using
+ attr->map_fd, attr->key, attr->value returns zero or negative error.
-- find and delete element by key in a given map
- err = bpf(BPF_MAP_DELETE_ELEM, union bpf_attr *attr, u32 size)
- using attr->map_fd, attr->key
+- find and delete element by key in a given map ``err =
+ bpf(BPF_MAP_DELETE_ELEM, union bpf_attr *attr, u32 size)`` using
+ attr->map_fd, attr->key.
-- to delete map: close(fd)
- Exiting process will delete maps automatically
+- to delete map: close(fd) Exiting process will delete maps automatically.
userspace programs use this syscall to create/access maps that eBPF programs
are concurrently updating.
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next 08/13] docs: net: Use double ticks instead of single tick
From: Tobin C. Harding @ 2018-08-01 5:09 UTC (permalink / raw)
To: Daniel Borkmann, Alexei Starovoitov
Cc: Tobin C. Harding, Jonathan Corbet, David S. Miller, linux-doc,
netdev, linux-kernel
In-Reply-To: <20180801050908.29970-1-me@tobin.cc>
Single tick around a command should be converted to a double
tick for RST files (excluding ticks in a code snippet section).
Use double ticks instead of single tick.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
---
Documentation/networking/filter.rst | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
index f9ec58144ed3..dda1afdb5f26 100644
--- a/Documentation/networking/filter.rst
+++ b/Documentation/networking/filter.rst
@@ -38,9 +38,9 @@ setup a socket, attach a filter, lock it then drop privileges and be
assured that the filter will be kept until the socket is closed.
The biggest user of this construct might be libpcap. Issuing a high-level
-filter command like `tcpdump -i em1 port 22` passes through the libpcap
+filter command like ``tcpdump -i em1 port 22`` passes through the libpcap
internal compiler that generates a structure that can eventually be loaded
-via SO_ATTACH_FILTER to the kernel. `tcpdump -i em1 port 22 -ddd`
+via SO_ATTACH_FILTER to the kernel. ``tcpdump -i em1 port 22 -ddd``
displays what is being placed into this structure.
Although we were only speaking about sockets here, BPF in Linux is used
@@ -375,7 +375,7 @@ Starting bpf_dbg is trivial and just requires issuing::
In case input and output do not equal stdin/stdout, bpf_dbg takes an
alternative stdin source as a first argument, and an alternative stdout
-sink as a second one, e.g. `./bpf_dbg test_in.txt test_out.txt`.
+sink as a second one, e.g. ``./bpf_dbg test_in.txt test_out.txt``.
Other than that, a particular libreadline configuration can be set via
file "~/.bpf_dbg_init" and the command history is stored in the file
@@ -388,7 +388,7 @@ The usual workflow would be to ...
> load bpf 6,40 0 0 12,21 0 3 2048,48 0 0 23,21 0 1 1,6 0 0 65535,6 0 0 0
Loads a BPF filter from standard output of bpf_asm, or transformed via
- e.g. `tcpdump -iem1 -ddd port 22 | tr '\n' ','`. Note that for JIT
+ e.g. tcpdump -iem1 -ddd port 22 | tr '\n' ','`. Note that for JIT
debugging (next section), this command creates a temporary socket and
loads the BPF code into the kernel. Thus, this will also be useful for
JIT developers.
@@ -523,7 +523,7 @@ generating disassembly out of the kernel log's hexdump::
44: leaveq
45: retq
-Issuing option `-o` will "annotate" opcodes to resulting assembler
+Issuing option ``-o`` will "annotate" opcodes to resulting assembler
instructions, which can be very useful for JIT developers::
# ./bpf_jit_disasm -o
--
2.17.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox