* [PATCH mlx5-next 13/15] net/mlx5: E-Switch, Use vport index when init rep
From: Saeed Mahameed @ 2019-06-17 19:23 UTC (permalink / raw)
To: Saeed Mahameed, Leon Romanovsky
Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Bodong Wang,
Parav Pandit, Mark Bloch
In-Reply-To: <20190617192247.25107-1-saeedm@mellanox.com>
From: Bodong Wang <bodong@mellanox.com>
Driver is referring to the array index when doing rep initialization,
using vport is confusing as it's normally interpreted as vport number.
This patch doesn't change any functionality.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 5124219a31de..f29b9e1f49ae 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1399,7 +1399,7 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
struct mlx5_core_dev *dev = esw->dev;
struct mlx5_eswitch_rep *rep;
u8 hw_id[ETH_ALEN], rep_type;
- int vport;
+ int vport_index;
esw->offloads.vport_reps = kcalloc(total_vports,
sizeof(struct mlx5_eswitch_rep),
@@ -1409,8 +1409,8 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
mlx5_query_nic_vport_mac_address(dev, 0, hw_id);
- mlx5_esw_for_all_reps(esw, vport, rep) {
- rep->vport = mlx5_eswitch_index_to_vport_num(esw, vport);
+ mlx5_esw_for_all_reps(esw, vport_index, rep) {
+ rep->vport = mlx5_eswitch_index_to_vport_num(esw, vport_index);
ether_addr_copy(rep->hw_id, hw_id);
for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++)
--
2.21.0
^ permalink raw reply related
* [PATCH mlx5-next 14/15] {IB, net}/mlx5: E-Switch, Use index of rep for vport to IB port mapping
From: Saeed Mahameed @ 2019-06-17 19:23 UTC (permalink / raw)
To: Saeed Mahameed, Leon Romanovsky
Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Bodong Wang,
Parav Pandit, Mark Bloch
In-Reply-To: <20190617192247.25107-1-saeedm@mellanox.com>
From: Bodong Wang <bodong@mellanox.com>
In the single IB device mode, the mapping between vport number and
rep relies on a counter. However for dynamic vport allocation, it is
desired to keep consistent map of eswitch vport and IB port.
Hence, simplify code to remove the free running counter and instead
use the available vport index during load/unload sequence from the
eswitch.
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Suggested-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/infiniband/hw/mlx5/ib_rep.c | 4 ++--
drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 -
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 1 +
include/linux/mlx5/eswitch.h | 2 ++
4 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
index d4ed611de35d..da4b936b3219 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.c
+++ b/drivers/infiniband/hw/mlx5/ib_rep.c
@@ -14,7 +14,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
int vport_index;
ibdev = mlx5_ib_get_uplink_ibdev(dev->priv.eswitch);
- vport_index = ibdev->free_port++;
+ vport_index = rep->vport_index;
ibdev->port[vport_index].rep = rep;
write_lock(&ibdev->port[vport_index].roce.netdev_lock);
@@ -50,7 +50,7 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
}
ibdev->is_rep = true;
- vport_index = ibdev->free_port++;
+ vport_index = rep->vport_index;
ibdev->port[vport_index].rep = rep;
ibdev->port[vport_index].roce.netdev =
mlx5_ib_get_rep_netdev(dev->priv.eswitch, rep->vport);
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 1c205c2bd486..ee73dc122d28 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -978,7 +978,6 @@ struct mlx5_ib_dev {
u16 devx_whitelist_uid;
struct mlx5_srq_table srq_table;
struct mlx5_async_ctx async_ctx;
- int free_port;
};
static inline struct mlx5_ib_cq *to_mibcq(struct mlx5_core_cq *mcq)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index f29b9e1f49ae..7fdea5600383 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1411,6 +1411,7 @@ int esw_offloads_init_reps(struct mlx5_eswitch *esw)
mlx5_esw_for_all_reps(esw, vport_index, rep) {
rep->vport = mlx5_eswitch_index_to_vport_num(esw, vport_index);
+ rep->vport_index = vport_index;
ether_addr_copy(rep->hw_id, hw_id);
for (rep_type = 0; rep_type < NUM_REP_TYPES; rep_type++)
diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
index d729f5e4d70a..d29abefa03c6 100644
--- a/include/linux/mlx5/eswitch.h
+++ b/include/linux/mlx5/eswitch.h
@@ -46,6 +46,8 @@ struct mlx5_eswitch_rep {
u16 vport;
u8 hw_id[ETH_ALEN];
u16 vlan;
+ /* Only IB rep is using vport_index */
+ u16 vport_index;
u32 vlan_refcount;
};
--
2.21.0
^ permalink raw reply related
* [PATCH mlx5-next 11/15] RDMA/mlx5: Add vport metadata matching for IB representors
From: Saeed Mahameed @ 2019-06-17 19:23 UTC (permalink / raw)
To: Saeed Mahameed, Leon Romanovsky
Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Jianbo Liu,
Roi Dayan, Mark Bloch
In-Reply-To: <20190617192247.25107-1-saeedm@mellanox.com>
From: Jianbo Liu <jianbol@mellanox.com>
If vport metadata matching is enabled in eswitch, the rule created
must be changed to match on the metadata, instead of source port.
Signed-off-by: Jianbo Liu <jianbol@mellanox.com>
Reviewed-by: Roi Dayan <roid@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/infiniband/hw/mlx5/ib_rep.c | 11 +++++++
drivers/infiniband/hw/mlx5/ib_rep.h | 16 ++++++++++
drivers/infiniband/hw/mlx5/main.c | 45 +++++++++++++++++++++++------
3 files changed, 63 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
index 22e651cb5534..d4ed611de35d 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.c
+++ b/drivers/infiniband/hw/mlx5/ib_rep.c
@@ -131,6 +131,17 @@ struct mlx5_eswitch_rep *mlx5_ib_vport_rep(struct mlx5_eswitch *esw, int vport)
return mlx5_eswitch_vport_rep(esw, vport);
}
+u32 mlx5_ib_eswitch_vport_match_metadata_enabled(struct mlx5_eswitch *esw)
+{
+ return mlx5_eswitch_vport_match_metadata_enabled(esw);
+}
+
+u32 mlx5_ib_eswitch_get_vport_metadata_for_match(struct mlx5_eswitch *esw,
+ u16 vport)
+{
+ return mlx5_eswitch_get_vport_metadata_for_match(esw, vport);
+}
+
struct mlx5_flow_handle *create_flow_rule_vport_sq(struct mlx5_ib_dev *dev,
struct mlx5_ib_sq *sq,
u16 port)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.h b/drivers/infiniband/hw/mlx5/ib_rep.h
index 22adce2d6795..65a04b6b0df3 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.h
+++ b/drivers/infiniband/hw/mlx5/ib_rep.h
@@ -25,6 +25,9 @@ struct mlx5_flow_handle *create_flow_rule_vport_sq(struct mlx5_ib_dev *dev,
u16 port);
struct net_device *mlx5_ib_get_rep_netdev(struct mlx5_eswitch *esw,
int vport_index);
+u32 mlx5_ib_eswitch_vport_match_metadata_enabled(struct mlx5_eswitch *esw);
+u32 mlx5_ib_eswitch_get_vport_metadata_for_match(struct mlx5_eswitch *esw,
+ u16 vport);
#else /* CONFIG_MLX5_ESWITCH */
static inline u8 mlx5_ib_eswitch_mode(struct mlx5_eswitch *esw)
{
@@ -67,6 +70,19 @@ struct net_device *mlx5_ib_get_rep_netdev(struct mlx5_eswitch *esw,
{
return NULL;
}
+
+static inline
+u32 mlx5_ib_eswitch_vport_match_metadata_enabled(struct mlx5_eswitch *esw)
+{
+ return 0;
+};
+
+static inline
+u32 mlx5_ib_eswitch_get_vport_metadata_for_match(struct mlx5_eswitch *esw,
+ u16 vport)
+{
+ return 0;
+};
#endif
static inline
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index be4c9a687df7..f97519c0c4df 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3469,6 +3469,37 @@ static int flow_counters_set_data(struct ib_counters *ibcounters,
return ret;
}
+static void mlx5_ib_set_rule_source_port(struct mlx5_ib_dev *dev,
+ struct mlx5_flow_spec *spec,
+ struct mlx5_eswitch_rep *rep)
+{
+ struct mlx5_eswitch *esw = dev->mdev->priv.eswitch;
+ void *misc;
+
+ if (mlx5_ib_eswitch_vport_match_metadata_enabled(esw)) {
+ misc = MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ misc_parameters_2);
+
+ MLX5_SET(fte_match_set_misc2, misc, metadata_reg_c_0,
+ mlx5_ib_eswitch_get_vport_metadata_for_match(esw,
+ rep->vport));
+ misc = MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
+ misc_parameters_2);
+
+ MLX5_SET_TO_ONES(fte_match_set_misc2, misc, metadata_reg_c_0);
+ } else {
+ misc = MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ misc_parameters);
+
+ MLX5_SET(fte_match_set_misc, misc, source_port, rep->vport);
+
+ misc = MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
+ misc_parameters);
+
+ MLX5_SET_TO_ONES(fte_match_set_misc, misc, source_port);
+ }
+}
+
static struct mlx5_ib_flow_handler *_create_flow_rule(struct mlx5_ib_dev *dev,
struct mlx5_ib_flow_prio *ft_prio,
const struct ib_flow_attr *flow_attr,
@@ -3523,19 +3554,15 @@ static struct mlx5_ib_flow_handler *_create_flow_rule(struct mlx5_ib_dev *dev,
set_underlay_qp(dev, spec, underlay_qpn);
if (dev->is_rep) {
- void *misc;
+ struct mlx5_eswitch_rep *rep;
- if (!dev->port[flow_attr->port - 1].rep) {
+ rep = dev->port[flow_attr->port - 1].rep;
+ if (!rep) {
err = -EINVAL;
goto free;
}
- misc = MLX5_ADDR_OF(fte_match_param, spec->match_value,
- misc_parameters);
- MLX5_SET(fte_match_set_misc, misc, source_port,
- dev->port[flow_attr->port - 1].rep->vport);
- misc = MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
- misc_parameters);
- MLX5_SET_TO_ONES(fte_match_set_misc, misc, source_port);
+
+ mlx5_ib_set_rule_source_port(dev, spec, rep);
}
spec->match_criteria_enable = get_match_criteria_enable(spec->match_criteria);
--
2.21.0
^ permalink raw reply related
* [PATCH mlx5-next 15/15] RDMA/mlx5: Cleanup rep when doing unload
From: Saeed Mahameed @ 2019-06-17 19:23 UTC (permalink / raw)
To: Saeed Mahameed, Leon Romanovsky
Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Bodong Wang,
Mark Bloch, Parav Pandit
In-Reply-To: <20190617192247.25107-1-saeedm@mellanox.com>
From: Bodong Wang <bodong@mellanox.com>
When an IB rep is loaded, netdev for the same vport is saved for later
reference. However, it's not cleaned up when doing unload. For ECPF,
kernel crashes when driver is referring to the already removed netdev.
Following steps lead to a shown call trace:
1. Create n VFs from host PF
2. Distroy the VFs
3. Run "rdma link" from ARM
Call trace:
mlx5_ib_get_netdev+0x9c/0xe8 [mlx5_ib]
mlx5_query_port_roce+0x268/0x558 [mlx5_ib]
mlx5_ib_rep_query_port+0x14/0x34 [mlx5_ib]
ib_query_port+0x9c/0xfc [ib_core]
fill_port_info+0x74/0x28c [ib_core]
nldev_port_get_doit+0x1a8/0x1e8 [ib_core]
rdma_nl_rcv_msg+0x16c/0x1c0 [ib_core]
rdma_nl_rcv+0xe8/0x144 [ib_core]
netlink_unicast+0x184/0x214
netlink_sendmsg+0x288/0x354
sock_sendmsg+0x18/0x2c
__sys_sendto+0xbc/0x138
__arm64_sys_sendto+0x28/0x34
el0_svc_common+0xb0/0x100
el0_svc_handler+0x6c/0x84
el0_svc+0x8/0xc
Cleanup the rep and netdev reference when unloading IB rep.
Fixes: 26628e2d58c9 ("RDMA/mlx5: Move to single device multiport ports in switchdev mode")
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Parav Pandit <parav@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/infiniband/hw/mlx5/ib_rep.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
index da4b936b3219..a4a54ddebb71 100644
--- a/drivers/infiniband/hw/mlx5/ib_rep.c
+++ b/drivers/infiniband/hw/mlx5/ib_rep.c
@@ -17,6 +17,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
vport_index = rep->vport_index;
ibdev->port[vport_index].rep = rep;
+ rep->rep_data[REP_IB].priv = ibdev;
write_lock(&ibdev->port[vport_index].roce.netdev_lock);
ibdev->port[vport_index].roce.netdev =
mlx5_ib_get_rep_netdev(dev->priv.eswitch, rep->vport);
@@ -68,15 +69,18 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
static void
mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep)
{
- struct mlx5_ib_dev *dev;
-
- if (!rep->rep_data[REP_IB].priv ||
- rep->vport != MLX5_VPORT_UPLINK)
- return;
+ struct mlx5_ib_dev *dev = mlx5_ib_rep_to_dev(rep);
+ struct mlx5_ib_port *port;
- dev = mlx5_ib_rep_to_dev(rep);
- __mlx5_ib_remove(dev, dev->profile, MLX5_IB_STAGE_MAX);
+ port = &dev->port[rep->vport_index];
+ write_lock(&port->roce.netdev_lock);
+ port->roce.netdev = NULL;
+ write_unlock(&port->roce.netdev_lock);
rep->rep_data[REP_IB].priv = NULL;
+ port->rep = NULL;
+
+ if (rep->vport == MLX5_VPORT_UPLINK)
+ __mlx5_ib_remove(dev, dev->profile, MLX5_IB_STAGE_MAX);
}
static void *mlx5_ib_vport_get_proto_dev(struct mlx5_eswitch_rep *rep)
--
2.21.0
^ permalink raw reply related
* [PATCH net v2] hvsock: fix epollout hang from race condition
From: Sunil Muthuswamy @ 2019-06-17 19:26 UTC (permalink / raw)
To: KY Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
David S. Miller, Dexuan Cui, Michael Kelley
Cc: netdev@vger.kernel.org, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org
Currently, hvsock can enter into a state where epoll_wait on EPOLLOUT will
not return even when the hvsock socket is writable, under some race
condition. This can happen under the following sequence:
- fd = socket(hvsocket)
- fd_out = dup(fd)
- fd_in = dup(fd)
- start a writer thread that writes data to fd_out with a combination of
epoll_wait(fd_out, EPOLLOUT) and
- start a reader thread that reads data from fd_in with a combination of
epoll_wait(fd_in, EPOLLIN)
- On the host, there are two threads that are reading/writing data to the
hvsocket
stack:
hvs_stream_has_space
hvs_notify_poll_out
vsock_poll
sock_poll
ep_poll
Race condition:
check for epollout from ep_poll():
assume no writable space in the socket
hvs_stream_has_space() returns 0
check for epollin from ep_poll():
assume socket has some free space < HVS_PKT_LEN(HVS_SEND_BUF_SIZE)
hvs_stream_has_space() will clear the channel pending send size
host will not notify the guest because the pending send size has
been cleared and so the hvsocket will never mark the
socket writable
Now, the EPOLLOUT will never return even if the socket write buffer is
empty.
The fix is to set the pending size to the default size and never change it.
This way the host will always notify the guest whenever the writable space
is bigger than the pending size. The host is already optimized to *only*
notify the guest when the pending size threshold boundary is crossed and
not everytime.
This change also reduces the cpu usage somewhat since hv_stream_has_space()
is in the hotpath of send:
vsock_stream_sendmsg()->hv_stream_has_space()
Earlier hv_stream_has_space was setting/clearing the pending size on every
call.
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Reviewed-by: Dexuan Cui <decui@microsoft.com>
---
- Resubmitting the patch after taking care of some spurious warnings.
net/vmw_vsock/hyperv_transport.c | 39 ++++++++-------------------------------
1 file changed, 8 insertions(+), 31 deletions(-)
diff --git a/net/vmw_vsock/hyperv_transport.c b/net/vmw_vsock/hyperv_transport.c
index e4801c7..cd3f47f 100644
--- a/net/vmw_vsock/hyperv_transport.c
+++ b/net/vmw_vsock/hyperv_transport.c
@@ -220,18 +220,6 @@ static void hvs_set_channel_pending_send_size(struct vmbus_channel *chan)
set_channel_pending_send_size(chan,
HVS_PKT_LEN(HVS_SEND_BUF_SIZE));
- /* See hvs_stream_has_space(): we must make sure the host has seen
- * the new pending send size, before we can re-check the writable
- * bytes.
- */
- virt_mb();
-}
-
-static void hvs_clear_channel_pending_send_size(struct vmbus_channel *chan)
-{
- set_channel_pending_send_size(chan, 0);
-
- /* Ditto */
virt_mb();
}
@@ -301,9 +289,6 @@ static void hvs_channel_cb(void *ctx)
if (hvs_channel_readable(chan))
sk->sk_data_ready(sk);
- /* See hvs_stream_has_space(): when we reach here, the writable bytes
- * may be already less than HVS_PKT_LEN(HVS_SEND_BUF_SIZE).
- */
if (hv_get_bytes_to_write(&chan->outbound) > 0)
sk->sk_write_space(sk);
}
@@ -404,6 +389,13 @@ static void hvs_open_connection(struct vmbus_channel *chan)
set_per_channel_state(chan, conn_from_host ? new : sk);
vmbus_set_chn_rescind_callback(chan, hvs_close_connection);
+ /* Set the pending send size to max packet size to always get
+ * notifications from the host when there is enough writable space.
+ * The host is optimized to send notifications only when the pending
+ * size boundary is crossed, and not always.
+ */
+ hvs_set_channel_pending_send_size(chan);
+
if (conn_from_host) {
new->sk_state = TCP_ESTABLISHED;
sk->sk_ack_backlog++;
@@ -697,23 +689,8 @@ static s64 hvs_stream_has_data(struct vsock_sock *vsk)
static s64 hvs_stream_has_space(struct vsock_sock *vsk)
{
struct hvsock *hvs = vsk->trans;
- struct vmbus_channel *chan = hvs->chan;
- s64 ret;
-
- ret = hvs_channel_writable_bytes(chan);
- if (ret > 0) {
- hvs_clear_channel_pending_send_size(chan);
- } else {
- /* See hvs_channel_cb() */
- hvs_set_channel_pending_send_size(chan);
-
- /* Re-check the writable bytes to avoid race */
- ret = hvs_channel_writable_bytes(chan);
- if (ret > 0)
- hvs_clear_channel_pending_send_size(chan);
- }
- return ret;
+ return hvs_channel_writable_bytes(hvs->chan);
}
static u64 hvs_stream_rcvhiwat(struct vsock_sock *vsk)
--
2.7.4
^ permalink raw reply related
* [PATCH v2 bpf-next 01/11] libbpf: add common min/max macro to libbpf_internal.h
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Multiple files in libbpf redefine their own definitions for min/max.
Let's define them in libbpf_internal.h and use those everywhere.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
---
tools/lib/bpf/bpf.c | 7 ++-----
tools/lib/bpf/bpf_prog_linfo.c | 5 +----
tools/lib/bpf/btf.c | 3 ---
tools/lib/bpf/btf_dump.c | 3 ---
tools/lib/bpf/libbpf_internal.h | 7 +++++++
5 files changed, 10 insertions(+), 15 deletions(-)
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 0d4b4fe10a84..c7d7993c44bb 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -26,10 +26,11 @@
#include <memory.h>
#include <unistd.h>
#include <asm/unistd.h>
+#include <errno.h>
#include <linux/bpf.h>
#include "bpf.h"
#include "libbpf.h"
-#include <errno.h>
+#include "libbpf_internal.h"
/*
* When building perf, unistd.h is overridden. __NR_bpf is
@@ -53,10 +54,6 @@
# endif
#endif
-#ifndef min
-#define min(x, y) ((x) < (y) ? (x) : (y))
-#endif
-
static inline __u64 ptr_to_u64(const void *ptr)
{
return (__u64) (unsigned long) ptr;
diff --git a/tools/lib/bpf/bpf_prog_linfo.c b/tools/lib/bpf/bpf_prog_linfo.c
index 6978314ea7f6..8c67561c93b0 100644
--- a/tools/lib/bpf/bpf_prog_linfo.c
+++ b/tools/lib/bpf/bpf_prog_linfo.c
@@ -6,10 +6,7 @@
#include <linux/err.h>
#include <linux/bpf.h>
#include "libbpf.h"
-
-#ifndef min
-#define min(x, y) ((x) < (y) ? (x) : (y))
-#endif
+#include "libbpf_internal.h"
struct bpf_prog_linfo {
void *raw_linfo;
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b2478e98c367..467224feb43b 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -16,9 +16,6 @@
#include "libbpf_internal.h"
#include "hashmap.h"
-#define max(a, b) ((a) > (b) ? (a) : (b))
-#define min(a, b) ((a) < (b) ? (a) : (b))
-
#define BTF_MAX_NR_TYPES 0x7fffffff
#define BTF_MAX_STR_OFFSET 0x7fffffff
diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c
index 4b22db77e2cc..7065bb5b2752 100644
--- a/tools/lib/bpf/btf_dump.c
+++ b/tools/lib/bpf/btf_dump.c
@@ -18,9 +18,6 @@
#include "libbpf.h"
#include "libbpf_internal.h"
-#define min(x, y) ((x) < (y) ? (x) : (y))
-#define max(x, y) ((x) < (y) ? (y) : (x))
-
static const char PREFIXES[] = "\t\t\t\t\t\t\t\t\t\t\t\t\t";
static const size_t PREFIX_CNT = sizeof(PREFIXES) - 1;
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 850f7bdec5cb..554a7856dc2d 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -23,6 +23,13 @@
#define BTF_PARAM_ENC(name, type) (name), (type)
#define BTF_VAR_SECINFO_ENC(type, offset, size) (type), (offset), (size)
+#ifndef min
+# define min(x, y) ((x) < (y) ? (x) : (y))
+#endif
+#ifndef max
+# define max(x, y) ((x) < (y) ? (y) : (x))
+#endif
+
extern void libbpf_print(enum libbpf_print_level level,
const char *format, ...)
__attribute__((format(printf, 2, 3)));
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 03/11] libbpf: streamline ELF parsing error-handling
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Simplify ELF parsing logic by exiting early, as there is no common clean
up path to execute. That makes it unnecessary to track when err was set
and when it was cleared. It also reduces nesting in some places.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/libbpf.c | 44 +++++++++++++++++++-----------------------
1 file changed, 20 insertions(+), 24 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 49d3a808e754..7ee44d8877c5 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -1154,24 +1154,21 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
if (gelf_getshdr(scn, &sh) != &sh) {
pr_warning("failed to get section(%d) header from %s\n",
idx, obj->path);
- err = -LIBBPF_ERRNO__FORMAT;
- goto out;
+ return -LIBBPF_ERRNO__FORMAT;
}
name = elf_strptr(elf, ep->e_shstrndx, sh.sh_name);
if (!name) {
pr_warning("failed to get section(%d) name from %s\n",
idx, obj->path);
- err = -LIBBPF_ERRNO__FORMAT;
- goto out;
+ return -LIBBPF_ERRNO__FORMAT;
}
data = elf_getdata(scn, 0);
if (!data) {
pr_warning("failed to get section(%d) data from %s(%s)\n",
idx, name, obj->path);
- err = -LIBBPF_ERRNO__FORMAT;
- goto out;
+ return -LIBBPF_ERRNO__FORMAT;
}
pr_debug("section(%d) %s, size %ld, link %d, flags %lx, type=%d\n",
idx, name, (unsigned long)data->d_size,
@@ -1182,10 +1179,14 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
err = bpf_object__init_license(obj,
data->d_buf,
data->d_size);
+ if (err)
+ return err;
} else if (strcmp(name, "version") == 0) {
err = bpf_object__init_kversion(obj,
data->d_buf,
data->d_size);
+ if (err)
+ return err;
} else if (strcmp(name, "maps") == 0) {
obj->efile.maps_shndx = idx;
} else if (strcmp(name, BTF_ELF_SEC) == 0) {
@@ -1196,11 +1197,10 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
if (obj->efile.symbols) {
pr_warning("bpf: multiple SYMTAB in %s\n",
obj->path);
- err = -LIBBPF_ERRNO__FORMAT;
- } else {
- obj->efile.symbols = data;
- obj->efile.strtabidx = sh.sh_link;
+ return -LIBBPF_ERRNO__FORMAT;
}
+ obj->efile.symbols = data;
+ obj->efile.strtabidx = sh.sh_link;
} else if (sh.sh_type == SHT_PROGBITS && data->d_size > 0) {
if (sh.sh_flags & SHF_EXECINSTR) {
if (strcmp(name, ".text") == 0)
@@ -1214,6 +1214,7 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
pr_warning("failed to alloc program %s (%s): %s",
name, obj->path, cp);
+ return err;
}
} else if (strcmp(name, ".data") == 0) {
obj->efile.data = data;
@@ -1225,8 +1226,8 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
pr_debug("skip section(%d) %s\n", idx, name);
}
} else if (sh.sh_type == SHT_REL) {
+ int nr_reloc = obj->efile.nr_reloc;
void *reloc = obj->efile.reloc;
- int nr_reloc = obj->efile.nr_reloc + 1;
int sec = sh.sh_info; /* points to other section */
/* Only do relo for section with exec instructions */
@@ -1236,28 +1237,24 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
continue;
}
- reloc = reallocarray(reloc, nr_reloc,
+ reloc = reallocarray(reloc, nr_reloc + 1,
sizeof(*obj->efile.reloc));
if (!reloc) {
pr_warning("realloc failed\n");
- err = -ENOMEM;
- } else {
- int n = nr_reloc - 1;
+ return -ENOMEM;
+ }
- obj->efile.reloc = reloc;
- obj->efile.nr_reloc = nr_reloc;
+ obj->efile.reloc = reloc;
+ obj->efile.nr_reloc++;
- obj->efile.reloc[n].shdr = sh;
- obj->efile.reloc[n].data = data;
- }
+ obj->efile.reloc[nr_reloc].shdr = sh;
+ obj->efile.reloc[nr_reloc].data = data;
} else if (sh.sh_type == SHT_NOBITS && strcmp(name, ".bss") == 0) {
obj->efile.bss = data;
obj->efile.bss_shndx = idx;
} else {
pr_debug("skip section(%d) %s\n", idx, name);
}
- if (err)
- goto out;
}
if (!obj->efile.strtabidx || obj->efile.strtabidx >= idx) {
@@ -1270,10 +1267,9 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
if (bpf_object__has_maps(obj)) {
err = bpf_object__init_maps(obj, flags);
if (err)
- goto out;
+ return err;
}
err = bpf_object__init_prog_names(obj);
-out:
return err;
}
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 02/11] libbpf: extract BTF loading logic
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
As a preparetion fro adding BTF-based BPF map loading, extract .BTF and
.BTF.ext loading logic.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/libbpf.c | 93 +++++++++++++++++++++++++-----------------
1 file changed, 55 insertions(+), 38 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index e725fa86b189..49d3a808e754 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -1078,6 +1078,58 @@ static void bpf_object__sanitize_btf_ext(struct bpf_object *obj)
}
}
+static int bpf_object__load_btf(struct bpf_object *obj,
+ Elf_Data *btf_data,
+ Elf_Data *btf_ext_data)
+{
+ int err = 0;
+
+ if (btf_data) {
+ obj->btf = btf__new(btf_data->d_buf, btf_data->d_size);
+ if (IS_ERR(obj->btf)) {
+ pr_warning("Error loading ELF section %s: %d.\n",
+ BTF_ELF_SEC, err);
+ goto out;
+ }
+ err = btf__finalize_data(obj, obj->btf);
+ if (err) {
+ pr_warning("Error finalizing %s: %d.\n",
+ BTF_ELF_SEC, err);
+ goto out;
+ }
+ bpf_object__sanitize_btf(obj);
+ err = btf__load(obj->btf);
+ if (err) {
+ pr_warning("Error loading %s into kernel: %d.\n",
+ BTF_ELF_SEC, err);
+ goto out;
+ }
+ }
+ if (btf_ext_data) {
+ if (!obj->btf) {
+ pr_debug("Ignore ELF section %s because its depending ELF section %s is not found.\n",
+ BTF_EXT_ELF_SEC, BTF_ELF_SEC);
+ goto out;
+ }
+ obj->btf_ext = btf_ext__new(btf_ext_data->d_buf,
+ btf_ext_data->d_size);
+ if (IS_ERR(obj->btf_ext)) {
+ pr_warning("Error loading ELF section %s: %ld. Ignored and continue.\n",
+ BTF_EXT_ELF_SEC, PTR_ERR(obj->btf_ext));
+ obj->btf_ext = NULL;
+ goto out;
+ }
+ bpf_object__sanitize_btf_ext(obj);
+ }
+out:
+ if (err || IS_ERR(obj->btf)) {
+ if (!IS_ERR_OR_NULL(obj->btf))
+ btf__free(obj->btf);
+ obj->btf = NULL;
+ }
+ return 0;
+}
+
static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
{
Elf *elf = obj->efile.elf;
@@ -1212,44 +1264,9 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
pr_warning("Corrupted ELF file: index of strtab invalid\n");
return -LIBBPF_ERRNO__FORMAT;
}
- if (btf_data) {
- obj->btf = btf__new(btf_data->d_buf, btf_data->d_size);
- if (IS_ERR(obj->btf)) {
- pr_warning("Error loading ELF section %s: %ld. Ignored and continue.\n",
- BTF_ELF_SEC, PTR_ERR(obj->btf));
- obj->btf = NULL;
- } else {
- err = btf__finalize_data(obj, obj->btf);
- if (!err) {
- bpf_object__sanitize_btf(obj);
- err = btf__load(obj->btf);
- }
- if (err) {
- pr_warning("Error finalizing and loading %s into kernel: %d. Ignored and continue.\n",
- BTF_ELF_SEC, err);
- btf__free(obj->btf);
- obj->btf = NULL;
- err = 0;
- }
- }
- }
- if (btf_ext_data) {
- if (!obj->btf) {
- pr_debug("Ignore ELF section %s because its depending ELF section %s is not found.\n",
- BTF_EXT_ELF_SEC, BTF_ELF_SEC);
- } else {
- obj->btf_ext = btf_ext__new(btf_ext_data->d_buf,
- btf_ext_data->d_size);
- if (IS_ERR(obj->btf_ext)) {
- pr_warning("Error loading ELF section %s: %ld. Ignored and continue.\n",
- BTF_EXT_ELF_SEC,
- PTR_ERR(obj->btf_ext));
- obj->btf_ext = NULL;
- } else {
- bpf_object__sanitize_btf_ext(obj);
- }
- }
- }
+ err = bpf_object__load_btf(obj, btf_data, btf_ext_data);
+ if (err)
+ return err;
if (bpf_object__has_maps(obj)) {
err = bpf_object__init_maps(obj, flags);
if (err)
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 04/11] libbpf: refactor map initialization
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
User and global data maps initialization has gotten pretty complicated
and unnecessarily convoluted. This patch splits out the logic for global
data map and user-defined map initialization. It also removes the
restriction of pre-calculating how many maps will be initialized,
instead allowing to keep adding new maps as they are discovered, which
will be used later for BTF-defined map definitions.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/libbpf.c | 247 ++++++++++++++++++++++-------------------
1 file changed, 133 insertions(+), 114 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 7ee44d8877c5..88609dca4f7d 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -234,6 +234,7 @@ struct bpf_object {
size_t nr_programs;
struct bpf_map *maps;
size_t nr_maps;
+ size_t maps_cap;
struct bpf_secdata sections;
bool loaded;
@@ -763,21 +764,51 @@ int bpf_object__variable_offset(const struct bpf_object *obj, const char *name,
return -ENOENT;
}
-static bool bpf_object__has_maps(const struct bpf_object *obj)
+static struct bpf_map *bpf_object__add_map(struct bpf_object *obj)
{
- return obj->efile.maps_shndx >= 0 ||
- obj->efile.data_shndx >= 0 ||
- obj->efile.rodata_shndx >= 0 ||
- obj->efile.bss_shndx >= 0;
+ struct bpf_map *new_maps;
+ size_t new_cap;
+ int i;
+
+ if (obj->nr_maps < obj->maps_cap)
+ return &obj->maps[obj->nr_maps++];
+
+ new_cap = max(4ul, obj->maps_cap * 3 / 2);
+ new_maps = realloc(obj->maps, new_cap * sizeof(*obj->maps));
+ if (!new_maps) {
+ pr_warning("alloc maps for object failed\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ obj->maps_cap = new_cap;
+ obj->maps = new_maps;
+
+ /* zero out new maps */
+ memset(obj->maps + obj->nr_maps, 0,
+ (obj->maps_cap - obj->nr_maps) * sizeof(*obj->maps));
+ /*
+ * fill all fd with -1 so won't close incorrect fd (fd=0 is stdin)
+ * when failure (zclose won't close negative fd)).
+ */
+ for (i = obj->nr_maps; i < obj->maps_cap; i++) {
+ obj->maps[i].fd = -1;
+ obj->maps[i].inner_map_fd = -1;
+ }
+
+ return &obj->maps[obj->nr_maps++];
}
static int
-bpf_object__init_internal_map(struct bpf_object *obj, struct bpf_map *map,
- enum libbpf_map_type type, Elf_Data *data,
- void **data_buff)
+bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type,
+ Elf_Data *data, void **data_buff)
{
- struct bpf_map_def *def = &map->def;
char map_name[BPF_OBJ_NAME_LEN];
+ struct bpf_map_def *def;
+ struct bpf_map *map;
+
+ map = bpf_object__add_map(obj);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
map->libbpf_type = type;
map->offset = ~(typeof(map->offset))0;
@@ -789,6 +820,7 @@ bpf_object__init_internal_map(struct bpf_object *obj, struct bpf_map *map,
return -ENOMEM;
}
+ def = &map->def;
def->type = BPF_MAP_TYPE_ARRAY;
def->key_size = sizeof(int);
def->value_size = data->d_size;
@@ -808,29 +840,58 @@ bpf_object__init_internal_map(struct bpf_object *obj, struct bpf_map *map,
return 0;
}
-static int bpf_object__init_maps(struct bpf_object *obj, int flags)
+static int bpf_object__init_global_data_maps(struct bpf_object *obj)
+{
+ int err;
+
+ if (!obj->caps.global_data)
+ return 0;
+ /*
+ * Populate obj->maps with libbpf internal maps.
+ */
+ if (obj->efile.data_shndx >= 0) {
+ err = bpf_object__init_internal_map(obj, LIBBPF_MAP_DATA,
+ obj->efile.data,
+ &obj->sections.data);
+ if (err)
+ return err;
+ }
+ if (obj->efile.rodata_shndx >= 0) {
+ err = bpf_object__init_internal_map(obj, LIBBPF_MAP_RODATA,
+ obj->efile.rodata,
+ &obj->sections.rodata);
+ if (err)
+ return err;
+ }
+ if (obj->efile.bss_shndx >= 0) {
+ err = bpf_object__init_internal_map(obj, LIBBPF_MAP_BSS,
+ obj->efile.bss, NULL);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+static int bpf_object__init_user_maps(struct bpf_object *obj, bool strict)
{
- int i, map_idx, map_def_sz = 0, nr_syms, nr_maps = 0, nr_maps_glob = 0;
- bool strict = !(flags & MAPS_RELAX_COMPAT);
Elf_Data *symbols = obj->efile.symbols;
+ int i, map_def_sz = 0, nr_maps = 0, nr_syms;
Elf_Data *data = NULL;
- int ret = 0;
+ Elf_Scn *scn;
+
+ if (obj->efile.maps_shndx < 0)
+ return 0;
if (!symbols)
return -EINVAL;
- nr_syms = symbols->d_size / sizeof(GElf_Sym);
-
- if (obj->efile.maps_shndx >= 0) {
- Elf_Scn *scn = elf_getscn(obj->efile.elf,
- obj->efile.maps_shndx);
- if (scn)
- data = elf_getdata(scn, NULL);
- if (!scn || !data) {
- pr_warning("failed to get Elf_Data from map section %d\n",
- obj->efile.maps_shndx);
- return -EINVAL;
- }
+ scn = elf_getscn(obj->efile.elf, obj->efile.maps_shndx);
+ if (scn)
+ data = elf_getdata(scn, NULL);
+ if (!scn || !data) {
+ pr_warning("failed to get Elf_Data from map section %d\n",
+ obj->efile.maps_shndx);
+ return -EINVAL;
}
/*
@@ -840,16 +901,8 @@ static int bpf_object__init_maps(struct bpf_object *obj, int flags)
*
* TODO: Detect array of map and report error.
*/
- if (obj->caps.global_data) {
- if (obj->efile.data_shndx >= 0)
- nr_maps_glob++;
- if (obj->efile.rodata_shndx >= 0)
- nr_maps_glob++;
- if (obj->efile.bss_shndx >= 0)
- nr_maps_glob++;
- }
-
- for (i = 0; data && i < nr_syms; i++) {
+ nr_syms = symbols->d_size / sizeof(GElf_Sym);
+ for (i = 0; i < nr_syms; i++) {
GElf_Sym sym;
if (!gelf_getsym(symbols, i, &sym))
@@ -858,79 +911,56 @@ static int bpf_object__init_maps(struct bpf_object *obj, int flags)
continue;
nr_maps++;
}
-
- if (!nr_maps && !nr_maps_glob)
- return 0;
-
/* Assume equally sized map definitions */
- if (data) {
- pr_debug("maps in %s: %d maps in %zd bytes\n", obj->path,
- nr_maps, data->d_size);
-
- map_def_sz = data->d_size / nr_maps;
- if (!data->d_size || (data->d_size % nr_maps) != 0) {
- pr_warning("unable to determine map definition size "
- "section %s, %d maps in %zd bytes\n",
- obj->path, nr_maps, data->d_size);
- return -EINVAL;
- }
- }
-
- nr_maps += nr_maps_glob;
- obj->maps = calloc(nr_maps, sizeof(obj->maps[0]));
- if (!obj->maps) {
- pr_warning("alloc maps for object failed\n");
- return -ENOMEM;
- }
- obj->nr_maps = nr_maps;
-
- for (i = 0; i < nr_maps; i++) {
- /*
- * fill all fd with -1 so won't close incorrect
- * fd (fd=0 is stdin) when failure (zclose won't close
- * negative fd)).
- */
- obj->maps[i].fd = -1;
- obj->maps[i].inner_map_fd = -1;
+ pr_debug("maps in %s: %d maps in %zd bytes\n",
+ obj->path, nr_maps, data->d_size);
+
+ map_def_sz = data->d_size / nr_maps;
+ if (!data->d_size || (data->d_size % nr_maps) != 0) {
+ pr_warning("unable to determine map definition size "
+ "section %s, %d maps in %zd bytes\n",
+ obj->path, nr_maps, data->d_size);
+ return -EINVAL;
}
- /*
- * Fill obj->maps using data in "maps" section.
- */
- for (i = 0, map_idx = 0; data && i < nr_syms; i++) {
+ /* Fill obj->maps using data in "maps" section. */
+ for (i = 0; i < nr_syms; i++) {
GElf_Sym sym;
const char *map_name;
struct bpf_map_def *def;
+ struct bpf_map *map;
if (!gelf_getsym(symbols, i, &sym))
continue;
if (sym.st_shndx != obj->efile.maps_shndx)
continue;
- map_name = elf_strptr(obj->efile.elf,
- obj->efile.strtabidx,
+ map = bpf_object__add_map(obj);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ map_name = elf_strptr(obj->efile.elf, obj->efile.strtabidx,
sym.st_name);
if (!map_name) {
pr_warning("failed to get map #%d name sym string for obj %s\n",
- map_idx, obj->path);
+ i, obj->path);
return -LIBBPF_ERRNO__FORMAT;
}
- obj->maps[map_idx].libbpf_type = LIBBPF_MAP_UNSPEC;
- obj->maps[map_idx].offset = sym.st_value;
+ map->libbpf_type = LIBBPF_MAP_UNSPEC;
+ map->offset = sym.st_value;
if (sym.st_value + map_def_sz > data->d_size) {
pr_warning("corrupted maps section in %s: last map \"%s\" too small\n",
obj->path, map_name);
return -EINVAL;
}
- obj->maps[map_idx].name = strdup(map_name);
- if (!obj->maps[map_idx].name) {
+ map->name = strdup(map_name);
+ if (!map->name) {
pr_warning("failed to alloc map name\n");
return -ENOMEM;
}
- pr_debug("map %d is \"%s\"\n", map_idx,
- obj->maps[map_idx].name);
+ pr_debug("map %d is \"%s\"\n", i, map->name);
def = (struct bpf_map_def *)(data->d_buf + sym.st_value);
/*
* If the definition of the map in the object file fits in
@@ -939,7 +969,7 @@ static int bpf_object__init_maps(struct bpf_object *obj, int flags)
* calloc above.
*/
if (map_def_sz <= sizeof(struct bpf_map_def)) {
- memcpy(&obj->maps[map_idx].def, def, map_def_sz);
+ memcpy(&map->def, def, map_def_sz);
} else {
/*
* Here the map structure being read is bigger than what
@@ -959,37 +989,30 @@ static int bpf_object__init_maps(struct bpf_object *obj, int flags)
return -EINVAL;
}
}
- memcpy(&obj->maps[map_idx].def, def,
- sizeof(struct bpf_map_def));
+ memcpy(&map->def, def, sizeof(struct bpf_map_def));
}
- map_idx++;
}
+ return 0;
+}
- if (!obj->caps.global_data)
- goto finalize;
+static int bpf_object__init_maps(struct bpf_object *obj, int flags)
+{
+ bool strict = !(flags & MAPS_RELAX_COMPAT);
+ int err;
- /*
- * Populate rest of obj->maps with libbpf internal maps.
- */
- if (obj->efile.data_shndx >= 0)
- ret = bpf_object__init_internal_map(obj, &obj->maps[map_idx++],
- LIBBPF_MAP_DATA,
- obj->efile.data,
- &obj->sections.data);
- if (!ret && obj->efile.rodata_shndx >= 0)
- ret = bpf_object__init_internal_map(obj, &obj->maps[map_idx++],
- LIBBPF_MAP_RODATA,
- obj->efile.rodata,
- &obj->sections.rodata);
- if (!ret && obj->efile.bss_shndx >= 0)
- ret = bpf_object__init_internal_map(obj, &obj->maps[map_idx++],
- LIBBPF_MAP_BSS,
- obj->efile.bss, NULL);
-finalize:
- if (!ret)
+ err = bpf_object__init_user_maps(obj, strict);
+ if (err)
+ return err;
+
+ err = bpf_object__init_global_data_maps(obj);
+ if (err)
+ return err;
+
+ if (obj->nr_maps) {
qsort(obj->maps, obj->nr_maps, sizeof(obj->maps[0]),
compare_bpf_map);
- return ret;
+ }
+ return 0;
}
static bool section_have_execinstr(struct bpf_object *obj, int idx)
@@ -1262,14 +1285,10 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
return -LIBBPF_ERRNO__FORMAT;
}
err = bpf_object__load_btf(obj, btf_data, btf_ext_data);
- if (err)
- return err;
- if (bpf_object__has_maps(obj)) {
+ if (!err)
err = bpf_object__init_maps(obj, flags);
- if (err)
- return err;
- }
- err = bpf_object__init_prog_names(obj);
+ if (!err)
+ err = bpf_object__init_prog_names(obj);
return err;
}
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 00/11] BTF-defined BPF map definitions
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
This patch set implements initial version (as discussed at LSF/MM2019
conference) of a new way to specify BPF maps, relying on BTF type information,
which allows for easy extensibility, preserving forward and backward
compatibility. See details and examples in description for patch #6.
[0] contains an outline of follow up extensions to be added after this basic
set of features lands. They are useful by itself, but also allows to bring
libbpf to feature-parity with iproute2 BPF loader. That should open a path
forward for BPF loaders unification.
Patch #1 centralizes commonly used min/max macro in libbpf_internal.h.
Patch #2 extracts .BTF and .BTF.ext loading loging from elf_collect().
Patch #3 simplifies elf_collect() error-handling logic.
Patch #4 refactors map initialization logic into user-provided maps and global
data maps, in preparation to adding another way (BTF-defined maps).
Patch #5 adds support for map definitions in multiple ELF sections and
deprecates bpf_object__find_map_by_offset() API which doesn't appear to be
used anymore and makes assumption that all map definitions reside in single
ELF section.
Patch #6 splits BTF intialization from sanitization/loading into kernel to
preserve original BTF at the time of map initialization.
Patch #7 adds support for BTF-defined maps.
Patch #8 adds new test for BTF-defined map definition.
Patches #9-11 convert test BPF map definitions to use BTF way.
[0] https://lore.kernel.org/bpf/CAEf4BzbfdG2ub7gCi0OYqBrUoChVHWsmOntWAkJt47=FE+km+A@mail.gmail.com/
v1->v2:
- more BTF-sanity checks in parsing map definitions (Song);
- removed confusing usage of "attribute", switched to "field;
- split off elf_collect() refactor from btf loading refactor (Song);
- split selftests conversion into 3 patches (Stanislav):
1. test already relying on BTF;
2. tests w/ custom types as key/value (so benefiting from BTF);
3. all the rest tests (integers as key/value, special maps w/o BTF support).
- smaller code improvements (Song);
rfc->v1:
- error out on unknown field by default (Stanislav, Jakub, Lorenz);
Andrii Nakryiko (11):
libbpf: add common min/max macro to libbpf_internal.h
libbpf: extract BTF loading logic
libbpf: streamline ELF parsing error-handling
libbpf: refactor map initialization
libbpf: identify maps by section index in addition to offset
libbpf: split initialization and loading of BTF
libbpf: allow specifying map definitions using BTF
selftests/bpf: add test for BTF-defined maps
selftests/bpf: switch BPF_ANNOTATE_KV_PAIR tests to BTF-defined maps
selftests/bpf: convert tests w/ custom values to BTF-defined maps
selftests/bpf: convert remaining selftests to BTF-defined maps
tools/lib/bpf/bpf.c | 7 +-
tools/lib/bpf/bpf_prog_linfo.c | 5 +-
tools/lib/bpf/btf.c | 3 -
tools/lib/bpf/btf.h | 1 +
tools/lib/bpf/btf_dump.c | 3 -
tools/lib/bpf/libbpf.c | 781 +++++++++++++-----
tools/lib/bpf/libbpf_internal.h | 7 +
tools/testing/selftests/bpf/progs/bpf_flow.c | 18 +-
.../selftests/bpf/progs/get_cgroup_id_kern.c | 18 +-
.../testing/selftests/bpf/progs/netcnt_prog.c | 22 +-
.../selftests/bpf/progs/sample_map_ret0.c | 18 +-
.../selftests/bpf/progs/socket_cookie_prog.c | 11 +-
.../bpf/progs/sockmap_verdict_prog.c | 36 +-
.../selftests/bpf/progs/test_btf_newkv.c | 73 ++
.../bpf/progs/test_get_stack_rawtp.c | 27 +-
.../selftests/bpf/progs/test_global_data.c | 27 +-
tools/testing/selftests/bpf/progs/test_l4lb.c | 45 +-
.../selftests/bpf/progs/test_l4lb_noinline.c | 45 +-
.../selftests/bpf/progs/test_map_in_map.c | 20 +-
.../selftests/bpf/progs/test_map_lock.c | 22 +-
.../testing/selftests/bpf/progs/test_obj_id.c | 9 +-
.../bpf/progs/test_select_reuseport_kern.c | 45 +-
.../bpf/progs/test_send_signal_kern.c | 22 +-
.../bpf/progs/test_skb_cgroup_id_kern.c | 9 +-
.../bpf/progs/test_sock_fields_kern.c | 60 +-
.../selftests/bpf/progs/test_spin_lock.c | 33 +-
.../bpf/progs/test_stacktrace_build_id.c | 44 +-
.../selftests/bpf/progs/test_stacktrace_map.c | 40 +-
.../testing/selftests/bpf/progs/test_tc_edt.c | 9 +-
.../bpf/progs/test_tcp_check_syncookie_kern.c | 9 +-
.../selftests/bpf/progs/test_tcp_estats.c | 9 +-
.../selftests/bpf/progs/test_tcpbpf_kern.c | 18 +-
.../selftests/bpf/progs/test_tcpnotify_kern.c | 18 +-
tools/testing/selftests/bpf/progs/test_xdp.c | 18 +-
.../selftests/bpf/progs/test_xdp_noinline.c | 60 +-
tools/testing/selftests/bpf/test_btf.c | 10 +-
.../selftests/bpf/test_queue_stack_map.h | 20 +-
.../testing/selftests/bpf/test_sockmap_kern.h | 72 +-
38 files changed, 1199 insertions(+), 495 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/test_btf_newkv.c
--
2.17.1
^ permalink raw reply
* [PATCH v2 bpf-next 05/11] libbpf: identify maps by section index in addition to offset
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
To support maps to be defined in multiple sections, it's important to
identify map not just by offset within its section, but section index as
well. This patch adds tracking of section index.
For global data, we record section index of corresponding
.data/.bss/.rodata ELF section for uniformity, and thus don't need
a special value of offset for those maps.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
---
tools/lib/bpf/libbpf.c | 40 +++++++++++++++++++++++++---------------
1 file changed, 25 insertions(+), 15 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 88609dca4f7d..b1f3ab4b39b3 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -207,7 +207,8 @@ static const char * const libbpf_type_to_btf_name[] = {
struct bpf_map {
int fd;
char *name;
- size_t offset;
+ int sec_idx;
+ size_t sec_offset;
int map_ifindex;
int inner_map_fd;
struct bpf_map_def def;
@@ -647,7 +648,9 @@ static int compare_bpf_map(const void *_a, const void *_b)
const struct bpf_map *a = _a;
const struct bpf_map *b = _b;
- return a->offset - b->offset;
+ if (a->sec_idx != b->sec_idx)
+ return a->sec_idx - b->sec_idx;
+ return a->sec_offset - b->sec_offset;
}
static bool bpf_map_type__is_map_in_map(enum bpf_map_type type)
@@ -800,7 +803,7 @@ static struct bpf_map *bpf_object__add_map(struct bpf_object *obj)
static int
bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type,
- Elf_Data *data, void **data_buff)
+ int sec_idx, Elf_Data *data, void **data_buff)
{
char map_name[BPF_OBJ_NAME_LEN];
struct bpf_map_def *def;
@@ -811,7 +814,8 @@ bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type,
return PTR_ERR(map);
map->libbpf_type = type;
- map->offset = ~(typeof(map->offset))0;
+ map->sec_idx = sec_idx;
+ map->sec_offset = 0;
snprintf(map_name, sizeof(map_name), "%.8s%.7s", obj->name,
libbpf_type_to_btf_name[type]);
map->name = strdup(map_name);
@@ -819,6 +823,8 @@ bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type,
pr_warning("failed to alloc map name\n");
return -ENOMEM;
}
+ pr_debug("map '%s' (global data): at sec_idx %d, offset %zu.\n",
+ map_name, map->sec_idx, map->sec_offset);
def = &map->def;
def->type = BPF_MAP_TYPE_ARRAY;
@@ -851,6 +857,7 @@ static int bpf_object__init_global_data_maps(struct bpf_object *obj)
*/
if (obj->efile.data_shndx >= 0) {
err = bpf_object__init_internal_map(obj, LIBBPF_MAP_DATA,
+ obj->efile.data_shndx,
obj->efile.data,
&obj->sections.data);
if (err)
@@ -858,6 +865,7 @@ static int bpf_object__init_global_data_maps(struct bpf_object *obj)
}
if (obj->efile.rodata_shndx >= 0) {
err = bpf_object__init_internal_map(obj, LIBBPF_MAP_RODATA,
+ obj->efile.rodata_shndx,
obj->efile.rodata,
&obj->sections.rodata);
if (err)
@@ -865,6 +873,7 @@ static int bpf_object__init_global_data_maps(struct bpf_object *obj)
}
if (obj->efile.bss_shndx >= 0) {
err = bpf_object__init_internal_map(obj, LIBBPF_MAP_BSS,
+ obj->efile.bss_shndx,
obj->efile.bss, NULL);
if (err)
return err;
@@ -948,7 +957,10 @@ static int bpf_object__init_user_maps(struct bpf_object *obj, bool strict)
}
map->libbpf_type = LIBBPF_MAP_UNSPEC;
- map->offset = sym.st_value;
+ map->sec_idx = sym.st_shndx;
+ map->sec_offset = sym.st_value;
+ pr_debug("map '%s' (legacy): at sec_idx %d, offset %zu.\n",
+ map_name, map->sec_idx, map->sec_offset);
if (sym.st_value + map_def_sz > data->d_size) {
pr_warning("corrupted maps section in %s: last map \"%s\" too small\n",
obj->path, map_name);
@@ -1448,9 +1460,13 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,
if (maps[map_idx].libbpf_type != type)
continue;
if (type != LIBBPF_MAP_UNSPEC ||
- maps[map_idx].offset == sym.st_value) {
- pr_debug("relocation: find map %zd (%s) for insn %u\n",
- map_idx, maps[map_idx].name, insn_idx);
+ (maps[map_idx].sec_idx == sym.st_shndx &&
+ maps[map_idx].sec_offset == sym.st_value)) {
+ pr_debug("relocation: found map %zd (%s, sec_idx %d, offset %zu) for insn %u\n",
+ map_idx, maps[map_idx].name,
+ maps[map_idx].sec_idx,
+ maps[map_idx].sec_offset,
+ insn_idx);
break;
}
}
@@ -3468,13 +3484,7 @@ bpf_object__find_map_fd_by_name(struct bpf_object *obj, const char *name)
struct bpf_map *
bpf_object__find_map_by_offset(struct bpf_object *obj, size_t offset)
{
- int i;
-
- for (i = 0; i < obj->nr_maps; i++) {
- if (obj->maps[i].offset == offset)
- return &obj->maps[i];
- }
- return ERR_PTR(-ENOENT);
+ return ERR_PTR(-ENOTSUP);
}
long libbpf_get_error(const void *ptr)
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 07/11] libbpf: allow specifying map definitions using BTF
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
This patch adds support for a new way to define BPF maps. It relies on
BTF to describe mandatory and optional attributes of a map, as well as
captures type information of key and value naturally. This eliminates
the need for BPF_ANNOTATE_KV_PAIR hack and ensures key/value sizes are
always in sync with the key/value type.
Relying on BTF, this approach allows for both forward and backward
compatibility w.r.t. extending supported map definition features. By
default, any unrecognized attributes are treated as an error, but it's
possible relax this using MAPS_RELAX_COMPAT flag. New attributes, added
in the future will need to be optional.
The outline of the new map definition (short, BTF-defined maps) is as follows:
1. All the maps should be defined in .maps ELF section. It's possible to
have both "legacy" map definitions in `maps` sections and BTF-defined
maps in .maps sections. Everything will still work transparently.
2. The map declaration and initialization is done through
a global/static variable of a struct type with few mandatory and
extra optional fields:
- type field is mandatory and specified type of BPF map;
- key/value fields are mandatory and capture key/value type/size information;
- max_entries attribute is optional; if max_entries is not specified or
initialized, it has to be provided in runtime through libbpf API
before loading bpf_object;
- map_flags is optional and if not defined, will be assumed to be 0.
3. Key/value fields should be **a pointer** to a type describing
key/value. The pointee type is assumed (and will be recorded as such
and used for size determination) to be a type describing key/value of
the map. This is done to save excessive amounts of space allocated in
corresponding ELF sections for key/value of big size.
4. As some maps disallow having BTF type ID associated with key/value,
it's possible to specify key/value size explicitly without
associating BTF type ID with it. Use key_size and value_size fields
to do that (see example below).
Here's an example of simple ARRAY map defintion:
struct my_value { int x, y, z; };
struct {
int type;
int max_entries;
int *key;
struct my_value *value;
} btf_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
.max_entries = 16,
};
This will define BPF ARRAY map 'btf_map' with 16 elements. The key will
be of type int and thus key size will be 4 bytes. The value is struct
my_value of size 12 bytes. This map can be used from C code exactly the
same as with existing maps defined through struct bpf_map_def.
Here's an example of STACKMAP definition (which currently disallows BTF type
IDs for key/value):
struct {
__u32 type;
__u32 max_entries;
__u32 map_flags;
__u32 key_size;
__u32 value_size;
} stackmap SEC(".maps") = {
.type = BPF_MAP_TYPE_STACK_TRACE,
.max_entries = 128,
.map_flags = BPF_F_STACK_BUILD_ID,
.key_size = sizeof(__u32),
.value_size = PERF_MAX_STACK_DEPTH * sizeof(struct bpf_stack_build_id),
};
This approach is naturally extended to support map-in-map, by making a value
field to be another struct that describes inner map. This feature is not
implemented yet. It's also possible to incrementally add features like pinning
with full backwards and forward compatibility. Support for static
initialization of BPF_MAP_TYPE_PROG_ARRAY using pointers to BPF programs
is also on the roadmap.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/btf.h | 1 +
tools/lib/bpf/libbpf.c | 353 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 345 insertions(+), 9 deletions(-)
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index ba4ffa831aa4..88a52ae56fc6 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -17,6 +17,7 @@ extern "C" {
#define BTF_ELF_SEC ".BTF"
#define BTF_EXT_ELF_SEC ".BTF.ext"
+#define MAPS_ELF_SEC ".maps"
struct btf;
struct btf_ext;
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index da942ab2f06a..585e3a2f1eb4 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -262,6 +262,7 @@ struct bpf_object {
} *reloc;
int nr_reloc;
int maps_shndx;
+ int btf_maps_shndx;
int text_shndx;
int data_shndx;
int rodata_shndx;
@@ -514,6 +515,7 @@ static struct bpf_object *bpf_object__new(const char *path,
obj->efile.obj_buf = obj_buf;
obj->efile.obj_buf_sz = obj_buf_sz;
obj->efile.maps_shndx = -1;
+ obj->efile.btf_maps_shndx = -1;
obj->efile.data_shndx = -1;
obj->efile.rodata_shndx = -1;
obj->efile.bss_shndx = -1;
@@ -1007,6 +1009,312 @@ static int bpf_object__init_user_maps(struct bpf_object *obj, bool strict)
return 0;
}
+static const struct btf_type *skip_mods_and_typedefs(const struct btf *btf,
+ __u32 id)
+{
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ while (true) {
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPEDEF:
+ t = btf__type_by_id(btf, t->type);
+ break;
+ default:
+ return t;
+ }
+ }
+}
+
+static bool get_map_field_int(const char *map_name,
+ const struct btf *btf,
+ const struct btf_type *def,
+ const struct btf_member *m,
+ const void *data, __u32 *res) {
+ const struct btf_type *t = skip_mods_and_typedefs(btf, m->type);
+ const char *name = btf__name_by_offset(btf, m->name_off);
+ __u32 int_info = *(const __u32 *)(const void *)(t + 1);
+
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_INT) {
+ pr_warning("map '%s': attr '%s': expected INT, got %u.\n",
+ map_name, name, BTF_INFO_KIND(t->info));
+ return false;
+ }
+ if (t->size != 4 || BTF_INT_BITS(int_info) != 32 ||
+ BTF_INT_OFFSET(int_info)) {
+ pr_warning("map '%s': attr '%s': expected 32-bit non-bitfield integer, "
+ "got %u-byte (%d-bit) one with bit offset %d.\n",
+ map_name, name, t->size, BTF_INT_BITS(int_info),
+ BTF_INT_OFFSET(int_info));
+ return false;
+ }
+ if (BTF_INFO_KFLAG(def->info) && BTF_MEMBER_BITFIELD_SIZE(m->offset)) {
+ pr_warning("map '%s': attr '%s': bitfield is not supported.\n",
+ map_name, name);
+ return false;
+ }
+ if (m->offset % 32) {
+ pr_warning("map '%s': attr '%s': unaligned fields are not supported.\n",
+ map_name, name);
+ return false;
+ }
+
+ *res = *(const __u32 *)(data + m->offset / 8);
+ return true;
+}
+
+static int bpf_object__init_user_btf_map(struct bpf_object *obj,
+ const struct btf_type *sec,
+ int var_idx, int sec_idx,
+ const Elf_Data *data, bool strict)
+{
+ const struct btf_type *var, *def, *t;
+ const struct btf_var_secinfo *vi;
+ const struct btf_var *var_extra;
+ const struct btf_member *m;
+ const void *def_data;
+ const char *map_name;
+ struct bpf_map *map;
+ int vlen, i;
+
+ vi = (const struct btf_var_secinfo *)(const void *)(sec + 1) + var_idx;
+ var = btf__type_by_id(obj->btf, vi->type);
+ var_extra = (const void *)(var + 1);
+ map_name = btf__name_by_offset(obj->btf, var->name_off);
+ vlen = BTF_INFO_VLEN(var->info);
+
+ if (map_name == NULL || map_name[0] == '\0') {
+ pr_warning("map #%d: empty name.\n", var_idx);
+ return -EINVAL;
+ }
+ if ((__u64)vi->offset + vi->size > data->d_size) {
+ pr_warning("map '%s' BTF data is corrupted.\n", map_name);
+ return -EINVAL;
+ }
+ if (BTF_INFO_KIND(var->info) != BTF_KIND_VAR) {
+ pr_warning("map '%s': unexpected var kind %u.\n",
+ map_name, BTF_INFO_KIND(var->info));
+ return -EINVAL;
+ }
+ if (var_extra->linkage != BTF_VAR_GLOBAL_ALLOCATED &&
+ var_extra->linkage != BTF_VAR_STATIC) {
+ pr_warning("map '%s': unsupported var linkage %u.\n",
+ map_name, var_extra->linkage);
+ return -EOPNOTSUPP;
+ }
+
+ def = skip_mods_and_typedefs(obj->btf, var->type);
+ if (BTF_INFO_KIND(def->info) != BTF_KIND_STRUCT) {
+ pr_warning("map '%s': unexpected def kind %u.\n",
+ map_name, BTF_INFO_KIND(var->info));
+ return -EINVAL;
+ }
+ if (def->size > vi->size) {
+ pr_warning("map '%s': invalid def size.\n", map_name);
+ return -EINVAL;
+ }
+
+ map = bpf_object__add_map(obj);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+ map->name = strdup(map_name);
+ if (!map->name) {
+ pr_warning("map '%s': failed to alloc map name.\n", map_name);
+ return -ENOMEM;
+ }
+ map->libbpf_type = LIBBPF_MAP_UNSPEC;
+ map->def.type = BPF_MAP_TYPE_UNSPEC;
+ map->sec_idx = sec_idx;
+ map->sec_offset = vi->offset;
+ pr_debug("map '%s': at sec_idx %d, offset %zu.\n",
+ map_name, map->sec_idx, map->sec_offset);
+
+ def_data = data->d_buf + vi->offset;
+ vlen = BTF_INFO_VLEN(def->info);
+ m = (const void *)(def + 1);
+ for (i = 0; i < vlen; i++, m++) {
+ const char *name = btf__name_by_offset(obj->btf, m->name_off);
+
+ if (!name) {
+ pr_warning("map '%s': invalid field #%d.\n",
+ map_name, i);
+ return -EINVAL;
+ }
+ if (strcmp(name, "type") == 0) {
+ if (!get_map_field_int(map_name, obj->btf, def, m,
+ def_data, &map->def.type))
+ return -EINVAL;
+ pr_debug("map '%s': found type = %u.\n",
+ map_name, map->def.type);
+ } else if (strcmp(name, "max_entries") == 0) {
+ if (!get_map_field_int(map_name, obj->btf, def, m,
+ def_data, &map->def.max_entries))
+ return -EINVAL;
+ pr_debug("map '%s': found max_entries = %u.\n",
+ map_name, map->def.max_entries);
+ } else if (strcmp(name, "map_flags") == 0) {
+ if (!get_map_field_int(map_name, obj->btf, def, m,
+ def_data, &map->def.map_flags))
+ return -EINVAL;
+ pr_debug("map '%s': found map_flags = %u.\n",
+ map_name, map->def.map_flags);
+ } else if (strcmp(name, "key_size") == 0) {
+ __u32 sz;
+
+ if (!get_map_field_int(map_name, obj->btf, def, m,
+ def_data, &sz))
+ return -EINVAL;
+ pr_debug("map '%s': found key_size = %u.\n",
+ map_name, sz);
+ if (map->def.key_size && map->def.key_size != sz) {
+ pr_warning("map '%s': conflictling key size %u != %u.\n",
+ map_name, map->def.key_size, sz);
+ return -EINVAL;
+ }
+ map->def.key_size = sz;
+ } else if (strcmp(name, "key") == 0) {
+ __s64 sz;
+
+ t = btf__type_by_id(obj->btf, m->type);
+ if (!t) {
+ pr_warning("map '%s': key type [%d] not found.\n",
+ map_name, m->type);
+ return -EINVAL;
+ }
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_PTR) {
+ pr_warning("map '%s': key spec is not PTR: %u.\n",
+ map_name, BTF_INFO_KIND(t->info));
+ return -EINVAL;
+ }
+ sz = btf__resolve_size(obj->btf, t->type);
+ if (sz < 0) {
+ pr_warning("map '%s': can't determine key size for type [%u]: %lld.\n",
+ map_name, t->type, sz);
+ return sz;
+ }
+ pr_debug("map '%s': found key [%u], sz = %lld.\n",
+ map_name, t->type, sz);
+ if (map->def.key_size && map->def.key_size != sz) {
+ pr_warning("map '%s': conflictling key size %u != %lld.\n",
+ map_name, map->def.key_size, sz);
+ return -EINVAL;
+ }
+ map->def.key_size = sz;
+ map->btf_key_type_id = t->type;
+ } else if (strcmp(name, "value_size") == 0) {
+ __u32 sz;
+
+ if (!get_map_field_int(map_name, obj->btf, def, m,
+ def_data, &sz))
+ return -EINVAL;
+ pr_debug("map '%s': found value_size = %u.\n",
+ map_name, sz);
+ if (map->def.value_size && map->def.value_size != sz) {
+ pr_warning("map '%s': conflictling value size %u != %u.\n",
+ map_name, map->def.value_size, sz);
+ return -EINVAL;
+ }
+ map->def.value_size = sz;
+ } else if (strcmp(name, "value") == 0) {
+ __s64 sz;
+
+ t = btf__type_by_id(obj->btf, m->type);
+ if (!t) {
+ pr_warning("map '%s': value type [%d] not found.\n",
+ map_name, m->type);
+ return -EINVAL;
+ }
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_PTR) {
+ pr_warning("map '%s': value spec is not PTR: %u.\n",
+ map_name, BTF_INFO_KIND(t->info));
+ return -EINVAL;
+ }
+ sz = btf__resolve_size(obj->btf, t->type);
+ if (sz < 0) {
+ pr_warning("map '%s': can't determine value size for type [%u]: %lld.\n",
+ map_name, t->type, sz);
+ return sz;
+ }
+ pr_debug("map '%s': found value [%u], sz = %lld.\n",
+ map_name, t->type, sz);
+ if (map->def.value_size && map->def.value_size != sz) {
+ pr_warning("map '%s': conflictling value size %u != %lld.\n",
+ map_name, map->def.value_size, sz);
+ return -EINVAL;
+ }
+ map->def.value_size = sz;
+ map->btf_value_type_id = t->type;
+ } else {
+ if (strict) {
+ pr_warning("map '%s': unknown field '%s'.\n",
+ map_name, name);
+ return -ENOTSUP;
+ }
+ pr_debug("map '%s': ignoring unknown field '%s'.\n",
+ map_name, name);
+ }
+ }
+
+ if (map->def.type == BPF_MAP_TYPE_UNSPEC) {
+ pr_warning("map '%s': map type isn't specified.\n", map_name);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int bpf_object__init_user_btf_maps(struct bpf_object *obj, bool strict)
+{
+ const struct btf_type *sec = NULL;
+ int nr_types, i, vlen, err;
+ const struct btf_type *t;
+ const char *name;
+ Elf_Data *data;
+ Elf_Scn *scn;
+
+ if (obj->efile.btf_maps_shndx < 0)
+ return 0;
+
+ scn = elf_getscn(obj->efile.elf, obj->efile.btf_maps_shndx);
+ if (scn)
+ data = elf_getdata(scn, NULL);
+ if (!scn || !data) {
+ pr_warning("failed to get Elf_Data from map section %d (%s)\n",
+ obj->efile.maps_shndx, MAPS_ELF_SEC);
+ return -EINVAL;
+ }
+
+ nr_types = btf__get_nr_types(obj->btf);
+ for (i = 1; i <= nr_types; i++) {
+ t = btf__type_by_id(obj->btf, i);
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_DATASEC)
+ continue;
+ name = btf__name_by_offset(obj->btf, t->name_off);
+ if (strcmp(name, MAPS_ELF_SEC) == 0) {
+ sec = t;
+ break;
+ }
+ }
+
+ if (!sec) {
+ pr_warning("DATASEC '%s' not found.\n", MAPS_ELF_SEC);
+ return -ENOENT;
+ }
+
+ vlen = BTF_INFO_VLEN(sec->info);
+ for (i = 0; i < vlen; i++) {
+ err = bpf_object__init_user_btf_map(obj, sec, i,
+ obj->efile.btf_maps_shndx,
+ data, strict);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
static int bpf_object__init_maps(struct bpf_object *obj, int flags)
{
bool strict = !(flags & MAPS_RELAX_COMPAT);
@@ -1016,6 +1324,10 @@ static int bpf_object__init_maps(struct bpf_object *obj, int flags)
if (err)
return err;
+ err = bpf_object__init_user_btf_maps(obj, strict);
+ if (err)
+ return err;
+
err = bpf_object__init_global_data_maps(obj);
if (err)
return err;
@@ -1113,10 +1425,16 @@ static void bpf_object__sanitize_btf_ext(struct bpf_object *obj)
}
}
+static bool bpf_object__is_btf_mandatory(const struct bpf_object *obj)
+{
+ return obj->efile.btf_maps_shndx >= 0;
+}
+
static int bpf_object__init_btf(struct bpf_object *obj,
Elf_Data *btf_data,
Elf_Data *btf_ext_data)
{
+ bool btf_required = bpf_object__is_btf_mandatory(obj);
int err = 0;
if (btf_data) {
@@ -1150,10 +1468,18 @@ static int bpf_object__init_btf(struct bpf_object *obj,
}
out:
if (err || IS_ERR(obj->btf)) {
+ if (btf_required)
+ err = err ? : PTR_ERR(obj->btf);
+ else
+ err = 0;
if (!IS_ERR_OR_NULL(obj->btf))
btf__free(obj->btf);
obj->btf = NULL;
}
+ if (btf_required && !obj->btf) {
+ pr_warning("BTF is required, but is missing or corrupted.\n");
+ return err == 0 ? -ENOENT : err;
+ }
return 0;
}
@@ -1173,6 +1499,8 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)
BTF_ELF_SEC, err);
btf__free(obj->btf);
obj->btf = NULL;
+ if (bpf_object__is_btf_mandatory(obj))
+ return err;
}
return 0;
}
@@ -1236,6 +1564,8 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
return err;
} else if (strcmp(name, "maps") == 0) {
obj->efile.maps_shndx = idx;
+ } else if (strcmp(name, MAPS_ELF_SEC) == 0) {
+ obj->efile.btf_maps_shndx = idx;
} else if (strcmp(name, BTF_ELF_SEC) == 0) {
btf_data = data;
} else if (strcmp(name, BTF_EXT_ELF_SEC) == 0) {
@@ -1355,7 +1685,8 @@ static bool bpf_object__shndx_is_data(const struct bpf_object *obj,
static bool bpf_object__shndx_is_maps(const struct bpf_object *obj,
int shndx)
{
- return shndx == obj->efile.maps_shndx;
+ return shndx == obj->efile.maps_shndx ||
+ shndx == obj->efile.btf_maps_shndx;
}
static bool bpf_object__relo_in_known_section(const struct bpf_object *obj,
@@ -1399,14 +1730,14 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,
prog->nr_reloc = nrels;
for (i = 0; i < nrels; i++) {
- GElf_Sym sym;
- GElf_Rel rel;
- unsigned int insn_idx;
- unsigned int shdr_idx;
struct bpf_insn *insns = prog->insns;
enum libbpf_map_type type;
+ unsigned int insn_idx;
+ unsigned int shdr_idx;
const char *name;
size_t map_idx;
+ GElf_Sym sym;
+ GElf_Rel rel;
if (!gelf_getrel(data, i, &rel)) {
pr_warning("relocation: failed to get %d reloc\n", i);
@@ -1500,14 +1831,18 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,
return 0;
}
-static int bpf_map_find_btf_info(struct bpf_map *map, const struct btf *btf)
+static int bpf_map_find_btf_info(struct bpf_object *obj, struct bpf_map *map)
{
struct bpf_map_def *def = &map->def;
__u32 key_type_id = 0, value_type_id = 0;
int ret;
+ /* if it's BTF-defined map, we don't need to search for type IDs */
+ if (map->sec_idx == obj->efile.btf_maps_shndx)
+ return 0;
+
if (!bpf_map__is_internal(map)) {
- ret = btf__get_map_kv_tids(btf, map->name, def->key_size,
+ ret = btf__get_map_kv_tids(obj->btf, map->name, def->key_size,
def->value_size, &key_type_id,
&value_type_id);
} else {
@@ -1515,7 +1850,7 @@ static int bpf_map_find_btf_info(struct bpf_map *map, const struct btf *btf)
* LLVM annotates global data differently in BTF, that is,
* only as '.data', '.bss' or '.rodata'.
*/
- ret = btf__find_by_name(btf,
+ ret = btf__find_by_name(obj->btf,
libbpf_type_to_btf_name[map->libbpf_type]);
}
if (ret < 0)
@@ -1805,7 +2140,7 @@ bpf_object__create_maps(struct bpf_object *obj)
map->inner_map_fd >= 0)
create_attr.inner_map_fd = map->inner_map_fd;
- if (obj->btf && !bpf_map_find_btf_info(map, obj->btf)) {
+ if (obj->btf && !bpf_map_find_btf_info(obj, map)) {
create_attr.btf_fd = btf__fd(obj->btf);
create_attr.btf_key_type_id = map->btf_key_type_id;
create_attr.btf_value_type_id = map->btf_value_type_id;
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 06/11] libbpf: split initialization and loading of BTF
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Libbpf does sanitization of BTF before loading it into kernel, if kernel
doesn't support some of newer BTF features. This removes some of the
important information from BTF (e.g., DATASEC and VAR description),
which will be used for map construction. This patch splits BTF
processing into initialization step, in which BTF is initialized from
ELF and all the original data is still preserved; and
sanitization/loading step, which ensures that BTF is safe to load into
kernel. This allows to use full BTF information to construct maps, while
still loading valid BTF into older kernels.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
---
tools/lib/bpf/libbpf.c | 34 ++++++++++++++++++++++++----------
1 file changed, 24 insertions(+), 10 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index b1f3ab4b39b3..da942ab2f06a 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -1113,7 +1113,7 @@ static void bpf_object__sanitize_btf_ext(struct bpf_object *obj)
}
}
-static int bpf_object__load_btf(struct bpf_object *obj,
+static int bpf_object__init_btf(struct bpf_object *obj,
Elf_Data *btf_data,
Elf_Data *btf_ext_data)
{
@@ -1132,13 +1132,6 @@ static int bpf_object__load_btf(struct bpf_object *obj,
BTF_ELF_SEC, err);
goto out;
}
- bpf_object__sanitize_btf(obj);
- err = btf__load(obj->btf);
- if (err) {
- pr_warning("Error loading %s into kernel: %d.\n",
- BTF_ELF_SEC, err);
- goto out;
- }
}
if (btf_ext_data) {
if (!obj->btf) {
@@ -1154,7 +1147,6 @@ static int bpf_object__load_btf(struct bpf_object *obj,
obj->btf_ext = NULL;
goto out;
}
- bpf_object__sanitize_btf_ext(obj);
}
out:
if (err || IS_ERR(obj->btf)) {
@@ -1165,6 +1157,26 @@ static int bpf_object__load_btf(struct bpf_object *obj,
return 0;
}
+static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)
+{
+ int err = 0;
+
+ if (!obj->btf)
+ return 0;
+
+ bpf_object__sanitize_btf(obj);
+ bpf_object__sanitize_btf_ext(obj);
+
+ err = btf__load(obj->btf);
+ if (err) {
+ pr_warning("Error loading %s into kernel: %d.\n",
+ BTF_ELF_SEC, err);
+ btf__free(obj->btf);
+ obj->btf = NULL;
+ }
+ return 0;
+}
+
static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
{
Elf *elf = obj->efile.elf;
@@ -1296,9 +1308,11 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags)
pr_warning("Corrupted ELF file: index of strtab invalid\n");
return -LIBBPF_ERRNO__FORMAT;
}
- err = bpf_object__load_btf(obj, btf_data, btf_ext_data);
+ err = bpf_object__init_btf(obj, btf_data, btf_ext_data);
if (!err)
err = bpf_object__init_maps(obj, flags);
+ if (!err)
+ err = bpf_object__sanitize_and_load_btf(obj);
if (!err)
err = bpf_object__init_prog_names(obj);
return err;
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 09/11] selftests/bpf: switch BPF_ANNOTATE_KV_PAIR tests to BTF-defined maps
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Switch tests that already rely on BTF to BTF-defined map definitions.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
.../testing/selftests/bpf/progs/netcnt_prog.c | 22 ++++---
.../selftests/bpf/progs/test_map_lock.c | 22 +++----
.../bpf/progs/test_send_signal_kern.c | 22 +++----
.../bpf/progs/test_sock_fields_kern.c | 60 +++++++++++--------
.../selftests/bpf/progs/test_spin_lock.c | 33 +++++-----
5 files changed, 87 insertions(+), 72 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/netcnt_prog.c b/tools/testing/selftests/bpf/progs/netcnt_prog.c
index 9f741e69cebe..a25c82a5b7c8 100644
--- a/tools/testing/selftests/bpf/progs/netcnt_prog.c
+++ b/tools/testing/selftests/bpf/progs/netcnt_prog.c
@@ -10,24 +10,22 @@
#define REFRESH_TIME_NS 100000000
#define NS_PER_SEC 1000000000
-struct bpf_map_def SEC("maps") percpu_netcnt = {
+struct {
+ __u32 type;
+ struct bpf_cgroup_storage_key *key;
+ struct percpu_net_cnt *value;
+} percpu_netcnt SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE,
- .key_size = sizeof(struct bpf_cgroup_storage_key),
- .value_size = sizeof(struct percpu_net_cnt),
};
-BPF_ANNOTATE_KV_PAIR(percpu_netcnt, struct bpf_cgroup_storage_key,
- struct percpu_net_cnt);
-
-struct bpf_map_def SEC("maps") netcnt = {
+struct {
+ __u32 type;
+ struct bpf_cgroup_storage_key *key;
+ struct net_cnt *value;
+} netcnt SEC(".maps") = {
.type = BPF_MAP_TYPE_CGROUP_STORAGE,
- .key_size = sizeof(struct bpf_cgroup_storage_key),
- .value_size = sizeof(struct net_cnt),
};
-BPF_ANNOTATE_KV_PAIR(netcnt, struct bpf_cgroup_storage_key,
- struct net_cnt);
-
SEC("cgroup/skb")
int bpf_nextcnt(struct __sk_buff *skb)
{
diff --git a/tools/testing/selftests/bpf/progs/test_map_lock.c b/tools/testing/selftests/bpf/progs/test_map_lock.c
index af8cc68ed2f9..40d9c2853393 100644
--- a/tools/testing/selftests/bpf/progs/test_map_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_map_lock.c
@@ -11,29 +11,31 @@ struct hmap_elem {
int var[VAR_NUM];
};
-struct bpf_map_def SEC("maps") hash_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct hmap_elem *value;
+} hash_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(int),
- .value_size = sizeof(struct hmap_elem),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(hash_map, int, struct hmap_elem);
-
struct array_elem {
struct bpf_spin_lock lock;
int var[VAR_NUM];
};
-struct bpf_map_def SEC("maps") array_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ struct array_elem *value;
+} array_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(struct array_elem),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(array_map, int, struct array_elem);
-
SEC("map_lock_demo")
int bpf_map_lock_test(struct __sk_buff *skb)
{
diff --git a/tools/testing/selftests/bpf/progs/test_send_signal_kern.c b/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
index 45a1a1a2c345..6ac68be5d68b 100644
--- a/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
@@ -4,24 +4,26 @@
#include <linux/version.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") info_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} info_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(info_map, __u32, __u64);
-
-struct bpf_map_def SEC("maps") status_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} status_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(status_map, __u32, __u64);
-
SEC("send_signal_demo")
int bpf_send_signal_test(void *ctx)
{
diff --git a/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c b/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
index 1c39e4ccb7f1..c3d383d650cb 100644
--- a/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
@@ -27,31 +27,43 @@ enum bpf_linum_array_idx {
__NR_BPF_LINUM_ARRAY_IDX,
};
-struct bpf_map_def SEC("maps") addr_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct sockaddr_in6 *value;
+} addr_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct sockaddr_in6),
.max_entries = __NR_BPF_ADDR_ARRAY_IDX,
};
-struct bpf_map_def SEC("maps") sock_result_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct bpf_sock *value;
+} sock_result_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct bpf_sock),
.max_entries = __NR_BPF_RESULT_ARRAY_IDX,
};
-struct bpf_map_def SEC("maps") tcp_sock_result_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct bpf_tcp_sock *value;
+} tcp_sock_result_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct bpf_tcp_sock),
.max_entries = __NR_BPF_RESULT_ARRAY_IDX,
};
-struct bpf_map_def SEC("maps") linum_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} linum_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = __NR_BPF_LINUM_ARRAY_IDX,
};
@@ -60,26 +72,26 @@ struct bpf_spinlock_cnt {
__u32 cnt;
};
-struct bpf_map_def SEC("maps") sk_pkt_out_cnt = {
+struct {
+ __u32 type;
+ __u32 map_flags;
+ int *key;
+ struct bpf_spinlock_cnt *value;
+} sk_pkt_out_cnt SEC(".maps") = {
.type = BPF_MAP_TYPE_SK_STORAGE,
- .key_size = sizeof(int),
- .value_size = sizeof(struct bpf_spinlock_cnt),
- .max_entries = 0,
.map_flags = BPF_F_NO_PREALLOC,
};
-BPF_ANNOTATE_KV_PAIR(sk_pkt_out_cnt, int, struct bpf_spinlock_cnt);
-
-struct bpf_map_def SEC("maps") sk_pkt_out_cnt10 = {
+struct {
+ __u32 type;
+ __u32 map_flags;
+ int *key;
+ struct bpf_spinlock_cnt *value;
+} sk_pkt_out_cnt10 SEC(".maps") = {
.type = BPF_MAP_TYPE_SK_STORAGE,
- .key_size = sizeof(int),
- .value_size = sizeof(struct bpf_spinlock_cnt),
- .max_entries = 0,
.map_flags = BPF_F_NO_PREALLOC,
};
-BPF_ANNOTATE_KV_PAIR(sk_pkt_out_cnt10, int, struct bpf_spinlock_cnt);
-
static bool is_loopback6(__u32 *a6)
{
return !a6[0] && !a6[1] && !a6[2] && a6[3] == bpf_htonl(1);
diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock.c b/tools/testing/selftests/bpf/progs/test_spin_lock.c
index 40f904312090..0a77ae36d981 100644
--- a/tools/testing/selftests/bpf/progs/test_spin_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_spin_lock.c
@@ -10,30 +10,29 @@ struct hmap_elem {
int test_padding;
};
-struct bpf_map_def SEC("maps") hmap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ struct hmap_elem *value;
+} hmap SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(int),
- .value_size = sizeof(struct hmap_elem),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(hmap, int, struct hmap_elem);
-
-
struct cls_elem {
struct bpf_spin_lock lock;
volatile int cnt;
};
-struct bpf_map_def SEC("maps") cls_map = {
+struct {
+ __u32 type;
+ struct bpf_cgroup_storage_key *key;
+ struct cls_elem *value;
+} cls_map SEC(".maps") = {
.type = BPF_MAP_TYPE_CGROUP_STORAGE,
- .key_size = sizeof(struct bpf_cgroup_storage_key),
- .value_size = sizeof(struct cls_elem),
};
-BPF_ANNOTATE_KV_PAIR(cls_map, struct bpf_cgroup_storage_key,
- struct cls_elem);
-
struct bpf_vqueue {
struct bpf_spin_lock lock;
/* 4 byte hole */
@@ -42,14 +41,16 @@ struct bpf_vqueue {
unsigned int rate;
};
-struct bpf_map_def SEC("maps") vqueue = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ struct bpf_vqueue *value;
+} vqueue SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(struct bpf_vqueue),
.max_entries = 1,
};
-BPF_ANNOTATE_KV_PAIR(vqueue, int, struct bpf_vqueue);
#define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20)
SEC("spin_lock_demo")
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 08/11] selftests/bpf: add test for BTF-defined maps
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Add file test for BTF-defined map definition.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
---
.../selftests/bpf/progs/test_btf_newkv.c | 73 +++++++++++++++++++
tools/testing/selftests/bpf/test_btf.c | 10 +--
2 files changed, 76 insertions(+), 7 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/test_btf_newkv.c
diff --git a/tools/testing/selftests/bpf/progs/test_btf_newkv.c b/tools/testing/selftests/bpf/progs/test_btf_newkv.c
new file mode 100644
index 000000000000..28c16bb583b6
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_btf_newkv.c
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+int _version SEC("version") = 1;
+
+struct ipv_counts {
+ unsigned int v4;
+ unsigned int v6;
+};
+
+/* just to validate we can handle maps in multiple sections */
+struct bpf_map_def SEC("maps") btf_map_legacy = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(long long),
+ .max_entries = 4,
+};
+
+BPF_ANNOTATE_KV_PAIR(btf_map_legacy, int, struct ipv_counts);
+
+struct {
+ int *key;
+ struct ipv_counts *value;
+ unsigned int type;
+ unsigned int max_entries;
+} btf_map SEC(".maps") = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .max_entries = 4,
+};
+
+struct dummy_tracepoint_args {
+ unsigned long long pad;
+ struct sock *sock;
+};
+
+__attribute__((noinline))
+static int test_long_fname_2(struct dummy_tracepoint_args *arg)
+{
+ struct ipv_counts *counts;
+ int key = 0;
+
+ if (!arg->sock)
+ return 0;
+
+ counts = bpf_map_lookup_elem(&btf_map, &key);
+ if (!counts)
+ return 0;
+
+ counts->v6++;
+
+ /* just verify we can reference both maps */
+ counts = bpf_map_lookup_elem(&btf_map_legacy, &key);
+ if (!counts)
+ return 0;
+
+ return 0;
+}
+
+__attribute__((noinline))
+static int test_long_fname_1(struct dummy_tracepoint_args *arg)
+{
+ return test_long_fname_2(arg);
+}
+
+SEC("dummy_tracepoint")
+int _dummy_tracepoint(struct dummy_tracepoint_args *arg)
+{
+ return test_long_fname_1(arg);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_btf.c b/tools/testing/selftests/bpf/test_btf.c
index 289daf54dec4..8351cb5f4a20 100644
--- a/tools/testing/selftests/bpf/test_btf.c
+++ b/tools/testing/selftests/bpf/test_btf.c
@@ -4016,13 +4016,9 @@ struct btf_file_test {
};
static struct btf_file_test file_tests[] = {
-{
- .file = "test_btf_haskv.o",
-},
-{
- .file = "test_btf_nokv.o",
- .btf_kv_notfound = true,
-},
+ { .file = "test_btf_haskv.o", },
+ { .file = "test_btf_newkv.o", },
+ { .file = "test_btf_nokv.o", .btf_kv_notfound = true, },
};
static int do_test_file(unsigned int test_num)
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 11/11] selftests/bpf: convert remaining selftests to BTF-defined maps
From: Andrii Nakryiko @ 2019-06-17 19:27 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Convert all the rest of selftests that use BPF maps. These are either
maps with integer key/value or special types of maps that don't event
allow BTF type information for key/value.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
.../selftests/bpf/progs/get_cgroup_id_kern.c | 18 +++--
.../selftests/bpf/progs/sample_map_ret0.c | 18 +++--
.../bpf/progs/sockmap_verdict_prog.c | 36 +++++++---
.../selftests/bpf/progs/test_map_in_map.c | 20 ++++--
.../testing/selftests/bpf/progs/test_obj_id.c | 9 ++-
.../bpf/progs/test_skb_cgroup_id_kern.c | 9 ++-
.../testing/selftests/bpf/progs/test_tc_edt.c | 9 ++-
.../bpf/progs/test_tcp_check_syncookie_kern.c | 9 ++-
.../selftests/bpf/test_queue_stack_map.h | 20 ++++--
.../testing/selftests/bpf/test_sockmap_kern.h | 72 +++++++++++++------
10 files changed, 154 insertions(+), 66 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c b/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
index 014dba10b8a5..87b202381088 100644
--- a/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
+++ b/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
@@ -4,17 +4,23 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") cg_ids = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} cg_ids SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 1,
};
-struct bpf_map_def SEC("maps") pidmap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} pidmap SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/sample_map_ret0.c b/tools/testing/selftests/bpf/progs/sample_map_ret0.c
index 0756303676ac..0f4d47cecd4d 100644
--- a/tools/testing/selftests/bpf/progs/sample_map_ret0.c
+++ b/tools/testing/selftests/bpf/progs/sample_map_ret0.c
@@ -2,17 +2,23 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") htab = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ long *value;
+} htab SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(long),
.max_entries = 2,
};
-struct bpf_map_def SEC("maps") array = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ long *value;
+} array SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(long),
.max_entries = 2,
};
diff --git a/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c b/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
index d85c874ef25e..983c4f6e4fad 100644
--- a/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
+++ b/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
@@ -4,31 +4,49 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") sock_map_rx = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map_rx SEC(".maps") = {
.type = BPF_MAP_TYPE_SOCKMAP,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_map_tx = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map_tx SEC(".maps") = {
.type = BPF_MAP_TYPE_SOCKMAP,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_map_msg = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map_msg SEC(".maps") = {
.type = BPF_MAP_TYPE_SOCKMAP,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_map_break = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_map_break SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 20,
};
diff --git a/tools/testing/selftests/bpf/progs/test_map_in_map.c b/tools/testing/selftests/bpf/progs/test_map_in_map.c
index 2985f262846e..7404bee7c26e 100644
--- a/tools/testing/selftests/bpf/progs/test_map_in_map.c
+++ b/tools/testing/selftests/bpf/progs/test_map_in_map.c
@@ -5,22 +5,30 @@
#include <linux/types.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") mim_array = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} mim_array SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY_OF_MAPS,
+ .max_entries = 1,
.key_size = sizeof(int),
/* must be sizeof(__u32) for map in map */
.value_size = sizeof(__u32),
- .max_entries = 1,
- .map_flags = 0,
};
-struct bpf_map_def SEC("maps") mim_hash = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} mim_hash SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH_OF_MAPS,
+ .max_entries = 1,
.key_size = sizeof(int),
/* must be sizeof(__u32) for map in map */
.value_size = sizeof(__u32),
- .max_entries = 1,
- .map_flags = 0,
};
SEC("xdp_mimtest")
diff --git a/tools/testing/selftests/bpf/progs/test_obj_id.c b/tools/testing/selftests/bpf/progs/test_obj_id.c
index 880d2963b472..2b1c2efdeed4 100644
--- a/tools/testing/selftests/bpf/progs/test_obj_id.c
+++ b/tools/testing/selftests/bpf/progs/test_obj_id.c
@@ -16,10 +16,13 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") test_map_id = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} test_map_id SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/test_skb_cgroup_id_kern.c b/tools/testing/selftests/bpf/progs/test_skb_cgroup_id_kern.c
index 68cf9829f5a7..af296b876156 100644
--- a/tools/testing/selftests/bpf/progs/test_skb_cgroup_id_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_skb_cgroup_id_kern.c
@@ -10,10 +10,13 @@
#define NUM_CGROUP_LEVELS 4
-struct bpf_map_def SEC("maps") cgroup_ids = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} cgroup_ids SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = NUM_CGROUP_LEVELS,
};
diff --git a/tools/testing/selftests/bpf/progs/test_tc_edt.c b/tools/testing/selftests/bpf/progs/test_tc_edt.c
index 3af64c470d64..c2781dd78617 100644
--- a/tools/testing/selftests/bpf/progs/test_tc_edt.c
+++ b/tools/testing/selftests/bpf/progs/test_tc_edt.c
@@ -16,10 +16,13 @@
#define THROTTLE_RATE_BPS (5 * 1000 * 1000)
/* flow_key => last_tstamp timestamp used */
-struct bpf_map_def SEC("maps") flow_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ uint32_t *key;
+ uint64_t *value;
+} flow_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(uint32_t),
- .value_size = sizeof(uint64_t),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/test_tcp_check_syncookie_kern.c b/tools/testing/selftests/bpf/progs/test_tcp_check_syncookie_kern.c
index 1ab095bcacd8..0f1725e25c44 100644
--- a/tools/testing/selftests/bpf/progs/test_tcp_check_syncookie_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcp_check_syncookie_kern.c
@@ -16,10 +16,13 @@
#include "bpf_helpers.h"
#include "bpf_endian.h"
-struct bpf_map_def SEC("maps") results = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} results SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/test_queue_stack_map.h b/tools/testing/selftests/bpf/test_queue_stack_map.h
index 295b9b3bc5c7..f284137a36c4 100644
--- a/tools/testing/selftests/bpf/test_queue_stack_map.h
+++ b/tools/testing/selftests/bpf/test_queue_stack_map.h
@@ -10,20 +10,28 @@
int _version SEC("version") = 1;
-struct bpf_map_def __attribute__ ((section("maps"), used)) map_in = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} map_in SEC(".maps") = {
.type = MAP_TYPE,
+ .max_entries = 32,
.key_size = 0,
.value_size = sizeof(__u32),
- .max_entries = 32,
- .map_flags = 0,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) map_out = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} map_out SEC(".maps") = {
.type = MAP_TYPE,
+ .max_entries = 32,
.key_size = 0,
.value_size = sizeof(__u32),
- .max_entries = 32,
- .map_flags = 0,
};
SEC("test")
diff --git a/tools/testing/selftests/bpf/test_sockmap_kern.h b/tools/testing/selftests/bpf/test_sockmap_kern.h
index 4e7d3da21357..70b9236cedb0 100644
--- a/tools/testing/selftests/bpf/test_sockmap_kern.h
+++ b/tools/testing/selftests/bpf/test_sockmap_kern.h
@@ -28,59 +28,89 @@
* are established and verdicts are decided.
*/
-struct bpf_map_def SEC("maps") sock_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map SEC(".maps") = {
.type = TEST_MAP_TYPE,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_map_txmsg = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map_txmsg SEC(".maps") = {
.type = TEST_MAP_TYPE,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_map_redir = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} sock_map_redir SEC(".maps") = {
.type = TEST_MAP_TYPE,
+ .max_entries = 20,
.key_size = sizeof(int),
.value_size = sizeof(int),
- .max_entries = 20,
};
-struct bpf_map_def SEC("maps") sock_apply_bytes = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_apply_bytes SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 1
};
-struct bpf_map_def SEC("maps") sock_cork_bytes = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_cork_bytes SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 1
};
-struct bpf_map_def SEC("maps") sock_bytes = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_bytes SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 6
};
-struct bpf_map_def SEC("maps") sock_redir_flags = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_redir_flags SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 1
};
-struct bpf_map_def SEC("maps") sock_skb_opts = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ int *key;
+ int *value;
+} sock_skb_opts SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
.max_entries = 1
};
--
2.17.1
^ permalink raw reply related
* [PATCH v2 bpf-next 10/11] selftests/bpf: convert tests w/ custom values to BTF-defined maps
From: Andrii Nakryiko @ 2019-06-17 19:26 UTC (permalink / raw)
To: andrii.nakryiko, ast, daniel, netdev, bpf, kernel-team; +Cc: Andrii Nakryiko
In-Reply-To: <20190617192700.2313445-1-andriin@fb.com>
Convert a bulk of selftests that have maps with custom (not integer) key
and/or value.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/testing/selftests/bpf/progs/bpf_flow.c | 18 ++++--
.../selftests/bpf/progs/socket_cookie_prog.c | 11 ++--
.../bpf/progs/test_get_stack_rawtp.c | 27 ++++++---
.../selftests/bpf/progs/test_global_data.c | 27 ++++++---
tools/testing/selftests/bpf/progs/test_l4lb.c | 45 +++++++++-----
.../selftests/bpf/progs/test_l4lb_noinline.c | 45 +++++++++-----
.../bpf/progs/test_select_reuseport_kern.c | 45 +++++++++-----
.../bpf/progs/test_stacktrace_build_id.c | 44 +++++++++-----
.../selftests/bpf/progs/test_stacktrace_map.c | 40 +++++++++----
.../selftests/bpf/progs/test_tcp_estats.c | 9 ++-
.../selftests/bpf/progs/test_tcpbpf_kern.c | 18 ++++--
.../selftests/bpf/progs/test_tcpnotify_kern.c | 18 ++++--
tools/testing/selftests/bpf/progs/test_xdp.c | 18 ++++--
.../selftests/bpf/progs/test_xdp_noinline.c | 60 ++++++++++++-------
14 files changed, 285 insertions(+), 140 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/bpf_flow.c b/tools/testing/selftests/bpf/progs/bpf_flow.c
index 81ad9a0b29d0..849f42e548b5 100644
--- a/tools/testing/selftests/bpf/progs/bpf_flow.c
+++ b/tools/testing/selftests/bpf/progs/bpf_flow.c
@@ -57,17 +57,25 @@ struct frag_hdr {
__be32 identification;
};
-struct bpf_map_def SEC("maps") jmp_table = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} jmp_table SEC(".maps") = {
.type = BPF_MAP_TYPE_PROG_ARRAY,
+ .max_entries = 8,
.key_size = sizeof(__u32),
.value_size = sizeof(__u32),
- .max_entries = 8
};
-struct bpf_map_def SEC("maps") last_dissection = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct bpf_flow_keys *value;
+} last_dissection SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct bpf_flow_keys),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/socket_cookie_prog.c b/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
index 0db15c3210ad..6aabb681fb9a 100644
--- a/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
+++ b/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
@@ -12,15 +12,16 @@ struct socket_cookie {
__u32 cookie_value;
};
-struct bpf_map_def SEC("maps") socket_cookies = {
+struct {
+ __u32 type;
+ __u32 map_flags;
+ int *key;
+ struct socket_cookie *value;
+} socket_cookies SEC(".maps") = {
.type = BPF_MAP_TYPE_SK_STORAGE,
- .key_size = sizeof(int),
- .value_size = sizeof(struct socket_cookie),
.map_flags = BPF_F_NO_PREALLOC,
};
-BPF_ANNOTATE_KV_PAIR(socket_cookies, int, struct socket_cookie);
-
SEC("cgroup/connect6")
int set_cookie(struct bpf_sock_addr *ctx)
{
diff --git a/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c b/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
index f6d9f238e00a..aaa6ec250e15 100644
--- a/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
+++ b/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
@@ -15,17 +15,25 @@ struct stack_trace_t {
struct bpf_stack_build_id user_stack_buildid[MAX_STACK_RAWTP];
};
-struct bpf_map_def SEC("maps") perfmap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} perfmap SEC(".maps") = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
+ .max_entries = 2,
.key_size = sizeof(int),
.value_size = sizeof(__u32),
- .max_entries = 2,
};
-struct bpf_map_def SEC("maps") stackdata_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct stack_trace_t *value;
+} stackdata_map SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct stack_trace_t),
.max_entries = 1,
};
@@ -47,10 +55,13 @@ struct bpf_map_def SEC("maps") stackdata_map = {
* issue and avoid complicated C programming massaging.
* This is an acceptable workaround since there is one entry here.
*/
-struct bpf_map_def SEC("maps") rawdata_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 (*value)[2 * MAX_STACK_RAWTP];
+} rawdata_map SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = MAX_STACK_RAWTP * sizeof(__u64) * 2,
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/test_global_data.c b/tools/testing/selftests/bpf/progs/test_global_data.c
index 5ab14e941980..866cc7ddbe43 100644
--- a/tools/testing/selftests/bpf/progs/test_global_data.c
+++ b/tools/testing/selftests/bpf/progs/test_global_data.c
@@ -7,17 +7,23 @@
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") result_number = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} result_number SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 11,
};
-struct bpf_map_def SEC("maps") result_string = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ const char (*value)[32];
+} result_string SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = 32,
.max_entries = 5,
};
@@ -27,10 +33,13 @@ struct foo {
__u64 c;
};
-struct bpf_map_def SEC("maps") result_struct = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct foo *value;
+} result_struct SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct foo),
.max_entries = 5,
};
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb.c b/tools/testing/selftests/bpf/progs/test_l4lb.c
index 1e10c9590991..848cbb90f581 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb.c
@@ -169,38 +169,53 @@ struct eth_hdr {
unsigned short eth_proto;
};
-struct bpf_map_def SEC("maps") vip_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ struct vip *key;
+ struct vip_meta *value;
+} vip_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(struct vip),
- .value_size = sizeof(struct vip_meta),
.max_entries = MAX_VIPS,
};
-struct bpf_map_def SEC("maps") ch_rings = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} ch_rings SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = CH_RINGS_SIZE,
};
-struct bpf_map_def SEC("maps") reals = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct real_definition *value;
+} reals SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct real_definition),
.max_entries = MAX_REALS,
};
-struct bpf_map_def SEC("maps") stats = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct vip_stats *value;
+} stats SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct vip_stats),
.max_entries = MAX_VIPS,
};
-struct bpf_map_def SEC("maps") ctl_array = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct ctl_value *value;
+} ctl_array SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct ctl_value),
.max_entries = CTL_MAP_SIZE,
};
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
index ba44a14e6dc4..c63ecf3ca573 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
@@ -165,38 +165,53 @@ struct eth_hdr {
unsigned short eth_proto;
};
-struct bpf_map_def SEC("maps") vip_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ struct vip *key;
+ struct vip_meta *value;
+} vip_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(struct vip),
- .value_size = sizeof(struct vip_meta),
.max_entries = MAX_VIPS,
};
-struct bpf_map_def SEC("maps") ch_rings = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} ch_rings SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = CH_RINGS_SIZE,
};
-struct bpf_map_def SEC("maps") reals = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct real_definition *value;
+} reals SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct real_definition),
.max_entries = MAX_REALS,
};
-struct bpf_map_def SEC("maps") stats = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct vip_stats *value;
+} stats SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct vip_stats),
.max_entries = MAX_VIPS,
};
-struct bpf_map_def SEC("maps") ctl_array = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct ctl_value *value;
+} ctl_array SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct ctl_value),
.max_entries = CTL_MAP_SIZE,
};
diff --git a/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c b/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
index 5b54ec637ada..435a9527733e 100644
--- a/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
@@ -21,38 +21,55 @@ int _version SEC("version") = 1;
#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
#endif
-struct bpf_map_def SEC("maps") outer_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} outer_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY_OF_MAPS,
+ .max_entries = 1,
.key_size = sizeof(__u32),
.value_size = sizeof(__u32),
- .max_entries = 1,
};
-struct bpf_map_def SEC("maps") result_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} result_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = NR_RESULTS,
};
-struct bpf_map_def SEC("maps") tmp_index_ovr_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ int *value;
+} tmp_index_ovr_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(int),
.max_entries = 1,
};
-struct bpf_map_def SEC("maps") linum_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} linum_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 1,
};
-struct bpf_map_def SEC("maps") data_check_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct data_check *value;
+} data_check_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct data_check),
.max_entries = 1,
};
diff --git a/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c b/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
index d86c281e957f..fcf2280bb60c 100644
--- a/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
+++ b/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
@@ -8,34 +8,50 @@
#define PERF_MAX_STACK_DEPTH 127
#endif
-struct bpf_map_def SEC("maps") control_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} control_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 1,
};
-struct bpf_map_def SEC("maps") stackid_hmap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} stackid_hmap SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 16384,
};
-struct bpf_map_def SEC("maps") stackmap = {
+typedef struct bpf_stack_build_id stack_trace_t[PERF_MAX_STACK_DEPTH];
+
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 map_flags;
+ __u32 key_size;
+ __u32 value_size;
+} stackmap SEC(".maps") = {
.type = BPF_MAP_TYPE_STACK_TRACE,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct bpf_stack_build_id)
- * PERF_MAX_STACK_DEPTH,
.max_entries = 128,
.map_flags = BPF_F_STACK_BUILD_ID,
+ .key_size = sizeof(__u32),
+ .value_size = sizeof(stack_trace_t),
};
-struct bpf_map_def SEC("maps") stack_amap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ /* there seems to be a bug in kernel not handling typedef properly */
+ struct bpf_stack_build_id (*value)[PERF_MAX_STACK_DEPTH];
+} stack_amap SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct bpf_stack_build_id)
- * PERF_MAX_STACK_DEPTH,
.max_entries = 128,
};
diff --git a/tools/testing/selftests/bpf/progs/test_stacktrace_map.c b/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
index af111af7ca1a..7ad09adbf648 100644
--- a/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
+++ b/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
@@ -8,31 +8,47 @@
#define PERF_MAX_STACK_DEPTH 127
#endif
-struct bpf_map_def SEC("maps") control_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} control_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 1,
};
-struct bpf_map_def SEC("maps") stackid_hmap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} stackid_hmap SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 16384,
};
-struct bpf_map_def SEC("maps") stackmap = {
+typedef __u64 stack_trace_t[PERF_MAX_STACK_DEPTH];
+
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} stackmap SEC(".maps") = {
.type = BPF_MAP_TYPE_STACK_TRACE,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64) * PERF_MAX_STACK_DEPTH,
.max_entries = 16384,
+ .key_size = sizeof(__u32),
+ .value_size = sizeof(stack_trace_t),
};
-struct bpf_map_def SEC("maps") stack_amap = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 (*value)[PERF_MAX_STACK_DEPTH];
+} stack_amap SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64) * PERF_MAX_STACK_DEPTH,
.max_entries = 16384,
};
diff --git a/tools/testing/selftests/bpf/progs/test_tcp_estats.c b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
index bee3bbecc0c4..df98f7e32832 100644
--- a/tools/testing/selftests/bpf/progs/test_tcp_estats.c
+++ b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
@@ -148,10 +148,13 @@ struct tcp_estats_basic_event {
struct tcp_estats_conn_id conn_id;
};
-struct bpf_map_def SEC("maps") ev_record_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct tcp_estats_basic_event *value;
+} ev_record_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct tcp_estats_basic_event),
.max_entries = 1024,
};
diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
index c7c3240e0dd4..38e10c9fd996 100644
--- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
@@ -14,17 +14,23 @@
#include "bpf_endian.h"
#include "test_tcpbpf.h"
-struct bpf_map_def SEC("maps") global_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct tcpbpf_globals *value;
+} global_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct tcpbpf_globals),
.max_entries = 4,
};
-struct bpf_map_def SEC("maps") sockopt_results = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ int *value;
+} sockopt_results SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(int),
.max_entries = 2,
};
diff --git a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
index ec6db6e64c41..d073d37d4e27 100644
--- a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
@@ -14,18 +14,26 @@
#include "bpf_endian.h"
#include "test_tcpnotify.h"
-struct bpf_map_def SEC("maps") global_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct tcpnotify_globals *value;
+} global_map SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct tcpnotify_globals),
.max_entries = 4,
};
-struct bpf_map_def SEC("maps") perf_event_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 key_size;
+ __u32 value_size;
+} perf_event_map SEC(".maps") = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
+ .max_entries = 2,
.key_size = sizeof(int),
.value_size = sizeof(__u32),
- .max_entries = 2,
};
int _version SEC("version") = 1;
diff --git a/tools/testing/selftests/bpf/progs/test_xdp.c b/tools/testing/selftests/bpf/progs/test_xdp.c
index 5e7df8bb5b5d..ec3d2c1c8cf9 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp.c
@@ -22,17 +22,23 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") rxcnt = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u64 *value;
+} rxcnt SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
.max_entries = 256,
};
-struct bpf_map_def SEC("maps") vip2tnl = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ struct vip *key;
+ struct iptnl_info *value;
+} vip2tnl SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(struct vip),
- .value_size = sizeof(struct iptnl_info),
.max_entries = MAX_IPTNL_ENTRIES,
};
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_noinline.c b/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
index 4fe6aaad22a4..d2eddb5553d1 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
@@ -163,52 +163,66 @@ struct lb_stats {
__u64 v1;
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) vip_map = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ struct vip_definition *key;
+ struct vip_meta *value;
+} vip_map SEC(".maps") = {
.type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(struct vip_definition),
- .value_size = sizeof(struct vip_meta),
.max_entries = 512,
- .map_flags = 0,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) lru_cache = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 map_flags;
+ struct flow_key *key;
+ struct real_pos_lru *value;
+} lru_cache SEC(".maps") = {
.type = BPF_MAP_TYPE_LRU_HASH,
- .key_size = sizeof(struct flow_key),
- .value_size = sizeof(struct real_pos_lru),
.max_entries = 300,
.map_flags = 1U << 1,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) ch_rings = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ __u32 *value;
+} ch_rings SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
.max_entries = 12 * 655,
- .map_flags = 0,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) reals = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct real_definition *value;
+} reals SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct real_definition),
.max_entries = 40,
- .map_flags = 0,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) stats = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct lb_stats *value;
+} stats SEC(".maps") = {
.type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct lb_stats),
.max_entries = 515,
- .map_flags = 0,
};
-struct bpf_map_def __attribute__ ((section("maps"), used)) ctl_array = {
+struct {
+ __u32 type;
+ __u32 max_entries;
+ __u32 *key;
+ struct ctl_value *value;
+} ctl_array SEC(".maps") = {
.type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct ctl_value),
.max_entries = 16,
- .map_flags = 0,
};
struct eth_hdr {
--
2.17.1
^ permalink raw reply related
* RE: [PATCH net] hvsock: fix epollout hang from race condition
From: Sunil Muthuswamy @ 2019-06-17 19:27 UTC (permalink / raw)
To: David Miller
Cc: Dexuan Cui, KY Srinivasan, Haiyang Zhang, Stephen Hemminger,
sashal@kernel.org, Michael Kelley, netdev@vger.kernel.org,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <20190617.115615.91633577273679753.davem@davemloft.net>
> -----Original Message-----
> From: David Miller <davem@davemloft.net>
> Sent: Monday, June 17, 2019 11:56 AM
> To: Sunil Muthuswamy <sunilmut@microsoft.com>
> Cc: Dexuan Cui <decui@microsoft.com>; KY Srinivasan <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Stephen
> Hemminger <sthemmin@microsoft.com>; sashal@kernel.org; Michael Kelley <mikelley@microsoft.com>; netdev@vger.kernel.org;
> linux-hyperv@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH net] hvsock: fix epollout hang from race condition
>
> From: Sunil Muthuswamy <sunilmut@microsoft.com>
> Date: Mon, 17 Jun 2019 18:47:08 +0000
>
> >
> >
> >> -----Original Message-----
> >> From: linux-hyperv-owner@vger.kernel.org <linux-hyperv-owner@vger.kernel.org> On Behalf Of David Miller
> >> Sent: Sunday, June 16, 2019 1:55 PM
> >> To: Dexuan Cui <decui@microsoft.com>
> >> Cc: Sunil Muthuswamy <sunilmut@microsoft.com>; KY Srinivasan <kys@microsoft.com>; Haiyang Zhang
> <haiyangz@microsoft.com>;
> >> Stephen Hemminger <sthemmin@microsoft.com>; sashal@kernel.org; Michael Kelley <mikelley@microsoft.com>;
> >> netdev@vger.kernel.org; linux-hyperv@vger.kernel.org; linux-kernel@vger.kernel.org
> >> Subject: Re: [PATCH net] hvsock: fix epollout hang from race condition
> >>
> >> From: Dexuan Cui <decui@microsoft.com>
> >> Date: Sat, 15 Jun 2019 03:22:32 +0000
> >>
> >> > These warnings are not introduced by this patch from Sunil.
> >> >
> >> > I'm not sure why I didn't notice these warnings before.
> >> > Probably my gcc version is not new eought?
> >> >
> >> > Actually these warnings are bogus, as I checked the related functions,
> >> > which may confuse the compiler's static analysis.
> >> >
> >> > I'm going to make a patch to initialize the pointers to NULL to suppress
> >> > the warnings. My patch will be based on the latest's net.git + this patch
> >> > from Sunil.
> >>
> >> Sunil should then resubmit his patch against something that has the
> >> warning suppression patch applied.
> >
> > David, Dexuan's patch to suppress the warnings seems to be applied now
> > to the 'net' branch. Can we please get this patch applied as well?
>
> I don't know how else to say "Suni should then resubmit his patch"
>
> Please just resubmit it!
The patch does not change at all. So, I was hoping we could reapply it. But, I have
resubmitted the patch. Thanks.
^ permalink raw reply
* Re: [PATCH bpf-next 8/8] selftests/bpf: switch tests to BTF-defined map definitions
From: Andrii Nakryiko @ 2019-06-17 19:30 UTC (permalink / raw)
To: Stanislav Fomichev
Cc: Andrii Nakryiko, bpf, Networking, Alexei Starovoitov,
Daniel Borkmann, Kernel Team
In-Reply-To: <20190615000104.GG9636@mini-arch>
On Fri, Jun 14, 2019 at 5:01 PM Stanislav Fomichev <sdf@fomichev.me> wrote:
>
> On 06/14, Andrii Nakryiko wrote:
> > On Fri, Jun 14, 2019 at 4:23 PM Stanislav Fomichev <sdf@fomichev.me> wrote:
> > >
> > > On 06/10, Andrii Nakryiko wrote:
> > > > Switch test map definition to new BTF-defined format.
> > > Reiterating my concerns on non-RFC version:
> > >
> > > Pretty please, let's not convert everything at once. Let's start
> > > with stuff that explicitly depends on BTF (spinlocks?).
> >
> > How about this approach. I can split last commit into two. One
> > converting all the stuff that needs BTF (spinlocks, etc). Another part
> > - everything else. If it's so important for your use case, you'll be
> > able to just back out my last commit. Or we just don't land last
> > commit.
> I can always rollback or do not backport internally; the issue is that
> it would be much harder to backport any future fixes/extensions to
> those tests. So splitting in two and not landing the last one is
> preferable ;-)
So I just posted v2 and I split all the test conversions into three parts:
1. tests that already rely on BTF
2. tests w/ custom key/value types
3. all the reset
I think we should definitely apply #1. I think #2 would be nice. And
we can probably hold off on #3. I'll let Alexei or Daniel decide, but
it shouldn't be hard for them to do that.
>
> > > One good argument (aside from the one that we'd like to be able to
> > > run tests internally without BTF for a while): libbpf doesn't
> > > have any tests as far as I'm aware. If we don't have 'legacy' maps in the
> > > selftests, libbpf may bit rot.
> >
> > I left few legacy maps exactly for that reason. See progs/test_btf_*.c.
> Damn it, you've destroyed my only good argument.
Heh :)
>
> > > (Andrii, feel free to ignore, since we've already discussed that)
> > >
> > > > Signed-off-by: Andrii Nakryiko <andriin@fb.com>
> > > > ---
> >
> >
> > <snip>
^ permalink raw reply
* [PATCH net-next v3 0/6] net/sched: Add txtime-assist support for taprio.
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
Changes in v3:
- Simplify implementation for taprio flags.
- txtime_delay can only be set if txtime-assist mode is enabled.
- txtime_delay and flags will only be visible in tc output if set by user.
- Minor changes in error reporting.
Changes in v2:
- Txtime-offload has now been renamed to txtime-assist mode.
- Renamed the offload parameter to flags.
- Removed the code which introduced the hardware offloading functionality.
Original Cover letter (with above changes included)
--------------------------------------------------
Currently, we are seeing packets being transmitted outside their
timeslices. We can confirm that the packets are being dequeued at the right
time. So, the delay is induced after the packet is dequeued, because
taprio, without any offloading, has no control of when a packet is actually
transmitted.
In order to solve this, we are making use of the txtime feature provided by
ETF qdisc. Hardware offloading needs to be supported by the ETF qdisc in
order to take advantage of this feature. The taprio qdisc will assign
txtime (in skb->tstamp) for all the packets which do not have the txtime
allocated via the SO_TXTIME socket option. For the packets which already
have SO_TXTIME set, taprio will validate whether the packet will be
transmitted in the correct interval.
In order to support this, the following parameters have been added:
- flags (taprio): This is added in order to support different offloading
modes which will be added in the future.
- txtime-delay (taprio): This indicates the minimum time it will take for
the packet to hit the wire after it reaches taprio_enqueue(). This is
useful in determining whether we can transmit the packet in the remaining
time if the gate corresponding to the packet is currently open.
- skip_skb_check (ETF): ETF currently drops any packet which does not have
the SO_TXTIME socket option set. This check can be skipped by specifying
this option.
Following is an example configuration:
tc qdisc replace dev $IFACE parent root handle 100 taprio \\
num_tc 3 \\
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \\
queues 1@0 1@0 1@0 \\
base-time $BASE_TIME \\
sched-entry S 01 300000 \\
sched-entry S 02 300000 \\
sched-entry S 04 400000 \\
flags 0x1 \\
txtime-delay 200000 \\
clockid CLOCK_TAI
tc qdisc replace dev $IFACE parent 100:1 etf \\
offload delta 200000 clockid CLOCK_TAI skip_skb_check
Here, the "flags" parameter is indicating that the txtime-assist mode is
enabled. Also, all the traffic classes have been assigned the same queue.
This is to prevent the traffic classes in the lower priority queues from
getting starved. Note that this configuration is specific to the i210
ethernet card. Other network cards where the hardware queues are given the
same priority, might be able to utilize more than one queue.
Following are some of the other highlights of the series:
- Fix a bug where hardware timestamping and SO_TXTIME options cannot be
used together. (Patch 1)
- Introduces the skip_skb_check option. (Patch 2)
- Make TxTime assist mode work with TCP packets (Patch 7).
The following changes are recommended to be done in order to get the best
performance from taprio in this mode:
# TSN in general does not allow Jumbo frames.
ip link set dev enp1s0 mtu 1514
# Disable segmentation offload. This is to prevent NIC from sending packets
# after the gate for a traffic class has closed.
ethtool -K eth0 gso off
ethtool -K eth0 tso off
# Disable energy efficient ethernet to make sure there are no latency
# spikes when NIC is trying to wake up when the packet is supposed to be
# sent.
ethtool --set-eee eth0 eee off
Thanks,
Vedang Patel
Vedang Patel (6):
igb: clear out tstamp after sending the packet.
etf: Add skip_sock_check
taprio: calculate cycle_time when schedule is installed
taprio: Add support for txtime-assist mode.
taprio: make clock reference conversions easier
taprio: Adjust timestamps for TCP packets.
drivers/net/ethernet/intel/igb/igb_main.c | 1 +
include/uapi/linux/pkt_sched.h | 5 +
net/sched/sch_etf.c | 10 +
net/sched/sch_taprio.c | 431 +++++++++++++++++++++++++++---
4 files changed, 413 insertions(+), 34 deletions(-)
--
2.7.3
^ permalink raw reply
* [PATCH net-next v3 1/6] igb: clear out tstamp after sending the packet.
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
In-Reply-To: <1560799870-18956-1-git-send-email-vedang.patel@intel.com>
skb->tstamp is being used at multiple places. On the transmit side, it
is used to determine the launchtime of the packet. It is also used to
determine the software timestamp after the packet has been transmitted.
So, clear out the tstamp value after it has been read so that we do not
report false software timestamp on the receive side.
Signed-off-by: Vedang Patel <vedang.patel@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index fc925adbd9fa..f66dae72fe37 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -5688,6 +5688,7 @@ static void igb_tx_ctxtdesc(struct igb_ring *tx_ring,
*/
if (tx_ring->launchtime_enable) {
ts = ns_to_timespec64(first->skb->tstamp);
+ first->skb->tstamp = 0;
context_desc->seqnum_seed = cpu_to_le32(ts.tv_nsec / 32);
} else {
context_desc->seqnum_seed = 0;
--
2.7.3
^ permalink raw reply related
* [PATCH net-next v3 2/6] etf: Add skip_sock_check
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
In-Reply-To: <1560799870-18956-1-git-send-email-vedang.patel@intel.com>
Currently, etf expects a socket with SO_TXTIME option set for each packet
it encounters. So, it will drop all other packets. But, in the future
commits we are planning to add functionality which where tstamp value will
be set by another qdisc. Also, some packets which are generated from within
the kernel (e.g. ICMP packets) do not have any socket associated with them.
So, this commit adds support for skip_sock_check. When this option is set,
etf will skip checking for a socket and other associated options for all
skbs.
Signed-off-by: Vedang Patel <vedang.patel@intel.com>
---
include/uapi/linux/pkt_sched.h | 1 +
net/sched/sch_etf.c | 10 ++++++++++
2 files changed, 11 insertions(+)
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 8b2f993cbb77..69fc52e4d6bd 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -990,6 +990,7 @@ struct tc_etf_qopt {
__u32 flags;
#define TC_ETF_DEADLINE_MODE_ON BIT(0)
#define TC_ETF_OFFLOAD_ON BIT(1)
+#define TC_ETF_SKIP_SOCK_CHECK BIT(2)
};
enum {
diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c
index db0c2ba1d156..cebfb65d8556 100644
--- a/net/sched/sch_etf.c
+++ b/net/sched/sch_etf.c
@@ -22,10 +22,12 @@
#define DEADLINE_MODE_IS_ON(x) ((x)->flags & TC_ETF_DEADLINE_MODE_ON)
#define OFFLOAD_IS_ON(x) ((x)->flags & TC_ETF_OFFLOAD_ON)
+#define SKIP_SOCK_CHECK_IS_SET(x) ((x)->flags & TC_ETF_SKIP_SOCK_CHECK)
struct etf_sched_data {
bool offload;
bool deadline_mode;
+ bool skip_sock_check;
int clockid;
int queue;
s32 delta; /* in ns */
@@ -77,6 +79,9 @@ static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb)
struct sock *sk = nskb->sk;
ktime_t now;
+ if (q->skip_sock_check)
+ goto skip;
+
if (!sk)
return false;
@@ -92,6 +97,7 @@ static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb)
if (sk->sk_txtime_deadline_mode != q->deadline_mode)
return false;
+skip:
now = q->get_time();
if (ktime_before(txtime, now) || ktime_before(txtime, q->last))
return false;
@@ -385,6 +391,7 @@ static int etf_init(struct Qdisc *sch, struct nlattr *opt,
q->clockid = qopt->clockid;
q->offload = OFFLOAD_IS_ON(qopt);
q->deadline_mode = DEADLINE_MODE_IS_ON(qopt);
+ q->skip_sock_check = SKIP_SOCK_CHECK_IS_SET(qopt);
switch (q->clockid) {
case CLOCK_REALTIME:
@@ -473,6 +480,9 @@ static int etf_dump(struct Qdisc *sch, struct sk_buff *skb)
if (q->deadline_mode)
opt.flags |= TC_ETF_DEADLINE_MODE_ON;
+ if (q->skip_sock_check)
+ opt.flags |= TC_ETF_SKIP_SOCK_CHECK;
+
if (nla_put(skb, TCA_ETF_PARMS, sizeof(opt), &opt))
goto nla_put_failure;
--
2.7.3
^ permalink raw reply related
* [PATCH net-next v3 3/6] taprio: calculate cycle_time when schedule is installed
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
In-Reply-To: <1560799870-18956-1-git-send-email-vedang.patel@intel.com>
cycle time for a particular schedule is calculated only when it is first
installed. So, it makes sense to just calculate it once right after the
'cycle_time' parameter has been parsed and store it in cycle_time.
Signed-off-by: Vedang Patel <vedang.patel@intel.com>
---
net/sched/sch_taprio.c | 29 +++++++++++------------------
1 file changed, 11 insertions(+), 18 deletions(-)
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 9ecfb8f5902a..a41d7d4434ee 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -108,22 +108,6 @@ static void switch_schedules(struct taprio_sched *q,
*admin = NULL;
}
-static ktime_t get_cycle_time(struct sched_gate_list *sched)
-{
- struct sched_entry *entry;
- ktime_t cycle = 0;
-
- if (sched->cycle_time != 0)
- return sched->cycle_time;
-
- list_for_each_entry(entry, &sched->entries, list)
- cycle = ktime_add_ns(cycle, entry->interval);
-
- sched->cycle_time = cycle;
-
- return cycle;
-}
-
static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff **to_free)
{
@@ -524,6 +508,15 @@ static int parse_taprio_schedule(struct nlattr **tb,
if (err < 0)
return err;
+ if (!new->cycle_time) {
+ struct sched_entry *entry;
+ ktime_t cycle = 0;
+
+ list_for_each_entry(entry, &new->entries, list)
+ cycle = ktime_add_ns(cycle, entry->interval);
+ new->cycle_time = cycle;
+ }
+
return 0;
}
@@ -605,7 +598,7 @@ static int taprio_get_start_time(struct Qdisc *sch,
return 0;
}
- cycle = get_cycle_time(sched);
+ cycle = sched->cycle_time;
/* The qdisc is expected to have at least one sched_entry. Moreover,
* any entry must have 'interval' > 0. Thus if the cycle time is zero,
@@ -632,7 +625,7 @@ static void setup_first_close_time(struct taprio_sched *q,
first = list_first_entry(&sched->entries,
struct sched_entry, list);
- cycle = get_cycle_time(sched);
+ cycle = sched->cycle_time;
/* FIXME: find a better place to do this */
sched->cycle_close_time = ktime_add_ns(base, cycle);
--
2.7.3
^ permalink raw reply related
* [PATCH net-next v3 5/6] taprio: make clock reference conversions easier
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
In-Reply-To: <1560799870-18956-1-git-send-email-vedang.patel@intel.com>
Later in this series we will need to transform from
CLOCK_MONOTONIC (used in TCP) to the clock reference used in TAPRIO.
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Vedang Patel <vedang.patel@intel.com>
---
net/sched/sch_taprio.c | 30 ++++++++++++++++++++++--------
1 file changed, 22 insertions(+), 8 deletions(-)
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index ab6080013666..f63cc3a9e69a 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -61,6 +61,7 @@ struct taprio_sched {
struct Qdisc **qdiscs;
struct Qdisc *root;
u32 flags;
+ enum tk_offsets tk_offset;
int clockid;
atomic64_t picos_per_byte; /* Using picoseconds because for 10Gbps+
* speeds it's sub-nanoseconds per byte
@@ -71,7 +72,6 @@ struct taprio_sched {
struct sched_entry __rcu *current_entry;
struct sched_gate_list __rcu *oper_sched;
struct sched_gate_list __rcu *admin_sched;
- ktime_t (*get_time)(void);
struct hrtimer advance_timer;
struct list_head taprio_list;
int txtime_delay;
@@ -85,6 +85,20 @@ static ktime_t sched_base_time(const struct sched_gate_list *sched)
return ns_to_ktime(sched->base_time);
}
+static inline ktime_t taprio_get_time(struct taprio_sched *q)
+{
+ ktime_t mono = ktime_get();
+
+ switch (q->tk_offset) {
+ case TK_OFFS_MAX:
+ return mono;
+ default:
+ return ktime_mono_to_any(mono, q->tk_offset);
+ }
+
+ return KTIME_MAX;
+}
+
static void taprio_free_sched_cb(struct rcu_head *head)
{
struct sched_gate_list *sched = container_of(head, struct sched_gate_list, rcu);
@@ -288,7 +302,7 @@ static long get_packet_txtime(struct sk_buff *skb, struct Qdisc *sch)
struct sched_entry *entry;
bool sched_changed;
- now = q->get_time();
+ now = taprio_get_time(q);
minimum_time = ktime_add_ns(now, q->txtime_delay);
rcu_read_lock();
@@ -479,7 +493,7 @@ static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
continue;
len = qdisc_pkt_len(skb);
- guard = ktime_add_ns(q->get_time(),
+ guard = ktime_add_ns(taprio_get_time(q),
length_to_duration(q, len));
/* In the case that there's no gate entry, there's no
@@ -848,7 +862,7 @@ static int taprio_get_start_time(struct Qdisc *sch,
s64 n;
base = sched_base_time(sched);
- now = q->get_time();
+ now = taprio_get_time(q);
if (ktime_after(base, now)) {
*start = base;
@@ -1094,16 +1108,16 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
switch (q->clockid) {
case CLOCK_REALTIME:
- q->get_time = ktime_get_real;
+ q->tk_offset = TK_OFFS_REAL;
break;
case CLOCK_MONOTONIC:
- q->get_time = ktime_get;
+ q->tk_offset = TK_OFFS_MAX;
break;
case CLOCK_BOOTTIME:
- q->get_time = ktime_get_boottime;
+ q->tk_offset = TK_OFFS_BOOT;
break;
case CLOCK_TAI:
- q->get_time = ktime_get_clocktai;
+ q->tk_offset = TK_OFFS_TAI;
break;
default:
NL_SET_ERR_MSG(extack, "Invalid 'clockid'");
--
2.7.3
^ permalink raw reply related
* [PATCH net-next v3 4/6] taprio: Add support for txtime-assist mode.
From: Vedang Patel @ 2019-06-17 19:31 UTC (permalink / raw)
To: netdev
Cc: jeffrey.t.kirsher, davem, jhs, xiyou.wangcong, jiri,
intel-wired-lan, vinicius.gomes, l, Vedang Patel
In-Reply-To: <1560799870-18956-1-git-send-email-vedang.patel@intel.com>
Currently, we are seeing non-critical packets being transmitted outside of
their timeslice. We can confirm that the packets are being dequeued at the
right time. So, the delay is induced in the hardware side. The most likely
reason is the hardware queues are starving the lower priority queues.
In order to improve the performance of taprio, we will be making use of the
txtime feature provided by the ETF qdisc. For all the packets which do not
have the SO_TXTIME option set, taprio will set the transmit timestamp (set
in skb->tstamp) in this mode. TAPrio Qdisc will ensure that the transmit
time for the packet is set to when the gate is open. If SO_TXTIME is set,
the TAPrio qdisc will validate whether the timestamp (in skb->tstamp)
occurs when the gate corresponding to skb's traffic class is open.
Following two parameters added to support this mode:
- flags: used to enable txtime-assist mode. Will also be used to enable
other modes (like hardware offloading) later.
- txtime-delay: This indicates the minimum time it will take for the packet
to hit the wire. This is useful in determining whether we can transmit
the packet in the remaining time if the gate corresponding to the packet is
currently open.
An example configuration for enabling txtime-assist:
tc qdisc replace dev eth0 parent root handle 100 taprio \\
num_tc 3 \\
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \\
queues 1@0 1@0 1@0 \\
base-time 1558653424279842568 \\
sched-entry S 01 300000 \\
sched-entry S 02 300000 \\
sched-entry S 04 400000 \\
flags 0x1 \\
txtime-delay 40000 \\
clockid CLOCK_TAI
tc qdisc replace dev $IFACE parent 100:1 etf skip_sock_check \\
offload delta 200000 clockid CLOCK_TAI
Note that all the traffic classes are mapped to the same queue. This is
only possible in taprio when txtime-assist is enabled. Also, note that the
ETF Qdisc is enabled with offload mode set.
In this mode, if the packet's traffic class is open and the complete packet
can be transmitted, taprio will try to transmit the packet immediately.
This will be done by setting skb->tstamp to current_time + the time delta
indicated in the txtime-delay parameter. This parameter indicates the time
taken (in software) for packet to reach the network adapter.
If the packet cannot be transmitted in the current interval or if the
packet's traffic is not currently transmitting, the skb->tstamp is set to
the next available timestamp value. This is tracked in the next_launchtime
parameter in the struct sched_entry.
The behaviour w.r.t admin and oper schedules is not changed from what is
present in software mode.
The transmit time is already known in advance. So, we do not need the HR
timers to advance the schedule and wakeup the dequeue side of taprio. So,
HR timer won't be run when this mode is enabled.
Signed-off-by: Vedang Patel <vedang.patel@intel.com>
---
include/uapi/linux/pkt_sched.h | 4 +
net/sched/sch_taprio.c | 351 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 338 insertions(+), 17 deletions(-)
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 69fc52e4d6bd..c085860ff637 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -1159,6 +1159,8 @@ enum {
* [TCA_TAPRIO_ATTR_SCHED_ENTRY_INTERVAL]
*/
+#define TCA_TAPRIO_ATTR_FLAG_TXTIME_ASSIST 0x1
+
enum {
TCA_TAPRIO_ATTR_UNSPEC,
TCA_TAPRIO_ATTR_PRIOMAP, /* struct tc_mqprio_qopt */
@@ -1170,6 +1172,8 @@ enum {
TCA_TAPRIO_ATTR_ADMIN_SCHED, /* The admin sched, only used in dump */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME, /* s64 */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME_EXTENSION, /* s64 */
+ TCA_TAPRIO_ATTR_FLAGS, /* u32 */
+ TCA_TAPRIO_ATTR_TXTIME_DELAY, /* s32 */
__TCA_TAPRIO_ATTR_MAX,
};
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index a41d7d4434ee..ab6080013666 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -21,12 +21,16 @@
#include <net/pkt_sched.h>
#include <net/pkt_cls.h>
#include <net/sch_generic.h>
+#include <net/sock.h>
static LIST_HEAD(taprio_list);
static DEFINE_SPINLOCK(taprio_list_lock);
#define TAPRIO_ALL_GATES_OPEN -1
+#define FLAGS_VALID(flags) (!((flags) & ~TCA_TAPRIO_ATTR_FLAG_TXTIME_ASSIST))
+#define TXTIME_ASSIST_IS_ENABLED(flags) ((flags) & TCA_TAPRIO_ATTR_FLAG_TXTIME_ASSIST)
+
struct sched_entry {
struct list_head list;
@@ -35,6 +39,7 @@ struct sched_entry {
* packet leaves after this time.
*/
ktime_t close_time;
+ ktime_t next_txtime;
atomic_t budget;
int index;
u32 gate_mask;
@@ -55,6 +60,7 @@ struct sched_gate_list {
struct taprio_sched {
struct Qdisc **qdiscs;
struct Qdisc *root;
+ u32 flags;
int clockid;
atomic64_t picos_per_byte; /* Using picoseconds because for 10Gbps+
* speeds it's sub-nanoseconds per byte
@@ -68,6 +74,7 @@ struct taprio_sched {
ktime_t (*get_time)(void);
struct hrtimer advance_timer;
struct list_head taprio_list;
+ int txtime_delay;
};
static ktime_t sched_base_time(const struct sched_gate_list *sched)
@@ -108,6 +115,237 @@ static void switch_schedules(struct taprio_sched *q,
*admin = NULL;
}
+/* Get how much time has been already elapsed in the current cycle. */
+static inline s32 get_cycle_time_elapsed(struct sched_gate_list *sched, ktime_t time)
+{
+ ktime_t time_since_sched_start;
+ s32 time_elapsed;
+
+ time_since_sched_start = ktime_sub(time, sched->base_time);
+ div_s64_rem(time_since_sched_start, sched->cycle_time, &time_elapsed);
+
+ return time_elapsed;
+}
+
+static ktime_t get_interval_end_time(struct sched_gate_list *sched,
+ struct sched_gate_list *admin,
+ struct sched_entry *entry,
+ ktime_t intv_start)
+{
+ s32 cycle_elapsed = get_cycle_time_elapsed(sched, intv_start);
+ ktime_t intv_end, cycle_ext_end, cycle_end;
+
+ cycle_end = ktime_add_ns(intv_start, sched->cycle_time - cycle_elapsed);
+ intv_end = ktime_add_ns(intv_start, entry->interval);
+ cycle_ext_end = ktime_add(cycle_end, sched->cycle_time_extension);
+
+ if (ktime_before(intv_end, cycle_end))
+ return intv_end;
+ else if (admin && admin != sched &&
+ ktime_after(admin->base_time, cycle_end) &&
+ ktime_before(admin->base_time, cycle_ext_end))
+ return admin->base_time;
+ else
+ return cycle_end;
+}
+
+static inline int length_to_duration(struct taprio_sched *q, int len)
+{
+ return (len * atomic64_read(&q->picos_per_byte)) / 1000;
+}
+
+/* Returns the entry corresponding to next available interval. If
+ * validate_interval is set, it only validates whether the timestamp occurs
+ * when the gate corresponding to the skb's traffic class is open.
+ */
+static struct sched_entry *find_entry_to_transmit(struct sk_buff *skb,
+ struct Qdisc *sch,
+ struct sched_gate_list *sched,
+ struct sched_gate_list *admin,
+ ktime_t time,
+ ktime_t *interval_start,
+ ktime_t *interval_end,
+ bool validate_interval)
+{
+ ktime_t curr_intv_start, curr_intv_end, cycle_end, packet_transmit_time;
+ ktime_t earliest_txtime = KTIME_MAX, txtime, cycle, transmit_end_time;
+ struct sched_entry *entry = NULL, *entry_found = NULL;
+ struct taprio_sched *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
+ bool entry_available = false;
+ s32 cycle_elapsed;
+ int tc, n;
+
+ tc = netdev_get_prio_tc_map(dev, skb->priority);
+ packet_transmit_time = length_to_duration(q, qdisc_pkt_len(skb));
+
+ *interval_start = 0;
+ *interval_end = 0;
+
+ if (!sched)
+ return NULL;
+
+ cycle = sched->cycle_time;
+ cycle_elapsed = get_cycle_time_elapsed(sched, time);
+ curr_intv_end = ktime_sub_ns(time, cycle_elapsed);
+ cycle_end = ktime_add_ns(curr_intv_end, cycle);
+
+ list_for_each_entry(entry, &sched->entries, list) {
+ curr_intv_start = curr_intv_end;
+ curr_intv_end = get_interval_end_time(sched, admin, entry,
+ curr_intv_start);
+
+ if (ktime_after(curr_intv_start, cycle_end))
+ break;
+
+ if (!(entry->gate_mask & BIT(tc)) ||
+ packet_transmit_time > entry->interval)
+ continue;
+
+ txtime = entry->next_txtime;
+
+ if (ktime_before(txtime, time) || validate_interval) {
+ transmit_end_time = ktime_add_ns(time, packet_transmit_time);
+ if ((ktime_before(curr_intv_start, time) &&
+ ktime_before(transmit_end_time, curr_intv_end)) ||
+ (ktime_after(curr_intv_start, time) && !validate_interval)) {
+ entry_found = entry;
+ *interval_start = curr_intv_start;
+ *interval_end = curr_intv_end;
+ break;
+ } else if (!entry_available && !validate_interval) {
+ /* Here, we are just trying to find out the
+ * first available interval in the next cycle.
+ */
+ entry_available = 1;
+ entry_found = entry;
+ *interval_start = ktime_add_ns(curr_intv_start, cycle);
+ *interval_end = ktime_add_ns(curr_intv_end, cycle);
+ }
+ } else if (ktime_before(txtime, earliest_txtime) &&
+ !entry_available) {
+ earliest_txtime = txtime;
+ entry_found = entry;
+ n = div_s64(ktime_sub(txtime, curr_intv_start), cycle);
+ *interval_start = ktime_add(curr_intv_start, n * cycle);
+ *interval_end = ktime_add(curr_intv_end, n * cycle);
+ }
+ }
+
+ return entry_found;
+}
+
+static bool is_valid_interval(struct sk_buff *skb, struct Qdisc *sch)
+{
+ struct taprio_sched *q = qdisc_priv(sch);
+ struct sched_gate_list *sched, *admin;
+ ktime_t interval_start, interval_end;
+ struct sched_entry *entry;
+
+ rcu_read_lock();
+ sched = rcu_dereference(q->oper_sched);
+ admin = rcu_dereference(q->admin_sched);
+
+ entry = find_entry_to_transmit(skb, sch, sched, admin, skb->tstamp,
+ &interval_start, &interval_end, true);
+ rcu_read_unlock();
+
+ return entry;
+}
+
+static inline ktime_t get_cycle_start(struct sched_gate_list *sched,
+ ktime_t time)
+{
+ ktime_t cycle_elapsed;
+
+ cycle_elapsed = get_cycle_time_elapsed(sched, time);
+
+ return ktime_sub(time, cycle_elapsed);
+}
+
+/* There are a few scenarios where we will have to modify the txtime from
+ * what is read from next_txtime in sched_entry. They are:
+ * 1. If txtime is in the past,
+ * a. The gate for the traffic class is currently open and packet can be
+ * transmitted before it closes, schedule the packet right away.
+ * b. If the gate corresponding to the traffic class is going to open later
+ * in the cycle, set the txtime of packet to the interval start.
+ * 2. If txtime is in the future, there are packets corresponding to the
+ * current traffic class waiting to be transmitted. So, the following
+ * possibilities exist:
+ * a. We can transmit the packet before the window containing the txtime
+ * closes.
+ * b. The window might close before the transmission can be completed
+ * successfully. So, schedule the packet in the next open window.
+ */
+static long get_packet_txtime(struct sk_buff *skb, struct Qdisc *sch)
+{
+ ktime_t transmit_end_time, interval_end, interval_start;
+ struct taprio_sched *q = qdisc_priv(sch);
+ struct sched_gate_list *sched, *admin;
+ ktime_t minimum_time, now, txtime;
+ int len, packet_transmit_time;
+ struct sched_entry *entry;
+ bool sched_changed;
+
+ now = q->get_time();
+ minimum_time = ktime_add_ns(now, q->txtime_delay);
+
+ rcu_read_lock();
+ admin = rcu_dereference(q->admin_sched);
+ sched = rcu_dereference(q->oper_sched);
+ if (admin && ktime_after(minimum_time, admin->base_time))
+ switch_schedules(q, &admin, &sched);
+
+ /* Until the schedule starts, all the queues are open */
+ if (!sched || ktime_before(minimum_time, sched->base_time)) {
+ txtime = minimum_time;
+ goto done;
+ }
+
+ len = qdisc_pkt_len(skb);
+ packet_transmit_time = length_to_duration(q, len);
+
+ do {
+ sched_changed = 0;
+
+ entry = find_entry_to_transmit(skb, sch, sched, admin,
+ minimum_time,
+ &interval_start, &interval_end,
+ false);
+ if (!entry) {
+ txtime = 0;
+ goto done;
+ }
+
+ txtime = entry->next_txtime;
+ txtime = max_t(ktime_t, txtime, minimum_time);
+ txtime = max_t(ktime_t, txtime, interval_start);
+
+ if (admin && admin != sched &&
+ ktime_after(txtime, admin->base_time)) {
+ sched = admin;
+ sched_changed = 1;
+ continue;
+ }
+
+ transmit_end_time = ktime_add(txtime, packet_transmit_time);
+ minimum_time = transmit_end_time;
+
+ /* Update the txtime of current entry to the next time it's
+ * interval starts.
+ */
+ if (ktime_after(transmit_end_time, interval_end))
+ entry->next_txtime = ktime_add(interval_start, sched->cycle_time);
+ } while (sched_changed || ktime_after(transmit_end_time, interval_end));
+
+ entry->next_txtime = transmit_end_time;
+
+done:
+ rcu_read_unlock();
+ return txtime;
+}
+
static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff **to_free)
{
@@ -121,6 +359,15 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
if (unlikely(!child))
return qdisc_drop(skb, sch, to_free);
+ if (skb->sk && sock_flag(skb->sk, SOCK_TXTIME)) {
+ if (!is_valid_interval(skb, sch))
+ return qdisc_drop(skb, sch, to_free);
+ } else if (TXTIME_ASSIST_IS_ENABLED(q->flags)) {
+ skb->tstamp = get_packet_txtime(skb, sch);
+ if (!skb->tstamp)
+ return qdisc_drop(skb, sch, to_free);
+ }
+
qdisc_qstats_backlog_inc(sch, skb);
sch->q.qlen++;
@@ -156,6 +403,9 @@ static struct sk_buff *taprio_peek(struct Qdisc *sch)
if (!skb)
continue;
+ if (TXTIME_ASSIST_IS_ENABLED(q->flags))
+ return skb;
+
prio = skb->priority;
tc = netdev_get_prio_tc_map(dev, prio);
@@ -168,11 +418,6 @@ static struct sk_buff *taprio_peek(struct Qdisc *sch)
return NULL;
}
-static inline int length_to_duration(struct taprio_sched *q, int len)
-{
- return div_u64(len * atomic64_read(&q->picos_per_byte), 1000);
-}
-
static void taprio_set_budget(struct taprio_sched *q, struct sched_entry *entry)
{
atomic_set(&entry->budget,
@@ -216,6 +461,13 @@ static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
if (unlikely(!child))
continue;
+ if (TXTIME_ASSIST_IS_ENABLED(q->flags)) {
+ skb = child->ops->dequeue(child);
+ if (!skb)
+ continue;
+ goto skb_found;
+ }
+
skb = child->ops->peek(child);
if (!skb)
continue;
@@ -246,6 +498,7 @@ static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
if (unlikely(!skb))
goto done;
+skb_found:
qdisc_bstats_update(sch, skb);
qdisc_qstats_backlog_dec(sch, skb);
sch->q.qlen--;
@@ -522,7 +775,8 @@ static int parse_taprio_schedule(struct nlattr **tb,
static int taprio_parse_mqprio_opt(struct net_device *dev,
struct tc_mqprio_qopt *qopt,
- struct netlink_ext_ack *extack)
+ struct netlink_ext_ack *extack,
+ u32 taprio_flags)
{
int i, j;
@@ -570,6 +824,9 @@ static int taprio_parse_mqprio_opt(struct net_device *dev,
return -EINVAL;
}
+ if (TXTIME_ASSIST_IS_ENABLED(taprio_flags))
+ continue;
+
/* Verify that the offset and counts do not overlap */
for (j = i + 1; j < qopt->num_tc; j++) {
if (last > qopt->offset[j]) {
@@ -700,6 +957,18 @@ static int taprio_dev_notifier(struct notifier_block *nb, unsigned long event,
return NOTIFY_DONE;
}
+static void setup_txtime(struct taprio_sched *q,
+ struct sched_gate_list *sched, ktime_t base)
+{
+ struct sched_entry *entry;
+ u32 interval = 0;
+
+ list_for_each_entry(entry, &sched->entries, list) {
+ entry->next_txtime = ktime_add_ns(base, interval);
+ interval += entry->interval;
+ }
+}
+
static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
@@ -708,6 +977,7 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
struct taprio_sched *q = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
struct tc_mqprio_qopt *mqprio = NULL;
+ u32 taprio_flags = 0;
int i, err, clockid;
unsigned long flags;
ktime_t start;
@@ -720,7 +990,21 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
if (tb[TCA_TAPRIO_ATTR_PRIOMAP])
mqprio = nla_data(tb[TCA_TAPRIO_ATTR_PRIOMAP]);
- err = taprio_parse_mqprio_opt(dev, mqprio, extack);
+ if (tb[TCA_TAPRIO_ATTR_FLAGS]) {
+ taprio_flags = nla_get_u32(tb[TCA_TAPRIO_ATTR_FLAGS]);
+
+ if (q->flags != 0 && q->flags != taprio_flags) {
+ NL_SET_ERR_MSG_MOD(extack, "Changing 'flags' of a running schedule is not supported");
+ return -EOPNOTSUPP;
+ } else if (!FLAGS_VALID(taprio_flags)) {
+ NL_SET_ERR_MSG_MOD(extack, "Specified 'flags' are not valid");
+ return -EINVAL;
+ }
+
+ q->flags = taprio_flags;
+ }
+
+ err = taprio_parse_mqprio_opt(dev, mqprio, extack, taprio_flags);
if (err < 0)
return err;
@@ -779,7 +1063,18 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
/* Protects against enqueue()/dequeue() */
spin_lock_bh(qdisc_lock(sch));
- if (!hrtimer_active(&q->advance_timer)) {
+ if (tb[TCA_TAPRIO_ATTR_TXTIME_DELAY]) {
+ if (!TXTIME_ASSIST_IS_ENABLED(q->flags)) {
+ NL_SET_ERR_MSG_MOD(extack, "txtime-delay can only be set when txtime-assist mode is enabled");
+ err = -EINVAL;
+ goto unlock;
+ }
+
+ q->txtime_delay = nla_get_s32(tb[TCA_TAPRIO_ATTR_TXTIME_DELAY]);
+ }
+
+ if (!TXTIME_ASSIST_IS_ENABLED(taprio_flags) &&
+ !hrtimer_active(&q->advance_timer)) {
hrtimer_init(&q->advance_timer, q->clockid, HRTIMER_MODE_ABS);
q->advance_timer.function = advance_sched;
}
@@ -822,20 +1117,35 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
goto unlock;
}
- setup_first_close_time(q, new_admin, start);
+ if (TXTIME_ASSIST_IS_ENABLED(taprio_flags)) {
+ setup_txtime(q, new_admin, start);
- /* Protects against advance_sched() */
- spin_lock_irqsave(&q->current_entry_lock, flags);
+ if (!oper) {
+ rcu_assign_pointer(q->oper_sched, new_admin);
+ err = 0;
+ new_admin = NULL;
+ goto unlock;
+ }
+
+ rcu_assign_pointer(q->admin_sched, new_admin);
+ if (admin)
+ call_rcu(&admin->rcu, taprio_free_sched_cb);
+ } else {
+ setup_first_close_time(q, new_admin, start);
- taprio_start_sched(sch, start, new_admin);
+ /* Protects against advance_sched() */
+ spin_lock_irqsave(&q->current_entry_lock, flags);
- rcu_assign_pointer(q->admin_sched, new_admin);
- if (admin)
- call_rcu(&admin->rcu, taprio_free_sched_cb);
- new_admin = NULL;
+ taprio_start_sched(sch, start, new_admin);
- spin_unlock_irqrestore(&q->current_entry_lock, flags);
+ rcu_assign_pointer(q->admin_sched, new_admin);
+ if (admin)
+ call_rcu(&admin->rcu, taprio_free_sched_cb);
+ spin_unlock_irqrestore(&q->current_entry_lock, flags);
+ }
+
+ new_admin = NULL;
err = 0;
unlock:
@@ -1073,6 +1383,13 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
if (nla_put_s32(skb, TCA_TAPRIO_ATTR_SCHED_CLOCKID, q->clockid))
goto options_error;
+ if (q->flags && nla_put_u32(skb, TCA_TAPRIO_ATTR_FLAGS, q->flags))
+ goto options_error;
+
+ if (q->txtime_delay &&
+ nla_put_s32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
+ goto options_error;
+
if (oper && dump_schedule(skb, oper))
goto options_error;
--
2.7.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox