* Re: [PATCH net-next] qed*: Utilize FW 8.33.11.0
From: Jason Gunthorpe @ 2018-03-26 21:18 UTC (permalink / raw)
To: Michal Kalderon
Cc: davem, netdev, linux-rdma, linux-scsi, Tomer Tayar,
Manish Rangankar, Ariel Elior
In-Reply-To: <1522083732-26560-1-git-send-email-Michal.Kalderon@cavium.com>
On Mon, Mar 26, 2018 at 08:02:12PM +0300, Michal Kalderon wrote:
> This FW contains several fixes and features
>
> RDMA Features
> - SRQ support
> - XRC support
> - Memory window support
> - RDMA low latency queue support
> - RDMA bonding support
>
> RDMA bug fixes
> - RDMA remote invalidate during retransmit fix
> - iWARP MPA connect interop issue with RTR fix
> - iWARP Legacy DPM support
> - Fix MPA reject flow
> - iWARP error handling
> - RQ WQE validation checks
>
> MISC
> - Fix some HSI types endianity
> - New Restriction: vlan insertion in core_tx_bd_data can't be set
> for LB packets
>
> ETH
> - HW QoS offload support
> - Fix vlan, dcb and sriov flow of VF sending a packet with
> inband VLAN tag instead of default VLAN
> - Allow GRE version 1 offloads in RX flow
> - Allow VXLAN steering
>
> iSCSI / FcoE
> - Fix bd availability checking flow
> - Support 256th sge proerly in iscsi/fcoe retransmit
> - Performance improvement
> - Fix handle iSCSI command arrival with AHS and with immediate
> - Fix ipv6 traffic class configuration
>
> DEBUG
> - Update debug utilities
>
> Advance version number
>
> Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
> Signed-off-by: Tomer Tayar <Tomer.Tayar@cavium.com>
> Signed-off-by: Manish Rangankar <Manish.Rangankar@cavium.com>
> Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
> drivers/infiniband/hw/qedr/main.c | 1 +
> drivers/infiniband/hw/qedr/qedr.h | 2 +
> drivers/infiniband/hw/qedr/qedr_hsi_rdma.h | 4 +-
> drivers/infiniband/hw/qedr/verbs.c | 4 +-
> drivers/net/ethernet/qlogic/qed/qed.h | 2 +-
> drivers/net/ethernet/qlogic/qed/qed_debug.c | 415 +++--
> drivers/net/ethernet/qlogic/qed/qed_dev.c | 4 +-
> drivers/net/ethernet/qlogic/qed/qed_hsi.h | 1892 ++++++++++----------
> .../net/ethernet/qlogic/qed/qed_init_fw_funcs.c | 103 +-
> drivers/net/ethernet/qlogic/qed/qed_iwarp.c | 7 -
> drivers/net/ethernet/qlogic/qed/qed_l2.c | 2 +-
> drivers/net/ethernet/qlogic/qed/qed_ll2.c | 13 -
> drivers/net/ethernet/qlogic/qede/qede.h | 2 +-
> drivers/scsi/qedf/qedf_version.h | 4 +-
> drivers/scsi/qedi/qedi_version.h | 4 +-
> include/linux/qed/common_hsi.h | 2 +-
> include/linux/qed/eth_common.h | 2 +-
> include/linux/qed/iscsi_common.h | 4 +-
> include/linux/qed/rdma_common.h | 2 +
> include/linux/qed/roce_common.h | 3 +
> 20 files changed, 1319 insertions(+), 1153 deletions(-)
>
> diff --git a/drivers/infiniband/hw/qedr/main.c b/drivers/infiniband/hw/qedr/main.c
> index db4bf97..7dbbe6d 100644
> +++ b/drivers/infiniband/hw/qedr/main.c
> @@ -51,6 +51,7 @@
> MODULE_DESCRIPTION("QLogic 40G/100G ROCE Driver");
> MODULE_AUTHOR("QLogic Corporation");
> MODULE_LICENSE("Dual BSD/GPL");
> +MODULE_VERSION(QEDR_MODULE_VERSION);
>
> #define QEDR_WQ_MULTIPLIER_DFT (3)
>
> diff --git a/drivers/infiniband/hw/qedr/qedr.h b/drivers/infiniband/hw/qedr/qedr.h
> index 86d4511..ab0d411 100644
> +++ b/drivers/infiniband/hw/qedr/qedr.h
> @@ -43,6 +43,8 @@
> #include "qedr_hsi_rdma.h"
>
> #define QEDR_NODE_DESC "QLogic 579xx RoCE HCA"
> +#define QEDR_MODULE_VERSION "8.33.11.20"
> +
I thought we had a general prohibition against versions like this in
mainline drivers? And what does this hunk have to do with supporting
new firmware?
Jason
^ permalink raw reply
* [PATCH v3 iproute2-next 0/8] cm_id, cq, mr, and pd resource tracking
From: Steve Wise @ 2018-03-26 20:59 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Hello,
This series enhances the iproute2 rdma tool to include dumping of
connection manager id (cm_id), completion queue (cq), memory region (mr),
and protection domain (pd) rdma resources. It is the user-space part
of the kernel resource tracking series merged into rdma-next for 4.17 [1].
Changes since v2:
- pull in rdma-core include/rdma/rdma_cma.h
- update the new local rdma/include/uapi/rdma/rdma_netlink.h
- 80 column reformat
- add reviewed-by tags
Changes since v1/RFC:
- removed RFC tag
- initialize rd properly to avoid passing a garbage port number
- revert accidental change to qp_valid_filters
- removed cm_id dev/network/transport types
- cm_id ip addrs now passed up as __kernel_sockaddr_storage
- cm_id ip address ports printed as "address:port" strings
- only parse/display memory keys and iova if available
- filter on "users" for cqs and pds
- fixed memory leaks
- removed PD_FLAGS attribute
- filter on "mrlen" for mrs
- filter on "poll-ctx" for cqs
- don't require addrs or qp_type for parsing cm_ids
- only filter optional attrs if they are present
- remove PGSIZE MR attr to match kernel
[1] https://www.spinics.net/lists/linux-rdma/msg61720.html
---
Steve Wise (8):
rdma: include rdma-core <rdma/rdma_cma.h>
rdma: Makefile change to find <rdma/rdma_cma.h>
rdma: update rdma_netlink.h
rdma: initialize the rd struct
rdma: Add CM_ID resource tracking information
rdma: Add CQ resource tracking information
rdma: Add MR resource tracking information
rdma: Add PD resource tracking information
include/json_writer.h | 2 +
lib/json_writer.c | 11 +
rdma/Makefile | 2 +-
rdma/include/rdma/rdma_cma.h | 728 ++++++++++++++++++++++++++++++++++
rdma/include/uapi/rdma/rdma_netlink.h | 38 ++
rdma/rdma.c | 2 +-
rdma/rdma.h | 2 +
rdma/res.c | 623 ++++++++++++++++++++++++++++-
rdma/utils.c | 16 +
9 files changed, 1421 insertions(+), 3 deletions(-)
create mode 100644 rdma/include/rdma/rdma_cma.h
--
1.8.3.1
^ permalink raw reply
* [PATCH v3 iproute2-next 1/8] rdma: include rdma-core <rdma/rdma_cma.h>
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
This avoids requiring rdma-core be installed on systems.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/include/rdma/rdma_cma.h | 728 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 728 insertions(+)
create mode 100644 rdma/include/rdma/rdma_cma.h
diff --git a/rdma/include/rdma/rdma_cma.h b/rdma/include/rdma/rdma_cma.h
new file mode 100644
index 0000000..456aa4e
--- /dev/null
+++ b/rdma/include/rdma/rdma_cma.h
@@ -0,0 +1,728 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005-2014 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#if !defined(_RDMA_CMA_H)
+#define _RDMA_CMA_H
+
+#include <netinet/in.h>
+#include <sys/socket.h>
+#include <infiniband/verbs.h>
+#include <infiniband/sa.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * Upon receiving a device removal event, users must destroy the associated
+ * RDMA identifier and release all resources allocated with the device.
+ */
+enum rdma_cm_event_type {
+ RDMA_CM_EVENT_ADDR_RESOLVED,
+ RDMA_CM_EVENT_ADDR_ERROR,
+ RDMA_CM_EVENT_ROUTE_RESOLVED,
+ RDMA_CM_EVENT_ROUTE_ERROR,
+ RDMA_CM_EVENT_CONNECT_REQUEST,
+ RDMA_CM_EVENT_CONNECT_RESPONSE,
+ RDMA_CM_EVENT_CONNECT_ERROR,
+ RDMA_CM_EVENT_UNREACHABLE,
+ RDMA_CM_EVENT_REJECTED,
+ RDMA_CM_EVENT_ESTABLISHED,
+ RDMA_CM_EVENT_DISCONNECTED,
+ RDMA_CM_EVENT_DEVICE_REMOVAL,
+ RDMA_CM_EVENT_MULTICAST_JOIN,
+ RDMA_CM_EVENT_MULTICAST_ERROR,
+ RDMA_CM_EVENT_ADDR_CHANGE,
+ RDMA_CM_EVENT_TIMEWAIT_EXIT
+};
+
+enum rdma_port_space {
+ RDMA_PS_IPOIB = 0x0002,
+ RDMA_PS_TCP = 0x0106,
+ RDMA_PS_UDP = 0x0111,
+ RDMA_PS_IB = 0x013F,
+};
+
+#define RDMA_IB_IP_PS_MASK 0xFFFFFFFFFFFF0000ULL
+#define RDMA_IB_IP_PORT_MASK 0x000000000000FFFFULL
+#define RDMA_IB_IP_PS_TCP 0x0000000001060000ULL
+#define RDMA_IB_IP_PS_UDP 0x0000000001110000ULL
+#define RDMA_IB_PS_IB 0x00000000013F0000ULL
+
+/*
+ * Global qkey value for UDP QPs and multicast groups created via the
+ * RDMA CM.
+ */
+#define RDMA_UDP_QKEY 0x01234567
+
+struct rdma_ib_addr {
+ union ibv_gid sgid;
+ union ibv_gid dgid;
+ __be16 pkey;
+};
+
+struct rdma_addr {
+ union {
+ struct sockaddr src_addr;
+ struct sockaddr_in src_sin;
+ struct sockaddr_in6 src_sin6;
+ struct sockaddr_storage src_storage;
+ };
+ union {
+ struct sockaddr dst_addr;
+ struct sockaddr_in dst_sin;
+ struct sockaddr_in6 dst_sin6;
+ struct sockaddr_storage dst_storage;
+ };
+ union {
+ struct rdma_ib_addr ibaddr;
+ } addr;
+};
+
+struct rdma_route {
+ struct rdma_addr addr;
+ struct ibv_sa_path_rec *path_rec;
+ int num_paths;
+};
+
+struct rdma_event_channel {
+ int fd;
+};
+
+struct rdma_cm_id {
+ struct ibv_context *verbs;
+ struct rdma_event_channel *channel;
+ void *context;
+ struct ibv_qp *qp;
+ struct rdma_route route;
+ enum rdma_port_space ps;
+ uint8_t port_num;
+ struct rdma_cm_event *event;
+ struct ibv_comp_channel *send_cq_channel;
+ struct ibv_cq *send_cq;
+ struct ibv_comp_channel *recv_cq_channel;
+ struct ibv_cq *recv_cq;
+ struct ibv_srq *srq;
+ struct ibv_pd *pd;
+ enum ibv_qp_type qp_type;
+};
+
+enum {
+ RDMA_MAX_RESP_RES = 0xFF,
+ RDMA_MAX_INIT_DEPTH = 0xFF
+};
+
+struct rdma_conn_param {
+ const void *private_data;
+ uint8_t private_data_len;
+ uint8_t responder_resources;
+ uint8_t initiator_depth;
+ uint8_t flow_control;
+ uint8_t retry_count; /* ignored when accepting */
+ uint8_t rnr_retry_count;
+ /* Fields below ignored if a QP is created on the rdma_cm_id. */
+ uint8_t srq;
+ uint32_t qp_num;
+};
+
+struct rdma_ud_param {
+ const void *private_data;
+ uint8_t private_data_len;
+ struct ibv_ah_attr ah_attr;
+ uint32_t qp_num;
+ uint32_t qkey;
+};
+
+struct rdma_cm_event {
+ struct rdma_cm_id *id;
+ struct rdma_cm_id *listen_id;
+ enum rdma_cm_event_type event;
+ int status;
+ union {
+ struct rdma_conn_param conn;
+ struct rdma_ud_param ud;
+ } param;
+};
+
+#define RAI_PASSIVE 0x00000001
+#define RAI_NUMERICHOST 0x00000002
+#define RAI_NOROUTE 0x00000004
+#define RAI_FAMILY 0x00000008
+
+struct rdma_addrinfo {
+ int ai_flags;
+ int ai_family;
+ int ai_qp_type;
+ int ai_port_space;
+ socklen_t ai_src_len;
+ socklen_t ai_dst_len;
+ struct sockaddr *ai_src_addr;
+ struct sockaddr *ai_dst_addr;
+ char *ai_src_canonname;
+ char *ai_dst_canonname;
+ size_t ai_route_len;
+ void *ai_route;
+ size_t ai_connect_len;
+ void *ai_connect;
+ struct rdma_addrinfo *ai_next;
+};
+
+/* Multicast join compatibility mask attributes */
+enum rdma_cm_join_mc_attr_mask {
+ RDMA_CM_JOIN_MC_ATTR_ADDRESS = 1 << 0,
+ RDMA_CM_JOIN_MC_ATTR_JOIN_FLAGS = 1 << 1,
+ RDMA_CM_JOIN_MC_ATTR_RESERVED = 1 << 2,
+};
+
+/* Multicast join flags */
+enum rdma_cm_mc_join_flags {
+ RDMA_MC_JOIN_FLAG_FULLMEMBER,
+ RDMA_MC_JOIN_FLAG_SENDONLY_FULLMEMBER,
+ RDMA_MC_JOIN_FLAG_RESERVED,
+};
+
+struct rdma_cm_join_mc_attr_ex {
+ /* Bitwise OR between "rdma_cm_join_mc_attr_mask" enum */
+ uint32_t comp_mask;
+ /* Use a flag from "rdma_cm_mc_join_flags" enum */
+ uint32_t join_flags;
+ /* Multicast address identifying the group to join */
+ struct sockaddr *addr;
+};
+
+/**
+ * rdma_create_event_channel - Open a channel used to report communication events.
+ * Description:
+ * Asynchronous events are reported to users through event channels. Each
+ * event channel maps to a file descriptor.
+ * Notes:
+ * All created event channels must be destroyed by calling
+ * rdma_destroy_event_channel. Users should call rdma_get_cm_event to
+ * retrieve events on an event channel.
+ * See also:
+ * rdma_get_cm_event, rdma_destroy_event_channel
+ */
+struct rdma_event_channel *rdma_create_event_channel(void);
+
+/**
+ * rdma_destroy_event_channel - Close an event communication channel.
+ * @channel: The communication channel to destroy.
+ * Description:
+ * Release all resources associated with an event channel and closes the
+ * associated file descriptor.
+ * Notes:
+ * All rdma_cm_id's associated with the event channel must be destroyed,
+ * and all returned events must be acked before calling this function.
+ * See also:
+ * rdma_create_event_channel, rdma_get_cm_event, rdma_ack_cm_event
+ */
+void rdma_destroy_event_channel(struct rdma_event_channel *channel);
+
+/**
+ * rdma_create_id - Allocate a communication identifier.
+ * @channel: The communication channel that events associated with the
+ * allocated rdma_cm_id will be reported on.
+ * @id: A reference where the allocated communication identifier will be
+ * returned.
+ * @context: User specified context associated with the rdma_cm_id.
+ * @ps: RDMA port space.
+ * Description:
+ * Creates an identifier that is used to track communication information.
+ * Notes:
+ * Rdma_cm_id's are conceptually equivalent to a socket for RDMA
+ * communication. The difference is that RDMA communication requires
+ * explicitly binding to a specified RDMA device before communication
+ * can occur, and most operations are asynchronous in nature. Communication
+ * events on an rdma_cm_id are reported through the associated event
+ * channel. Users must release the rdma_cm_id by calling rdma_destroy_id.
+ * See also:
+ * rdma_create_event_channel, rdma_destroy_id, rdma_get_devices,
+ * rdma_bind_addr, rdma_resolve_addr, rdma_connect, rdma_listen,
+ */
+int rdma_create_id(struct rdma_event_channel *channel,
+ struct rdma_cm_id **id, void *context,
+ enum rdma_port_space ps);
+
+/**
+ * rdma_create_ep - Allocate a communication identifier and qp.
+ * @id: A reference where the allocated communication identifier will be
+ * returned.
+ * @res: Result from rdma_getaddrinfo, which specifies the source and
+ * destination addresses, plus optional routing and connection information.
+ * @pd: Optional protection domain. This parameter is ignored if qp_init_attr
+ * is NULL.
+ * @qp_init_attr: Optional attributes for a QP created on the rdma_cm_id.
+ * Description:
+ * Create an identifier and option QP used for communication.
+ * Notes:
+ * If qp_init_attr is provided, then a queue pair will be allocated and
+ * associated with the rdma_cm_id. If a pd is provided, the QP will be
+ * created on that PD. Otherwise, the QP will be allocated on a default
+ * PD.
+ * The rdma_cm_id will be set to use synchronous operations (connect,
+ * listen, and get_request). To convert to asynchronous operation, the
+ * rdma_cm_id should be migrated to a user allocated event channel.
+ * See also:
+ * rdma_create_id, rdma_create_qp, rdma_migrate_id, rdma_connect,
+ * rdma_listen
+ */
+int rdma_create_ep(struct rdma_cm_id **id, struct rdma_addrinfo *res,
+ struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr);
+
+/**
+ * rdma_destroy_ep - Deallocates a communication identifier and qp.
+ * @id: The communication identifier to destroy.
+ * Description:
+ * Destroys the specified rdma_cm_id and any associated QP created
+ * on that id.
+ * See also:
+ * rdma_create_ep
+ */
+void rdma_destroy_ep(struct rdma_cm_id *id);
+
+/**
+ * rdma_destroy_id - Release a communication identifier.
+ * @id: The communication identifier to destroy.
+ * Description:
+ * Destroys the specified rdma_cm_id and cancels any outstanding
+ * asynchronous operation.
+ * Notes:
+ * Users must free any associated QP with the rdma_cm_id before
+ * calling this routine and ack an related events.
+ * See also:
+ * rdma_create_id, rdma_destroy_qp, rdma_ack_cm_event
+ */
+int rdma_destroy_id(struct rdma_cm_id *id);
+
+/**
+ * rdma_bind_addr - Bind an RDMA identifier to a source address.
+ * @id: RDMA identifier.
+ * @addr: Local address information. Wildcard values are permitted.
+ * Description:
+ * Associates a source address with an rdma_cm_id. The address may be
+ * wildcarded. If binding to a specific local address, the rdma_cm_id
+ * will also be bound to a local RDMA device.
+ * Notes:
+ * Typically, this routine is called before calling rdma_listen to bind
+ * to a specific port number, but it may also be called on the active side
+ * of a connection before calling rdma_resolve_addr to bind to a specific
+ * address.
+ * See also:
+ * rdma_create_id, rdma_listen, rdma_resolve_addr, rdma_create_qp
+ */
+int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr);
+
+/**
+ * rdma_resolve_addr - Resolve destination and optional source addresses.
+ * @id: RDMA identifier.
+ * @src_addr: Source address information. This parameter may be NULL.
+ * @dst_addr: Destination address information.
+ * @timeout_ms: Time to wait for resolution to complete.
+ * Description:
+ * Resolve destination and optional source addresses from IP addresses
+ * to an RDMA address. If successful, the specified rdma_cm_id will
+ * be bound to a local device.
+ * Notes:
+ * This call is used to map a given destination IP address to a usable RDMA
+ * address. If a source address is given, the rdma_cm_id is bound to that
+ * address, the same as if rdma_bind_addr were called. If no source
+ * address is given, and the rdma_cm_id has not yet been bound to a device,
+ * then the rdma_cm_id will be bound to a source address based on the
+ * local routing tables. After this call, the rdma_cm_id will be bound to
+ * an RDMA device. This call is typically made from the active side of a
+ * connection before calling rdma_resolve_route and rdma_connect.
+ * See also:
+ * rdma_create_id, rdma_resolve_route, rdma_connect, rdma_create_qp,
+ * rdma_get_cm_event, rdma_bind_addr
+ */
+int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms);
+
+/**
+ * rdma_resolve_route - Resolve the route information needed to establish a connection.
+ * @id: RDMA identifier.
+ * @timeout_ms: Time to wait for resolution to complete.
+ * Description:
+ * Resolves an RDMA route to the destination address in order to establish
+ * a connection. The destination address must have already been resolved
+ * by calling rdma_resolve_addr.
+ * Notes:
+ * This is called on the client side of a connection after calling
+ * rdma_resolve_addr, but before calling rdma_connect.
+ * See also:
+ * rdma_resolve_addr, rdma_connect, rdma_get_cm_event
+ */
+int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms);
+
+/**
+ * rdma_create_qp - Allocate a QP.
+ * @id: RDMA identifier.
+ * @pd: Optional protection domain for the QP.
+ * @qp_init_attr: initial QP attributes.
+ * Description:
+ * Allocate a QP associated with the specified rdma_cm_id and transition it
+ * for sending and receiving.
+ * Notes:
+ * The rdma_cm_id must be bound to a local RDMA device before calling this
+ * function, and the protection domain must be for that same device.
+ * QPs allocated to an rdma_cm_id are automatically transitioned by the
+ * librdmacm through their states. After being allocated, the QP will be
+ * ready to handle posting of receives. If the QP is unconnected, it will
+ * be ready to post sends.
+ * If pd is NULL, then the QP will be allocated using a default protection
+ * domain associated with the underlying RDMA device.
+ * See also:
+ * rdma_bind_addr, rdma_resolve_addr, rdma_destroy_qp, ibv_create_qp,
+ * ibv_modify_qp
+ */
+int rdma_create_qp(struct rdma_cm_id *id, struct ibv_pd *pd,
+ struct ibv_qp_init_attr *qp_init_attr);
+int rdma_create_qp_ex(struct rdma_cm_id *id,
+ struct ibv_qp_init_attr_ex *qp_init_attr);
+
+/**
+ * rdma_destroy_qp - Deallocate a QP.
+ * @id: RDMA identifier.
+ * Description:
+ * Destroy a QP allocated on the rdma_cm_id.
+ * Notes:
+ * Users must destroy any QP associated with an rdma_cm_id before
+ * destroying the ID.
+ * See also:
+ * rdma_create_qp, rdma_destroy_id, ibv_destroy_qp
+ */
+void rdma_destroy_qp(struct rdma_cm_id *id);
+
+/**
+ * rdma_connect - Initiate an active connection request.
+ * @id: RDMA identifier.
+ * @conn_param: optional connection parameters.
+ * Description:
+ * For a connected rdma_cm_id, this call initiates a connection request
+ * to a remote destination. For an unconnected rdma_cm_id, it initiates
+ * a lookup of the remote QP providing the datagram service.
+ * Notes:
+ * Users must have resolved a route to the destination address
+ * by having called rdma_resolve_route before calling this routine.
+ * A user may override the default connection parameters and exchange
+ * private data as part of the connection by using the conn_param parameter.
+ * See also:
+ * rdma_resolve_route, rdma_disconnect, rdma_listen, rdma_get_cm_event
+ */
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_listen - Listen for incoming connection requests.
+ * @id: RDMA identifier.
+ * @backlog: backlog of incoming connection requests.
+ * Description:
+ * Initiates a listen for incoming connection requests or datagram service
+ * lookup. The listen will be restricted to the locally bound source
+ * address.
+ * Notes:
+ * Users must have bound the rdma_cm_id to a local address by calling
+ * rdma_bind_addr before calling this routine. If the rdma_cm_id is
+ * bound to a specific IP address, the listen will be restricted to that
+ * address and the associated RDMA device. If the rdma_cm_id is bound
+ * to an RDMA port number only, the listen will occur across all RDMA
+ * devices.
+ * See also:
+ * rdma_bind_addr, rdma_connect, rdma_accept, rdma_reject, rdma_get_cm_event
+ */
+int rdma_listen(struct rdma_cm_id *id, int backlog);
+
+/**
+ * rdma_get_request
+ */
+int rdma_get_request(struct rdma_cm_id *listen, struct rdma_cm_id **id);
+
+/**
+ * rdma_accept - Called to accept a connection request.
+ * @id: Connection identifier associated with the request.
+ * @conn_param: Optional information needed to establish the connection.
+ * Description:
+ * Called from the listening side to accept a connection or datagram
+ * service lookup request.
+ * Notes:
+ * Unlike the socket accept routine, rdma_accept is not called on a
+ * listening rdma_cm_id. Instead, after calling rdma_listen, the user
+ * waits for a connection request event to occur. Connection request
+ * events give the user a newly created rdma_cm_id, similar to a new
+ * socket, but the rdma_cm_id is bound to a specific RDMA device.
+ * rdma_accept is called on the new rdma_cm_id.
+ * A user may override the default connection parameters and exchange
+ * private data as part of the connection by using the conn_param parameter.
+ * See also:
+ * rdma_listen, rdma_reject, rdma_get_cm_event
+ */
+int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_reject - Called to reject a connection request.
+ * @id: Connection identifier associated with the request.
+ * @private_data: Optional private data to send with the reject message.
+ * @private_data_len: Size of the private_data to send, in bytes.
+ * Description:
+ * Called from the listening side to reject a connection or datagram
+ * service lookup request.
+ * Notes:
+ * After receiving a connection request event, a user may call rdma_reject
+ * to reject the request. If the underlying RDMA transport supports
+ * private data in the reject message, the specified data will be passed to
+ * the remote side.
+ * See also:
+ * rdma_listen, rdma_accept, rdma_get_cm_event
+ */
+int rdma_reject(struct rdma_cm_id *id, const void *private_data,
+ uint8_t private_data_len);
+
+/**
+ * rdma_notify - Notifies the librdmacm of an asynchronous event.
+ * @id: RDMA identifier.
+ * @event: Asynchronous event.
+ * Description:
+ * Used to notify the librdmacm of asynchronous events that have occurred
+ * on a QP associated with the rdma_cm_id.
+ * Notes:
+ * Asynchronous events that occur on a QP are reported through the user's
+ * device event handler. This routine is used to notify the librdmacm of
+ * communication events. In most cases, use of this routine is not
+ * necessary, however if connection establishment is done out of band
+ * (such as done through Infiniband), it's possible to receive data on a
+ * QP that is not yet considered connected. This routine forces the
+ * connection into an established state in this case in order to handle
+ * the rare situation where the connection never forms on its own.
+ * Events that should be reported to the CM are: IB_EVENT_COMM_EST.
+ * See also:
+ * rdma_connect, rdma_accept, rdma_listen
+ */
+int rdma_notify(struct rdma_cm_id *id, enum ibv_event_type event);
+
+/**
+ * rdma_disconnect - This function disconnects a connection.
+ * @id: RDMA identifier.
+ * Description:
+ * Disconnects a connection and transitions any associated QP to the
+ * error state.
+ * See also:
+ * rdma_connect, rdma_listen, rdma_accept
+ */
+int rdma_disconnect(struct rdma_cm_id *id);
+
+/**
+ * rdma_join_multicast - Joins a multicast group.
+ * @id: Communication identifier associated with the request.
+ * @addr: Multicast address identifying the group to join.
+ * @context: User-defined context associated with the join request.
+ * Description:
+ * Joins a multicast group and attaches an associated QP to the group.
+ * Notes:
+ * Before joining a multicast group, the rdma_cm_id must be bound to
+ * an RDMA device by calling rdma_bind_addr or rdma_resolve_addr. Use of
+ * rdma_resolve_addr requires the local routing tables to resolve the
+ * multicast address to an RDMA device. The user must call
+ * rdma_leave_multicast to leave the multicast group and release any
+ * multicast resources. The context is returned to the user through
+ * the private_data field in the rdma_cm_event.
+ * See also:
+ * rdma_leave_multicast, rdma_bind_addr, rdma_resolve_addr, rdma_create_qp
+ */
+int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
+ void *context);
+
+/**
+ * rdma_leave_multicast - Leaves a multicast group.
+ * @id: Communication identifier associated with the request.
+ * @addr: Multicast address identifying the group to leave.
+ * Description:
+ * Leaves a multicast group and detaches an associated QP from the group.
+ * Notes:
+ * Calling this function before a group has been fully joined results in
+ * canceling the join operation. Users should be aware that messages
+ * received from the multicast group may stilled be queued for
+ * completion processing immediately after leaving a multicast group.
+ * Destroying an rdma_cm_id will automatically leave all multicast groups.
+ * See also:
+ * rdma_join_multicast, rdma_destroy_qp
+ */
+int rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr);
+
+/**
+ * rdma_multicast_ex - Joins a multicast group with options.
+ * @id: Communication identifier associated with the request.
+ * @mc_join_attr: Extensive struct containing multicast join parameters.
+ * @context: User-defined context associated with the join request.
+ * Description:
+ * Joins a multicast group with options. Currently supporting MC join flags.
+ * The QP will be attached based on the given join flag.
+ * Join message will be sent according to the join flag.
+ * Notes:
+ * Before joining a multicast group, the rdma_cm_id must be bound to
+ * an RDMA device by calling rdma_bind_addr or rdma_resolve_addr. Use of
+ * rdma_resolve_addr requires the local routing tables to resolve the
+ * multicast address to an RDMA device. The user must call
+ * rdma_leave_multicast to leave the multicast group and release any
+ * multicast resources. The context is returned to the user through
+ * the private_data field in the rdma_cm_event.
+ * See also:
+ * rdma_leave_multicast, rdma_bind_addr, rdma_resolve_addr, rdma_create_qp
+ */
+int rdma_join_multicast_ex(struct rdma_cm_id *id,
+ struct rdma_cm_join_mc_attr_ex *mc_join_attr,
+ void *context);
+
+/**
+ * rdma_get_cm_event - Retrieves the next pending communication event.
+ * @channel: Event channel to check for events.
+ * @event: Allocated information about the next communication event.
+ * Description:
+ * Retrieves a communication event. If no events are pending, by default,
+ * the call will block until an event is received.
+ * Notes:
+ * The default synchronous behavior of this routine can be changed by
+ * modifying the file descriptor associated with the given channel. All
+ * events that are reported must be acknowledged by calling rdma_ack_cm_event.
+ * Destruction of an rdma_cm_id will block until related events have been
+ * acknowledged.
+ * See also:
+ * rdma_ack_cm_event, rdma_create_event_channel, rdma_event_str
+ */
+int rdma_get_cm_event(struct rdma_event_channel *channel,
+ struct rdma_cm_event **event);
+
+/**
+ * rdma_ack_cm_event - Free a communication event.
+ * @event: Event to be released.
+ * Description:
+ * All events which are allocated by rdma_get_cm_event must be released,
+ * there should be a one-to-one correspondence between successful gets
+ * and acks.
+ * See also:
+ * rdma_get_cm_event, rdma_destroy_id
+ */
+int rdma_ack_cm_event(struct rdma_cm_event *event);
+
+__be16 rdma_get_src_port(struct rdma_cm_id *id);
+__be16 rdma_get_dst_port(struct rdma_cm_id *id);
+
+static inline struct sockaddr *rdma_get_local_addr(struct rdma_cm_id *id)
+{
+ return &id->route.addr.src_addr;
+}
+
+static inline struct sockaddr *rdma_get_peer_addr(struct rdma_cm_id *id)
+{
+ return &id->route.addr.dst_addr;
+}
+
+/**
+ * rdma_get_devices - Get list of RDMA devices currently available.
+ * @num_devices: If non-NULL, set to the number of devices returned.
+ * Description:
+ * Return a NULL-terminated array of opened RDMA devices. Callers can use
+ * this routine to allocate resources on specific RDMA devices that will be
+ * shared across multiple rdma_cm_id's.
+ * Notes:
+ * The returned array must be released by calling rdma_free_devices. Devices
+ * remain opened while the librdmacm is loaded.
+ * See also:
+ * rdma_free_devices
+ */
+struct ibv_context **rdma_get_devices(int *num_devices);
+
+/**
+ * rdma_free_devices - Frees the list of devices returned by rdma_get_devices.
+ * @list: List of devices returned from rdma_get_devices.
+ * Description:
+ * Frees the device array returned by rdma_get_devices.
+ * See also:
+ * rdma_get_devices
+ */
+void rdma_free_devices(struct ibv_context **list);
+
+/**
+ * rdma_event_str - Returns a string representation of an rdma cm event.
+ * @event: Asynchronous event.
+ * Description:
+ * Returns a string representation of an asynchronous event.
+ * See also:
+ * rdma_get_cm_event
+ */
+const char *rdma_event_str(enum rdma_cm_event_type event);
+
+/* Option levels */
+enum {
+ RDMA_OPTION_ID = 0,
+ RDMA_OPTION_IB = 1
+};
+
+/* Option details */
+enum {
+ RDMA_OPTION_ID_TOS = 0, /* uint8_t: RFC 2474 */
+ RDMA_OPTION_ID_REUSEADDR = 1, /* int: ~SO_REUSEADDR */
+ RDMA_OPTION_ID_AFONLY = 2, /* int: ~IPV6_V6ONLY */
+ RDMA_OPTION_IB_PATH = 1 /* struct ibv_path_data[] */
+};
+
+/**
+ * rdma_set_option - Set options for an rdma_cm_id.
+ * @id: Communication identifier to set option for.
+ * @level: Protocol level of the option to set.
+ * @optname: Name of the option to set.
+ * @optval: Reference to the option data.
+ * @optlen: The size of the %optval buffer.
+ */
+int rdma_set_option(struct rdma_cm_id *id, int level, int optname,
+ void *optval, size_t optlen);
+
+/**
+ * rdma_migrate_id - Move an rdma_cm_id to a new event channel.
+ * @id: Communication identifier to migrate.
+ * @channel: New event channel for rdma_cm_id events.
+ */
+int rdma_migrate_id(struct rdma_cm_id *id, struct rdma_event_channel *channel);
+
+/**
+ * rdma_getaddrinfo - RDMA address and route resolution service.
+ */
+int rdma_getaddrinfo(const char *node, const char *service,
+ const struct rdma_addrinfo *hints,
+ struct rdma_addrinfo **res);
+
+void rdma_freeaddrinfo(struct rdma_addrinfo *res);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RDMA_CMA_H */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 2/8] rdma: Makefile change to find <rdma/rdma_cma.h>
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rdma/Makefile b/rdma/Makefile
index 819fcbe..2cb46b6 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -4,7 +4,7 @@ include ../config.mk
TARGETS :=
ifeq ($(HAVE_MNL),y)
-CFLAGS += -I./include/uapi/
+CFLAGS += -I./include/uapi/ -I./include/
RDMA_OBJ = rdma.o utils.o dev.o link.o res.o
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 3/8] rdma: update rdma_netlink.h
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
From: Steve Wise <swise@opengridcomputing.com>
Pull in the latest rdma_netlink.h which has support for
the rdma nldev resource tracking objects being added
with this patch series.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/include/uapi/rdma/rdma_netlink.h | 38 +++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/rdma/include/uapi/rdma/rdma_netlink.h b/rdma/include/uapi/rdma/rdma_netlink.h
index dbac3b8..9446a72 100644
--- a/rdma/include/uapi/rdma/rdma_netlink.h
+++ b/rdma/include/uapi/rdma/rdma_netlink.h
@@ -238,6 +238,14 @@ enum rdma_nldev_command {
RDMA_NLDEV_CMD_RES_QP_GET, /* can dump */
+ RDMA_NLDEV_CMD_RES_CM_ID_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_CQ_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_MR_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_PD_GET, /* can dump */
+
RDMA_NLDEV_NUM_OPS
};
@@ -350,6 +358,36 @@ enum rdma_nldev_attr {
*/
RDMA_NLDEV_ATTR_RES_KERN_NAME, /* string */
+ RDMA_NLDEV_ATTR_RES_CM_ID, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY, /* nested table */
+ /*
+ * rdma_cm_id port space.
+ */
+ RDMA_NLDEV_ATTR_RES_PS, /* u32 */
+ /*
+ * Source and destination socket addresses
+ */
+ RDMA_NLDEV_ATTR_RES_SRC_ADDR, /* __kernel_sockaddr_storage */
+ RDMA_NLDEV_ATTR_RES_DST_ADDR, /* __kernel_sockaddr_storage */
+
+ RDMA_NLDEV_ATTR_RES_CQ, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CQ_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CQE, /* u32 */
+ RDMA_NLDEV_ATTR_RES_USECNT, /* u64 */
+ RDMA_NLDEV_ATTR_RES_POLL_CTX, /* u8 */
+
+ RDMA_NLDEV_ATTR_RES_MR, /* nested table */
+ RDMA_NLDEV_ATTR_RES_MR_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_RKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_LKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_IOVA, /* u64 */
+ RDMA_NLDEV_ATTR_RES_MRLEN, /* u64 */
+
+ RDMA_NLDEV_ATTR_RES_PD, /* nested table */
+ RDMA_NLDEV_ATTR_RES_PD_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY, /* u32 */
+
RDMA_NLDEV_ATTR_MAX
};
#endif /* _RDMA_NETLINK_H */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 4/8] rdma: initialize the rd struct
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Initialize the rd struct so port_idx is 0 unless set otherwise.
Otherwise, strict_port queries end up passing an uninitialized PORT
nlattr.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
rdma/rdma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rdma/rdma.c b/rdma/rdma.c
index ab2c960..b43e538 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -135,7 +135,7 @@ int main(int argc, char **argv)
bool json_output = false;
bool force = false;
char *filename;
- struct rd rd;
+ struct rd rd = {};
int opt;
int err;
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 5/8] rdma: Add CM_ID resource tracking information
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Sample output:
# rdma resource
2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7
3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7
# rdma resource show cm_id
link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP pid 30485 comm rping src-addr 0.0.0.0:7174
link cxgb4_0/2 lqpn 1048 qp-type RC state CONNECT ps TCP pid 30503 comm rping src-addr 172.16.2.1:7174 dst-addr 172.16.2.1:38246
link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP pid 30498 comm rping src-addr 172.16.2.1:38246 dst-addr 172.16.2.1:7174
link mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP pid 30485 comm rping src-addr 0.0.0.0:7174
link mlx4_0/1 lqpn 539 qp-type RC state CONNECT ps TCP pid 30494 comm rping src-addr 172.16.99.1:7174 dst-addr 172.16.99.1:43670
link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP pid 30492 comm rping src-addr 172.16.99.1:43670 dst-addr 172.16.99.1:7174
# rdma resource show cm_id dst-port 7174
link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP pid 30498 comm rping src-addr 172.16.2.1:38246 dst-addr 172.16.2.1:7174
link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP pid 30492 comm rping src-addr 172.16.99.1:43670 dst-addr 172.16.99.1:7174
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
rdma/rdma.h | 2 +
rdma/res.c | 262 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
rdma/utils.c | 5 ++
3 files changed, 268 insertions(+), 1 deletion(-)
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 5809f70..e55205b 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -18,10 +18,12 @@
#include <libmnl/libmnl.h>
#include <rdma/rdma_netlink.h>
#include <time.h>
+#include <net/if_arp.h>
#include "list.h"
#include "utils.h"
#include "json_writer.h"
+#include <rdma/rdma_cma.h>
#define pr_err(args...) fprintf(stderr, ##args)
#define pr_out(args...) fprintf(stdout, ##args)
diff --git a/rdma/res.c b/rdma/res.c
index 62f5c54..5506cf3 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -16,9 +16,11 @@ static int res_help(struct rd *rd)
{
pr_out("Usage: %s resource\n", rd->filename);
pr_out(" resource show [DEV]\n");
- pr_out(" resource show [qp]\n");
+ pr_out(" resource show [qp|cm_id]\n");
pr_out(" resource show qp link [DEV/PORT]\n");
pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
+ pr_out(" resource show cm_id link [DEV/PORT]\n");
+ pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
return 0;
}
@@ -433,6 +435,245 @@ static int res_qp_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_qp_type(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_string_field(rd->jw, "qp-type",
+ qp_types_to_str(val));
+ else
+ pr_out("qp-type %s ", qp_types_to_str(val));
+}
+
+static const char *cm_id_state_to_str(uint8_t idx)
+{
+ static const char * const cm_id_states_str[] = {
+ "IDLE", "ADDR_QUERY", "ADDR_RESOLVED", "ROUTE_QUERY",
+ "ROUTE_RESOLVED", "CONNECT", "DISCONNECT", "ADDR_BOUND",
+ "LISTEN", "DEVICE_REMOVAL", "DESTROYING" };
+
+ if (idx < ARRAY_SIZE(cm_id_states_str))
+ return cm_id_states_str[idx];
+ return "UNKNOWN";
+}
+
+static const char *cm_id_ps_to_str(uint32_t ps)
+{
+ switch (ps) {
+ case RDMA_PS_IPOIB:
+ return "IPoIB";
+ case RDMA_PS_IB:
+ return "IPoIB";
+ case RDMA_PS_TCP:
+ return "TCP";
+ case RDMA_PS_UDP:
+ return "UDP";
+ default:
+ return "---";
+ }
+}
+
+static void print_cm_id_state(struct rd *rd, uint8_t state)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "state", cm_id_state_to_str(state));
+ return;
+ }
+ pr_out("state %s ", cm_id_state_to_str(state));
+}
+
+static void print_ps(struct rd *rd, uint32_t ps)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "ps", cm_id_ps_to_str(ps));
+ return;
+ }
+ pr_out("ps %s ", cm_id_ps_to_str(ps));
+}
+
+static void print_ipaddr(struct rd *rd, const char *key, char *addrstr,
+ uint16_t port)
+{
+ if (rd->json_output) {
+ int name_size = INET6_ADDRSTRLEN+strlen(":65535");
+ char json_name[name_size];
+
+ snprintf(json_name, name_size, "%s:%u", addrstr, port);
+ jsonw_string_field(rd->jw, key, json_name);
+ return;
+ }
+ pr_out("%s %s:%u ", key, addrstr, port);
+}
+
+static int ss_ntop(struct nlattr *nla_line, char *addr_str, uint16_t *port)
+{
+ struct __kernel_sockaddr_storage *addr;
+
+ addr = (struct __kernel_sockaddr_storage *)
+ mnl_attr_get_payload(nla_line);
+ switch (addr->ss_family) {
+ case AF_INET: {
+ struct sockaddr_in *sin = (struct sockaddr_in *)addr;
+
+ if (!inet_ntop(AF_INET, (const void *)&sin->sin_addr, addr_str,
+ INET6_ADDRSTRLEN))
+ return -EINVAL;
+ *port = ntohs(sin->sin_port);
+ break;
+ }
+ case AF_INET6: {
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)addr;
+
+ if (!inet_ntop(AF_INET6, (const void *)&sin6->sin6_addr,
+ addr_str, INET6_ADDRSTRLEN))
+ return -EINVAL;
+ *port = ntohs(sin6->sin6_port);
+ break;
+ }
+ default:
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ int idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_CM_ID])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_CM_ID];
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ char src_addr_str[INET6_ADDRSTRLEN];
+ char dst_addr_str[INET6_ADDRSTRLEN];
+ uint16_t src_port, dst_port;
+ uint32_t port = 0, pid = 0;
+ uint8_t type = 0, state;
+ uint32_t lqpn = 0, ps;
+ char *comm = NULL;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return -EINVAL;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_STATE] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_PS] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_PORT_INDEX])
+ port = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]);
+
+ if (port && port != rd->port_idx)
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN]) {
+ lqpn = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_LQPN]);
+ if (rd_check_is_filtered(rd, "lqpn", lqpn))
+ continue;
+ }
+ if (nla_line[RDMA_NLDEV_ATTR_RES_TYPE]) {
+ type = mnl_attr_get_u8(
+ nla_line[RDMA_NLDEV_ATTR_RES_TYPE]);
+ if (rd_check_is_string_filtered(rd, "qp-type",
+ qp_types_to_str(type)))
+ continue;
+ }
+
+ ps = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PS]);
+ if (rd_check_is_string_filtered(rd, "ps", cm_id_ps_to_str(ps)))
+ continue;
+
+ state = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_STATE]);
+ if (rd_check_is_string_filtered(rd, "state",
+ cm_id_state_to_str(state)))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR]) {
+ if (ss_ntop(nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR],
+ src_addr_str, &src_port))
+ continue;
+ if (rd_check_is_string_filtered(rd, "src-addr",
+ src_addr_str))
+ continue;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR]) {
+ if (ss_ntop(nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR],
+ dst_addr_str, &dst_port))
+ continue;
+ if (rd_check_is_string_filtered(rd, "dst-addr",
+ dst_addr_str))
+ continue;
+ }
+
+ if (rd_check_is_filtered(rd, "src-port", src_port))
+ continue;
+
+ if (rd_check_is_filtered(rd, "dst-port", dst_port))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid)) {
+ free(comm);
+ continue;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) {
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(
+ nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+ }
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, port, nla_line);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN])
+ print_lqpn(rd, lqpn);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_TYPE])
+ print_qp_type(rd, type);
+ print_cm_id_state(rd, state);
+ print_ps(rd, ps);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR])
+ print_ipaddr(rd, "src-addr", src_addr_str, src_port);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR])
+ print_ipaddr(rd, "dst-addr", dst_addr_str, dst_port);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -457,11 +698,30 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
RES_FUNC(res_qp, RDMA_NLDEV_CMD_RES_QP_GET, qp_valid_filters, false);
+static const
+struct filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {
+ { .name = "link", .is_number = false },
+ { .name = "lqpn", .is_number = true },
+ { .name = "qp-type", .is_number = false },
+ { .name = "state", .is_number = false },
+ { .name = "ps", .is_number = false },
+ { .name = "dev-type", .is_number = false },
+ { .name = "transport-type", .is_number = false },
+ { .name = "pid", .is_number = true },
+ { .name = "src-addr", .is_number = false },
+ { .name = "src-port", .is_number = true },
+ { .name = "dst-addr", .is_number = false },
+ { .name = "dst-port", .is_number = true }
+};
+
+RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters, false);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
{ NULL, res_no_args },
{ "qp", res_qp },
+ { "cm_id", res_cm_id },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index f946016..ec81737 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -375,6 +375,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_STATE] = MNL_TYPE_U8,
[RDMA_NLDEV_ATTR_RES_PID] = MNL_TYPE_U32,
[RDMA_NLDEV_ATTR_RES_KERN_NAME] = MNL_TYPE_NUL_STRING,
+ [RDMA_NLDEV_ATTR_RES_CM_ID] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_SRC_ADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_DST_ADDR] = MNL_TYPE_UNSPEC,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 6/8] rdma: Add CQ resource tracking information
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Sample output:
# rdma resource show cq
link cxgb4_0/- cqe 46 users 2 pid 30503 comm rping
link cxgb4_0/- cqe 46 users 2 pid 30498 comm rping
link mlx4_0/- cqe 63 users 2 pid 30494 comm rping
link mlx4_0/- cqe 63 users 2 pid 30489 comm rping
link mlx4_0/- cqe 1023 users 2 poll_ctx WORKQUEUE pid 0 comm [ib_core]
# rdma resource show cq pid 30489
link mlx4_0/- cqe 63 users 2 pid 30489 comm rping
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
rdma/res.c | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
rdma/utils.c | 5 +++
2 files changed, 144 insertions(+)
diff --git a/rdma/res.c b/rdma/res.c
index 5506cf3..f1a90d3 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -21,6 +21,8 @@ static int res_help(struct rd *rd)
pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
pr_out(" resource show cm_id link [DEV/PORT]\n");
pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
+ pr_out(" resource show cq link [DEV/PORT]\n");
+ pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
return 0;
}
@@ -674,6 +676,132 @@ static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_cqe(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "cqe", val);
+ else
+ pr_out("cqe %u ", val);
+}
+
+static void print_users(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "users", val);
+ else
+ pr_out("users %" PRIu64 " ", val);
+}
+
+static const char *poll_ctx_to_str(uint8_t idx)
+{
+ static const char * const cm_id_states_str[] = {
+ "DIRECT", "SOFTIRQ", "WORKQUEUE"};
+
+ if (idx < ARRAY_SIZE(cm_id_states_str))
+ return cm_id_states_str[idx];
+ return "UNKNOWN";
+}
+
+static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "poll-ctx",
+ poll_ctx_to_str(poll_ctx));
+ return;
+ }
+ pr_out("poll-ctx %s ", poll_ctx_to_str(poll_ctx));
+}
+
+static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_CQ])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ char *comm = NULL;
+ uint32_t pid = 0;
+ uint8_t poll_ctx = 0;
+ uint64_t users;
+ uint32_t cqe;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ cqe = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]);
+
+ users = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
+ if (rd_check_is_filtered(rd, "users", users))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]) {
+ poll_ctx = mnl_attr_get_u8(
+ nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]);
+ if (rd_check_is_string_filtered(rd, "poll-ctx",
+ poll_ctx_to_str(poll_ctx)))
+ continue;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid)) {
+ free(comm);
+ continue;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(
+ nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ print_cqe(rd, cqe);
+ print_users(rd, users);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
+ print_poll_ctx(rd, poll_ctx);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -716,12 +844,23 @@ struct filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {
RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters, false);
+static const
+struct filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {
+ { .name = "link", .is_number = false },
+ { .name = "users", .is_number = true },
+ { .name = "poll-ctx", .is_number = false },
+ { .name = "pid", .is_number = true }
+};
+
+RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
{ NULL, res_no_args },
{ "qp", res_qp },
{ "cm_id", res_cm_id },
+ { "cq", res_cq },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index ec81737..5e79b62 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -380,6 +380,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32,
[RDMA_NLDEV_ATTR_RES_SRC_ADDR] = MNL_TYPE_UNSPEC,
[RDMA_NLDEV_ATTR_RES_DST_ADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
+ [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 7/8] rdma: Add MR resource tracking information
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Sample output:
Without CAP_NET_ADMIN:
$ rdma resource show mr mrlen 65536
link mlx4_0/- mrlen 65536 pid 0 comm [nvme_rdma]
link cxgb4_0/- mrlen 65536 pid 0 comm [nvme_rdma]
With CAP_NET_ADMIN:
# rdma resource show mr mrlen 65536
link mlx4_0/- rkey 0x12702 lkey 0x12702 iova 0x85724a000 mrlen 65536 pid 0 comm [nvme_rdma]
link cxgb4_0/- rkey 0x68fe4e9 lkey 0x68fe4e9 iova 0x835b91000 mrlen 65536 pid 0 comm [nvme_rdma]
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
include/json_writer.h | 2 +
lib/json_writer.c | 11 +++++
rdma/res.c | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++
rdma/utils.c | 6 +++
4 files changed, 146 insertions(+)
diff --git a/include/json_writer.h b/include/json_writer.h
index 45459fa..4b4dec2 100644
--- a/include/json_writer.h
+++ b/include/json_writer.h
@@ -35,6 +35,7 @@ void jsonw_bool(json_writer_t *self, bool value);
void jsonw_float(json_writer_t *self, double number);
void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num);
void jsonw_uint(json_writer_t *self, uint64_t number);
+void jsonw_xint(json_writer_t *self, uint64_t number);
void jsonw_hu(json_writer_t *self, unsigned short number);
void jsonw_int(json_writer_t *self, int64_t number);
void jsonw_null(json_writer_t *self);
@@ -45,6 +46,7 @@ void jsonw_string_field(json_writer_t *self, const char *prop, const char *val);
void jsonw_bool_field(json_writer_t *self, const char *prop, bool value);
void jsonw_float_field(json_writer_t *self, const char *prop, double num);
void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num);
+void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num);
void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num);
void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num);
void jsonw_null_field(json_writer_t *self, const char *prop);
diff --git a/lib/json_writer.c b/lib/json_writer.c
index 68401ae..0ad0421 100644
--- a/lib/json_writer.c
+++ b/lib/json_writer.c
@@ -225,6 +225,11 @@ void jsonw_uint(json_writer_t *self, uint64_t num)
jsonw_printf(self, "%"PRIu64, num);
}
+void jsonw_xint(json_writer_t *self, uint64_t num)
+{
+ jsonw_printf(self, "%"PRIx64, num);
+}
+
void jsonw_lluint(json_writer_t *self, unsigned long long int num)
{
jsonw_printf(self, "%llu", num);
@@ -269,6 +274,12 @@ void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num)
jsonw_uint(self, num);
}
+void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num)
+{
+ jsonw_name(self, prop);
+ jsonw_xint(self, num);
+}
+
void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num)
{
jsonw_name(self, prop);
diff --git a/rdma/res.c b/rdma/res.c
index f1a90d3..aa61e73 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -802,6 +802,121 @@ static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_key(struct rd *rd, const char *name, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_xint_field(rd->jw, name, val);
+ else
+ pr_out("%s 0x%x ", name, val);
+}
+
+static void print_iova(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_xint_field(rd->jw, "iova", val);
+ else
+ pr_out("iova 0x%" PRIx64 " ", val);
+}
+
+static void print_mrlen(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "mrlen", val);
+ else
+ pr_out("mrlen %" PRIu64 " ", val);
+}
+
+static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_MR])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_MR];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ uint32_t rkey = 0, lkey = 0;
+ uint64_t iova = 0, mrlen;
+ char *comm = NULL;
+ uint32_t pid = 0;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_MRLEN] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_RKEY])
+ rkey = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_RKEY]);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LKEY])
+ lkey = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_LKEY]);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_IOVA])
+ iova = mnl_attr_get_u64(
+ nla_line[RDMA_NLDEV_ATTR_RES_IOVA]);
+
+ mrlen = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_MRLEN]);
+ if (rd_check_is_filtered(rd, "mrlen", mrlen))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid)) {
+ free(comm);
+ continue;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(
+ nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_RKEY])
+ print_key(rd, "rkey", rkey);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LKEY])
+ print_key(rd, "lkey", lkey);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_IOVA])
+ print_iova(rd, iova);
+ print_mrlen(rd, mrlen);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -854,6 +969,17 @@ struct filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {
RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
+static const
+struct filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {
+ { .name = "link", .is_number = false },
+ { .name = "rkey", .is_number = true },
+ { .name = "lkey", .is_number = true },
+ { .name = "mrlen", .is_number = true },
+ { .name = "pid", .is_number = true }
+};
+
+RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
@@ -861,6 +987,7 @@ static int res_show(struct rd *rd)
{ "qp", res_qp },
{ "cm_id", res_cm_id },
{ "cq", res_cq },
+ { "mr", res_mr },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index 5e79b62..a2e08e9 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -385,6 +385,12 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
[RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
[RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
+ [RDMA_NLDEV_ATTR_RES_MR] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_MR_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_RKEY] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_LKEY] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_IOVA] = MNL_TYPE_U64,
+ [RDMA_NLDEV_ATTR_RES_MRLEN] = MNL_TYPE_U64,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 iproute2-next 8/8] rdma: Add PD resource tracking information
From: Steve Wise @ 2018-03-26 20:57 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
In-Reply-To: <cover.1522097991.git.swise@opengridcomputing.com>
Sample output:
Without CAP_NET_ADMIN capability:
link mlx4_0/- users 0 pid 0 comm [ib_srpt]
link mlx4_0/- users 0 pid 0 comm [ib_srp]
link mlx4_0/- users 1 pid 0 comm [ib_core]
link cxgb4_0/- users 0 pid 0 comm [ib_srp]
With CAP_NET_ADMIN capability:
link mlx4_0/- local_dma_lkey 0x8000 users 0 pid 0 comm [ib_srpt]
link mlx4_0/- local_dma_lkey 0x8000 users 0 pid 0 comm [ib_srp]
link mlx4_0/- local_dma_lkey 0x8000 users 1 pid 0 comm [ib_core]
link cxgb4_0/- local_dma_lkey 0x0 users 0 pid 0 comm [ib_srp]
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
rdma/res.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/rdma/res.c b/rdma/res.c
index aa61e73..6177e46 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -917,6 +917,91 @@ static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static int res_pd_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_PD])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_PD];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ uint32_t local_dma_lkey = 0, unsafe_global_rkey = 0;
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ char *comm = NULL;
+ uint32_t pid = 0;
+ uint64_t users;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY])
+ local_dma_lkey = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY]);
+
+ users = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
+ if (rd_check_is_filtered(rd, "users", users))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY])
+ unsafe_global_rkey = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY]);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(
+ nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(
+ nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY])
+ print_key(rd, "local_dma_lkey", local_dma_lkey);
+ print_users(rd, users);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY])
+ print_key(rd, "unsafe_global_rkey", unsafe_global_rkey);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -980,6 +1065,15 @@ struct filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {
RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true);
+static const
+struct filters pd_valid_filters[MAX_NUMBER_OF_FILTERS] = {
+ { .name = "link", .is_number = false },
+ { .name = "users", .is_number = true },
+ { .name = "pid", .is_number = true }
+};
+
+RES_FUNC(res_pd, RDMA_NLDEV_CMD_RES_PD_GET, pd_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
@@ -988,6 +1082,7 @@ static int res_show(struct rd *rd)
{ "cm_id", res_cm_id },
{ "cq", res_cq },
{ "mr", res_mr },
+ { "pd", res_pd },
{ 0 }
};
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH net-next 1/2] net: systemport: Implement adaptive interrupt coalescing
From: Tal Gilboa @ 2018-03-26 21:22 UTC (permalink / raw)
To: Florian Fainelli, netdev
Cc: davem, jaedon.shin, pgynther, opendmb, michal.chan, gospo, saeedm
In-Reply-To: <20180323011933.29748-2-f.fainelli@gmail.com>
On 3/23/2018 4:19 AM, Florian Fainelli wrote:
> Implement support for adaptive RX and TX interrupt coalescing using
> net_dim. We have each of our TX ring and our single RX ring implement a
> bcm_sysport_net_dim structure which holds an interrupt counter, number
> of packets, bytes, and a container for a net_dim instance.
>
> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
> drivers/net/ethernet/broadcom/bcmsysport.c | 141 ++++++++++++++++++++++++++---
> drivers/net/ethernet/broadcom/bcmsysport.h | 14 +++
> 2 files changed, 140 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
> index f15a8fc6dfc9..5a5a726bafa4 100644
> --- a/drivers/net/ethernet/broadcom/bcmsysport.c
> +++ b/drivers/net/ethernet/broadcom/bcmsysport.c
> @@ -15,6 +15,7 @@
> #include <linux/module.h>
> #include <linux/kernel.h>
> #include <linux/netdevice.h>
> +#include <linux/net_dim.h>
I don't think you need this include. You already include net_dim in
bcmsysport.h and include the bcmsysport.h here.
> #include <linux/etherdevice.h>
> #include <linux/platform_device.h>
> #include <linux/of.h>
> @@ -574,21 +575,55 @@ static int bcm_sysport_set_wol(struct net_device *dev,
> return 0;
> }
>
> +static void bcm_sysport_set_rx_coalesce(struct bcm_sysport_priv *priv)
> +{
> + u32 reg;
> +
> + reg = rdma_readl(priv, RDMA_MBDONE_INTR);
> + reg &= ~(RDMA_INTR_THRESH_MASK |
> + RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
> + reg |= priv->dim.coal_pkts;
> + reg |= DIV_ROUND_UP(priv->dim.coal_usecs * 1000, 8192) <<
> + RDMA_TIMEOUT_SHIFT;
> + rdma_writel(priv, reg, RDMA_MBDONE_INTR);
> +}
> +
> +static void bcm_sysport_set_tx_coalesce(struct bcm_sysport_tx_ring *ring)
> +{
> + struct bcm_sysport_priv *priv = ring->priv;
> + u32 reg;
> +
> + reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(ring->index));
> + reg &= ~(RING_INTR_THRESH_MASK |
> + RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
> + reg |= ring->dim.coal_pkts;
> + reg |= DIV_ROUND_UP(ring->dim.coal_usecs * 1000, 8192) <<
> + RING_TIMEOUT_SHIFT;
> + tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(ring->index));
> +}
> +
I wouldn't couple these functions with dim. This implies dim is always
used. IMO, would be more clear to use a generic method which takes usecs
and packets as an argument.
> static int bcm_sysport_get_coalesce(struct net_device *dev,
> struct ethtool_coalesce *ec)
> {
> struct bcm_sysport_priv *priv = netdev_priv(dev);
> + struct bcm_sysport_tx_ring *ring;
> + unsigned int i;
> u32 reg;
>
> reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(0));
>
> ec->tx_coalesce_usecs = (reg >> RING_TIMEOUT_SHIFT) * 8192 / 1000;
> ec->tx_max_coalesced_frames = reg & RING_INTR_THRESH_MASK;
> + for (i = 0; i < dev->num_tx_queues; i++) {
> + ring = &priv->tx_rings[i];
> + ec->use_adaptive_tx_coalesce |= ring->dim.use_dim;
> + }
>
> reg = rdma_readl(priv, RDMA_MBDONE_INTR);
>
> ec->rx_coalesce_usecs = (reg >> RDMA_TIMEOUT_SHIFT) * 8192 / 1000;
> ec->rx_max_coalesced_frames = reg & RDMA_INTR_THRESH_MASK;
> + ec->use_adaptive_rx_coalesce = priv->dim.use_dim;
>
> return 0;
> }
> @@ -597,8 +632,8 @@ static int bcm_sysport_set_coalesce(struct net_device *dev,
> struct ethtool_coalesce *ec)
> {
> struct bcm_sysport_priv *priv = netdev_priv(dev);
> + struct bcm_sysport_tx_ring *ring;
> unsigned int i;
> - u32 reg;
>
> /* Base system clock is 125Mhz, DMA timeout is this reference clock
> * divided by 1024, which yield roughly 8.192 us, our maximum value has
> @@ -615,22 +650,26 @@ static int bcm_sysport_set_coalesce(struct net_device *dev,
> return -EINVAL;
>
> for (i = 0; i < dev->num_tx_queues; i++) {
> - reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(i));
> - reg &= ~(RING_INTR_THRESH_MASK |
> - RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
> - reg |= ec->tx_max_coalesced_frames;
> - reg |= DIV_ROUND_UP(ec->tx_coalesce_usecs * 1000, 8192) <<
> - RING_TIMEOUT_SHIFT;
> - tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(i));
> + ring = &priv->tx_rings[i];
> + ring->dim.coal_pkts = ec->tx_max_coalesced_frames;
> + ring->dim.coal_usecs = ec->tx_coalesce_usecs;
> + if (!ec->use_adaptive_tx_coalesce && ring->dim.use_dim) {
> + ring->dim.coal_pkts = 1;
> + ring->dim.coal_usecs = 0;
> + }
> + ring->dim.use_dim = ec->use_adaptive_tx_coalesce;
> + bcm_sysport_set_tx_coalesce(ring);
> }
If I understand correctly, if I disable dim, moderation is set to
{usecs,packets}={0,1} regardless of the input from ethtool right?
Doesn't this break the wanted behavior? As mentioned above, I would
decouple dim from the set_tx/rx_coalesce() function. Also, when dim is
enabled, why change dim.coal_pkts/usecs? They would just be overwritten
in the next iteration of net_dim.
>
> - reg = rdma_readl(priv, RDMA_MBDONE_INTR);
> - reg &= ~(RDMA_INTR_THRESH_MASK |
> - RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
> - reg |= ec->rx_max_coalesced_frames;
> - reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192) <<
> - RDMA_TIMEOUT_SHIFT;
> - rdma_writel(priv, reg, RDMA_MBDONE_INTR);
> + priv->dim.coal_usecs = ec->rx_coalesce_usecs;
> + priv->dim.coal_pkts = ec->rx_max_coalesced_frames;
> +
> + if (!ec->use_adaptive_rx_coalesce && priv->dim.use_dim) {
> + priv->dim.coal_pkts = 1;
> + priv->dim.coal_usecs = 0;
> + }
> + priv->dim.use_dim = ec->use_adaptive_rx_coalesce;
> + bcm_sysport_set_rx_coalesce(priv);
Same comment as above.
>
> return 0;
> }
> @@ -709,6 +748,7 @@ static unsigned int bcm_sysport_desc_rx(struct bcm_sysport_priv *priv,
> struct bcm_sysport_stats64 *stats64 = &priv->stats64;
> struct net_device *ndev = priv->netdev;
> unsigned int processed = 0, to_process;
> + unsigned int processed_bytes = 0;
> struct bcm_sysport_cb *cb;
> struct sk_buff *skb;
> unsigned int p_index;
> @@ -800,6 +840,7 @@ static unsigned int bcm_sysport_desc_rx(struct bcm_sysport_priv *priv,
> */
> skb_pull(skb, sizeof(*rsb) + 2);
> len -= (sizeof(*rsb) + 2);
> + processed_bytes += len;
>
> /* UniMAC may forward CRC */
> if (priv->crc_fwd) {
> @@ -824,6 +865,9 @@ static unsigned int bcm_sysport_desc_rx(struct bcm_sysport_priv *priv,
> priv->rx_read_ptr = 0;
> }
>
> + priv->dim.packets = processed;
> + priv->dim.bytes = processed_bytes;
> +
> return processed;
> }
>
> @@ -900,6 +944,8 @@ static unsigned int __bcm_sysport_tx_reclaim(struct bcm_sysport_priv *priv,
> ring->packets += pkts_compl;
> ring->bytes += bytes_compl;
> u64_stats_update_end(&priv->syncp);
> + ring->dim.packets = pkts_compl;
> + ring->dim.bytes = bytes_compl;
>
> ring->c_index = c_index;
>
> @@ -945,6 +991,7 @@ static int bcm_sysport_tx_poll(struct napi_struct *napi, int budget)
> {
> struct bcm_sysport_tx_ring *ring =
> container_of(napi, struct bcm_sysport_tx_ring, napi);
> + struct net_dim_sample dim_sample;
> unsigned int work_done = 0;
>
> work_done = bcm_sysport_tx_reclaim(ring->priv, ring);
> @@ -961,6 +1008,12 @@ static int bcm_sysport_tx_poll(struct napi_struct *napi, int budget)
> return 0;
> }
>
> + if (ring->dim.use_dim) {
> + net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
> + ring->dim.bytes, &dim_sample);
> + net_dim(&ring->dim.dim, dim_sample);
> + }
> +
> return budget;
> }
>
> @@ -976,6 +1029,7 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
> {
> struct bcm_sysport_priv *priv =
> container_of(napi, struct bcm_sysport_priv, napi);
> + struct net_dim_sample dim_sample;
> unsigned int work_done = 0;
>
> work_done = bcm_sysport_desc_rx(priv, budget);
> @@ -998,6 +1052,12 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
> intrl2_0_mask_clear(priv, INTRL2_0_RDMA_MBDONE);
> }
>
> + if (priv->dim.use_dim) {
> + net_dim_sample(priv->dim.event_ctr, priv->dim.packets,
> + priv->dim.bytes, &dim_sample);
> + net_dim(&priv->dim.dim, dim_sample);
> + }
> +
> return work_done;
> }
>
> @@ -1016,6 +1076,40 @@ static void bcm_sysport_resume_from_wol(struct bcm_sysport_priv *priv)
> netif_dbg(priv, wol, priv->netdev, "resumed from WOL\n");
> }
>
> +static void bcm_sysport_dim_work(struct work_struct *work)
> +{
> + struct net_dim *dim = container_of(work, struct net_dim, work);
> + struct bcm_sysport_net_dim *ndim =
> + container_of(dim, struct bcm_sysport_net_dim, dim);
> + struct bcm_sysport_priv *priv =
> + container_of(ndim, struct bcm_sysport_priv, dim);
> + struct net_dim_cq_moder cur_profile =
> + net_dim_get_profile(dim->mode, dim->profile_ix);
> +
> + priv->dim.coal_usecs = cur_profile.usec;
> + priv->dim.coal_pkts = cur_profile.pkts;
> +
> + bcm_sysport_set_rx_coalesce(priv);
> + dim->state = NET_DIM_START_MEASURE;
> +}
> +
> +static void bcm_sysport_dim_tx_work(struct work_struct *work)
> +{
> + struct net_dim *dim = container_of(work, struct net_dim, work);
> + struct bcm_sysport_net_dim *ndim =
> + container_of(dim, struct bcm_sysport_net_dim, dim);
> + struct bcm_sysport_tx_ring *ring =
> + container_of(ndim, struct bcm_sysport_tx_ring, dim);
> + struct net_dim_cq_moder cur_profile =
> + net_dim_get_profile(dim->mode, dim->profile_ix);
> +
> + ring->dim.coal_usecs = cur_profile.usec;
> + ring->dim.coal_pkts = cur_profile.pkts;
> +
> + bcm_sysport_set_tx_coalesce(ring);
> + dim->state = NET_DIM_START_MEASURE;
> +}
> +
> /* RX and misc interrupt routine */
> static irqreturn_t bcm_sysport_rx_isr(int irq, void *dev_id)
> {
> @@ -1034,6 +1128,7 @@ static irqreturn_t bcm_sysport_rx_isr(int irq, void *dev_id)
> }
>
> if (priv->irq0_stat & INTRL2_0_RDMA_MBDONE) {
> + priv->dim.event_ctr++;
> if (likely(napi_schedule_prep(&priv->napi))) {
> /* disable RX interrupts */
> intrl2_0_mask_set(priv, INTRL2_0_RDMA_MBDONE);
> @@ -1061,6 +1156,7 @@ static irqreturn_t bcm_sysport_rx_isr(int irq, void *dev_id)
> continue;
>
> txr = &priv->tx_rings[ring];
> + txr->dim.event_ctr++;
>
> if (likely(napi_schedule_prep(&txr->napi))) {
> intrl2_0_mask_set(priv, ring_bit);
> @@ -1093,6 +1189,7 @@ static irqreturn_t bcm_sysport_tx_isr(int irq, void *dev_id)
> continue;
>
> txr = &priv->tx_rings[ring];
> + txr->dim.event_ctr++;
>
> if (likely(napi_schedule_prep(&txr->napi))) {
> intrl2_1_mask_set(priv, BIT(ring));
> @@ -1358,6 +1455,16 @@ static void bcm_sysport_adj_link(struct net_device *dev)
> phy_print_status(phydev);
> }
>
> +static void bcm_sysport_init_dim(struct bcm_sysport_net_dim *dim,
> + void (*cb)(struct work_struct *work))
> +{
> + INIT_WORK(&dim->dim.work, cb);
> + dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
> + dim->event_ctr = 0;
> + dim->packets = 0;
> + dim->bytes = 0;
> +}
What about default values for coal_usecs/pkts? dim supports it through
net_dim_get_def_profile(mode) function.
> +
> static int bcm_sysport_init_tx_ring(struct bcm_sysport_priv *priv,
> unsigned int index)
> {
> @@ -1447,6 +1554,7 @@ static int bcm_sysport_init_tx_ring(struct bcm_sysport_priv *priv,
> reg |= (1 << index);
> tdma_writel(priv, reg, TDMA_TIER1_ARB_0_QUEUE_EN);
>
> + bcm_sysport_init_dim(&ring->dim, bcm_sysport_dim_tx_work);
> napi_enable(&ring->napi);
>
> netif_dbg(priv, hw, priv->netdev,
> @@ -1477,6 +1585,7 @@ static void bcm_sysport_fini_tx_ring(struct bcm_sysport_priv *priv,
> return;
>
> napi_disable(&ring->napi);
> + cancel_work_sync(&ring->dim.dim.work);
> netif_napi_del(&ring->napi);
>
> bcm_sysport_tx_clean(priv, ring);
> @@ -1766,6 +1875,7 @@ static void bcm_sysport_netif_start(struct net_device *dev)
> struct bcm_sysport_priv *priv = netdev_priv(dev);
>
> /* Enable NAPI */
> + bcm_sysport_init_dim(&priv->dim, bcm_sysport_dim_work);
> napi_enable(&priv->napi);
>
> /* Enable RX interrupt and TX ring full interrupt */
> @@ -1951,6 +2061,7 @@ static void bcm_sysport_netif_stop(struct net_device *dev)
> /* stop all software from updating hardware */
> netif_tx_stop_all_queues(dev);
> napi_disable(&priv->napi);
> + cancel_work_sync(&priv->dim.dim.work);
> phy_stop(dev->phydev);
>
> /* mask all interrupts */
> diff --git a/drivers/net/ethernet/broadcom/bcmsysport.h b/drivers/net/ethernet/broadcom/bcmsysport.h
> index f5a984c1c986..9f48ad3cc38d 100644
> --- a/drivers/net/ethernet/broadcom/bcmsysport.h
> +++ b/drivers/net/ethernet/broadcom/bcmsysport.h
> @@ -12,6 +12,7 @@
> #define __BCM_SYSPORT_H
>
> #include <linux/if_vlan.h>
> +#include <linux/net_dim.h>
>
> /* Receive/transmit descriptor format */
> #define DESC_ADDR_HI_STATUS_LEN 0x00
> @@ -695,6 +696,16 @@ struct bcm_sysport_hw_params {
> unsigned int num_rx_desc_words;
> };
>
> +struct bcm_sysport_net_dim {
> + u16 use_dim;
> + u16 event_ctr;
> + unsigned long packets;
> + unsigned long bytes;
> + u32 coal_usecs;
> + u32 coal_pkts;
> + struct net_dim dim;
> +};
> +
> /* Software view of the TX ring */
> struct bcm_sysport_tx_ring {
> spinlock_t lock; /* Ring lock for tx reclaim/xmit */
> @@ -712,6 +723,7 @@ struct bcm_sysport_tx_ring {
> struct bcm_sysport_priv *priv; /* private context backpointer */
> unsigned long packets; /* packets statistics */
> unsigned long bytes; /* bytes statistics */
> + struct bcm_sysport_net_dim dim; /* Net DIM context */
> unsigned int switch_queue; /* switch port queue number */
> unsigned int switch_port; /* switch port queue number */
> bool inspect; /* inspect switch port and queue */
> @@ -743,6 +755,8 @@ struct bcm_sysport_priv {
> unsigned int rx_read_ptr;
> unsigned int rx_c_index;
>
> + struct bcm_sysport_net_dim dim;
> +
> /* PHY device */
> struct device_node *phy_dn;
> phy_interface_t phy_interface;
>
^ permalink raw reply
* Re: [PATCH net-next 2/2] net: bcmgenet: Add support for adaptive RX coalescing
From: Tal Gilboa @ 2018-03-26 21:23 UTC (permalink / raw)
To: Florian Fainelli, netdev
Cc: davem, jaedon.shin, pgynther, opendmb, michal.chan, gospo, saeedm
In-Reply-To: <20180323011933.29748-3-f.fainelli@gmail.com>
On 3/23/2018 4:19 AM, Florian Fainelli wrote:
> Unlike the moder modern SYSTEMPORT hardware, we do not have a
> configurable TDMA timeout, which limits us to implement adaptive RX
> interrupt coalescing only. We have each of our RX rings implement a
> bcmgenet_net_dim structure which holds an interrupt counter, number of
> packets, bytes, and a container for a net_dim instance.
>
> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
> drivers/net/ethernet/broadcom/genet/bcmgenet.c | 109 +++++++++++++++++++++----
> drivers/net/ethernet/broadcom/genet/bcmgenet.h | 12 +++
> 2 files changed, 103 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> index b1e35a9accf1..7db8edc643ec 100644
> --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> @@ -603,6 +603,8 @@ static int bcmgenet_get_coalesce(struct net_device *dev,
> struct ethtool_coalesce *ec)
> {
> struct bcmgenet_priv *priv = netdev_priv(dev);
> + struct bcmgenet_rx_ring *ring;
> + unsigned int i;
>
> ec->tx_max_coalesced_frames =
> bcmgenet_tdma_ring_readl(priv, DESC_INDEX,
> @@ -613,15 +615,37 @@ static int bcmgenet_get_coalesce(struct net_device *dev,
> ec->rx_coalesce_usecs =
> bcmgenet_rdma_readl(priv, DMA_RING16_TIMEOUT) * 8192 / 1000;
>
> + for (i = 0; i < priv->hw_params->rx_queues; i++) {
> + ring = &priv->rx_rings[i];
> + ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
> + }
> + ring = &priv->rx_rings[DESC_INDEX];
> + ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
> +
> return 0;
> }
>
> +static void bcmgenet_set_rx_coalesce(struct bcmgenet_rx_ring *ring)
> +{
> + struct bcmgenet_priv *priv = ring->priv;
> + unsigned int i = ring->index;
> + u32 reg;
> +
> + bcmgenet_rdma_ring_writel(priv, i, ring->dim.coal_pkts,
> + DMA_MBUF_DONE_THRESH);
> +
> + reg = bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT + i);
> + reg &= ~DMA_TIMEOUT_MASK;
> + reg |= DIV_ROUND_UP(ring->dim.coal_usecs * 1000, 8192);
> + bcmgenet_rdma_writel(priv, reg, DMA_RING0_TIMEOUT + i);
> +}
> +
Similar comments from path 1/2 apply here - wouldn't couple the genric
get_set_coalesce functions with dim.
> static int bcmgenet_set_coalesce(struct net_device *dev,
> struct ethtool_coalesce *ec)
> {
> struct bcmgenet_priv *priv = netdev_priv(dev);
> + struct bcmgenet_rx_ring *ring;
> unsigned int i;
> - u32 reg;
>
> /* Base system clock is 125Mhz, DMA timeout is this reference clock
> * divided by 1024, which yields roughly 8.192us, our maximum value
> @@ -641,7 +665,8 @@ static int bcmgenet_set_coalesce(struct net_device *dev,
> * transmitted, or when the ring is empty.
> */
> if (ec->tx_coalesce_usecs || ec->tx_coalesce_usecs_high ||
> - ec->tx_coalesce_usecs_irq || ec->tx_coalesce_usecs_low)
> + ec->tx_coalesce_usecs_irq || ec->tx_coalesce_usecs_low ||
> + ec->use_adaptive_tx_coalesce)
> return -EOPNOTSUPP;
>
> /* Program all TX queues with the same values, as there is no
> @@ -656,24 +681,26 @@ static int bcmgenet_set_coalesce(struct net_device *dev,
> DMA_MBUF_DONE_THRESH);
>
> for (i = 0; i < priv->hw_params->rx_queues; i++) {
> - bcmgenet_rdma_ring_writel(priv, i,
> - ec->rx_max_coalesced_frames,
> - DMA_MBUF_DONE_THRESH);
> -
> - reg = bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT + i);
> - reg &= ~DMA_TIMEOUT_MASK;
> - reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192);
> - bcmgenet_rdma_writel(priv, reg, DMA_RING0_TIMEOUT + i);
> + ring = &priv->rx_rings[i];
> + ring->dim.coal_usecs = ec->rx_coalesce_usecs;
> + ring->dim.coal_pkts = ec->rx_max_coalesced_frames;
> + if (!ec->use_adaptive_rx_coalesce && ring->dim.use_dim) {
> + ring->dim.coal_pkts = 1;
> + ring->dim.coal_usecs = 0;
> + }
> + ring->dim.use_dim = ec->use_adaptive_rx_coalesce;
> + bcmgenet_set_rx_coalesce(ring);
> }
>
> - bcmgenet_rdma_ring_writel(priv, DESC_INDEX,
> - ec->rx_max_coalesced_frames,
> - DMA_MBUF_DONE_THRESH);
> -
> - reg = bcmgenet_rdma_readl(priv, DMA_RING16_TIMEOUT);
> - reg &= ~DMA_TIMEOUT_MASK;
> - reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192);
> - bcmgenet_rdma_writel(priv, reg, DMA_RING16_TIMEOUT);
> + ring = &priv->rx_rings[DESC_INDEX];
> + ring->dim.coal_usecs = ec->rx_coalesce_usecs;
> + ring->dim.coal_pkts = ec->rx_max_coalesced_frames;
> + if (!ec->use_adaptive_rx_coalesce && ring->dim.use_dim) {
> + ring->dim.coal_pkts = 1;
> + ring->dim.coal_usecs = 0;
> + }
> + ring->dim.use_dim = ec->use_adaptive_rx_coalesce;
> + bcmgenet_set_rx_coalesce(ring);
>
> return 0;
> }
> @@ -1713,6 +1740,7 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
> unsigned long dma_flag;
> int len;
> unsigned int rxpktprocessed = 0, rxpkttoprocess;
> + unsigned int bytes_processed = 0;
> unsigned int p_index, mask;
> unsigned int discards;
> unsigned int chksum_ok = 0;
> @@ -1832,6 +1860,8 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
> len -= ETH_FCS_LEN;
> }
>
> + bytes_processed += len;
> +
> /*Finish setting up the received SKB and send it to the kernel*/
> skb->protocol = eth_type_trans(skb, priv->dev);
> ring->packets++;
> @@ -1854,6 +1884,9 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
> bcmgenet_rdma_ring_writel(priv, ring->index, ring->c_index, RDMA_CONS_INDEX);
> }
>
> + ring->dim.bytes = bytes_processed;
> + ring->dim.packets = rxpktprocessed;
> +
> return rxpktprocessed;
> }
>
> @@ -1862,6 +1895,7 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
> {
> struct bcmgenet_rx_ring *ring = container_of(napi,
> struct bcmgenet_rx_ring, napi);
> + struct net_dim_sample dim_sample;
> unsigned int work_done;
>
> work_done = bcmgenet_desc_rx(ring, budget);
> @@ -1871,9 +1905,32 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
> ring->int_enable(ring);
> }
>
> + if (ring->dim.use_dim) {
> + net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
> + ring->dim.bytes, &dim_sample);
> + net_dim(&ring->dim.dim, dim_sample);
> + }
> +
> return work_done;
> }
>
> +static void bcmgenet_dim_work(struct work_struct *work)
> +{
> + struct net_dim *dim = container_of(work, struct net_dim, work);
> + struct bcmgenet_net_dim *ndim =
> + container_of(dim, struct bcmgenet_net_dim, dim);
> + struct bcmgenet_rx_ring *ring =
> + container_of(ndim, struct bcmgenet_rx_ring, dim);
> + struct net_dim_cq_moder cur_profile =
> + net_dim_get_profile(dim->mode, dim->profile_ix);
> +
> + ring->dim.coal_usecs = cur_profile.usec;
> + ring->dim.coal_pkts = cur_profile.pkts;
> +
> + bcmgenet_set_rx_coalesce(ring);
> + dim->state = NET_DIM_START_MEASURE;
> +}
> +
> /* Assign skb to RX DMA descriptor. */
> static int bcmgenet_alloc_rx_buffers(struct bcmgenet_priv *priv,
> struct bcmgenet_rx_ring *ring)
> @@ -2022,6 +2079,16 @@ static void init_umac(struct bcmgenet_priv *priv)
> dev_dbg(kdev, "done init umac\n");
> }
>
> +static void bcmgenet_init_dim(struct bcmgenet_net_dim *dim,
> + void (*cb)(struct work_struct *work))
> +{
> + INIT_WORK(&dim->dim.work, cb);
> + dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
> + dim->event_ctr = 0;
> + dim->packets = 0;
> + dim->bytes = 0;
> +} > +
Similar comment from path 1/2 applies here - default values for
coal_usecs/pkts.
> /* Initialize a Tx ring along with corresponding hardware registers */
> static void bcmgenet_init_tx_ring(struct bcmgenet_priv *priv,
> unsigned int index, unsigned int size,
> @@ -2111,6 +2178,8 @@ static int bcmgenet_init_rx_ring(struct bcmgenet_priv *priv,
> if (ret)
> return ret;
>
> + bcmgenet_init_dim(&ring->dim, bcmgenet_dim_work);
> +
> /* Initialize Rx NAPI */
> netif_napi_add(priv->dev, &ring->napi, bcmgenet_rx_poll,
> NAPI_POLL_WEIGHT);
> @@ -2276,10 +2345,12 @@ static void bcmgenet_disable_rx_napi(struct bcmgenet_priv *priv)
> for (i = 0; i < priv->hw_params->rx_queues; ++i) {
> ring = &priv->rx_rings[i];
> napi_disable(&ring->napi);
> + cancel_work_sync(&ring->dim.dim.work);
> }
>
> ring = &priv->rx_rings[DESC_INDEX];
> napi_disable(&ring->napi);
> + cancel_work_sync(&ring->dim.dim.work);
> }
>
> static void bcmgenet_fini_rx_napi(struct bcmgenet_priv *priv)
> @@ -2557,6 +2628,7 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
> continue;
>
> rx_ring = &priv->rx_rings[index];
> + rx_ring->dim.event_ctr++;
>
> if (likely(napi_schedule_prep(&rx_ring->napi))) {
> rx_ring->int_disable(rx_ring);
> @@ -2601,6 +2673,7 @@ static irqreturn_t bcmgenet_isr0(int irq, void *dev_id)
>
> if (status & UMAC_IRQ_RXDMA_DONE) {
> rx_ring = &priv->rx_rings[DESC_INDEX];
> + rx_ring->dim.event_ctr++;
>
> if (likely(napi_schedule_prep(&rx_ring->napi))) {
> rx_ring->int_disable(rx_ring);
> diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.h b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> index 3c50431ccd2a..22c41e0430fb 100644
> --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> @@ -16,6 +16,7 @@
> #include <linux/mii.h>
> #include <linux/if_vlan.h>
> #include <linux/phy.h>
> +#include <linux/net_dim.h>
>
> /* total number of Buffer Descriptors, same for Rx/Tx */
> #define TOTAL_DESC 256
> @@ -572,6 +573,16 @@ struct bcmgenet_tx_ring {
> struct bcmgenet_priv *priv;
> };
>
> +struct bcmgenet_net_dim {
> + u16 use_dim;
> + u16 event_ctr;
> + unsigned long packets;
> + unsigned long bytes;
> + u32 coal_usecs;
> + u32 coal_pkts;
> + struct net_dim dim;
> +};
> +
> struct bcmgenet_rx_ring {
> struct napi_struct napi; /* Rx NAPI struct */
> unsigned long bytes;
> @@ -586,6 +597,7 @@ struct bcmgenet_rx_ring {
> unsigned int cb_ptr; /* Rx ring initial CB ptr */
> unsigned int end_ptr; /* Rx ring end CB ptr */
> unsigned int old_discards;
> + struct bcmgenet_net_dim dim;
> void (*int_enable)(struct bcmgenet_rx_ring *);
> void (*int_disable)(struct bcmgenet_rx_ring *);
> struct bcmgenet_priv *priv;
>
^ permalink raw reply
* Re: [PATCH v5 bpf-next 06/10] tracepoint: compute num_args at build time
From: Mathieu Desnoyers @ 2018-03-26 21:27 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: rostedt, David S. Miller, Daniel Borkmann, Linus Torvalds,
Peter Zijlstra, netdev, kernel-team, linux-api, Frank Ch. Eigler
In-Reply-To: <17073efa-d833-7348-bef1-79376ad43bc6@fb.com>
----- On Mar 26, 2018, at 1:55 PM, Alexei Starovoitov ast@fb.com wrote:
[...]
>
> correct. this set deals with in-kernel tracepoints only.
> No attempt to do anything with tracepoints inside modules.
Please endeavor to handle in-module tracepoints properly, then we'll
be able to pursue a more constructive discussion.
[...]
> Also I hope you noticed that the patch is doing:
> +++ b/include/linux/tracepoint-defs.h
> @@ -33,6 +33,7 @@ struct tracepoint {
> int (*regfunc)(void);
> void (*unregfunc)(void);
> struct tracepoint_func __rcu *funcs;
> + u32 num_args;
> };
>
> To make sure that bpf programs are safe I need to do a static check
> in the verifier that programs don't access arguments beyond
> those specified by the tracepoint.
>
> That was mentioned in the commit log of patch 6 too:
> "
> compute number of arguments passed into tracepoint
> at compile time and store it as part of 'struct tracepoint'.
> The number is necessary to check safety of bpf program access that
> is coming in subsequent patch.
> "
This part of the patch and its associated changelog is fine
with me. Please submit it as a separate commit from the rest
of the tracepoint.{c,h} changes.
[...]
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply
* Re: [PATCH v2 iproute2-next 3/6] rdma: Add CM_ID resource tracking information
From: Steve Wise @ 2018-03-26 21:34 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: David Ahern, leon, stephen, netdev, linux-rdma
In-Reply-To: <20180326211505.GF15554@ziepe.ca>
On 3/26/2018 4:15 PM, Jason Gunthorpe wrote:
> On Mon, Mar 26, 2018 at 09:30:41AM -0500, Steve Wise wrote:
>>
>> On 3/26/2018 9:17 AM, David Ahern wrote:
>>> On 2/27/18 9:07 AM, Steve Wise wrote:
>>>> diff --git a/rdma/rdma.h b/rdma/rdma.h
>>>> index 5809f70..e55205b 100644
>>>> +++ b/rdma/rdma.h
>>>> @@ -18,10 +18,12 @@
>>>> #include <libmnl/libmnl.h>
>>>> #include <rdma/rdma_netlink.h>
>>>> #include <time.h>
>>>> +#include <net/if_arp.h>
>>>>
>>>> #include "list.h"
>>>> #include "utils.h"
>>>> #include "json_writer.h"
>>>> +#include <rdma/rdma_cma.h>
>>>>
>>> did you forget to add rdma_cma.h? I don't see that file in my repo.
>> It is provided by the rdma-core package, upon which rdma tool now
>> depends for the rdma_port_space enum.
> It is a kernel bug that enum is not in an include/uapi/rdma header
>
> Fix it there and don't try to use rdma-core headers to get kernel ABI.
>
> Jason
I wish you'd commented on this just a little sooner. I just resent v3
of this series... with rdma_cma.h included. :)
How about the restrack/nldev code just translates the port space from
enum rdma_port_space to a new ABI enum, say nldev_rdma_port_space, that
i add to rdma_netlink.h? I'd hate to open the can of worms of trying to
split rdma_cma.h into uabi and no uabi headers. :(
Steve.
^ permalink raw reply
* Re: [PATCH net-next 1/2] net: systemport: Implement adaptive interrupt coalescing
From: Florian Fainelli @ 2018-03-26 21:36 UTC (permalink / raw)
To: Tal Gilboa, netdev
Cc: davem, jaedon.shin, pgynther, opendmb, michal.chan, gospo, saeedm
In-Reply-To: <58becbf1-baea-a679-ed32-b58fa5fd24bd@mellanox.com>
On 03/26/2018 02:22 PM, Tal Gilboa wrote:
> On 3/23/2018 4:19 AM, Florian Fainelli wrote:
>> Implement support for adaptive RX and TX interrupt coalescing using
>> net_dim. We have each of our TX ring and our single RX ring implement a
>> bcm_sysport_net_dim structure which holds an interrupt counter, number
>> of packets, bytes, and a container for a net_dim instance.
>>
>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>> ---
>> drivers/net/ethernet/broadcom/bcmsysport.c | 141
>> ++++++++++++++++++++++++++---
>> drivers/net/ethernet/broadcom/bcmsysport.h | 14 +++
>> 2 files changed, 140 insertions(+), 15 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c
>> b/drivers/net/ethernet/broadcom/bcmsysport.c
>> index f15a8fc6dfc9..5a5a726bafa4 100644
>> --- a/drivers/net/ethernet/broadcom/bcmsysport.c
>> +++ b/drivers/net/ethernet/broadcom/bcmsysport.c
>> @@ -15,6 +15,7 @@
>> #include <linux/module.h>
>> #include <linux/kernel.h>
>> #include <linux/netdevice.h>
>> +#include <linux/net_dim.h>
>
> I don't think you need this include. You already include net_dim in
> bcmsysport.h and include the bcmsysport.h here.
Indeed.
>
>> #include <linux/etherdevice.h>
>> #include <linux/platform_device.h>
>> #include <linux/of.h>
>> @@ -574,21 +575,55 @@ static int bcm_sysport_set_wol(struct net_device
>> *dev,
>> return 0;
>> }
>> +static void bcm_sysport_set_rx_coalesce(struct bcm_sysport_priv *priv)
>> +{
>> + u32 reg;
>> +
>> + reg = rdma_readl(priv, RDMA_MBDONE_INTR);
>> + reg &= ~(RDMA_INTR_THRESH_MASK |
>> + RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
>> + reg |= priv->dim.coal_pkts;
>> + reg |= DIV_ROUND_UP(priv->dim.coal_usecs * 1000, 8192) <<
>> + RDMA_TIMEOUT_SHIFT;
>> + rdma_writel(priv, reg, RDMA_MBDONE_INTR);
>> +}
>> +
>> +static void bcm_sysport_set_tx_coalesce(struct bcm_sysport_tx_ring
>> *ring)
>> +{
>> + struct bcm_sysport_priv *priv = ring->priv;
>> + u32 reg;
>> +
>> + reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(ring->index));
>> + reg &= ~(RING_INTR_THRESH_MASK |
>> + RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
>> + reg |= ring->dim.coal_pkts;
>> + reg |= DIV_ROUND_UP(ring->dim.coal_usecs * 1000, 8192) <<
>> + RING_TIMEOUT_SHIFT;
>> + tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(ring->index));
>> +}
>> +
>
> I wouldn't couple these functions with dim. This implies dim is always
> used. IMO, would be more clear to use a generic method which takes usecs
> and packets as an argument.
I did not want to create an additional structure for storing coalescing
parameters, but if you prefer I make this function take two parameters,
that sounds entirely reasonable.
>
>> static int bcm_sysport_get_coalesce(struct net_device *dev,
>> struct ethtool_coalesce *ec)
>> {
>> struct bcm_sysport_priv *priv = netdev_priv(dev);
>> + struct bcm_sysport_tx_ring *ring;
>> + unsigned int i;
>> u32 reg;
>> reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(0));
>> ec->tx_coalesce_usecs = (reg >> RING_TIMEOUT_SHIFT) * 8192 /
>> 1000;
>> ec->tx_max_coalesced_frames = reg & RING_INTR_THRESH_MASK;
>> + for (i = 0; i < dev->num_tx_queues; i++) {
>> + ring = &priv->tx_rings[i];
>> + ec->use_adaptive_tx_coalesce |= ring->dim.use_dim;
>> + }
>> reg = rdma_readl(priv, RDMA_MBDONE_INTR);
>> ec->rx_coalesce_usecs = (reg >> RDMA_TIMEOUT_SHIFT) * 8192 /
>> 1000;
>> ec->rx_max_coalesced_frames = reg & RDMA_INTR_THRESH_MASK;
>> + ec->use_adaptive_rx_coalesce = priv->dim.use_dim;
>> return 0;
>> }
>> @@ -597,8 +632,8 @@ static int bcm_sysport_set_coalesce(struct
>> net_device *dev,
>> struct ethtool_coalesce *ec)
>> {
>> struct bcm_sysport_priv *priv = netdev_priv(dev);
>> + struct bcm_sysport_tx_ring *ring;
>> unsigned int i;
>> - u32 reg;
>> /* Base system clock is 125Mhz, DMA timeout is this reference
>> clock
>> * divided by 1024, which yield roughly 8.192 us, our maximum
>> value has
>> @@ -615,22 +650,26 @@ static int bcm_sysport_set_coalesce(struct
>> net_device *dev,
>> return -EINVAL;
>> for (i = 0; i < dev->num_tx_queues; i++) {
>> - reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(i));
>> - reg &= ~(RING_INTR_THRESH_MASK |
>> - RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
>> - reg |= ec->tx_max_coalesced_frames;
>> - reg |= DIV_ROUND_UP(ec->tx_coalesce_usecs * 1000, 8192) <<
>> - RING_TIMEOUT_SHIFT;
>> - tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(i));
>> + ring = &priv->tx_rings[i];
>> + ring->dim.coal_pkts = ec->tx_max_coalesced_frames;
>> + ring->dim.coal_usecs = ec->tx_coalesce_usecs;
>> + if (!ec->use_adaptive_tx_coalesce && ring->dim.use_dim) {
>> + ring->dim.coal_pkts = 1;
>> + ring->dim.coal_usecs = 0;
>> + }
>> + ring->dim.use_dim = ec->use_adaptive_tx_coalesce;
>> + bcm_sysport_set_tx_coalesce(ring);
>> }
>
> If I understand correctly, if I disable dim, moderation is set to
> {usecs,packets}={0,1} regardless of the input from ethtool right?
Correct, these are the default coalescing parameters that the driver
sets. As mentioned before, since I am not storing any coalescing
parameters other than these two, there is no copy of what an user might
have previously provided, falling back to the defaults seemed reasonable.
> Doesn't this break the wanted behavior? As mentioned above, I would
> decouple dim from the set_tx/rx_coalesce() function. Also, when dim is
> enabled, why change dim.coal_pkts/usecs? They would just be overwritten
> in the next iteration of net_dim.
Indeed, that is not necessary.
>
>> - reg = rdma_readl(priv, RDMA_MBDONE_INTR);
>> - reg &= ~(RDMA_INTR_THRESH_MASK |
>> - RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
>> - reg |= ec->rx_max_coalesced_frames;
>> - reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192) <<
>> - RDMA_TIMEOUT_SHIFT;
>> - rdma_writel(priv, reg, RDMA_MBDONE_INTR);
>> + priv->dim.coal_usecs = ec->rx_coalesce_usecs;
>> + priv->dim.coal_pkts = ec->rx_max_coalesced_frames;
>> +
>> + if (!ec->use_adaptive_rx_coalesce && priv->dim.use_dim) {
>> + priv->dim.coal_pkts = 1;
>> + priv->dim.coal_usecs = 0;
>> + }
>> + priv->dim.use_dim = ec->use_adaptive_rx_coalesce;
>> + bcm_sysport_set_rx_coalesce(priv);
>
> Same comment as above.
>
>> return 0;
>> }
>> @@ -709,6 +748,7 @@ static unsigned int bcm_sysport_desc_rx(struct
>> bcm_sysport_priv *priv,
>> struct bcm_sysport_stats64 *stats64 = &priv->stats64;
>> struct net_device *ndev = priv->netdev;
>> unsigned int processed = 0, to_process;
>> + unsigned int processed_bytes = 0;
>> struct bcm_sysport_cb *cb;
>> struct sk_buff *skb;
>> unsigned int p_index;
>> @@ -800,6 +840,7 @@ static unsigned int bcm_sysport_desc_rx(struct
>> bcm_sysport_priv *priv,
>> */
>> skb_pull(skb, sizeof(*rsb) + 2);
>> len -= (sizeof(*rsb) + 2);
>> + processed_bytes += len;
>> /* UniMAC may forward CRC */
>> if (priv->crc_fwd) {
>> @@ -824,6 +865,9 @@ static unsigned int bcm_sysport_desc_rx(struct
>> bcm_sysport_priv *priv,
>> priv->rx_read_ptr = 0;
>> }
>> + priv->dim.packets = processed;
>> + priv->dim.bytes = processed_bytes;
>> +
>> return processed;
>> }
>> @@ -900,6 +944,8 @@ static unsigned int
>> __bcm_sysport_tx_reclaim(struct bcm_sysport_priv *priv,
>> ring->packets += pkts_compl;
>> ring->bytes += bytes_compl;
>> u64_stats_update_end(&priv->syncp);
>> + ring->dim.packets = pkts_compl;
>> + ring->dim.bytes = bytes_compl;
>> ring->c_index = c_index;
>> @@ -945,6 +991,7 @@ static int bcm_sysport_tx_poll(struct
>> napi_struct *napi, int budget)
>> {
>> struct bcm_sysport_tx_ring *ring =
>> container_of(napi, struct bcm_sysport_tx_ring, napi);
>> + struct net_dim_sample dim_sample;
>> unsigned int work_done = 0;
>> work_done = bcm_sysport_tx_reclaim(ring->priv, ring);
>> @@ -961,6 +1008,12 @@ static int bcm_sysport_tx_poll(struct
>> napi_struct *napi, int budget)
>> return 0;
>> }
>> + if (ring->dim.use_dim) {
>> + net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
>> + ring->dim.bytes, &dim_sample);
>> + net_dim(&ring->dim.dim, dim_sample);
>> + }
>> +
>> return budget;
>> }
>> @@ -976,6 +1029,7 @@ static int bcm_sysport_poll(struct napi_struct
>> *napi, int budget)
>> {
>> struct bcm_sysport_priv *priv =
>> container_of(napi, struct bcm_sysport_priv, napi);
>> + struct net_dim_sample dim_sample;
>> unsigned int work_done = 0;
>> work_done = bcm_sysport_desc_rx(priv, budget);
>> @@ -998,6 +1052,12 @@ static int bcm_sysport_poll(struct napi_struct
>> *napi, int budget)
>> intrl2_0_mask_clear(priv, INTRL2_0_RDMA_MBDONE);
>> }
>> + if (priv->dim.use_dim) {
>> + net_dim_sample(priv->dim.event_ctr, priv->dim.packets,
>> + priv->dim.bytes, &dim_sample);
>> + net_dim(&priv->dim.dim, dim_sample);
>> + }
>> +
>> return work_done;
>> }
>> @@ -1016,6 +1076,40 @@ static void
>> bcm_sysport_resume_from_wol(struct bcm_sysport_priv *priv)
>> netif_dbg(priv, wol, priv->netdev, "resumed from WOL\n");
>> }
>> +static void bcm_sysport_dim_work(struct work_struct *work)
>> +{
>> + struct net_dim *dim = container_of(work, struct net_dim, work);
>> + struct bcm_sysport_net_dim *ndim =
>> + container_of(dim, struct bcm_sysport_net_dim, dim);
>> + struct bcm_sysport_priv *priv =
>> + container_of(ndim, struct bcm_sysport_priv, dim);
>> + struct net_dim_cq_moder cur_profile =
>> + net_dim_get_profile(dim->mode, dim->profile_ix);
>> +
>> + priv->dim.coal_usecs = cur_profile.usec;
>> + priv->dim.coal_pkts = cur_profile.pkts;
>> +
>> + bcm_sysport_set_rx_coalesce(priv);
>> + dim->state = NET_DIM_START_MEASURE;
>> +}
>> +
>> +static void bcm_sysport_dim_tx_work(struct work_struct *work)
>> +{
>> + struct net_dim *dim = container_of(work, struct net_dim, work);
>> + struct bcm_sysport_net_dim *ndim =
>> + container_of(dim, struct bcm_sysport_net_dim, dim);
>> + struct bcm_sysport_tx_ring *ring =
>> + container_of(ndim, struct bcm_sysport_tx_ring, dim);
>> + struct net_dim_cq_moder cur_profile =
>> + net_dim_get_profile(dim->mode, dim->profile_ix);
>> +
>> + ring->dim.coal_usecs = cur_profile.usec;
>> + ring->dim.coal_pkts = cur_profile.pkts;
>> +
>> + bcm_sysport_set_tx_coalesce(ring);
>> + dim->state = NET_DIM_START_MEASURE;
>> +}
>> +
>> /* RX and misc interrupt routine */
>> static irqreturn_t bcm_sysport_rx_isr(int irq, void *dev_id)
>> {
>> @@ -1034,6 +1128,7 @@ static irqreturn_t bcm_sysport_rx_isr(int irq,
>> void *dev_id)
>> }
>> if (priv->irq0_stat & INTRL2_0_RDMA_MBDONE) {
>> + priv->dim.event_ctr++;
>> if (likely(napi_schedule_prep(&priv->napi))) {
>> /* disable RX interrupts */
>> intrl2_0_mask_set(priv, INTRL2_0_RDMA_MBDONE);
>> @@ -1061,6 +1156,7 @@ static irqreturn_t bcm_sysport_rx_isr(int irq,
>> void *dev_id)
>> continue;
>> txr = &priv->tx_rings[ring];
>> + txr->dim.event_ctr++;
>> if (likely(napi_schedule_prep(&txr->napi))) {
>> intrl2_0_mask_set(priv, ring_bit);
>> @@ -1093,6 +1189,7 @@ static irqreturn_t bcm_sysport_tx_isr(int irq,
>> void *dev_id)
>> continue;
>> txr = &priv->tx_rings[ring];
>> + txr->dim.event_ctr++;
>> if (likely(napi_schedule_prep(&txr->napi))) {
>> intrl2_1_mask_set(priv, BIT(ring));
>> @@ -1358,6 +1455,16 @@ static void bcm_sysport_adj_link(struct
>> net_device *dev)
>> phy_print_status(phydev);
>> }
>> +static void bcm_sysport_init_dim(struct bcm_sysport_net_dim *dim,
>> + void (*cb)(struct work_struct *work))
>> +{
>> + INIT_WORK(&dim->dim.work, cb);
>> + dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
>> + dim->event_ctr = 0;
>> + dim->packets = 0;
>> + dim->bytes = 0;
>> +}
>
> What about default values for coal_usecs/pkts? dim supports it through
> net_dim_get_def_profile(mode) function.
OK, thanks I did not know that.
--
Florian
^ permalink raw reply
* [net-next 01/10] i40e: move I40E_FLAG_FILTER_SYNC to a state bit
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
The I40E_FLAG_FILTER_SYNC flag is modified during run time possibly when
the RTNL lock is not held. Thus, it should not be part of pf->flags, and
instead should be using atomic bit operations in the pf->state field.
Create a __I40E_MACVLAN_SYNC_PENDING state bit, and use it instead of
the old I40E_FLAG_FILTER_SYNC flag.
This is part of a larger effort to remove the need for cmpxchg64 in
i40e_set_priv_flags().
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 3 ++-
drivers/net/ethernet/intel/i40e/i40e_main.c | 16 +++++++++-------
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 1d33a8b3ef54..52f99142244b 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -162,6 +162,7 @@ enum i40e_state_t {
__I40E_RESET_FAILED,
__I40E_PORT_SUSPENDED,
__I40E_VF_DISABLE,
+ __I40E_MACVLAN_SYNC_PENDING,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
@@ -516,7 +517,7 @@ struct i40e_pf {
#define I40E_FLAG_MSIX_ENABLED BIT_ULL(2)
#define I40E_FLAG_RSS_ENABLED BIT_ULL(3)
#define I40E_FLAG_VMDQ_ENABLED BIT_ULL(4)
-#define I40E_FLAG_FILTER_SYNC BIT_ULL(5)
+/* Gap for BIT_ULL(5) */
#define I40E_FLAG_SRIOV_ENABLED BIT_ULL(6)
#define I40E_FLAG_DCB_CAPABLE BIT_ULL(7)
#define I40E_FLAG_DCB_ENABLED BIT_ULL(8)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 536ed8e8a96f..7fc2c6d89637 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -1382,7 +1382,7 @@ struct i40e_mac_filter *i40e_add_filter(struct i40e_vsi *vsi,
hash_add(vsi->mac_filter_hash, &f->hlist, key);
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->back->state);
}
/* If we're asked to add a filter that has been marked for removal, it
@@ -1432,7 +1432,7 @@ void __i40e_del_filter(struct i40e_vsi *vsi, struct i40e_mac_filter *f)
}
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->state);
}
/**
@@ -1955,7 +1955,7 @@ static void i40e_set_rx_mode(struct net_device *netdev)
/* check for other flag changes */
if (vsi->current_netdev_flags != vsi->netdev->flags) {
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->back->state);
}
}
@@ -2577,9 +2577,10 @@ static void i40e_sync_filters_subtask(struct i40e_pf *pf)
{
int v;
- if (!pf || !(pf->flags & I40E_FLAG_FILTER_SYNC))
+ if (!pf)
+ return;
+ if (!test_and_clear_bit(__I40E_MACVLAN_SYNC_PENDING, pf->state))
return;
- pf->flags &= ~I40E_FLAG_FILTER_SYNC;
for (v = 0; v < pf->num_alloc_vsi; v++) {
if (pf->vsi[v] &&
@@ -2588,7 +2589,8 @@ static void i40e_sync_filters_subtask(struct i40e_pf *pf)
if (ret) {
/* come back and try again later */
- pf->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING,
+ pf->state);
break;
}
}
@@ -12240,7 +12242,7 @@ static int i40e_add_vsi(struct i40e_vsi *vsi)
if (f_count) {
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- pf->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, pf->state);
}
/* Update VSI BW information */
--
2.14.3
^ permalink raw reply related
* [net-next 00/10][pull request] 40GbE Intel Wired LAN Driver Updates 2018-03-26
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jeff Kirsher, netdev, nhorman, sassmann, jogreene
This series contains updates to i40e only.
Jake provides several patches which remove the need for cmpxchg64(),
starting with moving I40E_FLAG_[UDP]_FILTER_SYNC from pf->flags to pf->state
since they are modified during run time possibly when the RTNL lock is not
held so they should be a state bits and not flags. Moved additional
"flags" which should be state fields, into pf->state. Ensure we hold
the RTNL lock for the entire sequence of preparing for reset and when
resuming, which will protect the flags related to interrupt scheme under
RTNL lock so that their modification is properly threaded. Finally,
cleanup the use of cmpxchg64() since it is no longer needed. Cleaned up
the holes in the feature flags created my moving some flags to the state
field.
Björn Töpel adds XDP_REDIRECT support as well as tweaking the page
counting for XDP_REDIRECT so that it will function properly.
The following are changes since commit 996bfed118748c128ad4b6c05c09fd2f5fdfa1b4:
Merge tag 'wireless-drivers-next-for-davem-2018-03-24' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next
and are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue 40GbE
Björn Töpel (2):
i40e: tweak page counting for XDP_REDIRECT
i40e: add support for XDP_REDIRECT
Jacob Keller (8):
i40e: move I40E_FLAG_FILTER_SYNC to a state bit
i40e: move I40E_FLAG_UDP_FILTER_SYNC to the state field
i40e: move AUTO_DISABLED flags into the state field
i40e: move I40E_FLAG_TEMP_LINK_POLLING to state field
i40e: move client flags into state bits
i40e: hold the RTNL lock while changing interrupt schemes
i40e: stop using cmpxchg flow in i40e_set_priv_flags()
i40e: re-number feature flags to remove gaps
drivers/net/ethernet/intel/i40e/i40e.h | 68 ++++++++--------
drivers/net/ethernet/intel/i40e/i40e_client.c | 7 +-
drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 23 ++----
drivers/net/ethernet/intel/i40e/i40e_main.c | 99 ++++++++++++-----------
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 104 +++++++++++++++++++------
drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2 +
6 files changed, 181 insertions(+), 122 deletions(-)
--
2.14.3
^ permalink raw reply
* [net-next 04/10] i40e: move I40E_FLAG_TEMP_LINK_POLLING to state field
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
This flag is modified outside of the RTNL lock and thus should not be
part of the pf->flags variable.
Use a state bit instead, so that we can use atomic bit operations.
This is part of a larger effort to remove cmpxchg64 in
i40e_set_priv_flags()
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 3 ++-
drivers/net/ethernet/intel/i40e/i40e_main.c | 7 +++----
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 38633563dacd..1ffe802d489f 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -166,6 +166,7 @@ enum i40e_state_t {
__I40E_VF_DISABLE,
__I40E_MACVLAN_SYNC_PENDING,
__I40E_UDP_FILTER_SYNC_PENDING,
+ __I40E_TEMP_LINK_POLLING,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
@@ -534,7 +535,7 @@ struct i40e_pf {
#define I40E_FLAG_VEB_STATS_ENABLED BIT_ULL(17)
#define I40E_FLAG_LINK_POLLING_ENABLED BIT_ULL(18)
#define I40E_FLAG_TRUE_PROMISC_SUPPORT BIT_ULL(19)
-#define I40E_FLAG_TEMP_LINK_POLLING BIT_ULL(20)
+/* Gap for BIT_ULL(20) */
#define I40E_FLAG_LEGACY_RX BIT_ULL(21)
#define I40E_FLAG_PTP BIT_ULL(22)
#define I40E_FLAG_IWARP_ENABLED BIT_ULL(23)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index a478153818bc..307f5bf65708 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -8432,13 +8432,12 @@ static void i40e_link_event(struct i40e_pf *pf)
/* On success, disable temp link polling */
if (status == I40E_SUCCESS) {
- if (pf->flags & I40E_FLAG_TEMP_LINK_POLLING)
- pf->flags &= ~I40E_FLAG_TEMP_LINK_POLLING;
+ clear_bit(__I40E_TEMP_LINK_POLLING, pf->state);
} else {
/* Enable link polling temporarily until i40e_get_link_status
* returns I40E_SUCCESS
*/
- pf->flags |= I40E_FLAG_TEMP_LINK_POLLING;
+ set_bit(__I40E_TEMP_LINK_POLLING, pf->state);
dev_dbg(&pf->pdev->dev, "couldn't get link state, status: %d\n",
status);
return;
@@ -8490,7 +8489,7 @@ static void i40e_watchdog_subtask(struct i40e_pf *pf)
pf->service_timer_previous = jiffies;
if ((pf->flags & I40E_FLAG_LINK_POLLING_ENABLED) ||
- (pf->flags & I40E_FLAG_TEMP_LINK_POLLING))
+ test_bit(__I40E_TEMP_LINK_POLLING, pf->state))
i40e_link_event(pf);
/* Update the stats for active netdevs so the network stack
--
2.14.3
^ permalink raw reply related
* [net-next 03/10] i40e: move AUTO_DISABLED flags into the state field
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
The two Flow Directory auto disable flags are used at run time to mark
when the flow director features needed to be disabled. Thus the flags
could change even when the RTNL lock is not held.
They also have some code constructions which really should be
test_and_set or test_and_clear using atomic bit operations.
Create new state fields to mark this, and stop including them in
pf->flags.
This is part of a larger effort to remove the need for cmpxchg64 in
i40e_set_priv_flags().
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 5 +++--
drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 4 ++--
drivers/net/ethernet/intel/i40e/i40e_main.c | 23 +++++++++--------------
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 21 +++++++++++++--------
4 files changed, 27 insertions(+), 26 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index f557c3c36080..38633563dacd 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -159,6 +159,8 @@ enum i40e_state_t {
__I40E_BAD_EEPROM,
__I40E_DOWN_REQUESTED,
__I40E_FD_FLUSH_REQUESTED,
+ __I40E_FD_ATR_AUTO_DISABLED,
+ __I40E_FD_SB_AUTO_DISABLED,
__I40E_RESET_FAILED,
__I40E_PORT_SUSPENDED,
__I40E_VF_DISABLE,
@@ -524,8 +526,7 @@ struct i40e_pf {
#define I40E_FLAG_DCB_ENABLED BIT_ULL(8)
#define I40E_FLAG_FD_SB_ENABLED BIT_ULL(9)
#define I40E_FLAG_FD_ATR_ENABLED BIT_ULL(10)
-#define I40E_FLAG_FD_SB_AUTO_DISABLED BIT_ULL(11)
-#define I40E_FLAG_FD_ATR_AUTO_DISABLED BIT_ULL(12)
+/* Gap for BIT_ULL(11) and BIT_ULL(12) */
#define I40E_FLAG_MFP_ENABLED BIT_ULL(13)
/* Gap for BIT_ULL(14) */
#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT_ULL(15)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 846a9d597e01..2a9c93091e3d 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -3951,7 +3951,7 @@ static int i40e_add_fdir_ethtool(struct i40e_vsi *vsi,
if (!(pf->flags & I40E_FLAG_FD_SB_ENABLED))
return -EOPNOTSUPP;
- if (pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED)
+ if (test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
return -ENOSPC;
if (test_bit(__I40E_RESET_RECOVERY_PENDING, pf->state) ||
@@ -4460,7 +4460,7 @@ static int i40e_set_priv_flags(struct net_device *dev, u32 flags)
/* Flush current ATR settings if ATR was disabled */
if ((changed_flags & I40E_FLAG_FD_ATR_ENABLED) &&
!(pf->flags & I40E_FLAG_FD_ATR_ENABLED)) {
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
set_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
}
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 3177e82059e6..a478153818bc 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -1083,13 +1083,13 @@ static void i40e_update_pf_stats(struct i40e_pf *pf)
&osd->rx_lpi_count, &nsd->rx_lpi_count);
if (pf->flags & I40E_FLAG_FD_SB_ENABLED &&
- !(pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED))
+ !test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
nsd->fd_sb_status = true;
else
nsd->fd_sb_status = false;
if (pf->flags & I40E_FLAG_FD_ATR_ENABLED &&
- !(pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED))
+ !test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
nsd->fd_atr_status = true;
else
nsd->fd_atr_status = false;
@@ -8144,12 +8144,10 @@ u32 i40e_get_global_fd_count(struct i40e_pf *pf)
**/
static void i40e_reenable_fdir_sb(struct i40e_pf *pf)
{
- if (pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED) {
- pf->flags &= ~I40E_FLAG_FD_SB_AUTO_DISABLED;
+ if (test_and_clear_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
if ((pf->flags & I40E_FLAG_FD_SB_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "FD Sideband/ntuple is being enabled since we have space in the table now\n");
- }
}
/**
@@ -8158,7 +8156,7 @@ static void i40e_reenable_fdir_sb(struct i40e_pf *pf)
**/
static void i40e_reenable_fdir_atr(struct i40e_pf *pf)
{
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED) {
+ if (test_and_clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state)) {
/* ATR uses the same filtering logic as SB rules. It only
* functions properly if the input set mask is at the default
* settings. It is safe to restore the default input set
@@ -8168,7 +8166,6 @@ static void i40e_reenable_fdir_atr(struct i40e_pf *pf)
I40E_L3_SRC_MASK | I40E_L3_DST_MASK |
I40E_L4_SRC_MASK | I40E_L4_DST_MASK);
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "ATR is being enabled since we have space in the table and there are no conflicting ntuple rules\n");
@@ -8291,7 +8288,7 @@ static void i40e_fdir_flush_and_replay(struct i40e_pf *pf)
}
pf->fd_flush_timestamp = jiffies;
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
/* flush all filters */
wr32(&pf->hw, I40E_PFQF_CTL_1,
I40E_PFQF_CTL_1_CLEARFDTABLE_MASK);
@@ -8311,7 +8308,7 @@ static void i40e_fdir_flush_and_replay(struct i40e_pf *pf)
/* replay sideband filters */
i40e_fdir_filter_restore(pf->vsi[pf->lan_vsi]);
if (!disable_atr && !pf->fd_tcp4_filter_cnt)
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
clear_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
if (I40E_DEBUG_FD & pf->hw.debug_mask)
dev_info(&pf->pdev->dev, "FD Filter table flushed and FD-SB replayed.\n");
@@ -11291,20 +11288,18 @@ bool i40e_set_ntuple(struct i40e_pf *pf, netdev_features_t features)
need_reset = true;
i40e_fdir_filter_exit(pf);
}
- pf->flags &= ~(I40E_FLAG_FD_SB_ENABLED |
- I40E_FLAG_FD_SB_AUTO_DISABLED);
+ pf->flags &= ~I40E_FLAG_FD_SB_ENABLED;
+ clear_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state);
pf->flags |= I40E_FLAG_FD_SB_INACTIVE;
/* reset fd counters */
pf->fd_add_err = 0;
pf->fd_atr_cnt = 0;
/* if ATR was auto disabled it can be re-enabled. */
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED) {
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ if (test_and_clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "ATR re-enabled.\n");
- }
}
return need_reset;
}
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 7ccd05bf4b06..797bcdd3504e 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -336,7 +336,7 @@ static int i40e_add_del_fdir_tcpv4(struct i40e_vsi *vsi,
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
I40E_DEBUG_FD & pf->hw.debug_mask)
dev_info(&pf->pdev->dev, "Forcing ATR off, sideband rules for TCP/IPv4 flow being applied\n");
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
} else {
pf->fd_tcp4_filter_cnt--;
}
@@ -594,8 +594,14 @@ static void i40e_fd_handle_status(struct i40e_ring *rx_ring,
pf->fd_atr_cnt = i40e_get_current_atr_cnt(pf);
if ((rx_desc->wb.qword0.hi_dword.fd_id == 0) &&
- pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED) {
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state)) {
+ /* These set_bit() calls aren't atomic with the
+ * test_bit() here, but worse case we potentially
+ * disable ATR and queue a flush right after SB
+ * support is re-enabled. That shouldn't cause an
+ * issue in practice
+ */
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
set_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
}
@@ -608,11 +614,10 @@ static void i40e_fd_handle_status(struct i40e_ring *rx_ring,
*/
if (fcnt_prog >= (fcnt_avail - I40E_FDIR_BUFFER_FULL_MARGIN)) {
if ((pf->flags & I40E_FLAG_FD_SB_ENABLED) &&
- !(pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED)) {
- pf->flags |= I40E_FLAG_FD_SB_AUTO_DISABLED;
+ !test_and_set_bit(__I40E_FD_SB_AUTO_DISABLED,
+ pf->state))
if (I40E_DEBUG_FD & pf->hw.debug_mask)
dev_warn(&pdev->dev, "FD filter space full, new ntuple rules will not be added\n");
- }
}
} else if (error == BIT(I40E_RX_PROG_STATUS_DESC_NO_FD_ENTRY_SHIFT)) {
if (I40E_DEBUG_FD & pf->hw.debug_mask)
@@ -2647,7 +2652,7 @@ static void i40e_atr(struct i40e_ring *tx_ring, struct sk_buff *skb,
if (!(pf->flags & I40E_FLAG_FD_ATR_ENABLED))
return;
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED)
+ if (test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
return;
/* if sampling is disabled do nothing */
@@ -2687,7 +2692,7 @@ static void i40e_atr(struct i40e_ring *tx_ring, struct sk_buff *skb,
th = (struct tcphdr *)(hdr.network + hlen);
/* Due to lack of space, no more new filters can be programmed */
- if (th->syn && (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED))
+ if (th->syn && test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
return;
if (pf->flags & I40E_FLAG_HW_ATR_EVICT_ENABLED) {
/* HW ATR eviction will take care of removing filters on FIN
--
2.14.3
^ permalink raw reply related
* [net-next 02/10] i40e: move I40E_FLAG_UDP_FILTER_SYNC to the state field
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
This flag is modified during run time, possibly even when the RTNL lock
is not held. Additionally it has a few places which should be using
test_and_set or test_and_clear atomic bit operations.
Create a new state bit __I40E_UDP_SYNC_PENDING and use it instead of the
ole I40E_FLAG_UDP_FILTER_SYNC flag.
This is part of a larger effort to remove the need for using cmpxchg64
in i40e_set_priv_flags.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 3 ++-
drivers/net/ethernet/intel/i40e/i40e_main.c | 10 ++++------
2 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 52f99142244b..f557c3c36080 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -163,6 +163,7 @@ enum i40e_state_t {
__I40E_PORT_SUSPENDED,
__I40E_VF_DISABLE,
__I40E_MACVLAN_SYNC_PENDING,
+ __I40E_UDP_FILTER_SYNC_PENDING,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
@@ -526,7 +527,7 @@ struct i40e_pf {
#define I40E_FLAG_FD_SB_AUTO_DISABLED BIT_ULL(11)
#define I40E_FLAG_FD_ATR_AUTO_DISABLED BIT_ULL(12)
#define I40E_FLAG_MFP_ENABLED BIT_ULL(13)
-#define I40E_FLAG_UDP_FILTER_SYNC BIT_ULL(14)
+/* Gap for BIT_ULL(14) */
#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT_ULL(15)
#define I40E_FLAG_VEB_MODE_ENABLED BIT_ULL(16)
#define I40E_FLAG_VEB_STATS_ENABLED BIT_ULL(17)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 7fc2c6d89637..3177e82059e6 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -9721,7 +9721,7 @@ static void i40e_sync_udp_filters(struct i40e_pf *pf)
pf->pending_udp_bitmap |= BIT_ULL(i);
}
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
}
/**
@@ -9735,11 +9735,9 @@ static void i40e_sync_udp_filters_subtask(struct i40e_pf *pf)
u16 port;
int i;
- if (!(pf->flags & I40E_FLAG_UDP_FILTER_SYNC))
+ if (!test_and_clear_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state))
return;
- pf->flags &= ~I40E_FLAG_UDP_FILTER_SYNC;
-
for (i = 0; i < I40E_MAX_PF_UDP_OFFLOAD_PORTS; i++) {
if (pf->pending_udp_bitmap & BIT_ULL(i)) {
pf->pending_udp_bitmap &= ~BIT_ULL(i);
@@ -11439,7 +11437,7 @@ static void i40e_udp_tunnel_add(struct net_device *netdev,
/* New port: add it and mark its index in the bitmap */
pf->udp_ports[next_idx].port = port;
pf->pending_udp_bitmap |= BIT_ULL(next_idx);
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
}
/**
@@ -11480,7 +11478,7 @@ static void i40e_udp_tunnel_del(struct net_device *netdev,
*/
pf->udp_ports[idx].port = 0;
pf->pending_udp_bitmap |= BIT_ULL(idx);
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
return;
not_found:
--
2.14.3
^ permalink raw reply related
* [net-next 06/10] i40e: hold the RTNL lock while changing interrupt schemes
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
When we suspend and resume, we need to clear and re-enable the interrupt
scheme. This was previously not done while holding the RTNL lock, which
could be problematic, because we are actually destroying and re-creating
queues.
Hold the RTNL lock for the entire sequence of preparing for reset, and
when resuming. This additionally protects the flags related to interrupt
scheme under RTNL lock so that their modification is properly threaded.
This is part of a larger effort to remove the need for cmpxchg64 in
i40e_set_priv_flags().
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 68b51761c509..5efd6d7bfa59 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -14348,7 +14348,13 @@ static int __maybe_unused i40e_suspend(struct device *dev)
if (pf->wol_en && (pf->hw_features & I40E_HW_WOL_MC_MAGIC_PKT_WAKE))
i40e_enable_mc_magic_wake(pf);
- i40e_prep_for_reset(pf, false);
+ /* Since we're going to destroy queues during the
+ * i40e_clear_interrupt_scheme() we should hold the RTNL lock for this
+ * whole section
+ */
+ rtnl_lock();
+
+ i40e_prep_for_reset(pf, true);
wr32(hw, I40E_PFPM_APM, (pf->wol_en ? I40E_PFPM_APM_APME_MASK : 0));
wr32(hw, I40E_PFPM_WUFC, (pf->wol_en ? I40E_PFPM_WUFC_MAG_MASK : 0));
@@ -14360,6 +14366,8 @@ static int __maybe_unused i40e_suspend(struct device *dev)
*/
i40e_clear_interrupt_scheme(pf);
+ rtnl_unlock();
+
return 0;
}
@@ -14377,6 +14385,11 @@ static int __maybe_unused i40e_resume(struct device *dev)
if (!test_bit(__I40E_SUSPENDED, pf->state))
return 0;
+ /* We need to hold the RTNL lock prior to restoring interrupt schemes,
+ * since we're going to be restoring queues
+ */
+ rtnl_lock();
+
/* We cleared the interrupt scheme when we suspended, so we need to
* restore it now to resume device functionality.
*/
@@ -14387,7 +14400,9 @@ static int __maybe_unused i40e_resume(struct device *dev)
}
clear_bit(__I40E_DOWN, pf->state);
- i40e_reset_and_rebuild(pf, false, false);
+ i40e_reset_and_rebuild(pf, false, true);
+
+ rtnl_unlock();
/* Clear suspended state last after everything is recovered */
clear_bit(__I40E_SUSPENDED, pf->state);
--
2.14.3
^ permalink raw reply related
* [net-next 05/10] i40e: move client flags into state bits
From: Jeff Kirsher @ 2018-03-26 21:40 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
The iWarp client flags are all potentially changed when the RTNL lock is
not held, so they should not be part of the pf->flags variable. Instead,
move them into the state field so that we can use atomic bit operations.
This is part of a larger effort to remove cmpxchg64 in
i40e_set_priv_flags()
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 7 ++++---
drivers/net/ethernet/intel/i40e/i40e_client.c | 7 +++----
drivers/net/ethernet/intel/i40e/i40e_main.c | 22 ++++++++++------------
3 files changed, 17 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 1ffe802d489f..67518013ca4d 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -167,6 +167,9 @@ enum i40e_state_t {
__I40E_MACVLAN_SYNC_PENDING,
__I40E_UDP_FILTER_SYNC_PENDING,
__I40E_TEMP_LINK_POLLING,
+ __I40E_CLIENT_SERVICE_REQUESTED,
+ __I40E_CLIENT_L2_CHANGE,
+ __I40E_CLIENT_RESET,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
@@ -539,9 +542,7 @@ struct i40e_pf {
#define I40E_FLAG_LEGACY_RX BIT_ULL(21)
#define I40E_FLAG_PTP BIT_ULL(22)
#define I40E_FLAG_IWARP_ENABLED BIT_ULL(23)
-#define I40E_FLAG_SERVICE_CLIENT_REQUESTED BIT_ULL(24)
-#define I40E_FLAG_CLIENT_L2_CHANGE BIT_ULL(25)
-#define I40E_FLAG_CLIENT_RESET BIT_ULL(26)
+/* Gap for BIT_ULL(24) through BIT_ULL(26) */
#define I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED BIT_ULL(27)
#define I40E_FLAG_SOURCE_PRUNING_DISABLED BIT_ULL(28)
#define I40E_FLAG_TC_MQPRIO BIT_ULL(29)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_client.c b/drivers/net/ethernet/intel/i40e/i40e_client.c
index 999dea5a7c9e..d8ce4999864f 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_client.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_client.c
@@ -376,9 +376,8 @@ void i40e_client_subtask(struct i40e_pf *pf)
struct i40e_vsi *vsi = pf->vsi[pf->lan_vsi];
int ret = 0;
- if (!(pf->flags & I40E_FLAG_SERVICE_CLIENT_REQUESTED))
+ if (!test_and_clear_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state))
return;
- pf->flags &= ~I40E_FLAG_SERVICE_CLIENT_REQUESTED;
cdev = pf->cinst;
/* If we're down or resetting, just bail */
@@ -459,7 +458,7 @@ int i40e_lan_add_device(struct i40e_pf *pf)
* added, we can schedule a subtask to go initiate the clients if
* they can be launched at probe time.
*/
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
out:
@@ -554,7 +553,7 @@ static void i40e_client_prepare(struct i40e_client *client)
pf = ldev->pf;
i40e_client_add_instance(pf);
/* Start the client subtask */
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
}
mutex_unlock(&i40e_device_mutex);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 307f5bf65708..68b51761c509 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -2634,8 +2634,8 @@ static int i40e_change_mtu(struct net_device *netdev, int new_mtu)
netdev->mtu = new_mtu;
if (netif_running(netdev))
i40e_vsi_reinit_locked(vsi);
- pf->flags |= (I40E_FLAG_SERVICE_CLIENT_REQUESTED |
- I40E_FLAG_CLIENT_L2_CHANGE);
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
+ set_bit(__I40E_CLIENT_L2_CHANGE, pf->state);
return 0;
}
@@ -4722,9 +4722,9 @@ static void i40e_vsi_close(struct i40e_vsi *vsi)
i40e_vsi_free_tx_resources(vsi);
i40e_vsi_free_rx_resources(vsi);
vsi->current_netdev_flags = 0;
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
if (test_bit(__I40E_RESET_RECOVERY_PENDING, pf->state))
- pf->flags |= I40E_FLAG_CLIENT_RESET;
+ set_bit(__I40E_CLIENT_RESET, pf->state);
}
/**
@@ -6495,7 +6495,7 @@ static int i40e_up_complete(struct i40e_vsi *vsi)
/* On the next run of the service_task, notify any clients of the new
* opened netdev
*/
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
return 0;
@@ -8037,8 +8037,8 @@ static int i40e_handle_lldp_event(struct i40e_pf *pf,
i40e_service_event_schedule(pf);
} else {
i40e_pf_unquiesce_all_vsi(pf);
- pf->flags |= (I40E_FLAG_SERVICE_CLIENT_REQUESTED |
- I40E_FLAG_CLIENT_L2_CHANGE);
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
+ set_bit(__I40E_CLIENT_L2_CHANGE, pf->state);
}
exit:
@@ -9785,17 +9785,15 @@ static void i40e_service_task(struct work_struct *work)
i40e_vc_process_vflr_event(pf);
i40e_watchdog_subtask(pf);
i40e_fdir_reinit_subtask(pf);
- if (pf->flags & I40E_FLAG_CLIENT_RESET) {
+ if (test_and_clear_bit(__I40E_CLIENT_RESET, pf->state)) {
/* Client subtask will reopen next time through. */
i40e_notify_client_of_netdev_close(pf->vsi[pf->lan_vsi], true);
- pf->flags &= ~I40E_FLAG_CLIENT_RESET;
} else {
i40e_client_subtask(pf);
- if (pf->flags & I40E_FLAG_CLIENT_L2_CHANGE) {
+ if (test_and_clear_bit(__I40E_CLIENT_L2_CHANGE,
+ pf->state))
i40e_notify_client_of_l2_param_changes(
pf->vsi[pf->lan_vsi]);
- pf->flags &= ~I40E_FLAG_CLIENT_L2_CHANGE;
- }
}
i40e_sync_filters_subtask(pf);
i40e_sync_udp_filters_subtask(pf);
--
2.14.3
^ permalink raw reply related
* [net-next 09/10] i40e: tweak page counting for XDP_REDIRECT
From: Jeff Kirsher @ 2018-03-26 21:41 UTC (permalink / raw)
To: davem
Cc: Björn Töpel, netdev, nhorman, sassmann, jogreene,
Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Björn Töpel <bjorn.topel@intel.com>
This commit tweaks the page counting for XDP_REDIRECT to function
properly. XDP_REDIRECT support will be added in a future commit.
The current page counting scheme assumes that the reference count
cannot decrease until the received frame is sent to the upper layers
of the networking stack. This assumption does not hold for the
XDP_REDIRECT action, since a page (pointed out by xdp_buff) can have
its reference count decreased via the xdp_do_redirect call.
To work around that, we now start off by a large page count and then
don't allow a refcount less than two.
Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 797bcdd3504e..9c338cef8315 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -1588,9 +1588,8 @@ static bool i40e_alloc_mapped_page(struct i40e_ring *rx_ring,
bi->dma = dma;
bi->page = page;
bi->page_offset = i40e_rx_offset(rx_ring);
-
- /* initialize pagecnt_bias to 1 representing we fully own page */
- bi->pagecnt_bias = 1;
+ page_ref_add(page, USHRT_MAX - 1);
+ bi->pagecnt_bias = USHRT_MAX;
return true;
}
@@ -1956,8 +1955,8 @@ static bool i40e_can_reuse_rx_page(struct i40e_rx_buffer *rx_buffer)
* the pagecnt_bias and page count so that we fully restock the
* number of references the driver holds.
*/
- if (unlikely(!pagecnt_bias)) {
- page_ref_add(page, USHRT_MAX);
+ if (unlikely(pagecnt_bias == 1)) {
+ page_ref_add(page, USHRT_MAX - 1);
rx_buffer->pagecnt_bias = USHRT_MAX;
}
--
2.14.3
^ permalink raw reply related
* [net-next 08/10] i40e: re-number feature flags to remove gaps
From: Jeff Kirsher @ 2018-03-26 21:41 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
Remove the gaps created by the recent refactor of various feature flags
that have moved to the state field. Use only a u32 now that we have
fewer than 32 flags in the field.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e.h | 57 ++++++++++++++++------------------
1 file changed, 26 insertions(+), 31 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 67518013ca4d..a44139c1de80 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -518,37 +518,32 @@ struct i40e_pf {
#define I40E_HW_RESTART_AUTONEG BIT(18)
#define I40E_HW_STOPPABLE_FW_LLDP BIT(19)
- u64 flags;
-#define I40E_FLAG_RX_CSUM_ENABLED BIT_ULL(0)
-#define I40E_FLAG_MSI_ENABLED BIT_ULL(1)
-#define I40E_FLAG_MSIX_ENABLED BIT_ULL(2)
-#define I40E_FLAG_RSS_ENABLED BIT_ULL(3)
-#define I40E_FLAG_VMDQ_ENABLED BIT_ULL(4)
-/* Gap for BIT_ULL(5) */
-#define I40E_FLAG_SRIOV_ENABLED BIT_ULL(6)
-#define I40E_FLAG_DCB_CAPABLE BIT_ULL(7)
-#define I40E_FLAG_DCB_ENABLED BIT_ULL(8)
-#define I40E_FLAG_FD_SB_ENABLED BIT_ULL(9)
-#define I40E_FLAG_FD_ATR_ENABLED BIT_ULL(10)
-/* Gap for BIT_ULL(11) and BIT_ULL(12) */
-#define I40E_FLAG_MFP_ENABLED BIT_ULL(13)
-/* Gap for BIT_ULL(14) */
-#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT_ULL(15)
-#define I40E_FLAG_VEB_MODE_ENABLED BIT_ULL(16)
-#define I40E_FLAG_VEB_STATS_ENABLED BIT_ULL(17)
-#define I40E_FLAG_LINK_POLLING_ENABLED BIT_ULL(18)
-#define I40E_FLAG_TRUE_PROMISC_SUPPORT BIT_ULL(19)
-/* Gap for BIT_ULL(20) */
-#define I40E_FLAG_LEGACY_RX BIT_ULL(21)
-#define I40E_FLAG_PTP BIT_ULL(22)
-#define I40E_FLAG_IWARP_ENABLED BIT_ULL(23)
-/* Gap for BIT_ULL(24) through BIT_ULL(26) */
-#define I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED BIT_ULL(27)
-#define I40E_FLAG_SOURCE_PRUNING_DISABLED BIT_ULL(28)
-#define I40E_FLAG_TC_MQPRIO BIT_ULL(29)
-#define I40E_FLAG_FD_SB_INACTIVE BIT_ULL(30)
-#define I40E_FLAG_FD_SB_TO_CLOUD_FILTER BIT_ULL(31)
-#define I40E_FLAG_DISABLE_FW_LLDP BIT_ULL(32)
+ u32 flags;
+#define I40E_FLAG_RX_CSUM_ENABLED BIT(0)
+#define I40E_FLAG_MSI_ENABLED BIT(1)
+#define I40E_FLAG_MSIX_ENABLED BIT(2)
+#define I40E_FLAG_RSS_ENABLED BIT(3)
+#define I40E_FLAG_VMDQ_ENABLED BIT(4)
+#define I40E_FLAG_SRIOV_ENABLED BIT(5)
+#define I40E_FLAG_DCB_CAPABLE BIT(6)
+#define I40E_FLAG_DCB_ENABLED BIT(7)
+#define I40E_FLAG_FD_SB_ENABLED BIT(8)
+#define I40E_FLAG_FD_ATR_ENABLED BIT(9)
+#define I40E_FLAG_MFP_ENABLED BIT(10)
+#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT(11)
+#define I40E_FLAG_VEB_MODE_ENABLED BIT(12)
+#define I40E_FLAG_VEB_STATS_ENABLED BIT(13)
+#define I40E_FLAG_LINK_POLLING_ENABLED BIT(14)
+#define I40E_FLAG_TRUE_PROMISC_SUPPORT BIT(15)
+#define I40E_FLAG_LEGACY_RX BIT(16)
+#define I40E_FLAG_PTP BIT(17)
+#define I40E_FLAG_IWARP_ENABLED BIT(18)
+#define I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED BIT(19)
+#define I40E_FLAG_SOURCE_PRUNING_DISABLED BIT(20)
+#define I40E_FLAG_TC_MQPRIO BIT(21)
+#define I40E_FLAG_FD_SB_INACTIVE BIT(22)
+#define I40E_FLAG_FD_SB_TO_CLOUD_FILTER BIT(23)
+#define I40E_FLAG_DISABLE_FW_LLDP BIT(24)
struct i40e_client_instance *cinst;
bool stat_offsets_loaded;
--
2.14.3
^ permalink raw reply related
* [net-next 07/10] i40e: stop using cmpxchg flow in i40e_set_priv_flags()
From: Jeff Kirsher @ 2018-03-26 21:41 UTC (permalink / raw)
To: davem; +Cc: Jacob Keller, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180326214103.18218-1-jeffrey.t.kirsher@intel.com>
From: Jacob Keller <jacob.e.keller@intel.com>
Now that the only places which modify flags are either (a) during
initialization prior to creating a netdevice, or (b) while holding the
rtnl lock, we no longer need the cmpxchg64 call in i40e_set_priv_flags.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 2a9c93091e3d..b974482ff630 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -4436,21 +4436,12 @@ static int i40e_set_priv_flags(struct net_device *dev, u32 flags)
}
}
- /* Compare and exchange the new flags into place. If we failed, that
- * is if cmpxchg returns anything but the old value, this means that
- * something else has modified the flags variable since we copied it
- * originally. We'll just punt with an error and log something in the
- * message buffer.
- *
- * This is the point of no return for this function. We need to have
- * checked any discrepancies or misconfigurations and returned
- * EOPNOTSUPP before updating pf->flags here.
+ /* Now that we've checked to ensure that the new flags are valid, load
+ * them into place. Since we only modify flags either (a) during
+ * initialization or (b) while holding the RTNL lock, we don't need
+ * anything fancy here.
*/
- if (cmpxchg64(&pf->flags, orig_flags, new_flags) != orig_flags) {
- dev_warn(&pf->pdev->dev,
- "Unable to update pf->flags as it was modified by another thread...\n");
- return -EAGAIN;
- }
+ pf->flags = new_flags;
/* Process any additional changes needed as a result of flag changes.
* The changed_flags value reflects the list of bits that were
--
2.14.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox