* [PATCH V1 rdma-core 0/5] Licensing and cleanup issues
From: Yishai Hadas @ 2016-09-29 11:10 UTC (permalink / raw)
To: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, yishaih-VPRAkNaXOzVWk0Htik3J/w,
majd-VPRAkNaXOzVWk0Htik3J/w
Sending V1 to introduce few notes got from Jason, details below.
This series comes to solve some potential licensing issue around list.h
and ocrdma_list.h by using the CCAN functionality which has BSD license.
The CCAN required functionality was taken as is with minor touches as of
flattening the directory structure, its original header
with the licensing note was preserved.
In addition, the series drops some local definitions and use shared code instead.
Pull request was sent [1]
[1] https://github.com/linux-rdma/rdma-core/pull/5
Changes from V0:
Patch #2: Move to use min_t/max_t and fix typing issues among the project.
Add involved license files to ccan.
Sort list of files in ccan/CMakeLists.txt.
Use buildlib/config.h.in for ccan HAVE_XXX.
Patch #3: Add ccan/str.c and update relevant cmake files.
Sort list of files in ccan/CMakeLists.txt.
Yishai Hadas (5):
Remove container_of and offset local declarations
ccan: Add CCAN min and max functionality
ccan: Add list functionality
libmlx5: Move to use CCAN list functionality
libocrdma: Move to use CCAN list functionality
CMakeLists.txt | 1 +
buildlib/config.h.in | 10 +
buildlib/publish_headers.cmake | 18 +-
buildlib/rdma_functions.cmake | 8 +
ccan/BSD-MIT | 17 +
ccan/CC0 | 97 ++++
ccan/CMakeLists.txt | 16 +
ccan/build_assert.h | 40 ++
ccan/check_type.h | 64 +++
ccan/container_of.h | 146 ++++++
ccan/list.h | 842 +++++++++++++++++++++++++++++++++++
ccan/minmax.h | 65 +++
ccan/str.c | 13 +
ccan/str.h | 228 ++++++++++
ccan/str_debug.h | 30 ++
ibacm/CMakeLists.txt | 6 +-
ibacm/linux/osd.h | 8 +-
ibacm/prov/acmp/src/acmp.c | 2 +-
iwpmd/src/CMakeLists.txt | 2 +-
libibcm/examples/CMakeLists.txt | 2 +-
libibcm/src/cm.c | 5 -
libibumad/tests/CMakeLists.txt | 4 +-
libibverbs/examples/CMakeLists.txt | 16 +-
libibverbs/examples/rc_pingpong.c | 12 +-
libmlx5/src/buf.c | 11 +-
libmlx5/src/list.h | 331 --------------
libmlx5/src/mlx5.c | 2 +-
libmlx5/src/mlx5.h | 19 +-
libmlx5/src/qp.c | 4 +-
libmlx5/src/srq.c | 2 +-
libmlx5/src/verbs.c | 21 +-
libocrdma/src/ocrdma_list.h | 104 -----
libocrdma/src/ocrdma_main.c | 24 +-
libocrdma/src/ocrdma_main.h | 12 +-
libocrdma/src/ocrdma_verbs.c | 33 +-
librdmacm/examples/CMakeLists.txt | 26 +-
librdmacm/src/cma.h | 11 +-
librdmacm/src/rsocket.c | 13 +-
srp_daemon/srp_daemon/CMakeLists.txt | 2 +-
39 files changed, 1692 insertions(+), 575 deletions(-)
create mode 100644 ccan/BSD-MIT
create mode 100644 ccan/CC0
create mode 100644 ccan/CMakeLists.txt
create mode 100644 ccan/build_assert.h
create mode 100644 ccan/check_type.h
create mode 100644 ccan/container_of.h
create mode 100644 ccan/list.h
create mode 100644 ccan/minmax.h
create mode 100644 ccan/str.c
create mode 100644 ccan/str.h
create mode 100644 ccan/str_debug.h
delete mode 100644 libmlx5/src/list.h
delete mode 100644 libocrdma/src/ocrdma_list.h
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v1] mlx4: remove unused fields
From: Tariq Toukan @ 2016-09-29 7:44 UTC (permalink / raw)
To: David Decotigny, Yishai Hadas, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: edumazet-hpIqsD4AKlfQT0dZR+AlfA, David Decotigny
In-Reply-To: <1475085604-101493-1-git-send-email-ddecotig-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On 28/09/2016 9:00 PM, David Decotigny wrote:
> From: David Decotigny <decot-Ypc/8FJVVoBWk0Htik3J/w@public.gmane.org>
>
> This also can address following UBSAN warnings:
> [ 36.640343] ================================================================================
> [ 36.648772] UBSAN: Undefined behaviour in drivers/net/ethernet/mellanox/mlx4/fw.c:857:26
> [ 36.656853] shift exponent 64 is too large for 32-bit type 'int'
> [ 36.663348] ================================================================================
> [ 36.671783] ================================================================================
> [ 36.680213] UBSAN: Undefined behaviour in drivers/net/ethernet/mellanox/mlx4/fw.c:861:27
> [ 36.688297] shift exponent 35 is too large for 32-bit type 'int'
> [ 36.694702] ================================================================================
>
> Tested:
> reboot with UBSAN, no warning.
>
> Signed-off-by: David Decotigny <decot-Ypc/8FJVVoBWk0Htik3J/w@public.gmane.org>
> ---
> drivers/net/ethernet/mellanox/mlx4/fw.c | 4 ----
> drivers/net/ethernet/mellanox/mlx4/fw.h | 2 --
> 2 files changed, 6 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c
> index 090bf81..f9cbc67 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/fw.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/fw.c
> @@ -853,12 +853,8 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap)
> dev_cap->max_eqs = 1 << (field & 0xf);
> MLX4_GET(field, outbox, QUERY_DEV_CAP_RSVD_MTT_OFFSET);
> dev_cap->reserved_mtts = 1 << (field >> 4);
> - MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_MRW_SZ_OFFSET);
> - dev_cap->max_mrw_sz = 1 << field;
> MLX4_GET(field, outbox, QUERY_DEV_CAP_RSVD_MRW_OFFSET);
> dev_cap->reserved_mrws = 1 << (field & 0xf);
> - MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_MTT_SEG_OFFSET);
> - dev_cap->max_mtt_seg = 1 << (field & 0x3f);
> MLX4_GET(size, outbox, QUERY_DEV_CAP_NUM_SYS_EQ_OFFSET);
> dev_cap->num_sys_eqs = size & 0xfff;
> MLX4_GET(field, outbox, QUERY_DEV_CAP_MAX_REQ_QP_OFFSET);
> diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.h b/drivers/net/ethernet/mellanox/mlx4/fw.h
> index f11614f..5343a05 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/fw.h
> +++ b/drivers/net/ethernet/mellanox/mlx4/fw.h
> @@ -80,9 +80,7 @@ struct mlx4_dev_cap {
> int max_eqs;
> int num_sys_eqs;
> int reserved_mtts;
> - int max_mrw_sz;
> int reserved_mrws;
> - int max_mtt_seg;
> int max_requester_per_qp;
> int max_responder_per_qp;
> int max_rdma_global;
Reviewed-by: Tariq Toukan <tariqt-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v2 6/7] SRP transport: Port srp_wait_for_queuecommand() to scsi-mq
From: Hannes Reinecke @ 2016-09-29 5:54 UTC (permalink / raw)
To: Bart Van Assche, Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <bedf4b45-00b0-6844-047d-f1e05bb1092f@sandisk.com>
On 09/29/2016 02:01 AM, Bart Van Assche wrote:
> Ensure that if scsi-mq is enabled that srp_wait_for_queuecommand()
> waits until ongoing shost->hostt->queuecommand() calls have finished.
>
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Cc: James Bottomley <jejb@linux.vnet.ibm.com>
> Cc: Martin K. Petersen <martin.petersen@oracle.com>
> Cc: Doug Ledford <dledford@redhat.com>
> ---
> drivers/scsi/scsi_transport_srp.c | 24 ++++++++++++++++++++----
> 1 file changed, 20 insertions(+), 4 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
^ permalink raw reply
* Re: [PATCH v2 4/7] blk-mq: Introduce blk_quiesce_queue() and blk_resume_queue()
From: Hannes Reinecke @ 2016-09-29 5:52 UTC (permalink / raw)
To: Bart Van Assche, Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
In-Reply-To: <a9eb4e95-6d77-e527-c42a-fd2e6f104bed-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
On 09/29/2016 01:59 AM, Bart Van Assche wrote:
> blk_quiesce_queue() prevents that new queue_rq() invocations
> occur and waits until ongoing invocations have finished. This
> function does *not* wait until all outstanding requests have
> finished (this means invocation of request.end_io()).
> blk_resume_queue() resumes normal I/O processing.
>
> Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
> Cc: Hannes Reinecke <hare-IBi9RG/b67k@public.gmane.org>
> Cc: Johannes Thumshirn <jthumshirn-l3A5Bk7waGM@public.gmane.org>
> ---
> block/blk-mq.c | 137 ++++++++++++++++++++++++++++++++++++++++++++-----
> include/linux/blk-mq.h | 2 +
> include/linux/blkdev.h | 5 ++
> 3 files changed, 131 insertions(+), 13 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare-IBi9RG/b67k@public.gmane.org>
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare-l3A5Bk7waGM@public.gmane.org +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH rdma-core 5/5] libocrdma: Move to use CCAN list functionality
From: Devesh Sharma @ 2016-09-29 3:02 UTC (permalink / raw)
To: Yishai Hadas; +Cc: Jason Gunthorpe, Doug Ledford, linux-rdma, Majd Dibbiny
In-Reply-To: <1475076789-14359-6-git-send-email-yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Acked-By: Devesh Sharma <devesh.sharma-dY08KVG/lbpWk0Htik3J/w@public.gmane.org>
On Wed, Sep 28, 2016 at 9:03 PM, Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> Move to use CCAN list functionality which its license meets
> the project needs.
>
> Signed-off-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
> libocrdma/src/ocrdma_list.h | 104 -------------------------------------------
> libocrdma/src/ocrdma_main.c | 24 +++++-----
> libocrdma/src/ocrdma_main.h | 12 ++---
> libocrdma/src/ocrdma_verbs.c | 33 +++++++-------
> 4 files changed, 33 insertions(+), 140 deletions(-)
> delete mode 100644 libocrdma/src/ocrdma_list.h
>
> diff --git a/libocrdma/src/ocrdma_list.h b/libocrdma/src/ocrdma_list.h
> deleted file mode 100644
> index 1e0f1ff..0000000
> --- a/libocrdma/src/ocrdma_list.h
> +++ /dev/null
> @@ -1,104 +0,0 @@
> -/*
> - * Copyright (C) 2008-2013 Emulex. All rights reserved.
> - * This software is available to you under a choice of one of two
> - * licenses. You may choose to be licensed under the terms of the GNU
> - * General Public License (GPL) Version 2, available from the file
> - * COPYING in the main directory of this source tree, or the
> - * BSD license below:
> - *
> - * Redistribution and use in source and binary forms, with or
> - * without modification, are permitted provided that the following
> - * conditions are met:
> - *
> - * - Redistributions of source code must retain the above
> - * copyright notice, this list of conditions and the following
> - * disclaimer.
> - *
> - * - Redistributions in binary form must reproduce the above
> - * copyright notice, this list of conditions and the following
> - * disclaimer in the documentation and/or other materials
> - * provided with the distribution.
> - *
> - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
> - * THE POSSIBILITY OF SUCH DAMAGE.
> - */
> -
> -#ifndef __OCRDMA_LIST_H__
> -#define __OCRDMA_LIST_H__
> -
> -struct ocrdma_list_node {
> - struct ocrdma_list_node *next, *prev;
> -};
> -
> -struct ocrdma_list_head {
> - struct ocrdma_list_node node;
> - pthread_mutex_t lock;
> -};
> -
> -#define DBLY_LIST_HEAD_INIT(name) { { &(name.node), &(name.node) } , \
> - PTHREAD_MUTEX_INITIALIZER }
> -
> -#define DBLY_LIST_HEAD(name) \
> - struct ocrdma_list_head name = DBLY_LIST_HEAD_INIT(name); \
> -
> -#define INIT_DBLY_LIST_NODE(ptr) do { \
> - (ptr)->next = (ptr); (ptr)->prev = (ptr); \
> -} while (0)
> -
> -#define INIT_DBLY_LIST_HEAD(ptr) INIT_DBLY_LIST_NODE(ptr.node)
> -
> -static inline void __list_add_node(struct ocrdma_list_node *new,
> - struct ocrdma_list_node *prev,
> - struct ocrdma_list_node *next)
> -{
> - next->prev = new;
> - new->next = next;
> - new->prev = prev;
> - prev->next = new;
> -}
> -
> -static inline void list_add_node_tail(struct ocrdma_list_node *new,
> - struct ocrdma_list_head *head)
> -{
> - __list_add_node(new, head->node.prev, &head->node);
> -}
> -
> -static inline void __list_del_node(struct ocrdma_list_node *prev,
> - struct ocrdma_list_node *next)
> -{
> - next->prev = prev;
> - prev->next = next;
> -}
> -
> -static inline void list_del_node(struct ocrdma_list_node *entry)
> -{
> - __list_del_node(entry->prev, entry->next);
> - entry->next = entry->prev = 0;
> -}
> -
> -#define list_lock(head) pthread_mutex_lock(&((head)->lock))
> -#define list_unlock(head) pthread_mutex_unlock(&((head)->lock))
> -
> -#define list_node(ptr, type, member) \
> - ((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
> -
> -/**
> - * list_for_each_node_safe - iterate over a list safe against removal of list entry
> - * @pos: the &struct ocrdma_list_head to use as a loop counter.
> - * @n: another &struct ocrdma_list_head to use as temporary storage
> - * @head: the head for your list.
> - */
> -#define list_for_each_node_safe(pos, n, head) \
> - for (pos = (head)->node.next, n = pos->next; pos != &((head)->node); \
> - pos = n, n = pos->next)
> -
> -#endif /* __OCRDMA_LIST_H__ */
> diff --git a/libocrdma/src/ocrdma_main.c b/libocrdma/src/ocrdma_main.c
> index 5c494d8..064ecb3 100644
> --- a/libocrdma/src/ocrdma_main.c
> +++ b/libocrdma/src/ocrdma_main.c
> @@ -46,7 +46,7 @@
>
> #include "ocrdma_main.h"
> #include "ocrdma_abi.h"
> -#include "ocrdma_list.h"
> +#include <ccan/list.h>
>
> #include <sys/types.h>
> #include <sys/stat.h>
> @@ -68,7 +68,8 @@ struct {
> UCNA(EMULEX, GEN1), UCNA(EMULEX, GEN2), UCNA(EMULEX, GEN2_VF)
> };
>
> -static DBLY_LIST_HEAD(ocrdma_dev_list);
> +static LIST_HEAD(ocrdma_dev_list);
> +static pthread_mutex_t ocrdma_dev_list_lock = PTHREAD_MUTEX_INITIALIZER;
>
> static struct ibv_context *ocrdma_alloc_context(struct ibv_device *, int);
> static void ocrdma_free_context(struct ibv_context *);
> @@ -222,10 +223,10 @@ found:
> pthread_mutex_init(&dev->dev_lock, NULL);
> pthread_spin_init(&dev->flush_q_lock, PTHREAD_PROCESS_PRIVATE);
> dev->ibv_dev.ops = ocrdma_dev_ops;
> - INIT_DBLY_LIST_NODE(&dev->entry);
> - list_lock(&ocrdma_dev_list);
> - list_add_node_tail(&dev->entry, &ocrdma_dev_list);
> - list_unlock(&ocrdma_dev_list);
> + list_node_init(&dev->entry);
> + pthread_mutex_lock(&ocrdma_dev_list_lock);
> + list_add_tail(&ocrdma_dev_list, &dev->entry);
> + pthread_mutex_unlock(&ocrdma_dev_list_lock);
> return &dev->ibv_dev;
> qp_err:
> free(dev);
> @@ -244,14 +245,13 @@ void ocrdma_register_driver(void)
> static __attribute__ ((destructor))
> void ocrdma_unregister_driver(void)
> {
> - struct ocrdma_list_node *cur, *tmp;
> struct ocrdma_device *dev;
> - list_lock(&ocrdma_dev_list);
> - list_for_each_node_safe(cur, tmp, &ocrdma_dev_list) {
> - dev = list_node(cur, struct ocrdma_device, entry);
> + struct ocrdma_device *dev_tmp;
> + pthread_mutex_lock(&ocrdma_dev_list_lock);
> + list_for_each_safe(&ocrdma_dev_list, dev, dev_tmp, entry) {
> pthread_mutex_destroy(&dev->dev_lock);
> pthread_spin_destroy(&dev->flush_q_lock);
> - list_del_node(&dev->entry);
> + list_del(&dev->entry);
> /*
> * Avoid freeing the dev here since MPI get SIGSEGV
> * in few error cases because of reference to ib_dev
> @@ -260,5 +260,5 @@ void ocrdma_unregister_driver(void)
> */
> /* free(dev); */
> }
> - list_unlock(&ocrdma_dev_list);
> + pthread_mutex_unlock(&ocrdma_dev_list_lock);
> }
> diff --git a/libocrdma/src/ocrdma_main.h b/libocrdma/src/ocrdma_main.h
> index c81188b..b8be6e5 100644
> --- a/libocrdma/src/ocrdma_main.h
> +++ b/libocrdma/src/ocrdma_main.h
> @@ -42,7 +42,7 @@
> #include <infiniband/driver.h>
> #include <infiniband/arch.h>
>
> -#include "ocrdma_list.h"
> +#include <ccan/list.h>
>
> #define ocrdma_err(format, arg...) printf(format, ##arg)
>
> @@ -58,7 +58,7 @@ struct ocrdma_device {
> struct ocrdma_qp **qp_tbl;
> pthread_mutex_t dev_lock;
> pthread_spinlock_t flush_q_lock;
> - struct ocrdma_list_node entry;
> + struct list_node entry;
> int id;
> int gen;
> uint32_t wqe_size;
> @@ -106,8 +106,8 @@ struct ocrdma_cq {
> uint8_t deferred_arm;
> uint8_t deferred_sol;
> uint8_t first_arm;
> - struct ocrdma_list_head sq_head;
> - struct ocrdma_list_head rq_head;
> + struct list_head sq_head;
> + struct list_head rq_head;
> };
>
> enum {
> @@ -203,8 +203,8 @@ struct ocrdma_qp {
>
> enum ibv_qp_type qp_type;
> enum ocrdma_qp_state state;
> - struct ocrdma_list_node sq_entry;
> - struct ocrdma_list_node rq_entry;
> + struct list_node sq_entry;
> + struct list_node rq_entry;
> uint16_t id;
> uint16_t rsvd;
> uint32_t db_shift;
> diff --git a/libocrdma/src/ocrdma_verbs.c b/libocrdma/src/ocrdma_verbs.c
> index 6062626..413c706 100644
> --- a/libocrdma/src/ocrdma_verbs.c
> +++ b/libocrdma/src/ocrdma_verbs.c
> @@ -51,7 +51,7 @@
>
> #include "ocrdma_main.h"
> #include "ocrdma_abi.h"
> -#include "ocrdma_list.h"
> +#include <ccan/list.h>
>
> static void ocrdma_ring_cq_db(struct ocrdma_cq *cq, uint32_t armed,
> int solicited, uint32_t num_cqe);
> @@ -307,8 +307,8 @@ static struct ibv_cq *ocrdma_create_cq_common(struct ibv_context *context,
> ocrdma_ring_cq_db(cq, 0, 0, 0);
> }
> cq->ibv_cq.cqe = cqe;
> - INIT_DBLY_LIST_HEAD(&cq->sq_head);
> - INIT_DBLY_LIST_HEAD(&cq->rq_head);
> + list_head_init(&cq->sq_head);
> + list_head_init(&cq->rq_head);
> return &cq->ibv_cq;
> cq_err2:
> (void)ibv_cmd_destroy_cq(&cq->ibv_cq);
> @@ -621,8 +621,8 @@ struct ibv_qp *ocrdma_create_qp(struct ibv_pd *pd,
> }
> }
> qp->state = OCRDMA_QPS_RST;
> - INIT_DBLY_LIST_NODE(&qp->sq_entry);
> - INIT_DBLY_LIST_NODE(&qp->rq_entry);
> + list_node_init(&qp->sq_entry);
> + list_node_init(&qp->rq_entry);
> return &qp->ibv_qp;
>
> map_err:
> @@ -663,10 +663,9 @@ static int ocrdma_is_qp_in_sq_flushlist(struct ocrdma_cq *cq,
> struct ocrdma_qp *qp)
> {
> struct ocrdma_qp *list_qp;
> - struct ocrdma_list_node *cur, *tmp;
> + struct ocrdma_qp *list_qp_tmp;
> int found = 0;
> - list_for_each_node_safe(cur, tmp, &cq->sq_head) {
> - list_qp = list_node(cur, struct ocrdma_qp, sq_entry);
> + list_for_each_safe(&cq->sq_head, list_qp, list_qp_tmp, sq_entry) {
> if (qp == list_qp) {
> found = 1;
> break;
> @@ -679,10 +678,9 @@ static int ocrdma_is_qp_in_rq_flushlist(struct ocrdma_cq *cq,
> struct ocrdma_qp *qp)
> {
> struct ocrdma_qp *list_qp;
> - struct ocrdma_list_node *cur, *tmp;
> + struct ocrdma_qp *list_qp_tmp;
> int found = 0;
> - list_for_each_node_safe(cur, tmp, &cq->rq_head) {
> - list_qp = list_node(cur, struct ocrdma_qp, rq_entry);
> + list_for_each_safe(&cq->rq_head, list_qp, list_qp_tmp, rq_entry) {
> if (qp == list_qp) {
> found = 1;
> break;
> @@ -708,11 +706,11 @@ static void ocrdma_del_flush_qp(struct ocrdma_qp *qp)
> pthread_spin_lock(&dev->flush_q_lock);
> found = ocrdma_is_qp_in_sq_flushlist(qp->sq_cq, qp);
> if (found)
> - list_del_node(&qp->sq_entry);
> + list_del(&qp->sq_entry);
> if (!qp->srq) {
> found = ocrdma_is_qp_in_rq_flushlist(qp->rq_cq, qp);
> if (found)
> - list_del_node(&qp->rq_entry);
> + list_del(&qp->rq_entry);
> }
> pthread_spin_unlock(&dev->flush_q_lock);
> }
> @@ -724,11 +722,11 @@ static void ocrdma_flush_qp(struct ocrdma_qp *qp)
> pthread_spin_lock(&qp->dev->flush_q_lock);
> found = ocrdma_is_qp_in_sq_flushlist(qp->sq_cq, qp);
> if (!found)
> - list_add_node_tail(&qp->sq_entry, &qp->sq_cq->sq_head);
> + list_add_tail(&qp->sq_cq->sq_head, &qp->sq_entry);
> if (!qp->srq) {
> found = ocrdma_is_qp_in_rq_flushlist(qp->rq_cq, qp);
> if (!found)
> - list_add_node_tail(&qp->rq_entry, &qp->rq_cq->rq_head);
> + list_add_tail(&qp->rq_cq->rq_head, &qp->rq_entry);
> }
> pthread_spin_unlock(&qp->dev->flush_q_lock);
> }
> @@ -2034,7 +2032,7 @@ int ocrdma_poll_cq(struct ibv_cq *ibcq, int num_entries, struct ibv_wc *wc)
> int cqes_to_poll = num_entries;
> int num_os_cqe = 0, err_cqes = 0;
> struct ocrdma_qp *qp;
> - struct ocrdma_list_node *cur, *tmp;
> + struct ocrdma_qp *qp_tmp;
>
> cq = get_ocrdma_cq(ibcq);
> pthread_spin_lock(&cq->cq_lock);
> @@ -2045,8 +2043,7 @@ int ocrdma_poll_cq(struct ibv_cq *ibcq, int num_entries, struct ibv_wc *wc)
> if (cqes_to_poll) {
> wc = wc + num_os_cqe;
> pthread_spin_lock(&cq->dev->flush_q_lock);
> - list_for_each_node_safe(cur, tmp, &cq->sq_head) {
> - qp = list_node(cur, struct ocrdma_qp, sq_entry);
> + list_for_each_safe(&cq->sq_head, qp, qp_tmp, sq_entry) {
> if (cqes_to_poll == 0)
> break;
> err_cqes = ocrdma_add_err_cqe(cq, cqes_to_poll, qp, wc);
> --
> 1.8.3.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* 10740 linux-rdma
From: wwresson-epfaOiJH9AY @ 2016-09-29 3:01 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
[-- Attachment #1: EMAIL_477567171_linux-rdma.zip --]
[-- Type: application/zip, Size: 2335 bytes --]
^ permalink raw reply
* [PATCH v2 3/7] [RFC] nvme: Use BLK_MQ_S_STOPPED instead of QUEUE_FLAG_STOPPED in blk-mq code
From: Bart Van Assche @ 2016-09-29 0:02 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
Make nvme_requeue_req() check BLK_MQ_S_STOPPED instead of
QUEUE_FLAG_STOPPED. Remove the QUEUE_FLAG_STOPPED manipulations
that became superfluous because of this change. This patch fixes
a race condition: using queue_flag_clear_unlocked() is not safe
if any other function that manipulates the queue flags can be
called concurrently, e.g. blk_cleanup_queue(). Untested.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4669c05..d791fba 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -205,7 +205,7 @@ void nvme_requeue_req(struct request *req)
blk_mq_requeue_request(req);
spin_lock_irqsave(req->q->queue_lock, flags);
- if (!blk_queue_stopped(req->q))
+ if (!blk_mq_queue_stopped(req->q))
blk_mq_kick_requeue_list(req->q);
spin_unlock_irqrestore(req->q->queue_lock, flags);
}
@@ -2080,10 +2080,6 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
mutex_lock(&ctrl->namespaces_mutex);
list_for_each_entry(ns, &ctrl->namespaces, list) {
- spin_lock_irq(ns->queue->queue_lock);
- queue_flag_set(QUEUE_FLAG_STOPPED, ns->queue);
- spin_unlock_irq(ns->queue->queue_lock);
-
blk_mq_cancel_requeue_work(ns->queue);
blk_mq_stop_hw_queues(ns->queue);
}
@@ -2097,7 +2093,6 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
mutex_lock(&ctrl->namespaces_mutex);
list_for_each_entry(ns, &ctrl->namespaces, list) {
- queue_flag_clear_unlocked(QUEUE_FLAG_STOPPED, ns->queue);
blk_mq_start_stopped_hw_queues(ns->queue, true);
blk_mq_kick_requeue_list(ns->queue);
}
--
2.10.0
^ permalink raw reply related
* [PATCH v2 2/7] dm: Use BLK_MQ_S_STOPPED instead of QUEUE_FLAG_STOPPED in blk-mq code
From: Bart Van Assche @ 2016-09-29 0:02 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Mike Snitzer <snitzer@redhat.com>
---
drivers/md/dm-rq.c | 18 ++----------------
1 file changed, 2 insertions(+), 16 deletions(-)
diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c
index 182b679..f2c271a 100644
--- a/drivers/md/dm-rq.c
+++ b/drivers/md/dm-rq.c
@@ -75,12 +75,6 @@ static void dm_old_start_queue(struct request_queue *q)
static void dm_mq_start_queue(struct request_queue *q)
{
- unsigned long flags;
-
- spin_lock_irqsave(q->queue_lock, flags);
- queue_flag_clear(QUEUE_FLAG_STOPPED, q);
- spin_unlock_irqrestore(q->queue_lock, flags);
-
blk_mq_start_stopped_hw_queues(q, true);
blk_mq_kick_requeue_list(q);
}
@@ -105,16 +99,8 @@ static void dm_old_stop_queue(struct request_queue *q)
static void dm_mq_stop_queue(struct request_queue *q)
{
- unsigned long flags;
-
- spin_lock_irqsave(q->queue_lock, flags);
- if (blk_queue_stopped(q)) {
- spin_unlock_irqrestore(q->queue_lock, flags);
+ if (blk_mq_queue_stopped(q))
return;
- }
-
- queue_flag_set(QUEUE_FLAG_STOPPED, q);
- spin_unlock_irqrestore(q->queue_lock, flags);
/* Avoid that requeuing could restart the queue. */
blk_mq_cancel_requeue_work(q);
@@ -341,7 +327,7 @@ static void __dm_mq_kick_requeue_list(struct request_queue *q, unsigned long mse
unsigned long flags;
spin_lock_irqsave(q->queue_lock, flags);
- if (!blk_queue_stopped(q))
+ if (!blk_mq_queue_stopped(q))
blk_mq_delay_kick_requeue_list(q, msecs);
spin_unlock_irqrestore(q->queue_lock, flags);
}
--
2.10.0
^ permalink raw reply related
* [PATCH v2 7/7] [RFC] nvme: Fix a race condition
From: Bart Van Assche @ 2016-09-29 0:01 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
Avoid that nvme_queue_rq() is still running when nvme_stop_queues()
returns. Untested.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d791fba..98f1f29 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -201,13 +201,9 @@ fail:
void nvme_requeue_req(struct request *req)
{
- unsigned long flags;
-
blk_mq_requeue_request(req);
- spin_lock_irqsave(req->q->queue_lock, flags);
- if (!blk_mq_queue_stopped(req->q))
- blk_mq_kick_requeue_list(req->q);
- spin_unlock_irqrestore(req->q->queue_lock, flags);
+ WARN_ON_ONCE(blk_mq_queue_stopped(req->q));
+ blk_mq_kick_requeue_list(req->q);
}
EXPORT_SYMBOL_GPL(nvme_requeue_req);
@@ -2077,11 +2073,19 @@ EXPORT_SYMBOL_GPL(nvme_kill_queues);
void nvme_stop_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ struct request_queue *q;
mutex_lock(&ctrl->namespaces_mutex);
list_for_each_entry(ns, &ctrl->namespaces, list) {
- blk_mq_cancel_requeue_work(ns->queue);
- blk_mq_stop_hw_queues(ns->queue);
+ q = ns->queue;
+ blk_mq_quiesce_queue(q);
+ blk_mq_cancel_requeue_work(q);
+ blk_mq_stop_hw_queues(q);
+ /*
+ * Note: blk_mq_resume_queue() does not affect the value
+ * of the BLK_MQ_S_STOPPED bit.
+ */
+ blk_mq_resume_queue(q);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.10.0
^ permalink raw reply related
* [PATCH v2 6/7] SRP transport: Port srp_wait_for_queuecommand() to scsi-mq
From: Bart Van Assche @ 2016-09-29 0:01 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
Ensure that if scsi-mq is enabled that srp_wait_for_queuecommand()
waits until ongoing shost->hostt->queuecommand() calls have finished.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: James Bottomley <jejb@linux.vnet.ibm.com>
Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Doug Ledford <dledford@redhat.com>
---
drivers/scsi/scsi_transport_srp.c | 24 ++++++++++++++++++++----
1 file changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/scsi/scsi_transport_srp.c b/drivers/scsi/scsi_transport_srp.c
index e3cd3ec..f1d580e 100644
--- a/drivers/scsi/scsi_transport_srp.c
+++ b/drivers/scsi/scsi_transport_srp.c
@@ -25,6 +25,7 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
+#include <linux/blk-mq.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
@@ -405,8 +406,6 @@ static void srp_reconnect_work(struct work_struct *work)
/**
* scsi_request_fn_active() - number of kernel threads inside scsi_request_fn()
* @shost: SCSI host for which to count the number of scsi_request_fn() callers.
- *
- * To do: add support for scsi-mq in this function.
*/
static int scsi_request_fn_active(struct Scsi_Host *shost)
{
@@ -425,11 +424,28 @@ static int scsi_request_fn_active(struct Scsi_Host *shost)
return request_fn_active;
}
+static void srp_mq_wait_for_queuecommand(struct Scsi_Host *shost)
+{
+ struct scsi_device *sdev;
+ struct request_queue *q;
+
+ shost_for_each_device(sdev, shost) {
+ q = sdev->request_queue;
+
+ blk_mq_quiesce_queue(q);
+ blk_mq_resume_queue(q);
+ }
+}
+
/* Wait until ongoing shost->hostt->queuecommand() calls have finished. */
static void srp_wait_for_queuecommand(struct Scsi_Host *shost)
{
- while (scsi_request_fn_active(shost))
- msleep(20);
+ if (shost->use_blk_mq) {
+ srp_mq_wait_for_queuecommand(shost);
+ } else {
+ while (scsi_request_fn_active(shost))
+ msleep(20);
+ }
}
static void __rport_fail_io_fast(struct srp_rport *rport)
--
2.10.0
^ permalink raw reply related
* [PATCH v2 5/7] dm: Fix a race condition related to stopping and starting queues
From: Bart Van Assche @ 2016-09-29 0:00 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Ensure that all ongoing dm_mq_queue_rq() and dm_mq_requeue_request()
calls have stopped before setting the "queue stopped" flag. This
allows to remove the "queue stopped" test from dm_mq_queue_rq() and
dm_mq_requeue_request(). This patch fixes a race condition because
dm_mq_queue_rq() is called without holding the queue lock and hence
BLK_MQ_S_STOPPED can be set at any time while dm_mq_queue_rq() is
in progress. This patch prevents that the following hang occurs
sporadically when using dm-mq:
INFO: task systemd-udevd:10111 blocked for more than 480 seconds.
Not tainted 4.7.0-dbg+ #1
Call Trace:
[<ffffffff8161f397>] schedule+0x37/0x90
[<ffffffff816239ef>] schedule_timeout+0x27f/0x470
[<ffffffff8161e76f>] io_schedule_timeout+0x9f/0x110
[<ffffffff8161fb36>] bit_wait_io+0x16/0x60
[<ffffffff8161f929>] __wait_on_bit_lock+0x49/0xa0
[<ffffffff8114fe69>] __lock_page+0xb9/0xc0
[<ffffffff81165d90>] truncate_inode_pages_range+0x3e0/0x760
[<ffffffff81166120>] truncate_inode_pages+0x10/0x20
[<ffffffff81212a20>] kill_bdev+0x30/0x40
[<ffffffff81213d41>] __blkdev_put+0x71/0x360
[<ffffffff81214079>] blkdev_put+0x49/0x170
[<ffffffff812141c0>] blkdev_close+0x20/0x30
[<ffffffff811d48e8>] __fput+0xe8/0x1f0
[<ffffffff811d4a29>] ____fput+0x9/0x10
[<ffffffff810842d3>] task_work_run+0x83/0xb0
[<ffffffff8106606e>] do_exit+0x3ee/0xc40
[<ffffffff8106694b>] do_group_exit+0x4b/0xc0
[<ffffffff81073d9a>] get_signal+0x2ca/0x940
[<ffffffff8101bf43>] do_signal+0x23/0x660
[<ffffffff810022b3>] exit_to_usermode_loop+0x73/0xb0
[<ffffffff81002cb0>] syscall_return_slowpath+0xb0/0xc0
[<ffffffff81624e33>] entry_SYSCALL_64_fastpath+0xa6/0xa8
Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Reviewed-by: Hannes Reinecke <hare-IBi9RG/b67k@public.gmane.org>
Reviewed-by: Johannes Thumshirn <jthumshirn-l3A5Bk7waGM@public.gmane.org>
Cc: Mike Snitzer <snitzer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
drivers/md/dm-rq.c | 14 +++-----------
1 file changed, 3 insertions(+), 11 deletions(-)
diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c
index f2c271a..f7a8ba3 100644
--- a/drivers/md/dm-rq.c
+++ b/drivers/md/dm-rq.c
@@ -102,9 +102,12 @@ static void dm_mq_stop_queue(struct request_queue *q)
if (blk_mq_queue_stopped(q))
return;
+ /* Wait until dm_mq_queue_rq() has finished. */
+ blk_mq_quiesce_queue(q);
/* Avoid that requeuing could restart the queue. */
blk_mq_cancel_requeue_work(q);
blk_mq_stop_hw_queues(q);
+ blk_mq_resume_queue(q);
}
void dm_stop_queue(struct request_queue *q)
@@ -887,17 +890,6 @@ static int dm_mq_queue_rq(struct blk_mq_hw_ctx *hctx,
dm_put_live_table(md, srcu_idx);
}
- /*
- * On suspend dm_stop_queue() handles stopping the blk-mq
- * request_queue BUT: even though the hw_queues are marked
- * BLK_MQ_S_STOPPED at that point there is still a race that
- * is allowing block/blk-mq.c to call ->queue_rq against a
- * hctx that it really shouldn't. The following check guards
- * against this rarity (albeit _not_ race-free).
- */
- if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state)))
- return BLK_MQ_RQ_QUEUE_BUSY;
-
if (ti->type->busy && ti->type->busy(ti))
return BLK_MQ_RQ_QUEUE_BUSY;
--
2.10.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v2 4/7] blk-mq: Introduce blk_quiesce_queue() and blk_resume_queue()
From: Bart Van Assche @ 2016-09-28 23:59 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
blk_quiesce_queue() prevents that new queue_rq() invocations
occur and waits until ongoing invocations have finished. This
function does *not* wait until all outstanding requests have
finished (this means invocation of request.end_io()).
blk_resume_queue() resumes normal I/O processing.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Hannes Reinecke <hare@suse.com>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
---
block/blk-mq.c | 137 ++++++++++++++++++++++++++++++++++++++++++++-----
include/linux/blk-mq.h | 2 +
include/linux/blkdev.h | 5 ++
3 files changed, 131 insertions(+), 13 deletions(-)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index d8c45de..f5c71ad 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -58,15 +58,23 @@ static void blk_mq_hctx_clear_pending(struct blk_mq_hw_ctx *hctx,
sbitmap_clear_bit(&hctx->ctx_map, ctx->index_hw);
}
-void blk_mq_freeze_queue_start(struct request_queue *q)
+static bool __blk_mq_freeze_queue_start(struct request_queue *q,
+ bool kill_percpu_ref)
{
int freeze_depth;
freeze_depth = atomic_inc_return(&q->mq_freeze_depth);
if (freeze_depth == 1) {
- percpu_ref_kill(&q->q_usage_counter);
+ if (kill_percpu_ref)
+ percpu_ref_kill(&q->q_usage_counter);
blk_mq_run_hw_queues(q, false);
}
+ return freeze_depth == 1;
+}
+
+void blk_mq_freeze_queue_start(struct request_queue *q)
+{
+ __blk_mq_freeze_queue_start(q, true);
}
EXPORT_SYMBOL_GPL(blk_mq_freeze_queue_start);
@@ -102,19 +110,90 @@ void blk_mq_freeze_queue(struct request_queue *q)
}
EXPORT_SYMBOL_GPL(blk_mq_freeze_queue);
-void blk_mq_unfreeze_queue(struct request_queue *q)
+static bool __blk_mq_unfreeze_queue(struct request_queue *q,
+ bool reinit_percpu_ref)
{
int freeze_depth;
freeze_depth = atomic_dec_return(&q->mq_freeze_depth);
WARN_ON_ONCE(freeze_depth < 0);
if (!freeze_depth) {
- percpu_ref_reinit(&q->q_usage_counter);
+ if (reinit_percpu_ref)
+ percpu_ref_reinit(&q->q_usage_counter);
wake_up_all(&q->mq_freeze_wq);
}
+ return freeze_depth == 0;
+}
+
+void blk_mq_unfreeze_queue(struct request_queue *q)
+{
+ __blk_mq_unfreeze_queue(q, true);
}
EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue);
+/**
+ * blk_mq_quiesce_queue() - wait until all pending queue_rq calls have finished
+ *
+ * Prevent that new I/O requests are queued and wait until all pending
+ * queue_rq() calls have finished. Must not be called if the queue has already
+ * been frozen. Additionally, freezing the queue after having quiesced the
+ * queue and before resuming the queue is not allowed.
+ *
+ * Note: this function does not prevent that the struct request end_io()
+ * callback function is invoked.
+ */
+void blk_mq_quiesce_queue(struct request_queue *q)
+{
+ struct blk_mq_hw_ctx *hctx;
+ unsigned int i;
+ bool res, rcu = false;
+
+ spin_lock_irq(q->queue_lock);
+ WARN_ON_ONCE(blk_queue_quiescing(q));
+ queue_flag_set(QUEUE_FLAG_QUIESCING, q);
+ spin_unlock_irq(q->queue_lock);
+
+ res = __blk_mq_freeze_queue_start(q, false);
+ WARN_ON_ONCE(!res);
+ queue_for_each_hw_ctx(q, hctx, i) {
+ if (hctx->flags & BLK_MQ_F_BLOCKING) {
+ mutex_lock(&hctx->queue_rq_mutex);
+ mutex_unlock(&hctx->queue_rq_mutex);
+ } else {
+ rcu = true;
+ }
+ }
+ if (rcu)
+ synchronize_rcu();
+
+ spin_lock_irq(q->queue_lock);
+ WARN_ON_ONCE(!blk_queue_quiescing(q));
+ queue_flag_clear(QUEUE_FLAG_QUIESCING, q);
+ spin_unlock_irq(q->queue_lock);
+}
+EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
+
+/**
+ * blk_mq_resume_queue() - resume request processing
+ *
+ * Resume blk_queue_enter() calls that have been suspended by
+ * blk_mq_quiesce_queue().
+ *
+ * The caller is responsible for serializing blk_mq_quiesce_queue() and
+ * blk_mq_resume_queue().
+ */
+void blk_mq_resume_queue(struct request_queue *q)
+{
+ bool res;
+
+ res = __blk_mq_unfreeze_queue(q, false);
+ WARN_ON_ONCE(!res);
+ WARN_ON_ONCE(blk_queue_quiescing(q));
+
+ blk_mq_run_hw_queues(q, false);
+}
+EXPORT_SYMBOL_GPL(blk_mq_resume_queue);
+
void blk_mq_wake_waiters(struct request_queue *q)
{
struct blk_mq_hw_ctx *hctx;
@@ -488,6 +567,9 @@ static void blk_mq_requeue_work(struct work_struct *work)
struct request *rq, *next;
unsigned long flags;
+ if (blk_queue_quiescing(q))
+ return;
+
spin_lock_irqsave(&q->requeue_lock, flags);
list_splice_init(&q->requeue_list, &rq_list);
spin_unlock_irqrestore(&q->requeue_lock, flags);
@@ -782,7 +864,7 @@ static inline unsigned int queued_to_index(unsigned int queued)
* of IO. In particular, we'd like FIFO behaviour on handling existing
* items on the hctx->dispatch list. Ignore that for now.
*/
-static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
+static void blk_mq_process_rq_list(struct blk_mq_hw_ctx *hctx)
{
struct request_queue *q = hctx->queue;
struct request *rq;
@@ -791,9 +873,6 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
struct list_head *dptr;
int queued;
- if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state)))
- return;
-
WARN_ON(!cpumask_test_cpu(raw_smp_processor_id(), hctx->cpumask) &&
cpu_online(hctx->next_cpu));
@@ -883,7 +962,24 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
*
* blk_mq_run_hw_queue() already checks the STOPPED bit
**/
- blk_mq_run_hw_queue(hctx, true);
+ if (!blk_queue_quiescing(q))
+ blk_mq_run_hw_queue(hctx, true);
+ }
+}
+
+static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
+{
+ if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state)))
+ return;
+
+ if (hctx->flags & BLK_MQ_F_BLOCKING) {
+ mutex_lock(&hctx->queue_rq_mutex);
+ blk_mq_process_rq_list(hctx);
+ mutex_unlock(&hctx->queue_rq_mutex);
+ } else {
+ rcu_read_lock();
+ blk_mq_process_rq_list(hctx);
+ rcu_read_unlock();
}
}
@@ -1341,7 +1437,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
blk_mq_bio_to_request(rq, bio);
/*
- * We do limited pluging. If the bio can be merged, do that.
+ * We do limited plugging. If the bio can be merged, do that.
* Otherwise the existing request in the plug list will be
* issued. So the plug list will have one request at most
*/
@@ -1361,9 +1457,23 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio)
blk_mq_put_ctx(data.ctx);
if (!old_rq)
goto done;
- if (!blk_mq_direct_issue_request(old_rq, &cookie))
- goto done;
- blk_mq_insert_request(old_rq, false, true, true);
+
+ if (data.hctx->flags & BLK_MQ_F_BLOCKING) {
+ mutex_lock(&data.hctx->queue_rq_mutex);
+ if (blk_queue_quiescing(q) ||
+ blk_mq_direct_issue_request(old_rq, &cookie) != 0)
+ blk_mq_insert_request(old_rq, false, true,
+ true);
+ mutex_unlock(&data.hctx->queue_rq_mutex);
+ } else {
+ rcu_read_lock();
+ if (blk_queue_quiescing(q) ||
+ blk_mq_direct_issue_request(old_rq, &cookie) != 0)
+ blk_mq_insert_request(old_rq, false, true,
+ true);
+ rcu_read_unlock();
+ }
+
goto done;
}
@@ -1702,6 +1812,7 @@ static int blk_mq_init_hctx(struct request_queue *q,
INIT_DELAYED_WORK(&hctx->delay_work, blk_mq_delay_work_fn);
spin_lock_init(&hctx->lock);
INIT_LIST_HEAD(&hctx->dispatch);
+ mutex_init(&hctx->queue_rq_mutex);
hctx->queue = q;
hctx->queue_num = hctx_idx;
hctx->flags = set->flags & ~BLK_MQ_F_TAG_SHARED;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 368c460d..4b970f1 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -41,6 +41,8 @@ struct blk_mq_hw_ctx {
struct blk_mq_tags *tags;
+ struct mutex queue_rq_mutex;
+
unsigned long queued;
unsigned long run;
#define BLK_MQ_MAX_DISPATCH_ORDER 7
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index c47c358..d365cdf 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -505,6 +505,7 @@ struct request_queue {
#define QUEUE_FLAG_FUA 24 /* device supports FUA writes */
#define QUEUE_FLAG_FLUSH_NQ 25 /* flush not queueuable */
#define QUEUE_FLAG_DAX 26 /* device supports DAX */
+#define QUEUE_FLAG_QUIESCING 27
#define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \
(1 << QUEUE_FLAG_STACKABLE) | \
@@ -595,6 +596,8 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q)
#define blk_queue_secure_erase(q) \
(test_bit(QUEUE_FLAG_SECERASE, &(q)->queue_flags))
#define blk_queue_dax(q) test_bit(QUEUE_FLAG_DAX, &(q)->queue_flags)
+#define blk_queue_quiescing(q) test_bit(QUEUE_FLAG_QUIESCING, \
+ &(q)->queue_flags)
#define blk_noretry_request(rq) \
((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \
@@ -824,6 +827,8 @@ extern void __blk_run_queue(struct request_queue *q);
extern void __blk_run_queue_uncond(struct request_queue *q);
extern void blk_run_queue(struct request_queue *);
extern void blk_run_queue_async(struct request_queue *q);
+extern void blk_mq_quiesce_queue(struct request_queue *q);
+extern void blk_mq_resume_queue(struct request_queue *q);
extern int blk_rq_map_user(struct request_queue *, struct request *,
struct rq_map_data *, void __user *, unsigned long,
gfp_t);
--
2.10.0
^ permalink raw reply related
* [PATCH v2 1/7] blk-mq: Introduce blk_mq_queue_stopped()
From: Bart Van Assche @ 2016-09-28 23:57 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <f153ee47-e599-c333-92b0-d5da475012f3@sandisk.com>
The function blk_queue_stopped() allows to test whether or not a
traditional request queue has been stopped. Introduce a helper
function that allows block drivers to query easily whether or not
one or more hardware contexts of a blk-mq queue have been stopped.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
---
block/blk-mq.c | 20 ++++++++++++++++++++
include/linux/blk-mq.h | 1 +
2 files changed, 21 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index dc5f47f..d8c45de 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -950,6 +950,26 @@ void blk_mq_run_hw_queues(struct request_queue *q, bool async)
}
EXPORT_SYMBOL(blk_mq_run_hw_queues);
+/**
+ * blk_mq_queue_stopped() - check whether one or more hctxs have been stopped
+ * @q: request queue.
+ *
+ * The caller is responsible for serializing this function against
+ * blk_mq_{start,stop}_hw_queue().
+ */
+bool blk_mq_queue_stopped(struct request_queue *q)
+{
+ struct blk_mq_hw_ctx *hctx;
+ int i;
+
+ queue_for_each_hw_ctx(q, hctx, i)
+ if (test_bit(BLK_MQ_S_STOPPED, &hctx->state))
+ return true;
+
+ return false;
+}
+EXPORT_SYMBOL(blk_mq_queue_stopped);
+
void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx)
{
cancel_work(&hctx->run_work);
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 5daa0ef..368c460d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -233,6 +233,7 @@ void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs
void blk_mq_abort_requeue_list(struct request_queue *q);
void blk_mq_complete_request(struct request *rq, int error);
+bool blk_mq_queue_stopped(struct request_queue *q);
void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx);
void blk_mq_start_hw_queue(struct blk_mq_hw_ctx *hctx);
void blk_mq_stop_hw_queues(struct request_queue *q);
--
2.10.0
^ permalink raw reply related
* [PATCH v2 0/7] Introduce blk_quiesce_queue() and blk_resume_queue()
From: Bart Van Assche @ 2016-09-28 23:57 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, James Bottomley, Martin K. Petersen,
Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
Hello Jens,
Multiple block drivers need the functionality to stop a request queue
and to wait until all ongoing request_fn() / queue_rq() calls have
finished without waiting until all outstanding requests have finished.
Hence this patch series that introduces the blk_mq_quiesce_queue() and
blk_mq_resume_queue() functions. The dm-mq, SRP and nvme patches in this
patch series are three examples of where these functions are useful.
These patches apply on top of the September 21 version of your
for-4.9/block branch.
The changes compared to the previous version of this patch series are:
- Dropped the non-blk-mq changes from this patch series.
- Added support for harware queues with BLK_MQ_F_BLOCKING set.
- Added a call stack to the description of the dm race fix patch.
- Dropped the non-scsi-mq changes from the SRP patch.
- Added a patch that introduces blk_mq_queue_stopped() in the dm driver.
The individual patches in this series are:
0001-blk-mq-Introduce-blk_mq_queue_stopped.patch
0002-dm-Use-BLK_MQ_S_STOPPED-instead-of-QUEUE_FLAG_STOPPE.patch
0003-RFC-nvme-Use-BLK_MQ_S_STOPPED-instead-of-QUEUE_FLAG_.patch
0004-blk-mq-Introduce-blk_quiesce_queue-and-blk_resume_qu.patch
0005-dm-Fix-a-race-condition-related-to-stopping-and-star.patch
0006-SRP-transport-Port-srp_wait_for_queuecommand-to-scsi.patch
0007-RFC-nvme-Fix-a-race-condition.patch
Thanks,
Bart.
^ permalink raw reply
* [PATCH rdma-core] Add a .travis.yml file
From: Jason Gunthorpe @ 2016-09-28 23:28 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford
Necessary to use the Travis CI service.
Signed-off-by: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
---
.travis.yml | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
create mode 100644 .travis.yml
Now that travis is enabled on repo, we need a yml file to make it run.
Here is a basic one.
I don't test with distros as old as precise and I don't expect that glibc to
be new enough to build this (eg missing timerfd, O_CLOEXEC, etc), and the
cmake certainly isn't. So we use the trusty 'beta' environment and install
gcc 6.2.
See it in action:
https://github.com/linux-rdma/rdma-core/pull/7
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 000000000000..e09dd32ffa9f
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,28 @@
+language: c
+# We need at least cmake 2.12, this means we need to use trusty.
+# Precise's glibc, etc predates what we are willing to support.
+# No reason for sudo.
+sudo: required
+dist: trusty
+addons:
+ apt:
+ sources:
+ - ubuntu-toolchain-r-test
+ packages:
+ - build-essential
+ - cmake
+ - debhelper
+ - gcc
+ - gcc-6
+ - libnl-3-dev
+ - libnl-route-3-dev
+ - make
+ - ninja-build
+ - pkg-config
+ - python
+ - valgrind
+script:
+ - mkdir build
+ - cd build
+ - CC=gcc-6 cmake -GNinja ..
+ - ninja
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH 11/13] srp_daemon: Add the debian initscripts as an option
From: Jason Gunthorpe @ 2016-09-28 21:13 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Bart Van Assche
In-Reply-To: <57EC120F.7070506-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Wed, Sep 28, 2016 at 02:55:11PM -0400, Doug Ledford wrote:
> I'm in favor of providing a good, reliable, correct set of startup files
> for each of the major distro flavors.
Okay, how about a initscripts/{debian,centos6,suse..} directory tree,
and centralize all the initscripts there? Some one who understands
this can then try and eventually make sense of the duplication?
> > We are short unit files, could you
> > contribute yours?
>
> Absolutely. I'd like to basically import the entire redhat rdma package
> into this, but it will take a little sorting things out to get all of
> the files and such in the right place. And that's a precursor to our
> srpd unit file as it lists a specific dependency on the rdma unit file.
Yeah, that rdma unit file and udev integration really needs to be
upstream and in all distros if we are going to have any hope of having
upstream unit files...
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 11/13] srp_daemon: Add the debian initscripts as an option
From: Doug Ledford @ 2016-09-28 21:11 UTC (permalink / raw)
To: Jason Gunthorpe, Bart Van Assche
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <20160928210137.GB2533-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 1459 bytes --]
On 9/28/16 5:01 PM, Jason Gunthorpe wrote:
> On Wed, Sep 28, 2016 at 01:18:51PM -0700, Bart Van Assche wrote:
>> On 09/28/2016 11:47 AM, Jason Gunthorpe wrote:
>>> To be clear, there is already an initscript that (perhaps?) is for
>>> RedHat - but it isn't even close to the Debian version. So this patch
>>> introduces two scripts for srp_dameon, which I hated doing..
>>
>> The existing init script works reliably on multiple versions of RHEL, SLES,
>> Fedora and openSuSE. It is not specific to Red Hat distro's. So I'm
>> surprised that a separate initscript is needed for Debian?
>
> I suspect not being distro specific is the entire problem,
Indeed...
> eg the
> Debian initscript is using /etc/default/srptools to configure
> how the daemon is launched and it uses the start-stop-daemon tool.
>
> A good initscript should consistently follow the distro policies for
> initscripts..
Exactly why we often don't use the ones in the upstream package. Try to
be generically good, and you will be universally, generically bad for
any specific distro. If you want to provide one as an example, that's
fine. If you want to provide one that's actually used, then you need
one per distro (or a means of generating a distro specific one from a
template or some such).
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> GPG Key ID: 0E572FDD
Red Hat, Inc.
100 E. Davie St
Raleigh, NC 27601 USA
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 907 bytes --]
^ permalink raw reply
* Re: [PATCH 11/13] srp_daemon: Add the debian initscripts as an option
From: Jason Gunthorpe @ 2016-09-28 21:01 UTC (permalink / raw)
To: Bart Van Assche
Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <07810a28-9db1-6dca-0666-d811fe246b7e-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
On Wed, Sep 28, 2016 at 01:18:51PM -0700, Bart Van Assche wrote:
> On 09/28/2016 11:47 AM, Jason Gunthorpe wrote:
> >To be clear, there is already an initscript that (perhaps?) is for
> >RedHat - but it isn't even close to the Debian version. So this patch
> >introduces two scripts for srp_dameon, which I hated doing..
>
> The existing init script works reliably on multiple versions of RHEL, SLES,
> Fedora and openSuSE. It is not specific to Red Hat distro's. So I'm
> surprised that a separate initscript is needed for Debian?
I suspect not being distro specific is the entire problem, eg the
Debian initscript is using /etc/default/srptools to configure
how the daemon is launched and it uses the start-stop-daemon tool.
A good initscript should consistently follow the distro policies for
initscripts..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] IB/rxe: Fix honoring max IRD value for rd/atomic.
From: Parav Pandit @ 2016-09-28 20:26 UTC (permalink / raw)
To: monis-VPRAkNaXOzVWk0Htik3J/w, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: pandit.parav-Re5JQEeQqe8AvxtiuMwx3w
This patch fixes honoring max incoming read request count instead of
outgoing read req count
(a) during modify qp by allocating response queue metadata
(b) during incoming read request processing
Signed-off-by: Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
drivers/infiniband/sw/rxe/rxe_qp.c | 24 +++++++++++++-----------
drivers/infiniband/sw/rxe/rxe_resp.c | 2 +-
3 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
index 4a5484e..73849a5a 100644
--- a/drivers/infiniband/sw/rxe/rxe_loc.h
+++ b/drivers/infiniband/sw/rxe/rxe_loc.h
@@ -198,7 +198,7 @@ void free_rd_atomic_resource(struct rxe_qp *qp, struct resp_res *res);
static inline void rxe_advance_resp_resource(struct rxe_qp *qp)
{
qp->resp.res_head++;
- if (unlikely(qp->resp.res_head == qp->attr.max_rd_atomic))
+ if (unlikely(qp->resp.res_head == qp->attr.max_dest_rd_atomic))
qp->resp.res_head = 0;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c
index f2c0dc2..b8036cf 100644
--- a/drivers/infiniband/sw/rxe/rxe_qp.c
+++ b/drivers/infiniband/sw/rxe/rxe_qp.c
@@ -146,7 +146,7 @@ static void free_rd_atomic_resources(struct rxe_qp *qp)
if (qp->resp.resources) {
int i;
- for (i = 0; i < qp->attr.max_rd_atomic; i++) {
+ for (i = 0; i < qp->attr.max_dest_rd_atomic; i++) {
struct resp_res *res = &qp->resp.resources[i];
free_rd_atomic_resource(qp, res);
@@ -174,7 +174,7 @@ static void cleanup_rd_atomic_resources(struct rxe_qp *qp)
struct resp_res *res;
if (qp->resp.resources) {
- for (i = 0; i < qp->attr.max_rd_atomic; i++) {
+ for (i = 0; i < qp->attr.max_dest_rd_atomic; i++) {
res = &qp->resp.resources[i];
free_rd_atomic_resource(qp, res);
}
@@ -596,14 +596,21 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
if (mask & IB_QP_MAX_QP_RD_ATOMIC) {
int max_rd_atomic = __roundup_pow_of_two(attr->max_rd_atomic);
+ qp->attr.max_rd_atomic = max_rd_atomic;
+ atomic_set(&qp->req.rd_atomic, max_rd_atomic);
+ }
+
+ if (mask & IB_QP_MAX_DEST_RD_ATOMIC) {
+ int max_dest_rd_atomic =
+ __roundup_pow_of_two(attr->max_dest_rd_atomic);
+
+ qp->attr.max_dest_rd_atomic = max_dest_rd_atomic;
+
free_rd_atomic_resources(qp);
- err = alloc_rd_atomic_resources(qp, max_rd_atomic);
+ err = alloc_rd_atomic_resources(qp, max_dest_rd_atomic);
if (err)
return err;
-
- qp->attr.max_rd_atomic = max_rd_atomic;
- atomic_set(&qp->req.rd_atomic, max_rd_atomic);
}
if (mask & IB_QP_CUR_STATE)
@@ -704,11 +711,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
pr_debug("qp#%d set req psn = 0x%x\n", qp_num(qp), qp->req.psn);
}
- if (mask & IB_QP_MAX_DEST_RD_ATOMIC) {
- qp->attr.max_dest_rd_atomic =
- __roundup_pow_of_two(attr->max_dest_rd_atomic);
- }
-
if (mask & IB_QP_PATH_MIG_STATE)
qp->attr.path_mig_state = attr->path_mig_state;
diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c
index d0be8f2..dd3d88a 100644
--- a/drivers/infiniband/sw/rxe/rxe_resp.c
+++ b/drivers/infiniband/sw/rxe/rxe_resp.c
@@ -383,7 +383,7 @@ static enum resp_states check_resource(struct rxe_qp *qp,
* too many read/atomic ops, we just
* recycle the responder resource queue
*/
- if (likely(qp->attr.max_rd_atomic > 0))
+ if (likely(qp->attr.max_dest_rd_atomic > 0))
return RESPST_CHK_LENGTH;
else
return RESPST_ERR_TOO_MANY_RDMA_ATM_REQ;
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] IB/rxe: impoved debug prints & other code cleanup
From: Parav Pandit @ 2016-09-28 20:26 UTC (permalink / raw)
To: monis-VPRAkNaXOzVWk0Htik3J/w, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: pandit.parav-Re5JQEeQqe8AvxtiuMwx3w
1. Debugging qp state transitions and qp errors in loopback and
multiple QP tests is difficult without qp numbers in debug logs.
This patch adds qp number to important debug logs.
2. Instead of having rxe: prefix in few logs and not having in
few logs, using uniform module name prefix using pr_fmt macro.
3. Code cleanup for various warnings reported by checkpatch for
incomplete unsigned data type, line over 80 characters, return
statements.
Signed-off-by: Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rxe/rxe.c | 12 ++++++------
drivers/infiniband/sw/rxe/rxe.h | 5 +++++
drivers/infiniband/sw/rxe/rxe_av.c | 4 ++--
drivers/infiniband/sw/rxe/rxe_comp.c | 6 ++++--
drivers/infiniband/sw/rxe/rxe_mmap.c | 2 +-
drivers/infiniband/sw/rxe/rxe_mr.c | 2 +-
drivers/infiniband/sw/rxe/rxe_net.c | 16 +++++++---------
drivers/infiniband/sw/rxe/rxe_qp.c | 31 +++++++++++++++++--------------
drivers/infiniband/sw/rxe/rxe_recv.c | 3 ++-
drivers/infiniband/sw/rxe/rxe_req.c | 19 ++++++++++---------
drivers/infiniband/sw/rxe/rxe_resp.c | 25 +++++++++++++++++--------
drivers/infiniband/sw/rxe/rxe_sysfs.c | 12 ++++++------
drivers/infiniband/sw/rxe/rxe_verbs.c | 12 +++++++-----
13 files changed, 85 insertions(+), 64 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c
index ddd5927..006817b 100644
--- a/drivers/infiniband/sw/rxe/rxe.c
+++ b/drivers/infiniband/sw/rxe/rxe.c
@@ -358,31 +358,31 @@ static int __init rxe_module_init(void)
/* initialize slab caches for managed objects */
err = rxe_cache_init();
if (err) {
- pr_err("rxe: unable to init object pools\n");
+ pr_err("unable to init object pools\n");
return err;
}
err = rxe_net_ipv4_init();
if (err) {
- pr_err("rxe: unable to init ipv4 tunnel\n");
+ pr_err("unable to init ipv4 tunnel\n");
rxe_cache_exit();
goto exit;
}
err = rxe_net_ipv6_init();
if (err) {
- pr_err("rxe: unable to init ipv6 tunnel\n");
+ pr_err("unable to init ipv6 tunnel\n");
rxe_cache_exit();
goto exit;
}
err = register_netdevice_notifier(&rxe_net_notifier);
if (err) {
- pr_err("rxe: Failed to rigister netdev notifier\n");
+ pr_err("Failed to rigister netdev notifier\n");
goto exit;
}
- pr_info("rxe: loaded\n");
+ pr_info("loaded\n");
return 0;
@@ -398,7 +398,7 @@ static void __exit rxe_module_exit(void)
rxe_net_exit();
rxe_cache_exit();
- pr_info("rxe: unloaded\n");
+ pr_info("unloaded\n");
}
module_init(rxe_module_init);
diff --git a/drivers/infiniband/sw/rxe/rxe.h b/drivers/infiniband/sw/rxe/rxe.h
index 12c71c5..a696af8 100644
--- a/drivers/infiniband/sw/rxe/rxe.h
+++ b/drivers/infiniband/sw/rxe/rxe.h
@@ -34,6 +34,11 @@
#ifndef RXE_H
#define RXE_H
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/crc32.h>
diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c
index 5c94742..604f6fe 100644
--- a/drivers/infiniband/sw/rxe/rxe_av.c
+++ b/drivers/infiniband/sw/rxe/rxe_av.c
@@ -39,7 +39,7 @@ int rxe_av_chk_attr(struct rxe_dev *rxe, struct ib_ah_attr *attr)
struct rxe_port *port;
if (attr->port_num != 1) {
- pr_info("rxe: invalid port_num = %d\n", attr->port_num);
+ pr_info("invalid port_num = %d\n", attr->port_num);
return -EINVAL;
}
@@ -47,7 +47,7 @@ int rxe_av_chk_attr(struct rxe_dev *rxe, struct ib_ah_attr *attr)
if (attr->ah_flags & IB_AH_GRH) {
if (attr->grh.sgid_index > port->attr.gid_tbl_len) {
- pr_info("rxe: invalid sgid index = %d\n",
+ pr_info("invalid sgid index = %d\n",
attr->grh.sgid_index);
return -EINVAL;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c
index 1c59ef2..6c5e29d 100644
--- a/drivers/infiniband/sw/rxe/rxe_comp.c
+++ b/drivers/infiniband/sw/rxe/rxe_comp.c
@@ -567,7 +567,8 @@ int rxe_completer(void *arg)
state = COMPST_GET_ACK;
while (1) {
- pr_debug("state = %s\n", comp_state_name[state]);
+ pr_debug("qp#%d state = %s\n", qp_num(qp),
+ comp_state_name[state]);
switch (state) {
case COMPST_GET_ACK:
skb = skb_dequeue(&qp->resp_pkts);
@@ -709,7 +710,8 @@ int rxe_completer(void *arg)
qp->comp.rnr_retry--;
qp->req.need_retry = 1;
- pr_debug("set rnr nak timer\n");
+ pr_debug("qp#%d set rnr nak timer\n",
+ qp_num(qp));
mod_timer(&qp->rnr_nak_timer,
jiffies + rnrnak_jiffies(aeth_syn(pkt)
& ~AETH_TYPE_MASK));
diff --git a/drivers/infiniband/sw/rxe/rxe_mmap.c b/drivers/infiniband/sw/rxe/rxe_mmap.c
index 54b3c7c..c572a4c 100644
--- a/drivers/infiniband/sw/rxe/rxe_mmap.c
+++ b/drivers/infiniband/sw/rxe/rxe_mmap.c
@@ -126,7 +126,7 @@ found_it:
ret = remap_vmalloc_range(vma, ip->obj, 0);
if (ret) {
- pr_err("rxe: err %d from remap_vmalloc_range\n", ret);
+ pr_err("err %d from remap_vmalloc_range\n", ret);
goto done;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index f3dab65..1869152 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -39,7 +39,7 @@
*/
static u8 rxe_get_key(void)
{
- static unsigned key = 1;
+ static u32 key = 1;
key = key << 1;
diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
index 549aa84..f08609c 100644
--- a/drivers/infiniband/sw/rxe/rxe_net.c
+++ b/drivers/infiniband/sw/rxe/rxe_net.c
@@ -65,7 +65,7 @@ struct rxe_dev *net_to_rxe(struct net_device *ndev)
return found;
}
-struct rxe_dev *get_rxe_by_name(const char* name)
+struct rxe_dev *get_rxe_by_name(const char *name)
{
struct rxe_dev *rxe;
struct rxe_dev *found = NULL;
@@ -601,8 +601,7 @@ void rxe_port_up(struct rxe_dev *rxe)
port->attr.phys_state = IB_PHYS_STATE_LINK_UP;
rxe_port_event(rxe, IB_EVENT_PORT_ACTIVE);
- pr_info("rxe: set %s active\n", rxe->ib_dev.name);
- return;
+ pr_info("set %s active\n", rxe->ib_dev.name);
}
/* Caller must hold net_info_lock */
@@ -615,8 +614,7 @@ void rxe_port_down(struct rxe_dev *rxe)
port->attr.phys_state = IB_PHYS_STATE_LINK_DOWN;
rxe_port_event(rxe, IB_EVENT_PORT_ERR);
- pr_info("rxe: set %s down\n", rxe->ib_dev.name);
- return;
+ pr_info("set %s down\n", rxe->ib_dev.name);
}
static int rxe_notify(struct notifier_block *not_blk,
@@ -641,7 +639,7 @@ static int rxe_notify(struct notifier_block *not_blk,
rxe_port_down(rxe);
break;
case NETDEV_CHANGEMTU:
- pr_info("rxe: %s changed mtu to %d\n", ndev->name, ndev->mtu);
+ pr_info("%s changed mtu to %d\n", ndev->name, ndev->mtu);
rxe_set_mtu(rxe, ndev->mtu);
break;
case NETDEV_REBOOT:
@@ -651,7 +649,7 @@ static int rxe_notify(struct notifier_block *not_blk,
case NETDEV_CHANGENAME:
case NETDEV_FEAT_CHANGE:
default:
- pr_info("rxe: ignoring netdev event = %ld for %s\n",
+ pr_info("ignoring netdev event = %ld for %s\n",
event, ndev->name);
break;
}
@@ -671,7 +669,7 @@ int rxe_net_ipv4_init(void)
htons(ROCE_V2_UDP_DPORT), false);
if (IS_ERR(recv_sockets.sk4)) {
recv_sockets.sk4 = NULL;
- pr_err("rxe: Failed to create IPv4 UDP tunnel\n");
+ pr_err("Failed to create IPv4 UDP tunnel\n");
return -1;
}
@@ -688,7 +686,7 @@ int rxe_net_ipv6_init(void)
htons(ROCE_V2_UDP_DPORT), true);
if (IS_ERR(recv_sockets.sk6)) {
recv_sockets.sk6 = NULL;
- pr_err("rxe: Failed to create IPv6 UDP tunnel\n");
+ pr_err("Failed to create IPv6 UDP tunnel\n");
return -1;
}
#endif
diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c
index 22ba24f..f2c0dc2 100644
--- a/drivers/infiniband/sw/rxe/rxe_qp.c
+++ b/drivers/infiniband/sw/rxe/rxe_qp.c
@@ -298,8 +298,8 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp,
wqe_size = rcv_wqe_size(qp->rq.max_sge);
- pr_debug("max_wr = %d, max_sge = %d, wqe_size = %d\n",
- qp->rq.max_wr, qp->rq.max_sge, wqe_size);
+ pr_debug("qp#%d max_wr = %d, max_sge = %d, wqe_size = %d\n",
+ qp_num(qp), qp->rq.max_wr, qp->rq.max_sge, wqe_size);
qp->rq.queue = rxe_queue_init(rxe,
&qp->rq.max_wr,
@@ -673,24 +673,27 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
if (mask & IB_QP_RETRY_CNT) {
qp->attr.retry_cnt = attr->retry_cnt;
qp->comp.retry_cnt = attr->retry_cnt;
- pr_debug("set retry count = %d\n", attr->retry_cnt);
+ pr_debug("qp#%d set retry count = %d\n", qp_num(qp),
+ attr->retry_cnt);
}
if (mask & IB_QP_RNR_RETRY) {
qp->attr.rnr_retry = attr->rnr_retry;
qp->comp.rnr_retry = attr->rnr_retry;
- pr_debug("set rnr retry count = %d\n", attr->rnr_retry);
+ pr_debug("qp#%d set rnr retry count = %d\n", qp_num(qp),
+ attr->rnr_retry);
}
if (mask & IB_QP_RQ_PSN) {
qp->attr.rq_psn = (attr->rq_psn & BTH_PSN_MASK);
qp->resp.psn = qp->attr.rq_psn;
- pr_debug("set resp psn = 0x%x\n", qp->resp.psn);
+ pr_debug("qp#%d set resp psn = 0x%x\n", qp_num(qp),
+ qp->resp.psn);
}
if (mask & IB_QP_MIN_RNR_TIMER) {
qp->attr.min_rnr_timer = attr->min_rnr_timer;
- pr_debug("set min rnr timer = 0x%x\n",
+ pr_debug("qp#%d set min rnr timer = 0x%x\n", qp_num(qp),
attr->min_rnr_timer);
}
@@ -698,7 +701,7 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
qp->attr.sq_psn = (attr->sq_psn & BTH_PSN_MASK);
qp->req.psn = qp->attr.sq_psn;
qp->comp.psn = qp->attr.sq_psn;
- pr_debug("set req psn = 0x%x\n", qp->req.psn);
+ pr_debug("qp#%d set req psn = 0x%x\n", qp_num(qp), qp->req.psn);
}
if (mask & IB_QP_MAX_DEST_RD_ATOMIC) {
@@ -717,38 +720,38 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
switch (attr->qp_state) {
case IB_QPS_RESET:
- pr_debug("qp state -> RESET\n");
+ pr_debug("qp#%d state -> RESET\n", qp_num(qp));
rxe_qp_reset(qp);
break;
case IB_QPS_INIT:
- pr_debug("qp state -> INIT\n");
+ pr_debug("qp#%d state -> INIT\n", qp_num(qp));
qp->req.state = QP_STATE_INIT;
qp->resp.state = QP_STATE_INIT;
break;
case IB_QPS_RTR:
- pr_debug("qp state -> RTR\n");
+ pr_debug("qp#%d state -> RTR\n", qp_num(qp));
qp->resp.state = QP_STATE_READY;
break;
case IB_QPS_RTS:
- pr_debug("qp state -> RTS\n");
+ pr_debug("qp#%d state -> RTS\n", qp_num(qp));
qp->req.state = QP_STATE_READY;
break;
case IB_QPS_SQD:
- pr_debug("qp state -> SQD\n");
+ pr_debug("qp#%d state -> SQD\n", qp_num(qp));
rxe_qp_drain(qp);
break;
case IB_QPS_SQE:
- pr_warn("qp state -> SQE !!?\n");
+ pr_warn("qp#%d state -> SQE !!?\n", qp_num(qp));
/* Not possible from modify_qp. */
break;
case IB_QPS_ERR:
- pr_debug("qp state -> ERR\n");
+ pr_debug("qp#%d state -> ERR\n", qp_num(qp));
rxe_qp_error(qp);
break;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
index 144d2f1..46f0628 100644
--- a/drivers/infiniband/sw/rxe/rxe_recv.c
+++ b/drivers/infiniband/sw/rxe/rxe_recv.c
@@ -387,7 +387,8 @@ int rxe_rcv(struct sk_buff *skb)
pack_icrc = be32_to_cpu(*icrcp);
calc_icrc = rxe_icrc_hdr(pkt, skb);
- calc_icrc = crc32_le(calc_icrc, (u8 *)payload_addr(pkt), payload_size(pkt));
+ calc_icrc = crc32_le(calc_icrc, (u8 *)payload_addr(pkt),
+ payload_size(pkt));
calc_icrc = cpu_to_be32(~calc_icrc);
if (unlikely(calc_icrc != pack_icrc)) {
char saddr[sizeof(struct in6_addr)];
diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index 13a848a..832846b 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -38,7 +38,7 @@
#include "rxe_queue.h"
static int next_opcode(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
- unsigned opcode);
+ u32 opcode);
static inline void retry_first_write_send(struct rxe_qp *qp,
struct rxe_send_wqe *wqe,
@@ -121,7 +121,7 @@ void rnr_nak_timer(unsigned long data)
{
struct rxe_qp *qp = (struct rxe_qp *)data;
- pr_debug("rnr nak timer fired\n");
+ pr_debug("qp#%d rnr nak timer fired\n", qp_num(qp));
rxe_run_task(&qp->req.task, 1);
}
@@ -187,7 +187,7 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp)
return wqe;
}
-static int next_opcode_rc(struct rxe_qp *qp, unsigned opcode, int fits)
+static int next_opcode_rc(struct rxe_qp *qp, u32 opcode, int fits)
{
switch (opcode) {
case IB_WR_RDMA_WRITE:
@@ -259,7 +259,7 @@ static int next_opcode_rc(struct rxe_qp *qp, unsigned opcode, int fits)
return -EINVAL;
}
-static int next_opcode_uc(struct rxe_qp *qp, unsigned opcode, int fits)
+static int next_opcode_uc(struct rxe_qp *qp, u32 opcode, int fits)
{
switch (opcode) {
case IB_WR_RDMA_WRITE:
@@ -311,7 +311,7 @@ static int next_opcode_uc(struct rxe_qp *qp, unsigned opcode, int fits)
}
static int next_opcode(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
- unsigned opcode)
+ u32 opcode)
{
int fits = (wqe->dma.resid <= qp->mtu);
@@ -588,7 +588,7 @@ int rxe_requester(void *arg)
struct rxe_pkt_info pkt;
struct sk_buff *skb;
struct rxe_send_wqe *wqe;
- unsigned mask;
+ enum rxe_hdr_mask mask;
int payload;
int mtu;
int opcode;
@@ -626,7 +626,8 @@ next_wqe:
rmr = rxe_pool_get_index(&rxe->mr_pool,
wqe->wr.ex.invalidate_rkey >> 8);
if (!rmr) {
- pr_err("No mr for key %#x\n", wqe->wr.ex.invalidate_rkey);
+ pr_err("No mr for key %#x\n",
+ wqe->wr.ex.invalidate_rkey);
wqe->state = wqe_state_error;
wqe->status = IB_WC_MW_BIND_ERR;
goto exit;
@@ -702,12 +703,12 @@ next_wqe:
skb = init_req_packet(qp, wqe, opcode, payload, &pkt);
if (unlikely(!skb)) {
- pr_err("Failed allocating skb\n");
+ pr_err("qp#%d Failed allocating skb\n", qp_num(qp));
goto err;
}
if (fill_packet(qp, wqe, &pkt, skb, payload)) {
- pr_debug("Error during fill packet\n");
+ pr_debug("qp#%d Error during fill packet\n", qp_num(qp));
goto err;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c
index 3e0f0f2..d0be8f2 100644
--- a/drivers/infiniband/sw/rxe/rxe_resp.c
+++ b/drivers/infiniband/sw/rxe/rxe_resp.c
@@ -749,6 +749,18 @@ static enum resp_states read_reply(struct rxe_qp *qp,
return state;
}
+static void build_rdma_network_hdr(union rdma_network_hdr *hdr,
+ struct rxe_pkt_info *pkt)
+{
+ struct sk_buff *skb = PKT_TO_SKB(pkt);
+
+ memset(hdr, 0, sizeof(*hdr));
+ if (skb->protocol == htons(ETH_P_IP))
+ memcpy(&hdr->roce4grh, ip_hdr(skb), sizeof(hdr->roce4grh));
+ else if (skb->protocol == htons(ETH_P_IPV6))
+ memcpy(&hdr->ibgrh, ipv6_hdr(skb), sizeof(hdr->ibgrh));
+}
+
/* Executes a new request. A retried request never reach that function (send
* and writes are discarded, and reads and atomics are retried elsewhere.
*/
@@ -761,13 +773,8 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt)
qp_type(qp) == IB_QPT_SMI ||
qp_type(qp) == IB_QPT_GSI) {
union rdma_network_hdr hdr;
- struct sk_buff *skb = PKT_TO_SKB(pkt);
- memset(&hdr, 0, sizeof(hdr));
- if (skb->protocol == htons(ETH_P_IP))
- memcpy(&hdr.roce4grh, ip_hdr(skb), sizeof(hdr.roce4grh));
- else if (skb->protocol == htons(ETH_P_IPV6))
- memcpy(&hdr.ibgrh, ipv6_hdr(skb), sizeof(hdr.ibgrh));
+ build_rdma_network_hdr(&hdr, pkt);
err = send_data_in(qp, &hdr, sizeof(hdr));
if (err)
@@ -881,7 +888,8 @@ static enum resp_states do_complete(struct rxe_qp *qp,
rmr = rxe_pool_get_index(&rxe->mr_pool,
wc->ex.invalidate_rkey >> 8);
if (unlikely(!rmr)) {
- pr_err("Bad rkey %#x invalidation\n", wc->ex.invalidate_rkey);
+ pr_err("Bad rkey %#x invalidation\n",
+ wc->ex.invalidate_rkey);
return RESPST_ERROR;
}
rmr->state = RXE_MEM_STATE_FREE;
@@ -1208,7 +1216,8 @@ int rxe_responder(void *arg)
}
while (1) {
- pr_debug("state = %s\n", resp_state_name[state]);
+ pr_debug("qp#%d state = %s\n", qp_num(qp),
+ resp_state_name[state]);
switch (state) {
case RESPST_GET_REQ:
state = get_req(qp, &pkt);
diff --git a/drivers/infiniband/sw/rxe/rxe_sysfs.c b/drivers/infiniband/sw/rxe/rxe_sysfs.c
index cf8e778..d5ed757 100644
--- a/drivers/infiniband/sw/rxe/rxe_sysfs.c
+++ b/drivers/infiniband/sw/rxe/rxe_sysfs.c
@@ -79,7 +79,7 @@ static int rxe_param_set_add(const char *val, const struct kernel_param *kp)
len = sanitize_arg(val, intf, sizeof(intf));
if (!len) {
- pr_err("rxe: add: invalid interface name\n");
+ pr_err("add: invalid interface name\n");
err = -EINVAL;
goto err;
}
@@ -92,20 +92,20 @@ static int rxe_param_set_add(const char *val, const struct kernel_param *kp)
}
if (net_to_rxe(ndev)) {
- pr_err("rxe: already configured on %s\n", intf);
+ pr_err("already configured on %s\n", intf);
err = -EINVAL;
goto err;
}
rxe = rxe_net_add(ndev);
if (!rxe) {
- pr_err("rxe: failed to add %s\n", intf);
+ pr_err("failed to add %s\n", intf);
err = -EINVAL;
goto err;
}
rxe_set_port_state(ndev);
- pr_info("rxe: added %s to %s\n", rxe->ib_dev.name, intf);
+ pr_info("added %s to %s\n", rxe->ib_dev.name, intf);
err:
if (ndev)
dev_put(ndev);
@@ -120,7 +120,7 @@ static int rxe_param_set_remove(const char *val, const struct kernel_param *kp)
len = sanitize_arg(val, intf, sizeof(intf));
if (!len) {
- pr_err("rxe: add: invalid interface name\n");
+ pr_err("add: invalid interface name\n");
return -EINVAL;
}
@@ -133,7 +133,7 @@ static int rxe_param_set_remove(const char *val, const struct kernel_param *kp)
rxe = get_rxe_by_name(intf);
if (!rxe) {
- pr_err("rxe: not configured on %s\n", intf);
+ pr_err("not configured on %s\n", intf);
return -EINVAL;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index a5af691..19841c8 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -100,10 +100,12 @@ static int rxe_query_port(struct ib_device *dev,
rxe->ndev->ethtool_ops->get_settings(rxe->ndev, &cmd);
speed = cmd.speed;
} else {
- pr_warn("%s speed is unknown, defaulting to 1000\n", rxe->ndev->name);
+ pr_warn("%s speed is unknown, defaulting to 1000\n",
+ rxe->ndev->name);
speed = 1000;
}
- rxe_eth_speed_to_ib_speed(speed, &attr->active_speed, &attr->active_width);
+ rxe_eth_speed_to_ib_speed(speed, &attr->active_speed,
+ &attr->active_width);
mutex_unlock(&rxe->usdev_lock);
return 0;
@@ -761,7 +763,7 @@ static int init_send_wqe(struct rxe_qp *qp, struct ib_send_wr *ibwr,
}
static int post_one_send(struct rxe_qp *qp, struct ib_send_wr *ibwr,
- unsigned mask, u32 length)
+ unsigned int mask, u32 length)
{
int err;
struct rxe_sq *sq = &qp->sq;
@@ -1145,8 +1147,8 @@ static int rxe_set_page(struct ib_mr *ibmr, u64 addr)
return 0;
}
-static int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
- unsigned int *sg_offset)
+static int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
+ int sg_nents, unsigned int *sg_offset)
{
struct rxe_mem *mr = to_rmr(ibmr);
int n;
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] IB/rxe: IB/core: IB/rdmavt: Fix kernel crash for reg MR
From: Parav Pandit @ 2016-09-28 20:25 UTC (permalink / raw)
To: dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w,
monis-VPRAkNaXOzVWk0Htik3J/w, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: pandit.parav-Re5JQEeQqe8AvxtiuMwx3w
This patch fixes below kernel crash on memory registration for rxe
and other transport drivers which has dma_ops extension.
IB/core invokes ib_map_sg_attrs() in generic manner with dma attributes
which is used by mlx5 and mthca adapters.
However in doing so it ignored honoring dma_ops extension of software based
transports for sg map/unmap operation.
This results into calling dma_map_sg_attrs of hardware virtual device
resulting in crash for null reference.
Fix: It extends core to support sg_map/unmap_attrs and transport drivers
to implement those dma_ops callback functions.
Verified usign perftest applications.
Sep 27 06:54:44 kernel: [53869.549130] BUG: unable to handle kernel NULL pointer dereference at (null)
Sep 27 06:54:44 kernel: [53869.553005] IP: [<ffffffff81032a75>] check_addr+0x35/0x60
Sep 27 06:54:44 kernel: [53869.553005] PGD 3cad8067 PUD 3ce05067 PMD 0
Sep 27 06:54:44 kernel: [53869.553005] Oops: 0000 [#1] SMP
Sep 27 06:54:44 kernel: [53869.553005] Modules linked in: rdma_rxe rdma_ucm ib_umad rdma_cm configfs iw_cm ib_cm ib_uverbs udp_tunnel ib_core binfmt_misc dm_mirror dm_region_hash dm_log dm_mod mousedev evdev psmouse acpi_cpufreq pcspkr button ext4(E) crc16(E) jbd2(E) mbcache(E) ata_piix(E) libata(E) scsi_mod(E) [last unloaded: rdma_rxe]
Sep 27 06:54:44 kernel: [53869.553005] CPU: 0 PID: 32639 Comm: ib_send_bw Tainted: G E 4.8.0-rc6+ #12
Sep 27 06:54:44 kernel: [53869.553005] Hardware name: Xen HVM domU, BIOS 4.2.amazon 05/12/2016
Sep 27 06:54:44 kernel: [53869.553005] task: ffff88003ca45880 task.stack: ffff88003ce88000
Sep 27 06:54:44 kernel: [53869.553005] RIP: 0010:[<ffffffff81032a75>] [<ffffffff81032a75>] check_addr+0x35/0x60
Sep 27 06:54:44 kernel: [53869.553005] RSP: 0018:ffff88003ce8bb90 EFLAGS: 00010246
Sep 27 06:54:44 kernel: [53869.553005] RAX: 0000000000000000 RBX: ffff88003d4e1440 RCX: 0000000000001000
Sep 27 06:54:44 kernel: [53869.553005] RDX: 000000002cf3c000 RSI: ffff88003d012840 RDI: ffffffff81789d7e
Sep 27 06:54:44 kernel: [53869.553005] RBP: ffff88003ce8bbc0 R08: 0000000000000000 R09: ffff88003ce05048
Sep 27 06:54:44 kernel: [53869.553005] R10: 00000000013f4000 R11: 0000000000000002 R12: 0000000000000000
Sep 27 06:54:44 kernel: [53869.553005] R13: 0000000000000002 R14: ffff88003d012840 R15: 0000160000000000
Sep 27 06:54:44 kernel: [53869.553005] FS: 00007f19c7803740(0000) GS:ffff88003e200000(0000) knlGS:0000000000000000
Sep 27 06:54:44 kernel: [53869.553005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 27 06:54:44 kernel: [53869.553005] CR2: 0000000000000000 CR3: 0000000037cb1000 CR4: 00000000001406f0
Sep 27 06:54:44 kernel: [53869.553005] Stack:
Sep 27 06:54:44 kernel: [53869.553005] ffffffff81032b39 ffff880037a98c40 0000000000000000 ffff88003becf280
Sep 27 06:54:44 kernel: [53869.553005] ffff88003bf33000 0000000000000000 ffff88003ce8bc38 ffffffffa02b31c6
Sep 27 06:54:44 kernel: [53869.553005] ffff880037a98c40 ffff88003d012840 0000000000000002 0000000000000000
Sep 27 06:54:44 kernel: [53869.553005] Call Trace:
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff81032b39>] ? nommu_map_sg+0x99/0xd0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa02b31c6>] ib_umem_get+0x3d6/0x470 [ib_core]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa01cc329>] rxe_mem_init_user+0x49/0x270 [rdma_rxe]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa01c793a>] ? rxe_add_index+0xca/0x100 [rdma_rxe]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa01c995f>] rxe_reg_user_mr+0x9f/0x130 [rdma_rxe]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa00419fe>] ib_uverbs_reg_mr+0x14e/0x2c0 [ib_uverbs]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffffa003d3ab>] ib_uverbs_write+0x15b/0x3b0 [ib_uverbs]
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811e92a6>] ? mem_cgroup_commit_charge+0x76/0xe0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811af0a9>] ? page_add_new_anon_rmap+0x89/0xc0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff8117e6c9>] ? lru_cache_add_active_or_unevictable+0x39/0xc0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811f0da8>] __vfs_write+0x28/0x120
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811f1239>] ? rw_verify_area+0x49/0xb0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811f1492>] vfs_write+0xb2/0x1b0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff811f27d6>] SyS_write+0x46/0xa0
Sep 27 06:54:44 kernel: [53869.553005] [<ffffffff814f7d32>] entry_SYSCALL_64_fastpath+0x1a/0xa4
Signed-off-by: Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rdmavt/dma.c | 17 +++++++++++++++++
drivers/infiniband/sw/rxe/rxe_dma.c | 17 +++++++++++++++++
include/rdma/ib_verbs.h | 23 ++++++++++++++++++++---
3 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/sw/rdmavt/dma.c b/drivers/infiniband/sw/rdmavt/dma.c
index 33076a5..01f71ca 100644
--- a/drivers/infiniband/sw/rdmavt/dma.c
+++ b/drivers/infiniband/sw/rdmavt/dma.c
@@ -138,6 +138,21 @@ static void rvt_unmap_sg(struct ib_device *dev,
/* This is a stub, nothing to be done here */
}
+static int rvt_map_sg_attrs(struct ib_device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction direction,
+ unsigned long attrs)
+{
+ return rvt_map_sg(dev, sgl, nents, direction);
+}
+
+static void rvt_unmap_sg_attrs(struct ib_device *dev,
+ struct scatterlist *sg, int nents,
+ enum dma_data_direction direction,
+ unsigned long attrs)
+{
+ return rvt_unmap_sg(dev, sg, nents, direction);
+}
+
static void rvt_sync_single_for_cpu(struct ib_device *dev, u64 addr,
size_t size, enum dma_data_direction dir)
{
@@ -177,6 +192,8 @@ struct ib_dma_mapping_ops rvt_default_dma_mapping_ops = {
.unmap_page = rvt_dma_unmap_page,
.map_sg = rvt_map_sg,
.unmap_sg = rvt_unmap_sg,
+ .map_sg_attrs = rvt_map_sg_attrs,
+ .unmap_sg_attrs = rvt_unmap_sg_attrs,
.sync_single_for_cpu = rvt_sync_single_for_cpu,
.sync_single_for_device = rvt_sync_single_for_device,
.alloc_coherent = rvt_dma_alloc_coherent,
diff --git a/drivers/infiniband/sw/rxe/rxe_dma.c b/drivers/infiniband/sw/rxe/rxe_dma.c
index 7634c1a..a0f8af5 100644
--- a/drivers/infiniband/sw/rxe/rxe_dma.c
+++ b/drivers/infiniband/sw/rxe/rxe_dma.c
@@ -117,6 +117,21 @@ static void rxe_unmap_sg(struct ib_device *dev,
WARN_ON(!valid_dma_direction(direction));
}
+static int rxe_map_sg_attrs(struct ib_device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction direction,
+ unsigned long attrs)
+{
+ return rxe_map_sg(dev, sgl, nents, direction);
+}
+
+static void rxe_unmap_sg_attrs(struct ib_device *dev,
+ struct scatterlist *sg, int nents,
+ enum dma_data_direction direction,
+ unsigned long attrs)
+{
+ rxe_unmap_sg(dev, sg, nents, direction);
+}
+
static void rxe_sync_single_for_cpu(struct ib_device *dev,
u64 addr,
size_t size, enum dma_data_direction dir)
@@ -159,6 +174,8 @@ struct ib_dma_mapping_ops rxe_dma_mapping_ops = {
.unmap_page = rxe_dma_unmap_page,
.map_sg = rxe_map_sg,
.unmap_sg = rxe_unmap_sg,
+ .map_sg_attrs = rxe_map_sg_attrs,
+ .unmap_sg_attrs = rxe_unmap_sg_attrs,
.sync_single_for_cpu = rxe_sync_single_for_cpu,
.sync_single_for_device = rxe_sync_single_for_device,
.alloc_coherent = rxe_dma_alloc_coherent,
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index e1f9673..9e93565 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1739,6 +1739,14 @@ struct ib_dma_mapping_ops {
void (*unmap_sg)(struct ib_device *dev,
struct scatterlist *sg, int nents,
enum dma_data_direction direction);
+ int (*map_sg_attrs)(struct ib_device *dev,
+ struct scatterlist *sg, int nents,
+ enum dma_data_direction direction,
+ unsigned long attrs);
+ void (*unmap_sg_attrs)(struct ib_device *dev,
+ struct scatterlist *sg, int nents,
+ enum dma_data_direction direction,
+ unsigned long attrs);
void (*sync_single_for_cpu)(struct ib_device *dev,
u64 dma_handle,
size_t size,
@@ -3000,8 +3008,12 @@ static inline int ib_dma_map_sg_attrs(struct ib_device *dev,
enum dma_data_direction direction,
unsigned long dma_attrs)
{
- return dma_map_sg_attrs(dev->dma_device, sg, nents, direction,
- dma_attrs);
+ if (dev->dma_ops)
+ return dev->dma_ops->map_sg_attrs(dev, sg, nents, direction,
+ dma_attrs);
+ else
+ return dma_map_sg_attrs(dev->dma_device, sg, nents, direction,
+ dma_attrs);
}
static inline void ib_dma_unmap_sg_attrs(struct ib_device *dev,
@@ -3009,7 +3021,12 @@ static inline void ib_dma_unmap_sg_attrs(struct ib_device *dev,
enum dma_data_direction direction,
unsigned long dma_attrs)
{
- dma_unmap_sg_attrs(dev->dma_device, sg, nents, direction, dma_attrs);
+ if (dev->dma_ops)
+ return dev->dma_ops->unmap_sg_attrs(dev, sg, nents, direction,
+ dma_attrs);
+ else
+ dma_unmap_sg_attrs(dev->dma_device, sg, nents, direction,
+ dma_attrs);
}
/**
* ib_sg_dma_address - Return the DMA address from a scatter/gather entry
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] IB/rxe: Fix sending out loopback packet on netdev interface.
From: Parav Pandit @ 2016-09-28 20:24 UTC (permalink / raw)
To: monis-VPRAkNaXOzVWk0Htik3J/w, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: pandit.parav-Re5JQEeQqe8AvxtiuMwx3w
Bug: prepare4 and prepare6 sets loopback mask in pkt_info structure
instance of skb.
While xmit_packet and other requester side functions use
pkt_info from stack. Due to which loopback mask bit set to wrong
pkt_info structure. This resuilts into sending out packet to actual
netdev and loopback functionality is broken.
Fix: Similar to other requester side functions, prepare() variants to
set loopback mask bit on pkt_info structure used by other functions.
Verified with perftest applications.
Signed-off-by: Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rxe/rxe_net.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
index eedf2f1..549aa84 100644
--- a/drivers/infiniband/sw/rxe/rxe_net.c
+++ b/drivers/infiniband/sw/rxe/rxe_net.c
@@ -350,14 +350,14 @@ static void prepare_ipv6_hdr(struct dst_entry *dst, struct sk_buff *skb,
ip6h->payload_len = htons(skb->len - sizeof(*ip6h));
}
-static int prepare4(struct rxe_dev *rxe, struct sk_buff *skb, struct rxe_av *av)
+static int prepare4(struct rxe_dev *rxe, struct rxe_pkt_info *pkt,
+ struct sk_buff *skb, struct rxe_av *av)
{
struct dst_entry *dst;
bool xnet = false;
__be16 df = htons(IP_DF);
struct in_addr *saddr = &av->sgid_addr._sockaddr_in.sin_addr;
struct in_addr *daddr = &av->dgid_addr._sockaddr_in.sin_addr;
- struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
dst = rxe_find_route4(rxe->ndev, saddr, daddr);
if (!dst) {
@@ -376,12 +376,12 @@ static int prepare4(struct rxe_dev *rxe, struct sk_buff *skb, struct rxe_av *av)
return 0;
}
-static int prepare6(struct rxe_dev *rxe, struct sk_buff *skb, struct rxe_av *av)
+static int prepare6(struct rxe_dev *rxe, struct rxe_pkt_info *pkt,
+ struct sk_buff *skb, struct rxe_av *av)
{
struct dst_entry *dst;
struct in6_addr *saddr = &av->sgid_addr._sockaddr_in6.sin6_addr;
struct in6_addr *daddr = &av->dgid_addr._sockaddr_in6.sin6_addr;
- struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
dst = rxe_find_route6(rxe->ndev, saddr, daddr);
if (!dst) {
@@ -408,9 +408,9 @@ static int prepare(struct rxe_dev *rxe, struct rxe_pkt_info *pkt,
struct rxe_av *av = rxe_get_av(pkt);
if (av->network_type == RDMA_NETWORK_IPV4)
- err = prepare4(rxe, skb, av);
+ err = prepare4(rxe, pkt, skb, av);
else if (av->network_type == RDMA_NETWORK_IPV6)
- err = prepare6(rxe, skb, av);
+ err = prepare6(rxe, pkt, skb, av);
*crc = rxe_icrc_hdr(pkt, skb);
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] IB/rxe: Avoid scheduling tasklet for userspace QP
From: Parav Pandit @ 2016-09-28 20:24 UTC (permalink / raw)
To: monis-VPRAkNaXOzVWk0Htik3J/w, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: pandit.parav-Re5JQEeQqe8AvxtiuMwx3w
This patch avoids scheduing tasklet for WQE and protocol processing
for user space QP. It performs the task in calling process context.
To improve code readability kernel spacific post_send handling moved to
post_send_kernel() function.
Signed-off-by: Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rxe/rxe_verbs.c | 38 +++++++++++++++++++++++------------
1 file changed, 25 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 4552be9..a5af691 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -801,26 +801,15 @@ err1:
return err;
}
-static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
- struct ib_send_wr **bad_wr)
+static int rxe_post_send_kernel(struct rxe_qp *qp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr)
{
int err = 0;
- struct rxe_qp *qp = to_rqp(ibqp);
unsigned int mask;
unsigned int length = 0;
int i;
int must_sched;
- if (unlikely(!qp->valid)) {
- *bad_wr = wr;
- return -EINVAL;
- }
-
- if (unlikely(qp->req.state < QP_STATE_READY)) {
- *bad_wr = wr;
- return -EINVAL;
- }
-
while (wr) {
mask = wr_opcode_mask(wr->opcode, qp);
if (unlikely(!mask)) {
@@ -861,6 +850,29 @@ static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
return err;
}
+static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr)
+{
+ struct rxe_qp *qp = to_rqp(ibqp);
+
+ if (unlikely(!qp->valid)) {
+ *bad_wr = wr;
+ return -EINVAL;
+ }
+
+ if (unlikely(qp->req.state < QP_STATE_READY)) {
+ *bad_wr = wr;
+ return -EINVAL;
+ }
+
+ if (qp->is_user) {
+ /* Utilize process context to do protocol processing */
+ rxe_run_task(&qp->req.task, 0);
+ return 0;
+ } else
+ return rxe_post_send_kernel(qp, wr, bad_wr);
+}
+
static int rxe_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
struct ib_recv_wr **bad_wr)
{
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH 11/13] srp_daemon: Add the debian initscripts as an option
From: Bart Van Assche @ 2016-09-28 20:18 UTC (permalink / raw)
To: Jason Gunthorpe, Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <20160928184708.GA31472-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On 09/28/2016 11:47 AM, Jason Gunthorpe wrote:
> To be clear, there is already an initscript that (perhaps?) is for
> RedHat - but it isn't even close to the Debian version. So this patch
> introduces two scripts for srp_dameon, which I hated doing..
The existing init script works reliably on multiple versions of RHEL,
SLES, Fedora and openSuSE. It is not specific to Red Hat distro's. So
I'm surprised that a separate initscript is needed for Debian?
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Towards qedr patch series
From: Doug Ledford @ 2016-09-28 19:38 UTC (permalink / raw)
To: Amrani Ram
Cc: Elior Ariel, Kalderon Michal, Mintz Yuval, Borundia Rajesh,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <DM3PR07MB2201E202E535C987EEAA3BFDF8CA0-jBX7zO+Db/wI9Q/2ysJlLeFPX92sqiQdvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 2017 bytes --]
On 9/25/16 1:39 AM, Amrani, Ram wrote:
> Hi Doug,
> We will release qedr RFC v3 this week that will address all of the comments received on v1 and v2. Following this RFC we would like to generate a patch series.
> Note that the RFC contains patches to qed and qede that are maintained in netdev so the submission may be affected by netdev activity.
> Currently the RFC is based on netdev's net-next but we can rebase against your git (git://github.com/dledford/linux.git).
>
> What guidelines should we follow with this submission?
> What is the time frame to make it into kernel 4.9?
There are currently 5 (5!!!) entirely new drivers that people want to
get merged. Reviewing those monstrous beasts is a nightmare. The
chances that I will get to yours for 4.9 are slim at best.
However, on the off chance that I might get to it, base the qed patches
against Dave's net-next (up until the merge window opens, then base it
on his net tree as that's what will get merged). The qedr patches
should be based on my current k.o/for-4.9 branch off of k.o. When you
submit any patches, I would suggest doing it in two separate submissions
(if possible). One that is just the preparatory qed patches. They
should be sent to netdev@ and linux-rdma@. The submission should note
that they are intended to be taken with the qedr driver. If netdev@
reviews/acks the series, then Dave can let me know if he wants me to
take the series and keep with with the remaining qedr series, or if he
wants to take them himself and I just make sure my pull request of the
qedr patches goes to Linus after his. The second set would obviously be
the new qedr driver. It need not go to netdev. Once it's here, then
it's a matter of the netdev@ people signing off on the qed series
changes, me signing off on the qedr changes, then it can go in.
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> GPG Key ID: 0E572FDD
Red Hat, Inc.
100 E. Davie St
Raleigh, NC 27601 USA
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 907 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox