* Re: [PATCH infiniband-diags v3] ibportstate: Fixed switch peer port probing when using DR routing
From: Dag Moxnes @ 2016-10-24 14:37 UTC (permalink / raw)
To: Hal Rosenstock, Knut Omang, Ira Weiny
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Line Holen
In-Reply-To: <02e18a78-ba54-d7a2-c3fe-3e4d392f5593-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On 10/24/2016 03:50 PM, Hal Rosenstock wrote:
> On 10/24/2016 9:34 AM, Knut Omang wrote:
>> From: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>>
>> ibportstate queries to a remote peer port on a switch using direct
>> routing would result in timeouts. The reason for this is that the
>> DR path was not correctly constructed.
>>
>> Signed-off-by: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> Reviewed-by: Line Holen <line.holen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Knut Omang <knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> ---
>> src/ibportstate.c | 26 +++++++++++++++++---------
>> 1 file changed, 17 insertions(+), 9 deletions(-)
>>
>> diff --git a/src/ibportstate.c b/src/ibportstate.c
>> index cfb8be7..82cbcc2 100644
>> --- a/src/ibportstate.c
>> +++ b/src/ibportstate.c
>> @@ -1,6 +1,7 @@
>> /*
>> * Copyright (c) 2004-2009 Voltaire Inc. All rights reserved.
>> * Copyright (c) 2010,2011 Mellanox Technologies LTD. All rights reserved.
>> + * Copyright (c) 2011,2016 Oracle and/or its affiliates. All rights reserved.
>> *
>> * This software is available to you under a choice of one of two
>> * licenses. You may choose to be licensed under the terms of the GNU
>> @@ -655,15 +656,22 @@ int main(int argc, char **argv)
>>
>> /* Setup portid for peer port */
>> memcpy(&peerportid, &portid, sizeof(peerportid));
>> - peerportid.drpath.cnt = 1;
>> - peerportid.drpath.p[1] = (uint8_t) portnum;
>> -
>> - /* Set DrSLID to local lid */
>> - if (resolve_self(ibd_ca, ibd_ca_port, &selfportid,
>> - &selfport, 0) < 0)
>> - IBEXIT("could not resolve self");
>> - peerportid.drpath.drslid = (uint16_t) selfportid.lid;
>> - peerportid.drpath.drdlid = 0xffff;
>> + if (portid.lid == 0) {
>> + peerportid.drpath.cnt++;
>> + if (peerportid.drpath.cnt == IB_SUBNET_PATH_HOPS_MAX) {
>> + IBEXIT("Too many hops");
>> + }
>> + } else {
>> + peerportid.drpath.cnt = 1;
>> +
>> + /* Set DrSLID to local lid */
>> + if (resolve_self(ibd_ca, ibd_ca_port, &selfportid,
>> + &selfport, 0) < 0)
>> + IBEXIT("could not resolve self");
>> + peerportid.drpath.drslid = selfportid.lid;
> Why was casting of selfportid.lid to (uint16_t) dropped ?
That was not intentional. Thanks for pointing it out.
Regards,
-Dag
>
>> + peerportid.drpath.drdlid = 0xffff;
>> + }
>> + peerportid.drpath.p[peerportid.drpath.cnt] = (uint8_t) portnum;
>>
>> /* Get peer port NodeInfo to obtain peer port number */
>> is_peer_switch = get_node_info(&peerportid, data);
>>
>> base-commit: 17e03b4738913365a3f947719c4897fcb92df32c
>>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCH RFC v2 3/3] nvme-rdma: use rdma_reject_msg() to log connection rejects
From: Steve Wise @ 2016-10-24 14:57 UTC (permalink / raw)
To: 'Christoph Hellwig'
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagig-NQWnxTmZq1alnMjI0IkVqw, axboe-b10kYP2dOMg
In-Reply-To: <005701d22c7f$0c883ed0$2598bc70$@opengridcomputing.com>
> > Given the nasty casting issues in the current RDMA/CM API maybe we
> should
> > actually expand the scope of the rdma_consumer_reject helper to include
> > the above check, e.g. check that there is a private data len and then
> > return a pointer to the private data?
>
> An application could reject and not provide private data, so I think we
> need 3 helpers (so far):
>
> rdma_reject_msg() - protocol reject reason string
> rdma_is_consumer_reject() - true if the peer consumer/ulp rejected
> rdma_consumer_reject_data() - ptr to any private data
>
> Sound good?
Or these 3 could be rolled into one uber function that returns true if the
consumer rejected, and sets 2 pointers passed in: one to the protocol reject
string, and one to the private data, if any. If the something like like this:
bool rdma_reject_info(struct rdma_cm_id *id, int reason, char **protocol_msg,
char **consumer_data)
Kinda ugly, but only one call is needed vs 3 calls to get this info...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH RFC v2 3/3] nvme-rdma: use rdma_reject_msg() to log connection rejects
From: 'Christoph Hellwig' @ 2016-10-24 15:09 UTC (permalink / raw)
To: Steve Wise
Cc: 'Christoph Hellwig', dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagig-NQWnxTmZq1alnMjI0IkVqw, axboe-b10kYP2dOMg
In-Reply-To: <011d01d22e06$fd606480$f8212d80$@opengridcomputing.com>
On Mon, Oct 24, 2016 at 09:57:50AM -0500, Steve Wise wrote:
> > rdma_reject_msg() - protocol reject reason string
This one sounds useful on it's own.
> > rdma_is_consumer_reject() - true if the peer consumer/ulp rejected
> > rdma_consumer_reject_data() - ptr to any private data
I see why these are conceptually different, but why do we care
if something is a consumer reject except for printing what reject
we got (solved by rdma_reject_msg) or for getting consumer reject
data if there is any (solved by rdma_consumer_reject_data).
So all three helpers are fine with me, but rdma_is_consumer_reject
would be more of a low-level helper that drivers wouldn't use
directly, only through rdma_consumer_reject_data.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] IB/mlx5: avoid bogus -Wmaybe-uninitialized warning
From: Arnd Bergmann @ 2016-10-24 15:16 UTC (permalink / raw)
To: Matan Barak, Leon Romanovsky
Cc: Arnd Bergmann, Doug Ledford, Sean Hefty, Hal Rosenstock,
Sagi Grimberg, Bart Van Assche, Noa Osherovich, Saeed Mahameed,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
We get a false-positive warning in linux-next for the mlx5 driver:
infiniband/hw/mlx5/mr.c: In function ‘mlx5_ib_reg_user_mr’:
infiniband/hw/mlx5/mr.c:1172:5: error: ‘order’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1161:6: note: ‘order’ was declared here
infiniband/hw/mlx5/mr.c:1173:6: error: ‘ncont’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1160:6: note: ‘ncont’ was declared here
infiniband/hw/mlx5/mr.c:1173:6: error: ‘page_shift’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1158:6: note: ‘page_shift’ was declared here
infiniband/hw/mlx5/mr.c:1143:13: error: ‘npages’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1159:6: note: ‘npages’ was declared here
I had a trivial workaround for gcc-5 or higher, but that didn't work
on gcc-4.9 unfortunately.
The only way I found to avoid the warnings for gcc-4.9, short of
initializing each of the arguments first was to change the calling
conventions to separate the error code from the umem pointer. This
avoids casting the error codes from one pointer to another incompatible
pointer, and lets gcc figure out when that the data is actually valid
whenever we return successfully.
Signed-off-by: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
---
drivers/infiniband/hw/mlx5/mr.c | 39 +++++++++++++++++++++------------------
1 file changed, 21 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index d4ad672b905b..88d8d292677b 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -815,29 +815,33 @@ static void prep_umr_unreg_wqe(struct mlx5_ib_dev *dev,
umrwr->mkey = key;
}
-static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
- int access_flags, int *npages,
- int *page_shift, int *ncont, int *order)
+static int mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
+ int access_flags, struct ib_umem ** umem,
+ int *npages, int *page_shift, int *ncont,
+ int *order)
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
- struct ib_umem *umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags, 0);
- if (IS_ERR(umem)) {
+ int err;
+
+ *umem = ib_umem_get(pd->uobject->context, start, length,
+ access_flags, 0);
+ err = PTR_ERR_OR_ZERO(*umem);
+ if (err < 0) {
mlx5_ib_err(dev, "umem get failed (%ld)\n", PTR_ERR(umem));
- return (void *)umem;
+ return err;
}
- mlx5_ib_cont_pages(umem, start, npages, page_shift, ncont, order);
+ mlx5_ib_cont_pages(*umem, start, npages, page_shift, ncont, order);
if (!*npages) {
mlx5_ib_warn(dev, "avoid zero region\n");
- ib_umem_release(umem);
- return ERR_PTR(-EINVAL);
+ ib_umem_release(*umem);
+ return -EINVAL;
}
mlx5_ib_dbg(dev, "npages %d, ncont %d, order %d, page_shift %d\n",
*npages, *ncont, *order, *page_shift);
- return umem;
+ return 0;
}
static void mlx5_ib_umr_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -1163,11 +1167,11 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mlx5_ib_dbg(dev, "start 0x%llx, virt_addr 0x%llx, length 0x%llx, access_flags 0x%x\n",
start, virt_addr, length, access_flags);
- umem = mr_umem_get(pd, start, length, access_flags, &npages,
+ err = mr_umem_get(pd, start, length, access_flags, &umem, &npages,
&page_shift, &ncont, &order);
- if (IS_ERR(umem))
- return (void *)umem;
+ if (err < 0)
+ return ERR_PTR(err);
if (use_umr(order)) {
mr = reg_umr(pd, umem, virt_addr, length, ncont, page_shift,
@@ -1341,10 +1345,9 @@ int mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
*/
flags |= IB_MR_REREG_TRANS;
ib_umem_release(mr->umem);
- mr->umem = mr_umem_get(pd, addr, len, access_flags, &npages,
- &page_shift, &ncont, &order);
- if (IS_ERR(mr->umem)) {
- err = PTR_ERR(mr->umem);
+ err = mr_umem_get(pd, addr, len, access_flags, &mr->umem,
+ &npages, &page_shift, &ncont, &order);
+ if (err < 0) {
mr->umem = NULL;
return err;
}
--
2.9.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH infiniband-diags v4] ibportstate: Fixed switch peer port probing when using DR routing
From: Knut Omang @ 2016-10-24 15:38 UTC (permalink / raw)
To: Ira Weiny
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Line Holen, Knut Omang,
Dag Moxnes
From: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
ibportstate queries to a remote peer port on a switch using direct
routing would result in timeouts. The reason for this is that the
DR path was not correctly constructed.
Signed-off-by: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Reviewed-by: Line Holen <line.holen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Knut Omang <knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
src/ibportstate.c | 26 +++++++++++++++++---------
1 file changed, 17 insertions(+), 9 deletions(-)
diff --git a/src/ibportstate.c b/src/ibportstate.c
index cfb8be7..df50469 100644
--- a/src/ibportstate.c
+++ b/src/ibportstate.c
@@ -1,6 +1,7 @@
/*
* Copyright (c) 2004-2009 Voltaire Inc. All rights reserved.
* Copyright (c) 2010,2011 Mellanox Technologies LTD. All rights reserved.
+ * Copyright (c) 2011,2016 Oracle and/or its affiliates. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -655,15 +656,22 @@ int main(int argc, char **argv)
/* Setup portid for peer port */
memcpy(&peerportid, &portid, sizeof(peerportid));
- peerportid.drpath.cnt = 1;
- peerportid.drpath.p[1] = (uint8_t) portnum;
-
- /* Set DrSLID to local lid */
- if (resolve_self(ibd_ca, ibd_ca_port, &selfportid,
- &selfport, 0) < 0)
- IBEXIT("could not resolve self");
- peerportid.drpath.drslid = (uint16_t) selfportid.lid;
- peerportid.drpath.drdlid = 0xffff;
+ if (portid.lid == 0) {
+ peerportid.drpath.cnt++;
+ if (peerportid.drpath.cnt == IB_SUBNET_PATH_HOPS_MAX) {
+ IBEXIT("Too many hops");
+ }
+ } else {
+ peerportid.drpath.cnt = 1;
+
+ /* Set DrSLID to local lid */
+ if (resolve_self(ibd_ca, ibd_ca_port, &selfportid,
+ &selfport, 0) < 0)
+ IBEXIT("could not resolve self");
+ peerportid.drpath.drslid = (uint16_t) selfportid.lid;
+ peerportid.drpath.drdlid = 0xffff;
+ }
+ peerportid.drpath.p[peerportid.drpath.cnt] = (uint8_t) portnum;
/* Get peer port NodeInfo to obtain peer port number */
is_peer_switch = get_node_info(&peerportid, data);
base-commit: 17e03b4738913365a3f947719c4897fcb92df32c
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] IB/mlx5: avoid bogus -Wmaybe-uninitialized warning
From: Leon Romanovsky @ 2016-10-24 17:06 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Matan Barak, Doug Ledford, Sean Hefty, Hal Rosenstock,
Sagi Grimberg, Bart Van Assche, Noa Osherovich, Saeed Mahameed,
linux-rdma, linux-kernel
In-Reply-To: <20161024151658.2413803-1-arnd@arndb.de>
[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]
On Mon, Oct 24, 2016 at 05:16:42PM +0200, Arnd Bergmann wrote:
> We get a false-positive warning in linux-next for the mlx5 driver:
>
> infiniband/hw/mlx5/mr.c: In function ‘mlx5_ib_reg_user_mr’:
> infiniband/hw/mlx5/mr.c:1172:5: error: ‘order’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1161:6: note: ‘order’ was declared here
> infiniband/hw/mlx5/mr.c:1173:6: error: ‘ncont’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1160:6: note: ‘ncont’ was declared here
> infiniband/hw/mlx5/mr.c:1173:6: error: ‘page_shift’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1158:6: note: ‘page_shift’ was declared here
> infiniband/hw/mlx5/mr.c:1143:13: error: ‘npages’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1159:6: note: ‘npages’ was declared here
>
> I had a trivial workaround for gcc-5 or higher, but that didn't work
> on gcc-4.9 unfortunately.
>
> The only way I found to avoid the warnings for gcc-4.9, short of
> initializing each of the arguments first was to change the calling
> conventions to separate the error code from the umem pointer. This
> avoids casting the error codes from one pointer to another incompatible
> pointer, and lets gcc figure out when that the data is actually valid
> whenever we return successfully.
>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Thanks Arnd for fixing it.
I have a very small comment which is not related to functionality.
Rather than that,
Acked-by: Leon Romanovsky <leonro@mellanox.com>
> ---
> drivers/infiniband/hw/mlx5/mr.c | 39 +++++++++++++++++++++------------------
> 1 file changed, 21 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
> index d4ad672b905b..88d8d292677b 100644
> --- a/drivers/infiniband/hw/mlx5/mr.c
> +++ b/drivers/infiniband/hw/mlx5/mr.c
> @@ -815,29 +815,33 @@ static void prep_umr_unreg_wqe(struct mlx5_ib_dev *dev,
> umrwr->mkey = key;
> }
>
> -static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
> - int access_flags, int *npages,
> - int *page_shift, int *ncont, int *order)
> +static int mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
> + int access_flags, struct ib_umem ** umem,
I wonder if checkpatch does differentiate between "struct ib_umem ** umem"
and "struct ib_umem **umem". According to coding style, the second is preferable.
> + int *npages, int *page_shift, int *ncont,
> + int *order)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH infiniband-diags v4] ibportstate: Fixed switch peer port probing when using DR routing
From: Hal Rosenstock @ 2016-10-24 17:18 UTC (permalink / raw)
To: Knut Omang, Ira Weiny
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Line Holen, Dag Moxnes
In-Reply-To: <e4f9a5f6c7f55bd65574723956298f588a964f6f.1477323407.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
On 10/24/2016 11:38 AM, Knut Omang wrote:
> From: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>
> ibportstate queries to a remote peer port on a switch using direct
> routing would result in timeouts. The reason for this is that the
> DR path was not correctly constructed.
>
> Signed-off-by: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Line Holen <line.holen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Knut Omang <knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Reviewed-by: Hal Rosenstock <hal-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCH RFC v2 0/3] connect reject event helpers
From: Steve Wise @ 2016-10-24 17:44 UTC (permalink / raw)
To: 'Sagi Grimberg', dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, axboe-b10kYP2dOMg,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ, hch-jcswGhMUV9g
In-Reply-To: <001701d22be6$4854b8b0$d8fe2a10$@opengridcomputing.com>
> >
> > > While reviewing:
> > >
> > >
> http://lists.infradead.org/pipermail/linux-nvme/2016-October/006681.html
> > >
> > > I decided to propose transport-agnostic helper functions to better
> > > handle connection reject event information. I've included a nvme_rdma
> > > patch to utilize the new helpers.
> > >
> > > Thoughts?
> >
> > Hey Steve,
> >
> > This looks nice and useful. Would be great if you can
> > also help other ULPs that use this (e.g. srp/srpt)
>
> can-do!
Actually, srp uses the ib_cm directly. Further it already has logic to do
various actions based on the IB_CM Reject status. It really doesn't need these
helpers. The only one would be ibcm_reject_msg(). Looking at the iser
initiator, it logs nothing on a reject event. And nfsrdma doesn't log any
details about the reject reason also.
I could add a dev_warn() or something for iser and nfsrdma.
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH RFC v2 0/3] connect reject event helpers
From: Chuck Lever @ 2016-10-24 17:52 UTC (permalink / raw)
To: Steve Wise
Cc: Sagi Grimberg, Doug Ledford, Sean Hefty, List Linux RDMA Mailing,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, axboe-b10kYP2dOMg,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ, hch-jcswGhMUV9g
In-Reply-To: <01aa01d22e1e$46b0bc90$d41235b0$@opengridcomputing.com>
> On Oct 24, 2016, at 1:44 PM, Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> wrote:
>
>>>
>>>> While reviewing:
>>>>
>>>>
>> http://lists.infradead.org/pipermail/linux-nvme/2016-October/006681.html
>>>>
>>>> I decided to propose transport-agnostic helper functions to better
>>>> handle connection reject event information. I've included a nvme_rdma
>>>> patch to utilize the new helpers.
>>>>
>>>> Thoughts?
>>>
>>> Hey Steve,
>>>
>>> This looks nice and useful. Would be great if you can
>>> also help other ULPs that use this (e.g. srp/srpt)
>>
>> can-do!
>
> Actually, srp uses the ib_cm directly. Further it already has logic to do
> various actions based on the IB_CM Reject status. It really doesn't need these
> helpers. The only one would be ibcm_reject_msg(). Looking at the iser
> initiator, it logs nothing on a reject event. And nfsrdma doesn't log any
> details about the reject reason also.
>
> I could add a dev_warn() or something for iser and nfsrdma.
I'm fixing an issue in this area. I'd like to take care of
the xprtrdma connection upcall myself once you've got these
new APIs introduced.
--
Chuck Lever
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCH RFC v2 0/3] connect reject event helpers
From: Steve Wise @ 2016-10-24 17:57 UTC (permalink / raw)
To: 'Chuck Lever'
Cc: 'Sagi Grimberg', 'Doug Ledford',
'Sean Hefty', 'List Linux RDMA Mailing',
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, axboe-b10kYP2dOMg,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ, hch-jcswGhMUV9g
In-Reply-To: <2F5E122E-0B50-4264-8E44-3D484B0282F0-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> >>> Hey Steve,
> >>>
> >>> This looks nice and useful. Would be great if you can
> >>> also help other ULPs that use this (e.g. srp/srpt)
> >>
> >> can-do!
> >
> > Actually, srp uses the ib_cm directly. Further it already has logic to do
> > various actions based on the IB_CM Reject status. It really doesn't need
these
> > helpers. The only one would be ibcm_reject_msg(). Looking at the iser
> > initiator, it logs nothing on a reject event. And nfsrdma doesn't log any
> > details about the reject reason also.
> >
> > I could add a dev_warn() or something for iser and nfsrdma.
>
> I'm fixing an issue in this area. I'd like to take care of
> the xprtrdma connection upcall myself once you've got these
> new APIs introduced.
Sure, not a problem!
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH v3 1/6] rdma_cm: add rdma_reject_msg() helper function
From: Steve Wise @ 2016-10-24 18:59 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
rdma_reject_msg() returns a pointer to a string message associated with
the transport reject reason codes.
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/core/cm.c | 48 ++++++++++++++++++++++++++++++++++++++++++
drivers/infiniband/core/cma.c | 14 ++++++++++++
drivers/infiniband/core/iwcm.c | 21 ++++++++++++++++++
include/rdma/ib_cm.h | 6 ++++++
include/rdma/iw_cm.h | 6 ++++++
include/rdma/rdma_cm.h | 8 +++++++
6 files changed, 103 insertions(+)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index c995255..98bcc9c 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -57,6 +57,54 @@ MODULE_AUTHOR("Sean Hefty");
MODULE_DESCRIPTION("InfiniBand CM");
MODULE_LICENSE("Dual BSD/GPL");
+static const char * const ibcm_rej_reason_strs[] = {
+ [IB_CM_REJ_NO_QP] = "no qp",
+ [IB_CM_REJ_NO_EEC] = "no eec",
+ [IB_CM_REJ_NO_RESOURCES] = "no resources",
+ [IB_CM_REJ_TIMEOUT] = "timeout",
+ [IB_CM_REJ_UNSUPPORTED] = "unsupported",
+ [IB_CM_REJ_INVALID_COMM_ID] = "invalid comm id",
+ [IB_CM_REJ_INVALID_COMM_INSTANCE] = "invalid comm instance",
+ [IB_CM_REJ_INVALID_SERVICE_ID] = "invalid service id",
+ [IB_CM_REJ_INVALID_TRANSPORT_TYPE] = "invalid transport type",
+ [IB_CM_REJ_STALE_CONN] = "stale conn",
+ [IB_CM_REJ_RDC_NOT_EXIST] = "rdc not exist",
+ [IB_CM_REJ_INVALID_GID] = "invalid gid",
+ [IB_CM_REJ_INVALID_LID] = "invalid lid",
+ [IB_CM_REJ_INVALID_SL] = "invalid sl",
+ [IB_CM_REJ_INVALID_TRAFFIC_CLASS] = "invalid traffic class",
+ [IB_CM_REJ_INVALID_HOP_LIMIT] = "invalid hop limit",
+ [IB_CM_REJ_INVALID_PACKET_RATE] = "invalid packet rate",
+ [IB_CM_REJ_INVALID_ALT_GID] = "invalid alt gid",
+ [IB_CM_REJ_INVALID_ALT_LID] = "invalid alt lid",
+ [IB_CM_REJ_INVALID_ALT_SL] = "invalid alt sl",
+ [IB_CM_REJ_INVALID_ALT_TRAFFIC_CLASS] = "invalid alt traffic class",
+ [IB_CM_REJ_INVALID_ALT_HOP_LIMIT] = "invalid alt hop limit",
+ [IB_CM_REJ_INVALID_ALT_PACKET_RATE] = "invalid alt packet rate",
+ [IB_CM_REJ_PORT_CM_REDIRECT] = "port cm redirect",
+ [IB_CM_REJ_PORT_REDIRECT] = "port redirect",
+ [IB_CM_REJ_INVALID_MTU] = "invalid mtu",
+ [IB_CM_REJ_INSUFFICIENT_RESP_RESOURCES] = "insufficient resp resources",
+ [IB_CM_REJ_CONSUMER_DEFINED] = "consumer defined",
+ [IB_CM_REJ_INVALID_RNR_RETRY] = "invalid rnr retry",
+ [IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID] = "duplicate local comm id",
+ [IB_CM_REJ_INVALID_CLASS_VERSION] = "invalid class version",
+ [IB_CM_REJ_INVALID_FLOW_LABEL] = "invalid flow label",
+ [IB_CM_REJ_INVALID_ALT_FLOW_LABEL] = "invalid alt flow label",
+};
+
+const char *__attribute_const__ ibcm_reject_msg(int reason)
+{
+ size_t index = reason;
+
+ if (index >= ARRAY_SIZE(ibcm_rej_reason_strs) ||
+ !ibcm_rej_reason_strs[index])
+ return "unrecognized reason";
+ else
+ return ibcm_rej_reason_strs[index];
+}
+EXPORT_SYMBOL(ibcm_reject_msg);
+
static void cm_add_one(struct ib_device *device);
static void cm_remove_one(struct ib_device *device, void *client_data);
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 5f65a78..831e94b 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -101,6 +101,20 @@ const char *__attribute_const__ rdma_event_msg(enum rdma_cm_event_type event)
}
EXPORT_SYMBOL(rdma_event_msg);
+const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
+ int reason)
+{
+ if (rdma_ib_or_roce(id->device, id->port_num))
+ return ibcm_reject_msg(reason);
+
+ if (rdma_protocol_iwarp(id->device, id->port_num))
+ return iwcm_reject_msg(reason);
+
+ WARN_ON_ONCE(1);
+ return "unrecognized reason";
+}
+EXPORT_SYMBOL(rdma_reject_msg);
+
static void cma_add_one(struct ib_device *device);
static void cma_remove_one(struct ib_device *device, void *client_data);
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
index 357624f..8d01253 100644
--- a/drivers/infiniband/core/iwcm.c
+++ b/drivers/infiniband/core/iwcm.c
@@ -59,6 +59,27 @@ MODULE_AUTHOR("Tom Tucker");
MODULE_DESCRIPTION("iWARP CM");
MODULE_LICENSE("Dual BSD/GPL");
+static const char * const iwcm_rej_reason_strs[] = {
+ [ECONNRESET] = "reset by remote host",
+ [ECONNREFUSED] = "refused by remote application",
+ [ETIMEDOUT] = "setup timeout",
+};
+
+const char *__attribute_const__ iwcm_reject_msg(int reason)
+{
+ size_t index;
+
+ /* iWARP uses negative errnos */
+ index = -reason;
+
+ if (index >= ARRAY_SIZE(iwcm_rej_reason_strs) ||
+ !iwcm_rej_reason_strs[index])
+ return "unrecognized reason";
+ else
+ return iwcm_rej_reason_strs[index];
+}
+EXPORT_SYMBOL(iwcm_reject_msg);
+
static struct ibnl_client_cbs iwcm_nl_cb_table[] = {
[RDMA_NL_IWPM_REG_PID] = {.dump = iwpm_register_pid_cb},
[RDMA_NL_IWPM_ADD_MAPPING] = {.dump = iwpm_add_mapping_cb},
diff --git a/include/rdma/ib_cm.h b/include/rdma/ib_cm.h
index 92a7d85..b49258b 100644
--- a/include/rdma/ib_cm.h
+++ b/include/rdma/ib_cm.h
@@ -603,4 +603,10 @@ struct ib_cm_sidr_rep_param {
int ib_send_cm_sidr_rep(struct ib_cm_id *cm_id,
struct ib_cm_sidr_rep_param *param);
+/**
+ * ibcm_reject_msg - return a pointer to a reject message string.
+ * @reason: Value returned in the REJECT event status field.
+ */
+const char *__attribute_const__ ibcm_reject_msg(int reason);
+
#endif /* IB_CM_H */
diff --git a/include/rdma/iw_cm.h b/include/rdma/iw_cm.h
index 6d0065c..5cd7701 100644
--- a/include/rdma/iw_cm.h
+++ b/include/rdma/iw_cm.h
@@ -253,4 +253,10 @@ int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt);
int iw_cm_init_qp_attr(struct iw_cm_id *cm_id, struct ib_qp_attr *qp_attr,
int *qp_attr_mask);
+/**
+ * iwcm_reject_msg - return a pointer to a reject message string.
+ * @reason: Value returned in the REJECT event status field.
+ */
+const char *__attribute_const__ iwcm_reject_msg(int reason);
+
#endif /* IW_CM_H */
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index 81fb1d1..f11a768 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -388,4 +388,12 @@ int rdma_set_afonly(struct rdma_cm_id *id, int afonly);
*/
__be64 rdma_get_service_id(struct rdma_cm_id *id, struct sockaddr *addr);
+/**
+ * rdma_reject_msg - return a pointer to a reject message string.
+ * @id: Communication identifier that received the REJECT event.
+ * @reason: Value returned in the REJECT event status field.
+ */
+const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
+ int reason);
+
#endif /* RDMA_CM_H */
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 2/6] rdma_cm: add rdma_is_consumer_reject() helper function
From: Steve Wise @ 2016-10-24 18:59 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
Return true if the peer consumer application rejected the
connection attempt.
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/core/cma.c | 13 +++++++++++++
include/rdma/rdma_cm.h | 7 +++++++
2 files changed, 20 insertions(+)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 831e94b..0d88c9d 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -115,6 +115,19 @@ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
}
EXPORT_SYMBOL(rdma_reject_msg);
+bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason)
+{
+ if (rdma_ib_or_roce(id->device, id->port_num))
+ return reason == IB_CM_REJ_CONSUMER_DEFINED;
+
+ if (rdma_protocol_iwarp(id->device, id->port_num))
+ return reason == -ECONNREFUSED;
+
+ WARN_ON_ONCE(1);
+ return false;
+}
+EXPORT_SYMBOL(rdma_is_consumer_reject);
+
static void cma_add_one(struct ib_device *device);
static void cma_remove_one(struct ib_device *device, void *client_data);
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index f11a768..62039c2 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -395,5 +395,12 @@ __be64 rdma_get_service_id(struct rdma_cm_id *id, struct sockaddr *addr);
*/
const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
int reason);
+/**
+ * rdma_is_consumer_reject - return true if the consumer rejected the connect
+ * request.
+ * @id: Communication identifier that received the REJECT event.
+ * @reason: Value returned in the REJECT event status field.
+ */
+bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason);
#endif /* RDMA_CM_H */
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 3/6] rdma_cm: add rdma_consumer_reject_data helper function
From: Steve Wise @ 2016-10-24 18:59 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
rdma_consumer_reject_data() will return the private data pointer
and length if any is available.
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/core/cma.c | 16 ++++++++++++++++
include/rdma/rdma_cm.h | 10 ++++++++++
2 files changed, 26 insertions(+)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 0d88c9d..2536d42 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -128,6 +128,22 @@ bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason)
}
EXPORT_SYMBOL(rdma_is_consumer_reject);
+const void *rdma_consumer_reject_data(struct rdma_cm_id *id,
+ struct rdma_cm_event *ev, u8 *data_len)
+{
+ void *p;
+
+ if (rdma_is_consumer_reject(id, ev->status)) {
+ *data_len = ev->param.conn.private_data_len;
+ p = ev->param.conn.private_data;
+ } else {
+ *data_len = 0;
+ p = NULL;
+ }
+ return p;
+}
+EXPORT_SYMBOL(rdma_consumer_reject_data);
+
static void cma_add_one(struct ib_device *device);
static void cma_remove_one(struct ib_device *device, void *client_data);
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index 62039c2..d3968b5 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -403,4 +403,14 @@ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
*/
bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason);
+/**
+ * rdma_consumer_reject_data - return the consumer reject private data and
+ * length, if any.
+ * @id: Communication identifier that received the REJECT event.
+ * @ev: RDMA CM reject event.
+ * @data_len: Pointer to the resulting length of the consumer data.
+ */
+const void *rdma_consumer_reject_data(struct rdma_cm_id *id,
+ struct rdma_cm_event *ev, u8 *data_len);
+
#endif /* RDMA_CM_H */
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 4/6] nvme-rdma: use rdma connection reject helper functions
From: Steve Wise @ 2016-10-24 19:07 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
Also add nvme cm status strings and use them.
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/nvme/host/rdma.c | 46 ++++++++++++++++++++++++++++++++++++++--------
1 file changed, 38 insertions(+), 8 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index fbdb226..b6a6b50 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -43,6 +43,28 @@
#define NVME_RDMA_MAX_INLINE_SEGMENTS 1
+static const char *const nvme_rdma_cm_status_strs[] = {
+ [NVME_RDMA_CM_INVALID_LEN] = "invalid length",
+ [NVME_RDMA_CM_INVALID_RECFMT] = "invalid record format",
+ [NVME_RDMA_CM_INVALID_QID] = "invalid queue id",
+ [NVME_RDMA_CM_INVALID_HSQSIZE] = "invalid host sq size",
+ [NVME_RDMA_CM_INVALID_HRQSIZE] = "invalid host rq size",
+ [NVME_RDMA_CM_NO_RSC] = "resource not found",
+ [NVME_RDMA_CM_INVALID_IRD] = "invalid ird",
+ [NVME_RDMA_CM_INVALID_ORD] = "Invalid ord",
+};
+
+static const char *nvme_rdma_cm_msg(enum nvme_rdma_cm_status status)
+{
+ size_t index = status;
+
+ if (index >= ARRAY_SIZE(nvme_rdma_cm_status_strs) ||
+ !nvme_rdma_cm_status_strs[index])
+ return "unrecognized reason";
+ else
+ return nvme_rdma_cm_status_strs[index];
+};
+
/*
* We handle AEN commands ourselves and don't even let the
* block layer know about them.
@@ -1222,17 +1244,25 @@ out_destroy_queue_ib:
static int nvme_rdma_conn_rejected(struct nvme_rdma_queue *queue,
struct rdma_cm_event *ev)
{
- if (ev->param.conn.private_data_len) {
- struct nvme_rdma_cm_rej *rej =
- (struct nvme_rdma_cm_rej *)ev->param.conn.private_data;
+ struct rdma_cm_id *cm_id = queue->cm_id;
+ int status = ev->status;
+ const char *rej_msg;
+ struct nvme_rdma_cm_rej *rej_data;
+ u8 rej_data_len;
+
+ rej_msg = rdma_reject_msg(cm_id, status);
+ rej_data = (struct nvme_rdma_cm_rej *)
+ rdma_consumer_reject_data(cm_id, ev, &rej_data_len);
+
+ if (rej_data && rej_data_len >= sizeof(u16)) {
+ u16 sts = le16_to_cpu(rej_data->sts);
dev_err(queue->ctrl->ctrl.device,
- "Connect rejected, status %d.", le16_to_cpu(rej->sts));
- /* XXX: Think of something clever to do here... */
- } else {
+ "Connect rejected: status %d (%s) nvme status %d (%s).\n",
+ status, rej_msg, sts, nvme_rdma_cm_msg(sts));
+ } else
dev_err(queue->ctrl->ctrl.device,
- "Connect rejected, no private data.\n");
- }
+ "Connect rejected: status %d (%s).\n", status, rej_msg);
return -ECONNRESET;
}
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 5/6] ib_iser: log the connection reject message
From: Steve Wise @ 2016-10-24 19:07 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/ulp/iser/iser_verbs.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 1b49453..7970757 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -906,11 +906,14 @@ static int iser_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *eve
case RDMA_CM_EVENT_ESTABLISHED:
iser_connected_handler(cma_id, event->param.conn.private_data);
break;
+ case RDMA_CM_EVENT_REJECTED:
+ iser_info("Connection rejected: %s\n",
+ rdma_reject_msg(cma_id, event->status));
+ /*FALLTHROUGH*/
case RDMA_CM_EVENT_ADDR_ERROR:
case RDMA_CM_EVENT_ROUTE_ERROR:
case RDMA_CM_EVENT_CONNECT_ERROR:
case RDMA_CM_EVENT_UNREACHABLE:
- case RDMA_CM_EVENT_REJECTED:
iser_connect_error(cma_id);
break;
case RDMA_CM_EVENT_DISCONNECTED:
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 6/6] rds_rdma: log the connection reject message
From: Steve Wise @ 2016-10-24 19:07 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
In-Reply-To: <cover.1477336045.git.swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
net/rds/rdma_transport.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/net/rds/rdma_transport.c b/net/rds/rdma_transport.c
index 345f090..6bd2814 100644
--- a/net/rds/rdma_transport.c
+++ b/net/rds/rdma_transport.c
@@ -100,11 +100,14 @@ int rds_rdma_cm_event_handler(struct rdma_cm_id *cm_id,
trans->cm_connect_complete(conn, event);
break;
+ case RDMA_CM_EVENT_REJECTED:
+ rdsdebug("Connection rejected: %s\n",
+ rdma_reject_msg(cm_id, event->status));
+ /*FALLTHROUGH*/
case RDMA_CM_EVENT_ADDR_ERROR:
case RDMA_CM_EVENT_ROUTE_ERROR:
case RDMA_CM_EVENT_CONNECT_ERROR:
case RDMA_CM_EVENT_UNREACHABLE:
- case RDMA_CM_EVENT_REJECTED:
case RDMA_CM_EVENT_DEVICE_REMOVAL:
case RDMA_CM_EVENT_ADDR_CHANGE:
if (conn)
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v3 0/6] connect reject event helpers
From: Steve Wise @ 2016-10-24 19:07 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
sagi-NQWnxTmZq1alnMjI0IkVqw, hch-jcswGhMUV9g, axboe-b10kYP2dOMg,
santosh.shilimkar-QHcLZuEGTsvQT0dZR+AlfA
While reviewing:
http://lists.infradead.org/pipermail/linux-nvme/2016-October/006681.html
I decided to propose transport-agnostic helper functions to better
handle connection reject event information. Included are patches
for nvme_rdma, ib_ser, and rds_rdma to utilize the new helpers.
Changes since v2:
- reworked ibcm/iwcm_reject_msg() as per Christoph's recommendation
- use ibcm_ and iwcm_ prefix instead of ib_ and iw_ for reject_msg funcs
- change rdma_consumer_reject() to rdma_is_consumer_reject()
- add rdma_consumer_reject_data() helper function to return private
data/len
- use new helpers in nvme_rdma, ib_iser, and rdma_rds
- in nvme_rdma, add strings for nvme_rdma_cm_status values
---
Steve Wise (6):
rdma_cm: add rdma_reject_msg() helper function
rdma_cm: add rdma_is_consumer_reject() helper function
rdma_cm: add rdma_consumer_reject_data helper function
nvme-rdma: use rdma connection reject helper functions
ib_iser: log the connection reject message
rds_rdma: log the connection reject message
drivers/infiniband/core/cm.c | 48 ++++++++++++++++++++++++++++++++
drivers/infiniband/core/cma.c | 43 ++++++++++++++++++++++++++++
drivers/infiniband/core/iwcm.c | 21 ++++++++++++++
drivers/infiniband/ulp/iser/iser_verbs.c | 5 +++-
drivers/nvme/host/rdma.c | 46 ++++++++++++++++++++++++------
include/rdma/ib_cm.h | 6 ++++
include/rdma/iw_cm.h | 6 ++++
include/rdma/rdma_cm.h | 25 +++++++++++++++++
net/rds/rdma_transport.c | 5 +++-
9 files changed, 195 insertions(+), 10 deletions(-)
--
2.7.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/mlx5: avoid bogus -Wmaybe-uninitialized warning
From: Arnd Bergmann @ 2016-10-24 20:46 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Matan Barak, Doug Ledford, Sean Hefty, Hal Rosenstock,
Sagi Grimberg, Bart Van Assche, Noa Osherovich, Saeed Mahameed,
linux-rdma, linux-kernel
In-Reply-To: <20161024170642.GL25013@leon.nu>
On Monday, October 24, 2016 8:06:42 PM CEST Leon Romanovsky wrote:
>
> Acked-by: Leon Romanovsky <leonro@mellanox.com>
Thanks!
> > drivers/infiniband/hw/mlx5/mr.c | 39 +++++++++++++++++++++------------------
> > 1 file changed, 21 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
> > index d4ad672b905b..88d8d292677b 100644
> > --- a/drivers/infiniband/hw/mlx5/mr.c
> > +++ b/drivers/infiniband/hw/mlx5/mr.c
> > @@ -815,29 +815,33 @@ static void prep_umr_unreg_wqe(struct mlx5_ib_dev *dev,
> > umrwr->mkey = key;
> > }
> >
> > -static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
> > - int access_flags, int *npages,
> > - int *page_shift, int *ncont, int *order)
> > +static int mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
> > + int access_flags, struct ib_umem ** umem,
>
> I wonder if checkpatch does differentiate between "struct ib_umem ** umem"
> and "struct ib_umem **umem". According to coding style, the second is preferable.
It was unintended, I'll send a v2 patch in a minute.
Arnd
^ permalink raw reply
* [PATCH v2] IB/mlx5: avoid bogus -Wmaybe-uninitialized warning
From: Arnd Bergmann @ 2016-10-24 20:48 UTC (permalink / raw)
To: Doug Ledford, Sean Hefty, Hal Rosenstock
Cc: Arnd Bergmann, Matan Barak, Leon Romanovsky, Sagi Grimberg,
Bart Van Assche, Noa Osherovich, Saeed Mahameed, linux-rdma,
linux-kernel
We get a false-positive warning in linux-next for the mlx5 driver:
infiniband/hw/mlx5/mr.c: In function ‘mlx5_ib_reg_user_mr’:
infiniband/hw/mlx5/mr.c:1172:5: error: ‘order’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1161:6: note: ‘order’ was declared here
infiniband/hw/mlx5/mr.c:1173:6: error: ‘ncont’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1160:6: note: ‘ncont’ was declared here
infiniband/hw/mlx5/mr.c:1173:6: error: ‘page_shift’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1158:6: note: ‘page_shift’ was declared here
infiniband/hw/mlx5/mr.c:1143:13: error: ‘npages’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
infiniband/hw/mlx5/mr.c:1159:6: note: ‘npages’ was declared here
I had a trivial workaround for gcc-5 or higher, but that didn't work
on gcc-4.9 unfortunately.
The only way I found to avoid the warnings for gcc-4.9, short of
initializing each of the arguments first was to change the calling
conventions to separate the error code from the umem pointer. This
avoids casting the error codes from one pointer to another incompatible
pointer, and lets gcc figure out when that the data is actually valid
whenever we return successfully.
Acked-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
drivers/infiniband/hw/mlx5/mr.c | 39 +++++++++++++++++++++------------------
1 file changed, 21 insertions(+), 18 deletions(-)
v2: fix whitespace typo
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index d4ad672b905b..9ea74fca568a 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -815,29 +815,33 @@ static void prep_umr_unreg_wqe(struct mlx5_ib_dev *dev,
umrwr->mkey = key;
}
-static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
- int access_flags, int *npages,
- int *page_shift, int *ncont, int *order)
+static int mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
+ int access_flags, struct ib_umem **umem,
+ int *npages, int *page_shift, int *ncont,
+ int *order)
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
- struct ib_umem *umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags, 0);
- if (IS_ERR(umem)) {
+ int err;
+
+ *umem = ib_umem_get(pd->uobject->context, start, length,
+ access_flags, 0);
+ err = PTR_ERR_OR_ZERO(*umem);
+ if (err < 0) {
mlx5_ib_err(dev, "umem get failed (%ld)\n", PTR_ERR(umem));
- return (void *)umem;
+ return err;
}
- mlx5_ib_cont_pages(umem, start, npages, page_shift, ncont, order);
+ mlx5_ib_cont_pages(*umem, start, npages, page_shift, ncont, order);
if (!*npages) {
mlx5_ib_warn(dev, "avoid zero region\n");
- ib_umem_release(umem);
- return ERR_PTR(-EINVAL);
+ ib_umem_release(*umem);
+ return -EINVAL;
}
mlx5_ib_dbg(dev, "npages %d, ncont %d, order %d, page_shift %d\n",
*npages, *ncont, *order, *page_shift);
- return umem;
+ return 0;
}
static void mlx5_ib_umr_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -1163,11 +1167,11 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mlx5_ib_dbg(dev, "start 0x%llx, virt_addr 0x%llx, length 0x%llx, access_flags 0x%x\n",
start, virt_addr, length, access_flags);
- umem = mr_umem_get(pd, start, length, access_flags, &npages,
+ err = mr_umem_get(pd, start, length, access_flags, &umem, &npages,
&page_shift, &ncont, &order);
- if (IS_ERR(umem))
- return (void *)umem;
+ if (err < 0)
+ return ERR_PTR(err);
if (use_umr(order)) {
mr = reg_umr(pd, umem, virt_addr, length, ncont, page_shift,
@@ -1341,10 +1345,9 @@ int mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
*/
flags |= IB_MR_REREG_TRANS;
ib_umem_release(mr->umem);
- mr->umem = mr_umem_get(pd, addr, len, access_flags, &npages,
- &page_shift, &ncont, &order);
- if (IS_ERR(mr->umem)) {
- err = PTR_ERR(mr->umem);
+ err = mr_umem_get(pd, addr, len, access_flags, &mr->umem,
+ &npages, &page_shift, &ncont, &order);
+ if (err < 0) {
mr->umem = NULL;
return err;
}
--
2.9.0
^ permalink raw reply related
* Re: [PATCH v2] IB/mlx5: avoid bogus -Wmaybe-uninitialized warning
From: Leon Romanovsky @ 2016-10-25 5:40 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Doug Ledford, Sean Hefty, Hal Rosenstock, Matan Barak,
Sagi Grimberg, Bart Van Assche, Noa Osherovich, Saeed Mahameed,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161024204830.620592-1-arnd-r2nGTMty4D4@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1839 bytes --]
On Mon, Oct 24, 2016 at 10:48:21PM +0200, Arnd Bergmann wrote:
> We get a false-positive warning in linux-next for the mlx5 driver:
>
> infiniband/hw/mlx5/mr.c: In function ‘mlx5_ib_reg_user_mr’:
> infiniband/hw/mlx5/mr.c:1172:5: error: ‘order’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1161:6: note: ‘order’ was declared here
> infiniband/hw/mlx5/mr.c:1173:6: error: ‘ncont’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1160:6: note: ‘ncont’ was declared here
> infiniband/hw/mlx5/mr.c:1173:6: error: ‘page_shift’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1158:6: note: ‘page_shift’ was declared here
> infiniband/hw/mlx5/mr.c:1143:13: error: ‘npages’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
> infiniband/hw/mlx5/mr.c:1159:6: note: ‘npages’ was declared here
>
> I had a trivial workaround for gcc-5 or higher, but that didn't work
> on gcc-4.9 unfortunately.
>
> The only way I found to avoid the warnings for gcc-4.9, short of
> initializing each of the arguments first was to change the calling
> conventions to separate the error code from the umem pointer. This
> avoids casting the error codes from one pointer to another incompatible
> pointer, and lets gcc figure out when that the data is actually valid
> whenever we return successfully.
>
> Acked-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
> ---
> drivers/infiniband/hw/mlx5/mr.c | 39 +++++++++++++++++++++------------------
> 1 file changed, 21 insertions(+), 18 deletions(-)
>
> v2: fix whitespace typo
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: New providers in rdma-core
From: oulijun @ 2016-10-25 5:42 UTC (permalink / raw)
To: Leon Romanovsky, Amrani, Ram, aditr-pghWNbHTmq7QT0dZR+AlfA,
Knut Omang
Cc: Doug Ledford, Jason Gunthorpe, linux-rdma
In-Reply-To: <CALq1K=+nEP0jFOCxj30HA0+bg0L1bF5+xiBcAC2bRwUz+M6+eQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
在 2016/10/20 0:18, Leon Romanovsky 写道:
> Hi All,
>
> I want to remind you that latest kernel is exposing different vendor
> specific structures (include/uapi/rdma/*-abi.,h) and new coming user
> space drivers should use these headers directly and not copy/paste the
> code from them.
>
> In the future, we will convert already accepted drivers too.
>
> Thanks
>
> .
>
Hi, Leon
I am preparing for the userspace library of hns and have learned your email.
I have the same question with Jason Gunthorpe. Meanwhile, the new provider qedr don't
do this.
I think that your plan will have a more clever discussion coming forward. In order to
not be blocked, I will don't something more until it is a step closer. After that, i
will fix it seraparete separately.
thanks
Lijun Ou
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: NVMe Over Fabrics - Random Crash with SoftROCE
From: Leon Romanovsky @ 2016-10-25 5:47 UTC (permalink / raw)
To: Ripduman Sohan
Cc: Christoph Hellwig, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161024124625.GA2389-jcswGhMUV9g@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 10169 bytes --]
On Mon, Oct 24, 2016 at 02:46:25PM +0200, Christoph Hellwig wrote:
> Hi Ripduman,
>
> please report all NVMe issues to the linux-nvme list. I'm reading there
> as well, but it will allow for more people to follow the issue.
>
> I'm not even sure what the error is between all the traces, but maybe
> someone understands the rxe traces better there or on the linux-rdma
> list.
Hi Ripduman,
Please include Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> (RXE maintainer) in your
emails.
Thanks
>
> On Fri, Oct 21, 2016 at 10:30:15PM +0100, Ripduman Sohan wrote:
> > Hi,
> >
> > I'm trying to get NVMF going over SoftRoCE (rxe_rdma) and I get random
> > crashes. At the simplest reduction, if I connect the initiator to the
> > target, on an idle system I will on occasion get the error below on the
> > initiator (no data has been transferred between hosts at this point - and
> > this happens randomly, sometimes it takes hours, sometimes it happens
> > within 10 mins of boot).
> >
> > I'll probably start to debug this in a couple of weeks, but I thought it
> > might be passing it by you in case it's something you might have seen
> > before/have some clues?
> >
> > Thanks
> >
> > Rip
> >
> >
> > ---- log below ---- (initiator).
> >
> > rdma_rxe: loaded
> > rdma_rxe: set rxe0 active
> > rdma_rxe: added rxe0 to eth4
> > nvme nvme0: creating 8 I/O queues.
> > nvme nvme0: new ctrl: NQN "ramdisk", addr 172.16.139.22:4420
> > nvme nvme0: failed nvme_keep_alive_end_io error=16391
> > nvme nvme0: reconnecting in 10 seconds
> > nvme nvme0: Successfully reconnected
> >
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff8801389c6800
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff8801376d8000
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff8801369ee400
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff88013a9dc400
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff88013997d000
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff880137201c00
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff88013548f800
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff880138c0b800
> > 1346: nvme nvme0: disconnect received - connection closed
> > 1317: nvme nvme0: disconnected (10): status 0 id ffff880139936400
> > 1346: nvme nvme0: disconnect received - connection closed
> > 756: rdma_rxe: qp#26 state -> ERR
> > 756: rdma_rxe: qp#26 state -> ERR
> > 756: rdma_rxe: qp#26 state -> ERR
> > 756: rdma_rxe: qp#27 state -> ERR
> > 756: rdma_rxe: qp#27 state -> ERR
> > 756: rdma_rxe: qp#27 state -> ERR
> > 756: rdma_rxe: qp#28 state -> ERR
> > 756: rdma_rxe: qp#28 state -> ERR
> > 756: rdma_rxe: qp#28 state -> ERR
> > 756: rdma_rxe: qp#29 state -> ERR
> > 756: rdma_rxe: qp#29 state -> ERR
> > 756: rdma_rxe: qp#29 state -> ERR
> > 756: rdma_rxe: qp#30 state -> ERR
> > 756: rdma_rxe: qp#30 state -> ERR
> > 756: rdma_rxe: qp#30 state -> ERR
> > 756: rdma_rxe: qp#31 state -> ERR
> > 756: rdma_rxe: qp#31 state -> ERR
> > 756: rdma_rxe: qp#31 state -> ERR
> > 756: rdma_rxe: qp#32 state -> ERR
> > 756: rdma_rxe: qp#32 state -> ERR
> > 756: rdma_rxe: qp#32 state -> ERR
> > 756: rdma_rxe: qp#33 state -> ERR
> > 756: rdma_rxe: qp#33 state -> ERR
> > 756: rdma_rxe: qp#33 state -> ERR
> > 756: rdma_rxe: qp#25 state -> ERR
> > 756: rdma_rxe: qp#25 state -> ERR
> > 756: rdma_rxe: qp#25 state -> ERR
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff8801389c6800
> > 302: rdma_rxe: qp#33 max_wr = 33, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#33 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff8801389c6800
> > 730: rdma_rxe: qp#33 state -> INIT
> > 698: rdma_rxe: qp#33 set resp psn = 0x7a0c05
> > 704: rdma_rxe: qp#33 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#33 state -> RTR
> > 684: rdma_rxe: qp#33 set retry count = 7
> > 691: rdma_rxe: qp#33 set rnr retry count = 7
> > 711: rdma_rxe: qp#33 set req psn = 0x2c631
> > 741: rdma_rxe: qp#33 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff8801389c6800
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff88013a461800
> > 302: rdma_rxe: qp#34 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#34 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff88013a461800
> > 730: rdma_rxe: qp#34 state -> INIT
> > 698: rdma_rxe: qp#34 set resp psn = 0x4e6c1c
> > 704: rdma_rxe: qp#34 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#34 state -> RTR
> > 684: rdma_rxe: qp#34 set retry count = 7
> > 691: rdma_rxe: qp#34 set rnr retry count = 7
> > 711: rdma_rxe: qp#34 set req psn = 0x186e10
> > 741: rdma_rxe: qp#34 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff88013a461800
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff88013997dc00
> > 302: rdma_rxe: qp#35 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#35 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff88013997dc00
> > 730: rdma_rxe: qp#35 state -> INIT
> > 698: rdma_rxe: qp#35 set resp psn = 0xd727f8
> > 704: rdma_rxe: qp#35 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#35 state -> RTR
> > 684: rdma_rxe: qp#35 set retry count = 7
> > 691: rdma_rxe: qp#35 set rnr retry count = 7
> > 711: rdma_rxe: qp#35 set req psn = 0xd8e512
> > 741: rdma_rxe: qp#35 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff88013997dc00
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880139d81000
> > 302: rdma_rxe: qp#36 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#36 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880139d81000
> > 730: rdma_rxe: qp#36 state -> INIT
> > 698: rdma_rxe: qp#36 set resp psn = 0x7978ee
> > 704: rdma_rxe: qp#36 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#36 state -> RTR
> > 684: rdma_rxe: qp#36 set retry count = 7
> > 691: rdma_rxe: qp#36 set rnr retry count = 7
> > 711: rdma_rxe: qp#36 set req psn = 0xc5b0ef
> > 741: rdma_rxe: qp#36 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880139d81000
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880137201800
> > 302: rdma_rxe: qp#37 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#37 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880137201800
> > 730: rdma_rxe: qp#37 state -> INIT
> > 698: rdma_rxe: qp#37 set resp psn = 0x970dd5
> > 704: rdma_rxe: qp#37 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#37 state -> RTR
> > 684: rdma_rxe: qp#37 set retry count = 7
> > 691: rdma_rxe: qp#37 set rnr retry count = 7
> > 711: rdma_rxe: qp#37 set req psn = 0x71f2a2
> > 741: rdma_rxe: qp#37 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880137201800
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880139e34c00
> > 302: rdma_rxe: qp#38 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#38 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880139e34c00
> > 730: rdma_rxe: qp#38 state -> INIT
> > 698: rdma_rxe: qp#38 set resp psn = 0x542d56
> > 704: rdma_rxe: qp#38 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#38 state -> RTR
> > 684: rdma_rxe: qp#38 set retry count = 7
> > 691: rdma_rxe: qp#38 set rnr retry count = 7
> > 711: rdma_rxe: qp#38 set req psn = 0x71fad4
> > 741: rdma_rxe: qp#38 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880139e34c00
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880134e43800
> > 302: rdma_rxe: qp#39 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#39 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880134e43800
> > 730: rdma_rxe: qp#39 state -> INIT
> > 698: rdma_rxe: qp#39 set resp psn = 0xdbca4
> > 704: rdma_rxe: qp#39 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#39 state -> RTR
> > 684: rdma_rxe: qp#39 set retry count = 7
> > 691: rdma_rxe: qp#39 set rnr retry count = 7
> > 711: rdma_rxe: qp#39 set req psn = 0xd84ac0
> > 741: rdma_rxe: qp#39 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880134e43800
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880138d15400
> > 302: rdma_rxe: qp#40 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#40 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880138d15400
> > 730: rdma_rxe: qp#40 state -> INIT
> > 698: rdma_rxe: qp#40 set resp psn = 0x6afd31
> > 704: rdma_rxe: qp#40 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#40 state -> RTR
> > 684: rdma_rxe: qp#40 set retry count = 7
> > 691: rdma_rxe: qp#40 set rnr retry count = 7
> > 711: rdma_rxe: qp#40 set req psn = 0xb917ed
> > 741: rdma_rxe: qp#40 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880138d15400
> > 1317: nvme nvme0: address resolved (0): status 0 id ffff880134f45400
> > 302: rdma_rxe: qp#41 max_wr = 129, max_sge = 1, wqe_size = 56
> > 730: rdma_rxe: qp#41 state -> INIT
> > 1317: nvme nvme0: route resolved (2): status 0 id ffff880134f45400
> > 730: rdma_rxe: qp#41 state -> INIT
> > 698: rdma_rxe: qp#41 set resp psn = 0x8a6989
> > 704: rdma_rxe: qp#41 set min rnr timer = 0x0
> > 736: rdma_rxe: qp#41 state -> RTR
> > 684: rdma_rxe: qp#41 set retry count = 7
> > 691: rdma_rxe: qp#41 set rnr retry count = 7
> > 711: rdma_rxe: qp#41 set req psn = 0x23c909
> > 741: rdma_rxe: qp#41 state -> RTS
> > 1317: nvme nvme0: established (9): status 0 id ffff880134f45400
> > nvme nvme0: Successfully reconnected
> >
> > --
> > --rip
> ---end quoted text---
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: New providers in rdma-core
From: Leon Romanovsky @ 2016-10-25 6:08 UTC (permalink / raw)
To: oulijun
Cc: Amrani, Ram, aditr-pghWNbHTmq7QT0dZR+AlfA, Knut Omang,
Doug Ledford, Jason Gunthorpe, linux-rdma
In-Reply-To: <580EF0D4.9050209-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 937 bytes --]
On Tue, Oct 25, 2016 at 01:42:44PM +0800, oulijun wrote:
> 在 2016/10/20 0:18, Leon Romanovsky 写道:
> > Hi All,
> >
> > I want to remind you that latest kernel is exposing different vendor
> > specific structures (include/uapi/rdma/*-abi.,h) and new coming user
> > space drivers should use these headers directly and not copy/paste the
> > code from them.
> >
> > In the future, we will convert already accepted drivers too.
> >
> > Thanks
> >
> > .
> >
> Hi, Leon
> I am preparing for the userspace library of hns and have learned your email.
> I have the same question with Jason Gunthorpe. Meanwhile, the new provider qedr don't
> do this.
> I think that your plan will have a more clever discussion coming forward. In order to
> not be blocked, I will don't something more until it is a step closer. After that, i
> will fix it seraparete separately.
Sure, no problem.
>
> thanks
> Lijun Ou
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH] build: Fix build script to use correct cmake cmd
From: Leon Romanovsky @ 2016-10-25 10:12 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, Dennis Dalessandro,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161021193034.GA616-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]
On Fri, Oct 21, 2016 at 01:30:34PM -0600, Jason Gunthorpe wrote:
> On Fri, Oct 21, 2016 at 03:03:23PM -0400, Doug Ledford wrote:
> > According to github, this change broke the travis CI build on your pull
> > request when the previous patches all worked. However, travis is down
> > right now, so I suspect this didn't *actually* break it, that the travis
> > CI was down anyway and github logged a failure.
>
> Yes, travis does not use build.sh
>
> Maybe this is a good time to ask if anyone is interested in the docker
> stuff I have - eg should I make it pushable? It is easy to use, but
> you need to have docker installed.
I would be happy to get it and be more confident in my local tests.
>
> There are several other container/chroot/docker based toolkits out
> there for doing this sort of work, but I haven't studied that area in
> a while to see if there is something workable.
>
> The docker script is able to run almost-travis locally, as well as do
> clean package builds for all distros.
It looks like it goes beyond rdma-core definition. Do we want to put it
separately or anyway integrate into main library?
>
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH 0/3] iopmem : A block device for PCIe memory
From: Stephen Bates @ 2016-10-25 11:50 UTC (permalink / raw)
To: Dave Chinner
Cc: Christoph Hellwig, Dan Williams, linux-kernel@vger.kernel.org,
linux-nvdimm@lists.01.org, linux-rdma, linux-block, Linux MM,
Ross Zwisler, Matthew Wilcox, jgunthorpe, haggaie, Jens Axboe,
Jonathan Corbet, jim.macdonald, sbates, Logan Gunthorpe,
David Woodhouse, Raj, Ashok
In-Reply-To: <20161021111253.GQ14023@dastard>
Hi Dave and Christoph
On Fri, Oct 21, 2016 at 10:12:53PM +1100, Dave Chinner wrote:
> On Fri, Oct 21, 2016 at 02:57:14AM -0700, Christoph Hellwig wrote:
> > On Fri, Oct 21, 2016 at 10:22:39AM +1100, Dave Chinner wrote:
> > > You do realise that local filesystems can silently change the
> > > location of file data at any point in time, so there is no such
> > > thing as a "stable mapping" of file data to block device addresses
> > > in userspace?
> > >
> > > If you want remote access to the blocks owned and controlled by a
> > > filesystem, then you need to use a filesystem with a remote locking
> > > mechanism to allow co-ordinated, coherent access to the data in
> > > those blocks. Anything else is just asking for ongoing, unfixable
> > > filesystem corruption or data leakage problems (i.e. security
> > > issues).
> >
Dave are you saying that even for local mappings of files on a DAX
capable system it is possible for the mappings to move on you unless
the FS supports locking? Does that not mean DAX on such FS is
inherently broken?
> > And at least for XFS we have such a mechanism :) E.g. I have a
> > prototype of a pNFS layout that uses XFS+DAX to allow clients to do
> > RDMA directly to XFS files, with the same locking mechanism we use
> > for the current block and scsi layout in xfs_pnfs.c.
>
Thanks for fixing this issue on XFS Christoph! I assume this problem
continues to exist on the other DAX capable FS?
One more reason to consider a move to /dev/dax I guess ;-)...
Stephen
> Oh, that's good to know - pNFS over XFS was exactly what I was
> thinking of when I wrote my earlier reply. A few months ago someone
> else was trying to use file mappings in userspace for direct remote
> client access on fabric connected devices. I told them "pNFS on XFS
> and write an efficient transport for you hardware"....
>
> Now that I know we've got RDMA support for pNFS on XFS in the
> pipeline, I can just tell them "just write an rdma driver for your
> hardware" instead. :P
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox