* Re: build failure on linus' latest master branch
From: Leon Romanovsky @ 2016-10-11 12:02 UTC (permalink / raw)
To: Steve Wise; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <01ca01d2230c$b8ade6f0$2a09b4d0$@opengridcomputing.com>
[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]
On Mon, Oct 10, 2016 at 10:41:09AM -0500, Steve Wise wrote:
> Hey Leon, I hit this trying to compile Linus' latest master branch.
>
> CC [M] drivers/net/ethernet/mellanox/mlx5/core/pagealloc.o
> drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c: In function
> âreclaim_pages_cmd.clone.0â:
> drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c:347: error: call to
> â__compiletime_assert_347â declared with attribute error: BUILD_BUG_ON failed:
> __mlx5_bit_off(manage_pages_out, pas[i]) % 64
> drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c: In function âgive_pagesâ:
> drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c:290: error: call to
> â__compiletime_assert_290â declared with attribute error: BUILD_BUG_ON failed:
> __mlx5_bit_off(manage_pages_in, pas[i]) % 64
> make[5]: *** [drivers/net/ethernet/mellanox/mlx5/core/pagealloc.o] Error 1
>
> Is there a fix for this?
We investigated a lot this failure and it appears on specific and very
old GCC 4.4.
http://www.spinics.net/lists/linux-rdma/msg39135.html
http://marc.info/?t=147614305200001&r=1&w=2
And the fix is http://marc.info/?l=linux-netdev&m=147615976207362&w=2
>
> Thanks,
>
> Steve.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* RE: build failure on linus' latest master branch
From: Steve Wise @ 2016-10-11 14:30 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011120219.GM9282-2ukJVAZIZ/Y@public.gmane.org>
> > Is there a fix for this?
>
> We investigated a lot this failure and it appears on specific and very
> old GCC 4.4.
> http://www.spinics.net/lists/linux-rdma/msg39135.html
> http://marc.info/?t=147614305200001&r=1&w=2
>
> And the fix is http://marc.info/?l=linux-netdev&m=147615976207362&w=2
>
Thanks. So was it ever merged?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: switch iwpm to ccan list.h
From: Steve Wise @ 2016-10-11 14:57 UTC (permalink / raw)
To: 'Christoph Hellwig',
Tatyana.E.Nikolova-ral2JQCrhuEAvxtiuMwx3w,
robert.o.sharp-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1476092514-18188-1-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
>
> I've started running sparse on the rdma-core code, and one of the
> worst offenders is the iwpmd list code, so this (untested) patch
> set converts it over to the ccan list.h helpers. Reviews and
> testing appreciated.
Looks good...tests out ok.
Acked-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
Tested-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/9] Introduce blk_quiesce_queue() and blk_resume_queue()
From: Laurence Oberman @ 2016-10-11 16:27 UTC (permalink / raw)
To: Bart Van Assche
Cc: Jens Axboe, Christoph Hellwig, James Bottomley,
Martin K. Petersen, Mike Snitzer, Doug Ledford, Keith Busch,
linux-block, linux-scsi, linux-rdma, linux-nvme
In-Reply-To: <7948dbb8-6333-dc62-2673-4da35b4dfdbc@sandisk.com>
----- Original Message -----
> From: "Bart Van Assche" <bart.vanassche@sandisk.com>
> To: "Jens Axboe" <axboe@fb.com>
> Cc: "Christoph Hellwig" <hch@lst.de>, "James Bottomley" <jejb@linux.vnet.ibm.com>, "Martin K. Petersen"
> <martin.petersen@oracle.com>, "Mike Snitzer" <snitzer@redhat.com>, "Doug Ledford" <dledford@redhat.com>, "Keith
> Busch" <keith.busch@intel.com>, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, linux-rdma@vger.kernel.org,
> linux-nvme@lists.infradead.org
> Sent: Monday, September 26, 2016 2:25:54 PM
> Subject: [PATCH 0/9] Introduce blk_quiesce_queue() and blk_resume_queue()
>
> Hello Jens,
>
> Multiple block drivers need the functionality to stop a request queue
> and to wait until all ongoing request_fn() / queue_rq() calls have
> finished without waiting until all outstanding requests have finished.
> Hence this patch series that introduces the blk_quiesce_queue() and
> blk_resume_queue() functions. The dm-mq, SRP and nvme patches in this
> patch series are three examples of where these functions are useful.
> These patches apply on top of the September 21 version of your
> for-4.9/block branch. The individual patches in this series are:
>
> 0001-blk-mq-Introduce-blk_mq_queue_stopped.patch
> 0002-dm-Fix-a-race-condition-related-to-stopping-and-star.patch
> 0003-RFC-nvme-Use-BLK_MQ_S_STOPPED-instead-of-QUEUE_FLAG_.patch
> 0004-block-Move-blk_freeze_queue-and-blk_unfreeze_queue-c.patch
> 0005-block-Extend-blk_freeze_queue_start-to-the-non-blk-m.patch
> 0006-block-Rename-mq_freeze_wq-and-mq_freeze_depth.patch
> 0007-blk-mq-Introduce-blk_quiesce_queue-and-blk_resume_qu.patch
> 0008-SRP-transport-Port-srp_wait_for_queuecommand-to-scsi.patch
> 0009-RFC-nvme-Fix-a-race-condition.patch
>
> Thanks,
>
> Bart.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Hello
I took Bart's latest patches from his tree and tested all the SRP/RDMA and as many of the nvme tests.
Everything is passing my tests, including SRP port resets etc.
The nvme tests were all on a small intel nvme card.
Tested-by: Laurence Oberman <loberman@redhat.com>
^ permalink raw reply
* Re: [PATCH v2 1/7] blk-mq: Introduce blk_mq_queue_stopped()
From: Christoph Hellwig @ 2016-10-11 16:40 UTC (permalink / raw)
To: Bart Van Assche
Cc: Jens Axboe, Christoph Hellwig, James Bottomley,
Martin K. Petersen, Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <26c7db22-bac8-6d8c-ba6e-3cbf0cec3368@sandisk.com>
Looks fine,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply
* Re: [PATCH v2 6/7] SRP transport: Port srp_wait_for_queuecommand() to scsi-mq
From: Christoph Hellwig @ 2016-10-11 16:44 UTC (permalink / raw)
To: Bart Van Assche
Cc: Sagi Grimberg, Jens Axboe, Christoph Hellwig, James Bottomley,
Martin K. Petersen, Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <619e3ffc-2123-4cb0-361a-4025ba47b991@sandisk.com>
On Wed, Oct 05, 2016 at 02:51:50PM -0700, Bart Van Assche wrote:
> There are multiple direct blk_*() calls in other SCSI transport drivers. So
> my proposal is to wait with moving this code into scsi_lib.c until there is
> a second user of this code.
I still don't think these low-level difference for blk-mq vs legacy
request belong into a scsi LLDD. So I concur with Sagi that this
should go into the core SCSI code.
In fact I suspect we should just call it directly from
scsi_internal_device_block, and maybe even scsi_internal_device_unblock
for case of setting the device offline.
^ permalink raw reply
* Re: [PATCH v2 7/7] [RFC] nvme: Fix a race condition
From: Christoph Hellwig @ 2016-10-11 16:46 UTC (permalink / raw)
To: Bart Van Assche
Cc: Jens Axboe, Christoph Hellwig, James Bottomley,
Martin K. Petersen, Mike Snitzer, Doug Ledford, Keith Busch,
linux-block@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org
In-Reply-To: <ff412bfb-548c-c4fc-5180-78ede8a19ef2@sandisk.com>
On Wed, Sep 28, 2016 at 05:01:45PM -0700, Bart Van Assche wrote:
> Avoid that nvme_queue_rq() is still running when nvme_stop_queues()
> returns. Untested.
>
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Cc: Keith Busch <keith.busch@intel.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Sagi Grimberg <sagi@grimberg.me>
> ---
> drivers/nvme/host/core.c | 20 ++++++++++++--------
> 1 file changed, 12 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index d791fba..98f1f29 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -201,13 +201,9 @@ fail:
>
> void nvme_requeue_req(struct request *req)
> {
> - unsigned long flags;
> -
> blk_mq_requeue_request(req);
> - spin_lock_irqsave(req->q->queue_lock, flags);
> - if (!blk_mq_queue_stopped(req->q))
> - blk_mq_kick_requeue_list(req->q);
> - spin_unlock_irqrestore(req->q->queue_lock, flags);
> + WARN_ON_ONCE(blk_mq_queue_stopped(req->q));
> + blk_mq_kick_requeue_list(req->q);
> }
> EXPORT_SYMBOL_GPL(nvme_requeue_req);
Can we just add a 'bool kick' argument to blk_mq_requeue_request and
move all this handling to the core?
^ permalink raw reply
* [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Paolo Abeni @ 2016-10-11 17:15 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: Doug Ledford, Sean Hefty, Hal Rosenstock,
netdev-u79uwXL29TY76Z2rM5mHXA
After the commit 9207f9d45b0a ("net: preserve IP control block
during GSO segmentation"), the GSO CB and the IPoIB CB conflict.
That destroy the IPoIB address information cached there,
causing a severe performance regression, as better described here:
http://marc.info/?l=linux-kernel&m=146787279825501&w=2
This change moves the data cached by the IPoIB driver from the
skb control lock into the IPoIB hard header, as done before
the commit 936d7de3d736 ("IPoIB: Stop lying about hard_header_len
and use skb->cb to stash LL addresses").
In order to avoid GRO issue, on packet reception, the IPoIB driver
stash into the skb a dummy pseudo header, so that the received
packets have actually a hard header matching the declared length.
Also the connected mode maximum mtu is reduced by 16 bytes to
cope with the increased hard header len.
After this commit, IPoIB performances are back to pre-regression
value.
Fixes: 9207f9d45b0a ("net: preserve IP control block during GSO segmentation")
Signed-off-by: Paolo Abeni <pabeni-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
drivers/infiniband/ulp/ipoib/ipoib.h | 24 ++++++++----
drivers/infiniband/ulp/ipoib/ipoib_cm.c | 17 ++++----
drivers/infiniband/ulp/ipoib/ipoib_ib.c | 12 +++---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 54 ++++++++++++++++----------
drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 6 ++-
5 files changed, 67 insertions(+), 46 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index 9dbfcc0..5dd01fa 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -63,12 +63,14 @@ enum ipoib_flush_level {
enum {
IPOIB_ENCAP_LEN = 4,
+ IPOIB_PSEUDO_LEN = 20,
+ IPOIB_HARD_LEN = IPOIB_ENCAP_LEN + IPOIB_PSEUDO_LEN,
IPOIB_UD_HEAD_SIZE = IB_GRH_BYTES + IPOIB_ENCAP_LEN,
IPOIB_UD_RX_SG = 2, /* max buffer needed for 4K mtu */
- IPOIB_CM_MTU = 0x10000 - 0x10, /* padding to align header to 16 */
- IPOIB_CM_BUF_SIZE = IPOIB_CM_MTU + IPOIB_ENCAP_LEN,
+ IPOIB_CM_MTU = 0x10000 - 0x20, /* padding to align header to 16 */
+ IPOIB_CM_BUF_SIZE = IPOIB_CM_MTU + IPOIB_HARD_LEN,
IPOIB_CM_HEAD_SIZE = IPOIB_CM_BUF_SIZE % PAGE_SIZE,
IPOIB_CM_RX_SG = ALIGN(IPOIB_CM_BUF_SIZE, PAGE_SIZE) / PAGE_SIZE,
IPOIB_RX_RING_SIZE = 256,
@@ -134,15 +136,21 @@ struct ipoib_header {
u16 reserved;
};
-struct ipoib_cb {
- struct qdisc_skb_cb qdisc_cb;
- u8 hwaddr[INFINIBAND_ALEN];
+struct ipoib_pseudo_header {
+ u8 hwaddr[INFINIBAND_ALEN];
};
-static inline struct ipoib_cb *ipoib_skb_cb(const struct sk_buff *skb)
+static inline void skb_add_pseudo_hdr(struct sk_buff *skb)
{
- BUILD_BUG_ON(sizeof(skb->cb) < sizeof(struct ipoib_cb));
- return (struct ipoib_cb *)skb->cb;
+ char *data = skb_push(skb, IPOIB_PSEUDO_LEN);
+
+ /*
+ * only the ipoib header is present now, make room for a dummy
+ * pseudo header and set skb field accordingly
+ */
+ memset(data, 0, IPOIB_PSEUDO_LEN);
+ skb_reset_mac_header(skb);
+ skb_pull(skb, IPOIB_HARD_LEN);
}
/* Used for all multicast joins (broadcast, IPv4 mcast and IPv6 mcast) */
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 4ad297d..1b04c8a 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -63,6 +63,8 @@ MODULE_PARM_DESC(cm_data_debug_level,
#define IPOIB_CM_RX_DELAY (3 * 256 * HZ)
#define IPOIB_CM_RX_UPDATE_MASK (0x3)
+#define IPOIB_CM_RX_RESERVE (ALIGN(IPOIB_HARD_LEN, 16) - IPOIB_ENCAP_LEN)
+
static struct ib_qp_attr ipoib_cm_err_attr = {
.qp_state = IB_QPS_ERR
};
@@ -146,15 +148,15 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
struct sk_buff *skb;
int i;
- skb = dev_alloc_skb(IPOIB_CM_HEAD_SIZE + 12);
+ skb = dev_alloc_skb(ALIGN(IPOIB_CM_HEAD_SIZE, 16));
if (unlikely(!skb))
return NULL;
/*
- * IPoIB adds a 4 byte header. So we need 12 more bytes to align the
+ * IPoIB adds a IPOIB_ENCAP_LEN byte header, this will align the
* IP header to a multiple of 16.
*/
- skb_reserve(skb, 12);
+ skb_reserve(skb, IPOIB_CM_RX_RESERVE);
mapping[0] = ib_dma_map_single(priv->ca, skb->data, IPOIB_CM_HEAD_SIZE,
DMA_FROM_DEVICE);
@@ -624,9 +626,9 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
if (wc->byte_len < IPOIB_CM_COPYBREAK) {
int dlen = wc->byte_len;
- small_skb = dev_alloc_skb(dlen + 12);
+ small_skb = dev_alloc_skb(dlen + IPOIB_CM_RX_RESERVE);
if (small_skb) {
- skb_reserve(small_skb, 12);
+ skb_reserve(small_skb, IPOIB_CM_RX_RESERVE);
ib_dma_sync_single_for_cpu(priv->ca, rx_ring[wr_id].mapping[0],
dlen, DMA_FROM_DEVICE);
skb_copy_from_linear_data(skb, small_skb->data, dlen);
@@ -663,8 +665,7 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
copied:
skb->protocol = ((struct ipoib_header *) skb->data)->proto;
- skb_reset_mac_header(skb);
- skb_pull(skb, IPOIB_ENCAP_LEN);
+ skb_add_pseudo_hdr(skb);
++dev->stats.rx_packets;
dev->stats.rx_bytes += skb->len;
@@ -1583,7 +1584,7 @@ int ipoib_cm_dev_init(struct net_device *dev)
max_srq_sge = min_t(int, IPOIB_CM_RX_SG, priv->ca->attrs.max_srq_sge);
ipoib_cm_create_srq(dev, max_srq_sge);
if (ipoib_cm_has_srq(dev)) {
- priv->cm.max_cm_mtu = max_srq_sge * PAGE_SIZE - 0x10;
+ priv->cm.max_cm_mtu = max_srq_sge * PAGE_SIZE - 0x20;
priv->cm.num_frags = max_srq_sge;
ipoib_dbg(priv, "max_cm_mtu = 0x%x, num_frags=%d\n",
priv->cm.max_cm_mtu, priv->cm.num_frags);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index be11d5d..830fecb 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -128,16 +128,15 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id)
buf_size = IPOIB_UD_BUF_SIZE(priv->max_ib_mtu);
- skb = dev_alloc_skb(buf_size + IPOIB_ENCAP_LEN);
+ skb = dev_alloc_skb(buf_size + IPOIB_HARD_LEN);
if (unlikely(!skb))
return NULL;
/*
- * IB will leave a 40 byte gap for a GRH and IPoIB adds a 4 byte
- * header. So we need 4 more bytes to get to 48 and align the
- * IP header to a multiple of 16.
+ * the IP header will be at IPOIP_HARD_LEN + IB_GRH_BYTES, that is
+ * 64 bytes aligned
*/
- skb_reserve(skb, 4);
+ skb_reserve(skb, sizeof(struct ipoib_pseudo_header));
mapping = priv->rx_ring[id].mapping;
mapping[0] = ib_dma_map_single(priv->ca, skb->data, buf_size,
@@ -253,8 +252,7 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
skb_pull(skb, IB_GRH_BYTES);
skb->protocol = ((struct ipoib_header *) skb->data)->proto;
- skb_reset_mac_header(skb);
- skb_pull(skb, IPOIB_ENCAP_LEN);
+ skb_add_pseudo_hdr(skb);
++dev->stats.rx_packets;
dev->stats.rx_bytes += skb->len;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index cc1c1b0..823a528 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -925,9 +925,12 @@ static void neigh_add_path(struct sk_buff *skb, u8 *daddr,
ipoib_neigh_free(neigh);
goto err_drop;
}
- if (skb_queue_len(&neigh->queue) < IPOIB_MAX_PATH_REC_QUEUE)
+ if (skb_queue_len(&neigh->queue) <
+ IPOIB_MAX_PATH_REC_QUEUE) {
+ /* put pseudoheader back on for next time */
+ skb_push(skb, IPOIB_PSEUDO_LEN);
__skb_queue_tail(&neigh->queue, skb);
- else {
+ } else {
ipoib_warn(priv, "queue length limit %d. Packet drop.\n",
skb_queue_len(&neigh->queue));
goto err_drop;
@@ -964,7 +967,7 @@ err_drop:
}
static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
- struct ipoib_cb *cb)
+ struct ipoib_pseudo_header *phdr)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ipoib_path *path;
@@ -972,16 +975,18 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
spin_lock_irqsave(&priv->lock, flags);
- path = __path_find(dev, cb->hwaddr + 4);
+ path = __path_find(dev, phdr->hwaddr + 4);
if (!path || !path->valid) {
int new_path = 0;
if (!path) {
- path = path_rec_create(dev, cb->hwaddr + 4);
+ path = path_rec_create(dev, phdr->hwaddr + 4);
new_path = 1;
}
if (path) {
if (skb_queue_len(&path->queue) < IPOIB_MAX_PATH_REC_QUEUE) {
+ /* put pseudoheader back on for next time */
+ skb_push(skb, IPOIB_PSEUDO_LEN);
__skb_queue_tail(&path->queue, skb);
} else {
++dev->stats.tx_dropped;
@@ -1009,10 +1014,12 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
be16_to_cpu(path->pathrec.dlid));
spin_unlock_irqrestore(&priv->lock, flags);
- ipoib_send(dev, skb, path->ah, IPOIB_QPN(cb->hwaddr));
+ ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr));
return;
} else if ((path->query || !path_rec_start(dev, path)) &&
skb_queue_len(&path->queue) < IPOIB_MAX_PATH_REC_QUEUE) {
+ /* put pseudoheader back on for next time */
+ skb_push(skb, IPOIB_PSEUDO_LEN);
__skb_queue_tail(&path->queue, skb);
} else {
++dev->stats.tx_dropped;
@@ -1026,13 +1033,15 @@ static int ipoib_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ipoib_neigh *neigh;
- struct ipoib_cb *cb = ipoib_skb_cb(skb);
+ struct ipoib_pseudo_header *phdr;
struct ipoib_header *header;
unsigned long flags;
+ phdr = (struct ipoib_pseudo_header *) skb->data;
+ skb_pull(skb, sizeof(*phdr));
header = (struct ipoib_header *) skb->data;
- if (unlikely(cb->hwaddr[4] == 0xff)) {
+ if (unlikely(phdr->hwaddr[4] == 0xff)) {
/* multicast, arrange "if" according to probability */
if ((header->proto != htons(ETH_P_IP)) &&
(header->proto != htons(ETH_P_IPV6)) &&
@@ -1045,13 +1054,13 @@ static int ipoib_start_xmit(struct sk_buff *skb, struct net_device *dev)
return NETDEV_TX_OK;
}
/* Add in the P_Key for multicast*/
- cb->hwaddr[8] = (priv->pkey >> 8) & 0xff;
- cb->hwaddr[9] = priv->pkey & 0xff;
+ phdr->hwaddr[8] = (priv->pkey >> 8) & 0xff;
+ phdr->hwaddr[9] = priv->pkey & 0xff;
- neigh = ipoib_neigh_get(dev, cb->hwaddr);
+ neigh = ipoib_neigh_get(dev, phdr->hwaddr);
if (likely(neigh))
goto send_using_neigh;
- ipoib_mcast_send(dev, cb->hwaddr, skb);
+ ipoib_mcast_send(dev, phdr->hwaddr, skb);
return NETDEV_TX_OK;
}
@@ -1060,16 +1069,16 @@ static int ipoib_start_xmit(struct sk_buff *skb, struct net_device *dev)
case htons(ETH_P_IP):
case htons(ETH_P_IPV6):
case htons(ETH_P_TIPC):
- neigh = ipoib_neigh_get(dev, cb->hwaddr);
+ neigh = ipoib_neigh_get(dev, phdr->hwaddr);
if (unlikely(!neigh)) {
- neigh_add_path(skb, cb->hwaddr, dev);
+ neigh_add_path(skb, phdr->hwaddr, dev);
return NETDEV_TX_OK;
}
break;
case htons(ETH_P_ARP):
case htons(ETH_P_RARP):
/* for unicast ARP and RARP should always perform path find */
- unicast_arp_send(skb, dev, cb);
+ unicast_arp_send(skb, dev, phdr);
return NETDEV_TX_OK;
default:
/* ethertype not supported by IPoIB */
@@ -1086,11 +1095,13 @@ send_using_neigh:
goto unref;
}
} else if (neigh->ah) {
- ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(cb->hwaddr));
+ ipoib_send(dev, skb, neigh->ah, IPOIB_QPN(phdr->hwaddr));
goto unref;
}
if (skb_queue_len(&neigh->queue) < IPOIB_MAX_PATH_REC_QUEUE) {
+ /* put pseudoheader back on for next time */
+ skb_push(skb, sizeof(*phdr));
spin_lock_irqsave(&priv->lock, flags);
__skb_queue_tail(&neigh->queue, skb);
spin_unlock_irqrestore(&priv->lock, flags);
@@ -1122,8 +1133,8 @@ static int ipoib_hard_header(struct sk_buff *skb,
unsigned short type,
const void *daddr, const void *saddr, unsigned len)
{
+ struct ipoib_pseudo_header *phdr;
struct ipoib_header *header;
- struct ipoib_cb *cb = ipoib_skb_cb(skb);
header = (struct ipoib_header *) skb_push(skb, sizeof *header);
@@ -1132,12 +1143,13 @@ static int ipoib_hard_header(struct sk_buff *skb,
/*
* we don't rely on dst_entry structure, always stuff the
- * destination address into skb->cb so we can figure out where
+ * destination address into skb hard header so we can figure out where
* to send the packet later.
*/
- memcpy(cb->hwaddr, daddr, INFINIBAND_ALEN);
+ phdr = (struct ipoib_pseudo_header *) skb_push(skb, sizeof(*phdr));
+ memcpy(phdr->hwaddr, daddr, INFINIBAND_ALEN);
- return sizeof *header;
+ return IPOIB_HARD_LEN;
}
static void ipoib_set_mcast_list(struct net_device *dev)
@@ -1759,7 +1771,7 @@ void ipoib_setup(struct net_device *dev)
dev->flags |= IFF_BROADCAST | IFF_MULTICAST;
- dev->hard_header_len = IPOIB_ENCAP_LEN;
+ dev->hard_header_len = IPOIB_HARD_LEN;
dev->addr_len = INFINIBAND_ALEN;
dev->type = ARPHRD_INFINIBAND;
dev->tx_queue_len = ipoib_sendq_size * 2;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index d3394b6..1909dd2 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -796,9 +796,11 @@ void ipoib_mcast_send(struct net_device *dev, u8 *daddr, struct sk_buff *skb)
__ipoib_mcast_add(dev, mcast);
list_add_tail(&mcast->list, &priv->multicast_list);
}
- if (skb_queue_len(&mcast->pkt_queue) < IPOIB_MAX_MCAST_QUEUE)
+ if (skb_queue_len(&mcast->pkt_queue) < IPOIB_MAX_MCAST_QUEUE) {
+ /* put pseudoheader back on for next time */
+ skb_push(skb, sizeof(struct ipoib_pseudo_header));
skb_queue_tail(&mcast->pkt_queue, skb);
- else {
+ } else {
++dev->stats.tx_dropped;
dev_kfree_skb_any(skb);
}
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Jason Gunthorpe @ 2016-10-11 17:32 UTC (permalink / raw)
To: Paolo Abeni
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1dbd83dfe7f435eecc5bc460e901b47758280f30.1476206016.git.pabeni-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Tue, Oct 11, 2016 at 07:15:44PM +0200, Paolo Abeni wrote:
> Also the connected mode maximum mtu is reduced by 16 bytes to
> cope with the increased hard header len.
Changing the MTU is going to cause annoying interop problems, can you
avoid this?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Paolo Abeni @ 2016-10-11 17:37 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: linux-rdma, Doug Ledford, Sean Hefty, Hal Rosenstock, netdev
In-Reply-To: <20161011173216.GA16892@obsidianresearch.com>
On Tue, 2016-10-11 at 11:32 -0600, Jason Gunthorpe wrote:
> On Tue, Oct 11, 2016 at 07:15:44PM +0200, Paolo Abeni wrote:
>
> > Also the connected mode maximum mtu is reduced by 16 bytes to
> > cope with the increased hard header len.
>
> Changing the MTU is going to cause annoying interop problems, can you
> avoid this?
I don't like changing the maximum MTU value, too, but I was unable to
find an alternative solution. The PMTU detection should protect against
such issues.
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Doug Ledford @ 2016-10-11 17:41 UTC (permalink / raw)
To: Jason Gunthorpe, Paolo Abeni
Cc: linux-rdma, Sean Hefty, Hal Rosenstock, netdev
In-Reply-To: <20161011173216.GA16892@obsidianresearch.com>
[-- Attachment #1.1: Type: text/plain, Size: 945 bytes --]
On 10/11/2016 1:32 PM, Jason Gunthorpe wrote:
> On Tue, Oct 11, 2016 at 07:15:44PM +0200, Paolo Abeni wrote:
>
>> Also the connected mode maximum mtu is reduced by 16 bytes to
>> cope with the increased hard header len.
>
> Changing the MTU is going to cause annoying interop problems, can you
> avoid this?
(Paolo did the work I'm describing here, I'm just giving the explanation
he gave me):
Not using this particular solution I don't think. We tried it without
increasing the declared hard header length and it broke when dealing
with skb_clone/GSO paths. In order to make the LL pseudo header get
copied along with the rest of the encap and data on clone, we had to
declare the header. The problem then became that the sg setup is such
that we are limited to 16 4k pages for the sg array, so that header had
to come out of the 64k maximum mtu.
--
Doug Ledford <dledford@redhat.com>
GPG Key ID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Jason Gunthorpe @ 2016-10-11 17:42 UTC (permalink / raw)
To: Paolo Abeni
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1476207452.448.2.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Tue, Oct 11, 2016 at 07:37:32PM +0200, Paolo Abeni wrote:
> On Tue, 2016-10-11 at 11:32 -0600, Jason Gunthorpe wrote:
> > On Tue, Oct 11, 2016 at 07:15:44PM +0200, Paolo Abeni wrote:
> >
> > > Also the connected mode maximum mtu is reduced by 16 bytes to
> > > cope with the increased hard header len.
> >
> > Changing the MTU is going to cause annoying interop problems, can you
> > avoid this?
>
> I don't like changing the maximum MTU value, too, but I was unable to
> find an alternative solution. The PMTU detection should protect against
> such issues.
It is more that PMTU, we have instructed all users that is the MTU
number needed to enable CM mode, so it appears in documentation,
scripts, etc.
There is really no way to re-use some of the existing alignment
padding or exceed 64k?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Paolo Abeni @ 2016-10-11 17:52 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011174224.GA17319-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On Tue, 2016-10-11 at 11:42 -0600, Jason Gunthorpe wrote:
> On Tue, Oct 11, 2016 at 07:37:32PM +0200, Paolo Abeni wrote:
> > On Tue, 2016-10-11 at 11:32 -0600, Jason Gunthorpe wrote:
> > > On Tue, Oct 11, 2016 at 07:15:44PM +0200, Paolo Abeni wrote:
> > >
> > > > Also the connected mode maximum mtu is reduced by 16 bytes to
> > > > cope with the increased hard header len.
> > >
> > > Changing the MTU is going to cause annoying interop problems, can you
> > > avoid this?
> >
> > I don't like changing the maximum MTU value, too, but I was unable to
> > find an alternative solution. The PMTU detection should protect against
> > such issues.
>
> It is more that PMTU, we have instructed all users that is the MTU
> number needed to enable CM mode, so it appears in documentation,
> scripts, etc.
AFAICS the max mtu is already underlying h/w dependent, how does such
differences are currently coped by ? (I'm sorry I lack some/a lot of IB
back-ground)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Jason Gunthorpe @ 2016-10-11 18:01 UTC (permalink / raw)
To: Doug Ledford; +Cc: Paolo Abeni, linux-rdma, Sean Hefty, Hal Rosenstock, netdev
In-Reply-To: <22f61258-9bcf-0adc-f23f-79a4f1d50c6a@redhat.com>
On Tue, Oct 11, 2016 at 01:41:56PM -0400, Doug Ledford wrote:
> declare the header. The problem then became that the sg setup is such
> that we are limited to 16 4k pages for the sg array, so that header had
> to come out of the 64k maximum mtu.
Oh, that clarifies things..
Hum, so various options become:
- Use >=17 SGL entries when creating the QP. Is this possible
on common adapters?
- Use the FRWR infrastructure when necessary. Is there any chance
the majority of skbs will have at least two physically
continuous pages to make this overhead rare? Perhaps as a fall
back if many adaptors can do >=17 SGLs
- Pad the hard header out to 4k and discard the first page
when building the sgl
- Memcopy the first ~8k into a contiguous 8k region on send
- Move the pseudo header to the end so it can cross the page
barrier without needing a sgl entry. (probably impossible?)
>From Paolo
> AFAICS the max mtu is already underlying h/w dependent, how does such
> differences are currently coped by ? (I'm sorry I lack some/a lot of IB
> back-ground)
It isn't h/w dependent. In CM mode the MTU is 65520 because that is
what is hard coded into the ipoib driver. We tell everyone to use that
number. Eg see RH's docs on the subject:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Configuring_IPoIB.html
AFAIK, today everyone just wires that number into their scripts, so we
have to mass change everything to the smaller number. That sounds
really hard, IMHO if there is any way to avoid it we should, even if
it is a little costly.
Jason
^ permalink raw reply
* Re: [PATCH rdma-core 1/5] Pull uninitialized_var into util/compiler.h
From: Jason Gunthorpe @ 2016-10-11 18:05 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161010040930.GF9282-2ukJVAZIZ/Y@public.gmane.org>
On Mon, Oct 10, 2016 at 07:09:30AM +0300, Leon Romanovsky wrote:
> > I think people would complain about the extra stores. gcc 6 will
> > eliminate them, but older compilers will not.
>
> This unintialized_var(x) adds extra store too (... x = x ...).
I have confirmed that the unintialized_var macro does not impact code
generation on the old gccs while the =0 approach does.
So no extra store.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Paolo Abeni @ 2016-10-11 18:10 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, linux-rdma, Sean Hefty, Hal Rosenstock, netdev
In-Reply-To: <20161011180119.GB17319@obsidianresearch.com>
On Tue, 2016-10-11 at 12:01 -0600, Jason Gunthorpe wrote:
> > AFAICS the max mtu is already underlying h/w dependent, how does such
> > differences are currently coped by ? (I'm sorry I lack some/a lot of IB
> > back-ground)
>
> It isn't h/w dependent. In CM mode the MTU is 65520 because that is
> what is hard coded into the ipoib driver. We tell everyone to use that
> number. Eg see RH's docs on the subject:
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Configuring_IPoIB.html
>
> AFAIK, today everyone just wires that number into their scripts, so we
> have to mass change everything to the smaller number. That sounds
> really hard, IMHO if there is any way to avoid it we should, even if
> it is a little costly.
Thank you for the details!
The first s/g fragment (the head buffer) is not allocated with the page
allocator, so perhaps there is some not too difficult/costly way out of
this.
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Doug Ledford @ 2016-10-11 18:17 UTC (permalink / raw)
To: Paolo Abeni, Jason Gunthorpe
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1476209407.448.9.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 1622 bytes --]
On 10/11/2016 2:10 PM, Paolo Abeni wrote:
> On Tue, 2016-10-11 at 12:01 -0600, Jason Gunthorpe wrote:
>>> AFAICS the max mtu is already underlying h/w dependent, how does such
>>> differences are currently coped by ? (I'm sorry I lack some/a lot of IB
>>> back-ground)
>>
>> It isn't h/w dependent. In CM mode the MTU is 65520 because that is
>> what is hard coded into the ipoib driver. We tell everyone to use that
>> number. Eg see RH's docs on the subject:
>>
>> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Configuring_IPoIB.html
>>
>> AFAIK, today everyone just wires that number into their scripts, so we
>> have to mass change everything to the smaller number.
Well, not exactly. Even if we put 65520 into the scripts, the kernel
will silently drop it down to 65504. It actually won't require anyone
change anything, they just won't get the full value. I experimented
with this in the past for other reasons and an overly large MTU setting
just resulted in the max MTU. I don't know if that's changed, but if it
still works that way, this is much less of an issue than it might
otherwise be.
>> That sounds
>> really hard, IMHO if there is any way to avoid it we should, even if
>> it is a little costly.
>
> Thank you for the details!
>
> The first s/g fragment (the head buffer) is not allocated with the page
> allocator, so perhaps there is some not too difficult/costly way out of
> this.
>
>
>
>
>
>
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
GPG Key ID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Jason Gunthorpe @ 2016-10-11 18:27 UTC (permalink / raw)
To: Paolo Abeni
Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1476209407.448.9.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Tue, Oct 11, 2016 at 08:10:07PM +0200, Paolo Abeni wrote:
> The first s/g fragment (the head buffer) is not allocated with the page
> allocator, so perhaps there is some not too difficult/costly way out of
> this.
Keep in mind, there is nothing magic about the 16 SGL limit, other
than we know all hardware supports it. That can be bumped up and most
hardware will support a higher value.
We'd just have to figure out if any hardware breaks, Mellanox and Intel
should be able to respond to that question.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Jason Gunthorpe @ 2016-10-11 18:30 UTC (permalink / raw)
To: Doug Ledford
Cc: Paolo Abeni, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <073e0007-a43b-134d-ba7e-b290304be585-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Tue, Oct 11, 2016 at 02:17:51PM -0400, Doug Ledford wrote:
> Well, not exactly. Even if we put 65520 into the scripts, the kernel
> will silently drop it down to 65504. It actually won't require anyone
> change anything, they just won't get the full value. I experimented
> with this in the past for other reasons and an overly large MTU setting
> just resulted in the max MTU. I don't know if that's changed, but if it
> still works that way, this is much less of an issue than it might
> otherwise be.
So it is just docs and relying on PMTU? That is not as bad..
Still would be nice to avoid if at all possible..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] IB/ipoib: move back the IB LL address into the hard header
From: Doug Ledford @ 2016-10-11 18:50 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Paolo Abeni, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Sean Hefty,
Hal Rosenstock, netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011183019.GC20253-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 1038 bytes --]
On 10/11/2016 2:30 PM, Jason Gunthorpe wrote:
> On Tue, Oct 11, 2016 at 02:17:51PM -0400, Doug Ledford wrote:
>
>> Well, not exactly. Even if we put 65520 into the scripts, the kernel
>> will silently drop it down to 65504. It actually won't require anyone
>> change anything, they just won't get the full value. I experimented
>> with this in the past for other reasons and an overly large MTU setting
>> just resulted in the max MTU. I don't know if that's changed, but if it
>> still works that way, this is much less of an issue than it might
>> otherwise be.
>
> So it is just docs and relying on PMTU? That is not as bad..
>
> Still would be nice to avoid if at all possible..
I agree, but we have a test getting ready to commence. We'll know
shortly how much the reduced MTU effects things because they aren't
going to alter any of their setup, just put the new kernel in place, and
see what happens.
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
GPG Key ID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply
* Re: [RFC PATCH 03/11] IB/sa: Modify SM Address handle to program GRH when using large lids
From: Chandramouli, Dasaratharaman @ 2016-10-11 19:19 UTC (permalink / raw)
To: Jason Gunthorpe, ira.weiny-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Don Hiatt
In-Reply-To: <20160923183217.GD13920-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
The reason for doing this in the kernel was that we wanted a more
uniform approach in specifying extended LIDs across the user and the
kernel space. This also meant we could leave the ah_attr.dlid field in
the kernel to be 16 bits.
We will extend ah_attr.dlid to 32 bits in the kernel per your
suggestion. This should make specifying extended LID information in the
GRH of the address handle unnecessary.
Thanks,
Dasa
On 9/23/2016 11:32 AM, Jason Gunthorpe wrote:
> On Fri, Sep 23, 2016 at 01:44:26PM -0400, ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org wrote:
>> From: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>>
>> When either the port lid or the sm lid is above the ib unicast lid
>> space, the SMI creates an address handle with the revelant GRH
>> information.
>
> Woah, I though you were only using that horrible GID hack for the
> uABI.
>
> Don't do such crazy things in the kernel. Fix the internal kernel to
> support your larger lid.
>
> Jason
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: build failure on linus' latest master branch
From: Leon Romanovsky @ 2016-10-11 19:36 UTC (permalink / raw)
To: Steve Wise; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <004701d223cc$0772cbb0$16586310$@opengridcomputing.com>
[-- Attachment #1: Type: text/plain, Size: 735 bytes --]
On Tue, Oct 11, 2016 at 09:30:35AM -0500, Steve Wise wrote:
> > > Is there a fix for this?
> >
> > We investigated a lot this failure and it appears on specific and very
> > old GCC 4.4.
> > http://www.spinics.net/lists/linux-rdma/msg39135.html
> > http://marc.info/?t=147614305200001&r=1&w=2
> >
> > And the fix is http://marc.info/?l=linux-netdev&m=147615976207362&w=2
> >
>
> Thanks. So was it ever merged?
Not yet, I hope that Dave or Doug will forward that patch to Linus
before -rc1 is released.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH rdma-core 1/5] Pull uninitialized_var into util/compiler.h
From: Leon Romanovsky @ 2016-10-11 19:46 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011180517.GA17866-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 693 bytes --]
On Tue, Oct 11, 2016 at 12:05:17PM -0600, Jason Gunthorpe wrote:
> On Mon, Oct 10, 2016 at 07:09:30AM +0300, Leon Romanovsky wrote:
>
> > > I think people would complain about the extra stores. gcc 6 will
> > > eliminate them, but older compilers will not.
> >
> > This unintialized_var(x) adds extra store too (... x = x ...).
>
> I have confirmed that the unintialized_var macro does not impact code
> generation on the old gccs while the =0 approach does.
Ohh, thanks.
But I'm still left under impression of this article [1] that using such
macro is a bad thing and we are "punishing" all users of modern compilers.
[1] https://lwn.net/Articles/529954/
>
> So no extra store.
>
> Jason
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH rdma-core 1/5] Pull uninitialized_var into util/compiler.h
From: Jason Gunthorpe @ 2016-10-11 20:06 UTC (permalink / raw)
To: Leon Romanovsky; +Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011194633.GP9282-2ukJVAZIZ/Y@public.gmane.org>
On Tue, Oct 11, 2016 at 10:46:33PM +0300, Leon Romanovsky wrote:
> But I'm still left under impression of this article [1] that using such
> macro is a bad thing and we are "punishing" all users of modern compilers.
I agree with the article, it is a bad idea. This is why my version is
disabling the macro entirely if gcc 6 or clang is used - aka the
compilers that run in Travis.
So, new code must not introduce control flow that is more complex than
gcc 6 can understand, and the macro is used only for gcc 4.x and 5.x
compatability to provide warning free compile on popular distros
without a performance hit.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: build failure on linus' latest master branch
From: Doug Ledford @ 2016-10-11 20:47 UTC (permalink / raw)
To: Leon Romanovsky, Steve Wise; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161011193633.GN9282-2ukJVAZIZ/Y@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 715 bytes --]
On 10/11/2016 3:36 PM, Leon Romanovsky wrote:
> On Tue, Oct 11, 2016 at 09:30:35AM -0500, Steve Wise wrote:
>>>> Is there a fix for this?
>>>
>>> We investigated a lot this failure and it appears on specific and very
>>> old GCC 4.4.
>>> http://www.spinics.net/lists/linux-rdma/msg39135.html
>>> http://marc.info/?t=147614305200001&r=1&w=2
>>>
>>> And the fix is http://marc.info/?l=linux-netdev&m=147615976207362&w=2
>>>
>>
>> Thanks. So was it ever merged?
>
> Not yet, I hope that Dave or Doug will forward that patch to Linus
> before -rc1 is released.
That patch isn't even on my radar.
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
GPG Key ID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox