* Respected Sir/Madam
From: مايكروسوفت @ 2016-11-06 19:14 UTC (permalink / raw)
To: INF-5s8wZthzITI
وقد ظهرت مايكروسوفت آسيا بك البريد الإلكتروني معرف الفائز من $ 900،000،00 دولار أمريكي ارسال
اسم:
رقم الهاتف المحمول:
الاحتلال :
عنوان:
الرد على هذا رقم البريد الإلكتروني: mox151@outlook.com
^ permalink raw reply
* Re: [PATCH rdma-rc 5/9] IB/mlx4: Handle well-known-gid in mad_demux processing
From: Or Gerlitz @ 2016-11-06 20:46 UTC (permalink / raw)
To: jackm
Cc: Hal Rosenstock, Leon Romanovsky, Doug Ledford,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Daniel Jurgens, Leon Romanovsky, eitan-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <20161106204107.00005742-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On Sun, Nov 6, 2016 at 8:41 PM, jackm <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> On Sat, 5 Nov 2016 17:03:25 -0400
> Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>
>> Isn't the SA well-known GID the concatenation of the subnet prefix
>> (which isn't necessarily the default link local one) and the GUID
>> 0x0200000000000002 ?
>
> You are correct regarding the subnet prefix (it should be the port's
> current subnet prefix, and not the default subnet prefix).
>
> Regarding the SM GUID, the version of the Virtualization Annex that I
> worked with (v17 -- Feb 16, 2015) states:
> SM GID A well-known GID that is associated with the SM,
> comprising the concatenation of the Subnet prefix and the GUID 0x2. The
> SM GID is never present in any GID Table.
>
> Has the SM GUID value changed (0x2 -->0x0200000000000002) in the Annex
> since v17? (I'm not a member of the MGTWG workgroup, so I don't have
> access to the most recent version).
Jack, I see that in Linux we've picked the value Hal is mentioning,
see Eli Cohen's commit a0c1b2a3 "IB/core: Support accessing SA in
virtualized environment". You probably can talk to Eli to get where
things stand.
Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH rdma-rc 5/9] IB/mlx4: Handle well-known-gid in mad_demux processing
From: Or Gerlitz @ 2016-11-06 20:51 UTC (permalink / raw)
To: jackm
Cc: Hal Rosenstock, Leon Romanovsky, Doug Ledford,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Daniel Jurgens, Leon Romanovsky, eitan-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <CAJ3xEMjLvo+LVFS0WL-K=m34-_1Tc0=pArOgELXmxqdtN871TA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Sun, Nov 6, 2016 at 10:46 PM, Or Gerlitz <gerlitz.or-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Sun, Nov 6, 2016 at 8:41 PM, jackm <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>> On Sat, 5 Nov 2016 17:03:25 -0400
>> Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>>
>>> Isn't the SA well-known GID the concatenation of the subnet prefix
>>> (which isn't necessarily the default link local one) and the GUID
>>> 0x0200000000000002 ?
>>
>> You are correct regarding the subnet prefix (it should be the port's
>> current subnet prefix, and not the default subnet prefix).
>>
>> Regarding the SM GUID, the version of the Virtualization Annex that I
>> worked with (v17 -- Feb 16, 2015) states:
>> SM GID A well-known GID that is associated with the SM,
>> comprising the concatenation of the Subnet prefix and the GUID 0x2. The
>> SM GID is never present in any GID Table.
>>
>> Has the SM GUID value changed (0x2 -->0x0200000000000002) in the Annex
>> since v17? (I'm not a member of the MGTWG workgroup, so I don't have
>> access to the most recent version).
>
> Jack, I see that in Linux we've picked the value Hal is mentioning,
> see Eli Cohen's commit a0c1b2a3 "IB/core: Support accessing SA in
> virtualized environment". You probably can talk to Eli to get where
> things stand.
yep, looking now on the annex I see the following:
SM GID -- A well-known GID that is associated with the SM, comprising
the concatenation of the Subnet prefix and the GUID
0x0200000000000002. The
SM GID is never present in any GID Table. If the SubnetPrefix is modified
by the SM, the SM GID is updated implicitly
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH for-next 01/11] IB/hns: Add the interface for querying QP1
From: Anurup M @ 2016-11-07 5:45 UTC (permalink / raw)
To: Salil Mehta, dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
mehta.salil.lnk-Re5JQEeQqe8AvxtiuMwx3w,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linuxarm-hv44wF8Li93QT0dZR+AlfA
In-Reply-To: <20161104163633.141880-2-salil.mehta-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
On 11/4/2016 10:06 PM, Salil Mehta wrote:
> From: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>
> In old code, It only added the interface for querying non-specific
> QP. This patch mainly adds an interface for querying QP1.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Salil Mehta <salil.mehta-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 87 +++++++++++++++++++++++++++-
> drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 6 +-
> 2 files changed, 90 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> index 71232e5..ca8b784 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> @@ -2630,8 +2630,82 @@ static int hns_roce_v1_query_qpc(struct hns_roce_dev *hr_dev,
> return ret;
> }
>
> -int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
> - int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr)
> +static int hns_roce_v1_q_sqp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
> + int qp_attr_mask,
> + struct ib_qp_init_attr *qp_init_attr)
> +{
> + struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> + struct hns_roce_sqp_context *context;
> + u32 addr;
> +
> + context = kzalloc(sizeof(*context), GFP_KERNEL);
> + if (!context)
> + return -ENOMEM;
> +
Do we really need dynamic alloc here as the size is fixed and this memory scope is
only inside this function. I think better to use a static allocation.
> + mutex_lock(&hr_qp->mutex);
> +
> + if (hr_qp->state == IB_QPS_RESET) {
I think alloc can be moved after this check (if dynamic alloc is really needed).
> + qp_attr->qp_state = IB_QPS_RESET;
> + goto done;
> + }
> +
> + addr = ROCEE_QP1C_CFG0_0_REG + hr_qp->port * sizeof(*context);
> + context->qp1c_bytes_4 = roce_read(hr_dev, addr);
> + context->sq_rq_bt_l = roce_read(hr_dev, addr + 1);
> + context->qp1c_bytes_12 = roce_read(hr_dev, addr + 2);
> + context->qp1c_bytes_16 = roce_read(hr_dev, addr + 3);
> + context->qp1c_bytes_20 = roce_read(hr_dev, addr + 4);
> + context->cur_rq_wqe_ba_l = roce_read(hr_dev, addr + 5);
> + context->qp1c_bytes_28 = roce_read(hr_dev, addr + 6);
> + context->qp1c_bytes_32 = roce_read(hr_dev, addr + 7);
> + context->cur_sq_wqe_ba_l = roce_read(hr_dev, addr + 8);
> + context->qp1c_bytes_40 = roce_read(hr_dev, addr + 9);
> +
> + hr_qp->state = roce_get_field(context->qp1c_bytes_4,
> + QP1C_BYTES_4_QP_STATE_M,
> + QP1C_BYTES_4_QP_STATE_S);
> + qp_attr->qp_state = hr_qp->state;
> + qp_attr->path_mtu = IB_MTU_256;
> + qp_attr->path_mig_state = IB_MIG_ARMED;
> + qp_attr->qkey = QKEY_VAL;
> + qp_attr->rq_psn = 0;
> + qp_attr->sq_psn = 0;
> + qp_attr->dest_qp_num = 1;
> + qp_attr->qp_access_flags = 6;
> +
> + qp_attr->pkey_index = roce_get_field(context->qp1c_bytes_20,
> + QP1C_BYTES_20_PKEY_IDX_M,
> + QP1C_BYTES_20_PKEY_IDX_S);
> + qp_attr->port_num = hr_qp->port + 1;
> + qp_attr->sq_draining = 0;
> + qp_attr->max_rd_atomic = 0;
> + qp_attr->max_dest_rd_atomic = 0;
> + qp_attr->min_rnr_timer = 0;
> + qp_attr->timeout = 0;
> + qp_attr->retry_cnt = 0;
> + qp_attr->rnr_retry = 0;
> + qp_attr->alt_timeout = 0;
> +
> +done:
> + qp_attr->cur_qp_state = qp_attr->qp_state;
> + qp_attr->cap.max_recv_wr = hr_qp->rq.wqe_cnt;
> + qp_attr->cap.max_recv_sge = hr_qp->rq.max_gs;
> + qp_attr->cap.max_send_wr = hr_qp->sq.wqe_cnt;
> + qp_attr->cap.max_send_sge = hr_qp->sq.max_gs;
> + qp_attr->cap.max_inline_data = 0;
> + qp_init_attr->cap = qp_attr->cap;
> + qp_init_attr->create_flags = 0;
> +
> + mutex_unlock(&hr_qp->mutex);
> + kfree(context);
> +
> + return 0;
> +}
> +
> +static int hns_roce_v1_q_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
> + int qp_attr_mask,
> + struct ib_qp_init_attr *qp_init_attr)
> {
> struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> @@ -2767,6 +2841,15 @@ int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
> return ret;
> }
>
> +int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
> + int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr)
> +{
> + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> +
> + return hr_qp->doorbell_qpn <= 1 ?
> + hns_roce_v1_q_sqp(ibqp, qp_attr, qp_attr_mask, qp_init_attr) :
> + hns_roce_v1_q_qp(ibqp, qp_attr, qp_attr_mask, qp_init_attr);
> +}
> static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
> struct hns_roce_qp *hr_qp,
> int is_user)
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> index 539b0a3b..2e1878b 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> @@ -480,13 +480,17 @@ struct hns_roce_sqp_context {
> u32 qp1c_bytes_12;
> u32 qp1c_bytes_16;
> u32 qp1c_bytes_20;
> - u32 qp1c_bytes_28;
> u32 cur_rq_wqe_ba_l;
> + u32 qp1c_bytes_28;
> u32 qp1c_bytes_32;
> u32 cur_sq_wqe_ba_l;
> u32 qp1c_bytes_40;
> };
>
> +#define QP1C_BYTES_4_QP_STATE_S 0
> +#define QP1C_BYTES_4_QP_STATE_M \
> + (((1UL << 3) - 1) << QP1C_BYTES_4_QP_STATE_S)
> +
> #define QP1C_BYTES_4_SQ_WQE_SHIFT_S 8
> #define QP1C_BYTES_4_SQ_WQE_SHIFT_M \
> (((1UL << 4) - 1) << QP1C_BYTES_4_SQ_WQE_SHIFT_S)
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v4 10/10] IB/mlx5: Simplify completion into a wait_event
From: Binoy Jayan @ 2016-11-07 5:51 UTC (permalink / raw)
To: Linus Torvalds
Cc: Doug Ledford, Sean Hefty, Hal Rosenstock, Arnd Bergmann,
linux-rdma@vger.kernel.org, Linux Kernel Mailing List
In-Reply-To: <CA+55aFxUA8r8_-yYtZfijWK_2WB+eCxMiHS-bVDN45-98V6Hdw@mail.gmail.com>
Hi Linus,
On 3 November 2016 at 21:07, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> This is wrong.
Will change it back.
> Since that "umr_context" variable is on the stack, and you are waiting
> for it to be fully done, it really should be a completion.
Thank you for sharing your insight with wait_event and completion.
-Binoy
^ permalink raw reply
* [RESEND REQEUST] consult with the libhns upstreaming into rdma-core
From: oulijun @ 2016-11-07 6:43 UTC (permalink / raw)
To: Doug Ledford, linux-rdma; +Cc: Linuxarm
Hi Jason/Doug,
Following your comments with I have fixed several issues.
Thank you for pointing them out.
Do I need to resend the PATCH v2 for libhns?
Lijun Ou
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH for-next 10/11] IB/hns: Implement the add_gid/del_gid and optimize the GIDs management
From: Anurup M @ 2016-11-07 8:17 UTC (permalink / raw)
To: Salil Mehta, dledford
Cc: linux-rdma, netdev, mehta.salil.lnk, linux-kernel, linuxarm
In-Reply-To: <20161104163633.141880-11-salil.mehta@huawei.com>
On 11/4/2016 10:06 PM, Salil Mehta wrote:
> From: Shaobo Xu <xushaobo2@huawei.com>
>
> IB core has implemented the calculation of GIDs and the management
> of GID tables, and it is now responsible to supply query function
> for GIDs. So the calculation of GIDs and the management of GID
> tables in the RoCE driver is redundant.
>
> The patch is to implement the add_gid/del_gid to set the GIDs in
> the RoCE driver, remove the redundant calculation and management of
> GIDs in the notifier call of the net device and the inet, and
> update the query_gid.
>
> Signed-off-by: Shaobo Xu <xushaobo2@huawei.com>
> Reviewed-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> ---
> drivers/infiniband/hw/hns/hns_roce_device.h | 2 -
> drivers/infiniband/hw/hns/hns_roce_main.c | 270 +++++----------------------
> 2 files changed, 48 insertions(+), 224 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
> index 593a42a..9ef1cc3 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_device.h
> +++ b/drivers/infiniband/hw/hns/hns_roce_device.h
> @@ -429,8 +429,6 @@ struct hns_roce_ib_iboe {
> struct net_device *netdevs[HNS_ROCE_MAX_PORTS];
> struct notifier_block nb;
> struct notifier_block nb_inet;
> - /* 16 GID is shared by 6 port in v1 engine. */
> - union ib_gid gid_table[HNS_ROCE_MAX_GID_NUM];
> u8 phy_port[HNS_ROCE_MAX_PORTS];
> };
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
> index 6770171..795ef97 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_main.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_main.c
> @@ -35,52 +35,13 @@
> #include <rdma/ib_addr.h>
> #include <rdma/ib_smi.h>
> #include <rdma/ib_user_verbs.h>
> +#include <rdma/ib_cache.h>
> #include "hns_roce_common.h"
> #include "hns_roce_device.h"
> #include "hns_roce_user.h"
> #include "hns_roce_hem.h"
>
> /**
> - * hns_roce_addrconf_ifid_eui48 - Get default gid.
> - * @eui: eui.
> - * @vlan_id: gid
> - * @dev: net device
> - * Description:
> - * MAC convert to GID
> - * gid[0..7] = fe80 0000 0000 0000
> - * gid[8] = mac[0] ^ 2
> - * gid[9] = mac[1]
> - * gid[10] = mac[2]
> - * gid[11] = ff (VLAN ID high byte (4 MS bits))
> - * gid[12] = fe (VLAN ID low byte)
> - * gid[13] = mac[3]
> - * gid[14] = mac[4]
> - * gid[15] = mac[5]
> - */
> -static void hns_roce_addrconf_ifid_eui48(u8 *eui, u16 vlan_id,
> - struct net_device *dev)
> -{
> - memcpy(eui, dev->dev_addr, 3);
> - memcpy(eui + 5, dev->dev_addr + 3, 3);
> - if (vlan_id < 0x1000) {
> - eui[3] = vlan_id >> 8;
> - eui[4] = vlan_id & 0xff;
> - } else {
> - eui[3] = 0xff;
> - eui[4] = 0xfe;
> - }
> - eui[0] ^= 2;
> -}
> -
> -static void hns_roce_make_default_gid(struct net_device *dev, union ib_gid *gid)
> -{
> - memset(gid, 0, sizeof(*gid));
> - gid->raw[0] = 0xFE;
> - gid->raw[1] = 0x80;
> - hns_roce_addrconf_ifid_eui48(&gid->raw[8], 0xffff, dev);
> -}
> -
> -/**
> * hns_get_gid_index - Get gid index.
> * @hr_dev: pointer to structure hns_roce_dev.
> * @port: port, value range: 0 ~ MAX
> @@ -96,30 +57,6 @@ int hns_get_gid_index(struct hns_roce_dev *hr_dev, u8 port, int gid_index)
> return gid_index * hr_dev->caps.num_ports + port;
> }
>
> -static int hns_roce_set_gid(struct hns_roce_dev *hr_dev, u8 port, int gid_index,
> - union ib_gid *gid)
> -{
> - struct device *dev = &hr_dev->pdev->dev;
> - u8 gid_idx = 0;
> -
> - if (gid_index >= hr_dev->caps.gid_table_len[port]) {
> - dev_err(dev, "gid_index %d illegal, port %d gid range: 0~%d\n",
> - gid_index, port, hr_dev->caps.gid_table_len[port] - 1);
> - return -EINVAL;
> - }
> -
> - gid_idx = hns_get_gid_index(hr_dev, port, gid_index);
> -
> - if (!memcmp(gid, &hr_dev->iboe.gid_table[gid_idx], sizeof(*gid)))
> - return -EINVAL;
> -
> - memcpy(&hr_dev->iboe.gid_table[gid_idx], gid, sizeof(*gid));
> -
> - hr_dev->hw->set_gid(hr_dev, port, gid_index, gid);
> -
> - return 0;
> -}
> -
> static void hns_roce_set_mac(struct hns_roce_dev *hr_dev, u8 port, u8 *addr)
> {
> u8 phy_port;
> @@ -147,15 +84,44 @@ static void hns_roce_set_mtu(struct hns_roce_dev *hr_dev, u8 port, int mtu)
> hr_dev->hw->set_mtu(hr_dev, phy_port, tmp);
> }
>
> -static void hns_roce_update_gids(struct hns_roce_dev *hr_dev, int port)
> +static int hns_roce_add_gid(struct ib_device *device, u8 port_num,
> + unsigned int index, const union ib_gid *gid,
> + const struct ib_gid_attr *attr, void **context)
> +{
> + struct hns_roce_dev *hr_dev = to_hr_dev(device);
> + u8 port = port_num - 1;
> + unsigned long flags;
> +
> + if (port >= hr_dev->caps.num_ports)
> + return -EINVAL;
> +
> + spin_lock_irqsave(&hr_dev->iboe.lock, flags);
> +
> + hr_dev->hw->set_gid(hr_dev, port, index, (union ib_gid *)gid);
> +
> + spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
> +
> + return 0;
> +}
> +
> +static int hns_roce_del_gid(struct ib_device *device, u8 port_num,
> + unsigned int index, void **context)
> {
> - struct ib_event event;
> + struct hns_roce_dev *hr_dev = to_hr_dev(device);
> + union ib_gid zgid = { {0} };
> + u8 port = port_num - 1;
> + unsigned long flags;
> +
> + if (port >= hr_dev->caps.num_ports)
> + return -EINVAL;
>
> - /* Refresh gid in ib_cache */
> - event.device = &hr_dev->ib_dev;
> - event.element.port_num = port + 1;
> - event.event = IB_EVENT_GID_CHANGE;
> - ib_dispatch_event(&event);
> + spin_lock_irqsave(&hr_dev->iboe.lock, flags);
> +
> + hr_dev->hw->set_gid(hr_dev, port, index, &zgid);
zgid has value zero. and after this call, where is zgid used?
> +
> + spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
> +
> + return 0;
> }
>
> static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
> @@ -164,8 +130,6 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
> struct device *dev = &hr_dev->pdev->dev;
> struct net_device *netdev;
> unsigned long flags;
> - union ib_gid gid;
> - int ret = 0;
>
> netdev = hr_dev->iboe.netdevs[port];
> if (!netdev) {
> @@ -181,10 +145,6 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
> case NETDEV_REGISTER:
> case NETDEV_CHANGEADDR:
> hns_roce_set_mac(hr_dev, port, netdev->dev_addr);
> - hns_roce_make_default_gid(netdev, &gid);
> - ret = hns_roce_set_gid(hr_dev, port, 0, &gid);
> - if (!ret)
> - hns_roce_update_gids(hr_dev, port);
> break;
> case NETDEV_DOWN:
> /*
> @@ -197,7 +157,7 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
> }
>
> spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
> - return ret;
> + return 0;
> }
>
> static int hns_roce_netdev_event(struct notifier_block *self,
> @@ -224,118 +184,17 @@ static int hns_roce_netdev_event(struct notifier_block *self,
> return NOTIFY_DONE;
> }
>
> -static void hns_roce_addr_event(int event, struct net_device *event_netdev,
> - struct hns_roce_dev *hr_dev, union ib_gid *gid)
> -{
> - struct hns_roce_ib_iboe *iboe = NULL;
> - int gid_table_len = 0;
> - unsigned long flags;
> - union ib_gid zgid;
> - u8 gid_idx = 0;
> - u8 port = 0;
> - int i = 0;
> - int free;
> - struct net_device *real_dev = rdma_vlan_dev_real_dev(event_netdev) ?
> - rdma_vlan_dev_real_dev(event_netdev) :
> - event_netdev;
> -
> - if (event != NETDEV_UP && event != NETDEV_DOWN)
> - return;
> -
> - iboe = &hr_dev->iboe;
> - while (port < hr_dev->caps.num_ports) {
> - if (real_dev == iboe->netdevs[port])
> - break;
> - port++;
> - }
> -
> - if (port >= hr_dev->caps.num_ports) {
> - dev_dbg(&hr_dev->pdev->dev, "can't find netdev\n");
> - return;
> - }
> -
> - memset(zgid.raw, 0, sizeof(zgid.raw));
> - free = -1;
> - gid_table_len = hr_dev->caps.gid_table_len[port];
> -
> - spin_lock_irqsave(&hr_dev->iboe.lock, flags);
> -
> - for (i = 0; i < gid_table_len; i++) {
> - gid_idx = hns_get_gid_index(hr_dev, port, i);
> - if (!memcmp(gid->raw, iboe->gid_table[gid_idx].raw,
> - sizeof(gid->raw)))
> - break;
> - if (free < 0 && !memcmp(zgid.raw,
> - iboe->gid_table[gid_idx].raw, sizeof(zgid.raw)))
> - free = i;
> - }
> -
> - if (i >= gid_table_len) {
> - if (free < 0) {
> - spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
> - dev_dbg(&hr_dev->pdev->dev,
> - "gid_index overflow, port(%d)\n", port);
> - return;
> - }
> - if (!hns_roce_set_gid(hr_dev, port, free, gid))
> - hns_roce_update_gids(hr_dev, port);
> - } else if (event == NETDEV_DOWN) {
> - if (!hns_roce_set_gid(hr_dev, port, i, &zgid))
> - hns_roce_update_gids(hr_dev, port);
> - }
> -
> - spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
> -}
> -
> -static int hns_roce_inet_event(struct notifier_block *self, unsigned long event,
> - void *ptr)
> -{
> - struct in_ifaddr *ifa = ptr;
> - struct hns_roce_dev *hr_dev;
> - struct net_device *dev = ifa->ifa_dev->dev;
> - union ib_gid gid;
> -
> - ipv6_addr_set_v4mapped(ifa->ifa_address, (struct in6_addr *)&gid);
> -
> - hr_dev = container_of(self, struct hns_roce_dev, iboe.nb_inet);
> -
> - hns_roce_addr_event(event, dev, hr_dev, &gid);
> -
> - return NOTIFY_DONE;
> -}
> -
> -static int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
> +static int hns_roce_setup_mtu_mac(struct hns_roce_dev *hr_dev)
> {
> - struct in_ifaddr *ifa_list = NULL;
> - union ib_gid gid = {{0} };
> - u32 ipaddr = 0;
> - int index = 0;
> - int ret = 0;
> - u8 i = 0;
> + u8 i;
>
> for (i = 0; i < hr_dev->caps.num_ports; i++) {
> hns_roce_set_mtu(hr_dev, i,
> ib_mtu_enum_to_int(hr_dev->caps.max_mtu));
> hns_roce_set_mac(hr_dev, i, hr_dev->iboe.netdevs[i]->dev_addr);
> -
> - if (hr_dev->iboe.netdevs[i]->ip_ptr) {
> - ifa_list = hr_dev->iboe.netdevs[i]->ip_ptr->ifa_list;
> - index = 1;
> - while (ifa_list) {
> - ipaddr = ifa_list->ifa_address;
> - ipv6_addr_set_v4mapped(ipaddr,
> - (struct in6_addr *)&gid);
> - ret = hns_roce_set_gid(hr_dev, i, index, &gid);
> - if (ret)
> - break;
> - index++;
> - ifa_list = ifa_list->ifa_next;
> - }
> - hns_roce_update_gids(hr_dev, i);
> - }
> }
>
> - return ret;
> + return 0;
> }
>
> static int hns_roce_query_device(struct ib_device *ib_dev,
> @@ -444,31 +303,6 @@ static enum rdma_link_layer hns_roce_get_link_layer(struct ib_device *device,
> static int hns_roce_query_gid(struct ib_device *ib_dev, u8 port_num, int index,
> union ib_gid *gid)
> {
> - struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
> - struct device *dev = &hr_dev->pdev->dev;
> - u8 gid_idx = 0;
> - u8 port;
> -
> - if (port_num < 1 || port_num > hr_dev->caps.num_ports ||
> - index >= hr_dev->caps.gid_table_len[port_num - 1]) {
> - dev_err(dev,
> - "port_num %d index %d illegal! correct range: port_num 1~%d index 0~%d!\n",
> - port_num, index, hr_dev->caps.num_ports,
> - hr_dev->caps.gid_table_len[port_num - 1] - 1);
> - return -EINVAL;
> - }
> -
> - port = port_num - 1;
> - gid_idx = hns_get_gid_index(hr_dev, port, index);
> - if (gid_idx >= HNS_ROCE_MAX_GID_NUM) {
> - dev_err(dev, "port_num %d index %d illegal! total gid num %d!\n",
> - port_num, index, HNS_ROCE_MAX_GID_NUM);
> - return -EINVAL;
> - }
> -
> - memcpy(gid->raw, hr_dev->iboe.gid_table[gid_idx].raw,
> - HNS_ROCE_GID_SIZE);
> -
> return 0;
> }
>
> @@ -646,6 +480,8 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
> ib_dev->get_link_layer = hns_roce_get_link_layer;
> ib_dev->get_netdev = hns_roce_get_netdev;
> ib_dev->query_gid = hns_roce_query_gid;
> + ib_dev->add_gid = hns_roce_add_gid;
> + ib_dev->del_gid = hns_roce_del_gid;
> ib_dev->query_pkey = hns_roce_query_pkey;
> ib_dev->alloc_ucontext = hns_roce_alloc_ucontext;
> ib_dev->dealloc_ucontext = hns_roce_dealloc_ucontext;
> @@ -688,32 +524,22 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
> return ret;
> }
>
> - ret = hns_roce_setup_mtu_gids(hr_dev);
> + ret = hns_roce_setup_mtu_mac(hr_dev);
> if (ret) {
> - dev_err(dev, "roce_setup_mtu_gids failed!\n");
> - goto error_failed_setup_mtu_gids;
> + dev_err(dev, "setup_mtu_mac failed!\n");
> + goto error_failed_setup_mtu_mac;
> }
>
> iboe->nb.notifier_call = hns_roce_netdev_event;
> ret = register_netdevice_notifier(&iboe->nb);
> if (ret) {
> dev_err(dev, "register_netdevice_notifier failed!\n");
> - goto error_failed_setup_mtu_gids;
> - }
> -
> - iboe->nb_inet.notifier_call = hns_roce_inet_event;
> - ret = register_inetaddr_notifier(&iboe->nb_inet);
> - if (ret) {
> - dev_err(dev, "register inet addr notifier failed!\n");
> - goto error_failed_register_inetaddr_notifier;
> + goto error_failed_setup_mtu_mac;
> }
>
> return 0;
>
> -error_failed_register_inetaddr_notifier:
> - unregister_netdevice_notifier(&iboe->nb);
> -
> -error_failed_setup_mtu_gids:
> +error_failed_setup_mtu_mac:
> ib_unregister_device(ib_dev);
>
> return ret;
>
^ permalink raw reply
* Re: [PATCH for-next 10/11] IB/hns: Implement the add_gid/del_gid and optimize the GIDs management
From: Shaobo Xu @ 2016-11-07 10:04 UTC (permalink / raw)
To: Anurup M, Salil Mehta, dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
mehta.salil.lnk-Re5JQEeQqe8AvxtiuMwx3w,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linuxarm-hv44wF8Li93QT0dZR+AlfA
In-Reply-To: <58203884.6040808-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
On 2016/11/7 16:17, Anurup M wrote:
>
>
> On 11/4/2016 10:06 PM, Salil Mehta wrote:
>> From: Shaobo Xu <xushaobo2-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>
>> IB core has implemented the calculation of GIDs and the management
>> of GID tables, and it is now responsible to supply query function
>> for GIDs. So the calculation of GIDs and the management of GID
>> tables in the RoCE driver is redundant.
>>
>> The patch is to implement the add_gid/del_gid to set the GIDs in
>> the RoCE driver, remove the redundant calculation and management of
>> GIDs in the notifier call of the net device and the inet, and
>> update the query_gid.
>>
>> Signed-off-by: Shaobo Xu <xushaobo2-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Reviewed-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Salil Mehta <salil.mehta-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> ---
>> drivers/infiniband/hw/hns/hns_roce_device.h | 2 -
>> drivers/infiniband/hw/hns/hns_roce_main.c | 270 +++++----------------------
>> 2 files changed, 48 insertions(+), 224 deletions(-)
>>
>> diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
>> index 593a42a..9ef1cc3 100644
>> --- a/drivers/infiniband/hw/hns/hns_roce_device.h
>> +++ b/drivers/infiniband/hw/hns/hns_roce_device.h
>> @@ -429,8 +429,6 @@ struct hns_roce_ib_iboe {
>> struct net_device *netdevs[HNS_ROCE_MAX_PORTS];
>> struct notifier_block nb;
>> struct notifier_block nb_inet;
>> - /* 16 GID is shared by 6 port in v1 engine. */
>> - union ib_gid gid_table[HNS_ROCE_MAX_GID_NUM];
>> u8 phy_port[HNS_ROCE_MAX_PORTS];
>> };
>>
>> diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
>> index 6770171..795ef97 100644
>> --- a/drivers/infiniband/hw/hns/hns_roce_main.c
>> +++ b/drivers/infiniband/hw/hns/hns_roce_main.c
>> @@ -35,52 +35,13 @@
>> #include <rdma/ib_addr.h>
>> #include <rdma/ib_smi.h>
>> #include <rdma/ib_user_verbs.h>
>> +#include <rdma/ib_cache.h>
>> #include "hns_roce_common.h"
>> #include "hns_roce_device.h"
>> #include "hns_roce_user.h"
>> #include "hns_roce_hem.h"
>>
>> /**
>> - * hns_roce_addrconf_ifid_eui48 - Get default gid.
>> - * @eui: eui.
>> - * @vlan_id: gid
>> - * @dev: net device
>> - * Description:
>> - * MAC convert to GID
>> - * gid[0..7] = fe80 0000 0000 0000
>> - * gid[8] = mac[0] ^ 2
>> - * gid[9] = mac[1]
>> - * gid[10] = mac[2]
>> - * gid[11] = ff (VLAN ID high byte (4 MS bits))
>> - * gid[12] = fe (VLAN ID low byte)
>> - * gid[13] = mac[3]
>> - * gid[14] = mac[4]
>> - * gid[15] = mac[5]
>> - */
>> -static void hns_roce_addrconf_ifid_eui48(u8 *eui, u16 vlan_id,
>> - struct net_device *dev)
>> -{
>> - memcpy(eui, dev->dev_addr, 3);
>> - memcpy(eui + 5, dev->dev_addr + 3, 3);
>> - if (vlan_id < 0x1000) {
>> - eui[3] = vlan_id >> 8;
>> - eui[4] = vlan_id & 0xff;
>> - } else {
>> - eui[3] = 0xff;
>> - eui[4] = 0xfe;
>> - }
>> - eui[0] ^= 2;
>> -}
>> -
>> -static void hns_roce_make_default_gid(struct net_device *dev, union ib_gid *gid)
>> -{
>> - memset(gid, 0, sizeof(*gid));
>> - gid->raw[0] = 0xFE;
>> - gid->raw[1] = 0x80;
>> - hns_roce_addrconf_ifid_eui48(&gid->raw[8], 0xffff, dev);
>> -}
>> -
>> -/**
>> * hns_get_gid_index - Get gid index.
>> * @hr_dev: pointer to structure hns_roce_dev.
>> * @port: port, value range: 0 ~ MAX
>> @@ -96,30 +57,6 @@ int hns_get_gid_index(struct hns_roce_dev *hr_dev, u8 port, int gid_index)
>> return gid_index * hr_dev->caps.num_ports + port;
>> }
>>
>> -static int hns_roce_set_gid(struct hns_roce_dev *hr_dev, u8 port, int gid_index,
>> - union ib_gid *gid)
>> -{
>> - struct device *dev = &hr_dev->pdev->dev;
>> - u8 gid_idx = 0;
>> -
>> - if (gid_index >= hr_dev->caps.gid_table_len[port]) {
>> - dev_err(dev, "gid_index %d illegal, port %d gid range: 0~%d\n",
>> - gid_index, port, hr_dev->caps.gid_table_len[port] - 1);
>> - return -EINVAL;
>> - }
>> -
>> - gid_idx = hns_get_gid_index(hr_dev, port, gid_index);
>> -
>> - if (!memcmp(gid, &hr_dev->iboe.gid_table[gid_idx], sizeof(*gid)))
>> - return -EINVAL;
>> -
>> - memcpy(&hr_dev->iboe.gid_table[gid_idx], gid, sizeof(*gid));
>> -
>> - hr_dev->hw->set_gid(hr_dev, port, gid_index, gid);
>> -
>> - return 0;
>> -}
>> -
>> static void hns_roce_set_mac(struct hns_roce_dev *hr_dev, u8 port, u8 *addr)
>> {
>> u8 phy_port;
>> @@ -147,15 +84,44 @@ static void hns_roce_set_mtu(struct hns_roce_dev *hr_dev, u8 port, int mtu)
>> hr_dev->hw->set_mtu(hr_dev, phy_port, tmp);
>> }
>>
>> -static void hns_roce_update_gids(struct hns_roce_dev *hr_dev, int port)
>> +static int hns_roce_add_gid(struct ib_device *device, u8 port_num,
>> + unsigned int index, const union ib_gid *gid,
>> + const struct ib_gid_attr *attr, void **context)
>> +{
>> + struct hns_roce_dev *hr_dev = to_hr_dev(device);
>> + u8 port = port_num - 1;
>> + unsigned long flags;
>> +
>> + if (port >= hr_dev->caps.num_ports)
>> + return -EINVAL;
>> +
>> + spin_lock_irqsave(&hr_dev->iboe.lock, flags);
>> +
>> + hr_dev->hw->set_gid(hr_dev, port, index, (union ib_gid *)gid);
>> +
>> + spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
>> +
>> + return 0;
>> +}
>> +
>> +static int hns_roce_del_gid(struct ib_device *device, u8 port_num,
>> + unsigned int index, void **context)
>> {
>> - struct ib_event event;
>> + struct hns_roce_dev *hr_dev = to_hr_dev(device);
>> + union ib_gid zgid = { {0} };
>> + u8 port = port_num - 1;
>> + unsigned long flags;
>> +
>> + if (port >= hr_dev->caps.num_ports)
>> + return -EINVAL;
>>
>> - /* Refresh gid in ib_cache */
>> - event.device = &hr_dev->ib_dev;
>> - event.element.port_num = port + 1;
>> - event.event = IB_EVENT_GID_CHANGE;
>> - ib_dispatch_event(&event);
>> + spin_lock_irqsave(&hr_dev->iboe.lock, flags);
>> +
>> + hr_dev->hw->set_gid(hr_dev, port, index, &zgid);
> zgid has value zero. and after this call, where is zgid used?
While deleting the GIDs of the specified port, it needs to set zgid to the port GID configure register,
which means no GID to this port.
Best Regards,
>> +
>> + spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
>> +
>> + return 0;
>> }
>>
>> static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
>> @@ -164,8 +130,6 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
>> struct device *dev = &hr_dev->pdev->dev;
>> struct net_device *netdev;
>> unsigned long flags;
>> - union ib_gid gid;
>> - int ret = 0;
>>
>> netdev = hr_dev->iboe.netdevs[port];
>> if (!netdev) {
>> @@ -181,10 +145,6 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
>> case NETDEV_REGISTER:
>> case NETDEV_CHANGEADDR:
>> hns_roce_set_mac(hr_dev, port, netdev->dev_addr);
>> - hns_roce_make_default_gid(netdev, &gid);
>> - ret = hns_roce_set_gid(hr_dev, port, 0, &gid);
>> - if (!ret)
>> - hns_roce_update_gids(hr_dev, port);
>> break;
>> case NETDEV_DOWN:
>> /*
>> @@ -197,7 +157,7 @@ static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
>> }
>>
>> spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
>> - return ret;
>> + return 0;
>> }
>>
>> static int hns_roce_netdev_event(struct notifier_block *self,
>> @@ -224,118 +184,17 @@ static int hns_roce_netdev_event(struct notifier_block *self,
>> return NOTIFY_DONE;
>> }
>>
>> -static void hns_roce_addr_event(int event, struct net_device *event_netdev,
>> - struct hns_roce_dev *hr_dev, union ib_gid *gid)
>> -{
>> - struct hns_roce_ib_iboe *iboe = NULL;
>> - int gid_table_len = 0;
>> - unsigned long flags;
>> - union ib_gid zgid;
>> - u8 gid_idx = 0;
>> - u8 port = 0;
>> - int i = 0;
>> - int free;
>> - struct net_device *real_dev = rdma_vlan_dev_real_dev(event_netdev) ?
>> - rdma_vlan_dev_real_dev(event_netdev) :
>> - event_netdev;
>> -
>> - if (event != NETDEV_UP && event != NETDEV_DOWN)
>> - return;
>> -
>> - iboe = &hr_dev->iboe;
>> - while (port < hr_dev->caps.num_ports) {
>> - if (real_dev == iboe->netdevs[port])
>> - break;
>> - port++;
>> - }
>> -
>> - if (port >= hr_dev->caps.num_ports) {
>> - dev_dbg(&hr_dev->pdev->dev, "can't find netdev\n");
>> - return;
>> - }
>> -
>> - memset(zgid.raw, 0, sizeof(zgid.raw));
>> - free = -1;
>> - gid_table_len = hr_dev->caps.gid_table_len[port];
>> -
>> - spin_lock_irqsave(&hr_dev->iboe.lock, flags);
>> -
>> - for (i = 0; i < gid_table_len; i++) {
>> - gid_idx = hns_get_gid_index(hr_dev, port, i);
>> - if (!memcmp(gid->raw, iboe->gid_table[gid_idx].raw,
>> - sizeof(gid->raw)))
>> - break;
>> - if (free < 0 && !memcmp(zgid.raw,
>> - iboe->gid_table[gid_idx].raw, sizeof(zgid.raw)))
>> - free = i;
>> - }
>> -
>> - if (i >= gid_table_len) {
>> - if (free < 0) {
>> - spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
>> - dev_dbg(&hr_dev->pdev->dev,
>> - "gid_index overflow, port(%d)\n", port);
>> - return;
>> - }
>> - if (!hns_roce_set_gid(hr_dev, port, free, gid))
>> - hns_roce_update_gids(hr_dev, port);
>> - } else if (event == NETDEV_DOWN) {
>> - if (!hns_roce_set_gid(hr_dev, port, i, &zgid))
>> - hns_roce_update_gids(hr_dev, port);
>> - }
>> -
>> - spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
>> -}
>> -
>> -static int hns_roce_inet_event(struct notifier_block *self, unsigned long event,
>> - void *ptr)
>> -{
>> - struct in_ifaddr *ifa = ptr;
>> - struct hns_roce_dev *hr_dev;
>> - struct net_device *dev = ifa->ifa_dev->dev;
>> - union ib_gid gid;
>> -
>> - ipv6_addr_set_v4mapped(ifa->ifa_address, (struct in6_addr *)&gid);
>> -
>> - hr_dev = container_of(self, struct hns_roce_dev, iboe.nb_inet);
>> -
>> - hns_roce_addr_event(event, dev, hr_dev, &gid);
>> -
>> - return NOTIFY_DONE;
>> -}
>> -
>> -static int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
>> +static int hns_roce_setup_mtu_mac(struct hns_roce_dev *hr_dev)
>> {
>> - struct in_ifaddr *ifa_list = NULL;
>> - union ib_gid gid = {{0} };
>> - u32 ipaddr = 0;
>> - int index = 0;
>> - int ret = 0;
>> - u8 i = 0;
>> + u8 i;
>>
>> for (i = 0; i < hr_dev->caps.num_ports; i++) {
>> hns_roce_set_mtu(hr_dev, i,
>> ib_mtu_enum_to_int(hr_dev->caps.max_mtu));
>> hns_roce_set_mac(hr_dev, i, hr_dev->iboe.netdevs[i]->dev_addr);
>> -
>> - if (hr_dev->iboe.netdevs[i]->ip_ptr) {
>> - ifa_list = hr_dev->iboe.netdevs[i]->ip_ptr->ifa_list;
>> - index = 1;
>> - while (ifa_list) {
>> - ipaddr = ifa_list->ifa_address;
>> - ipv6_addr_set_v4mapped(ipaddr,
>> - (struct in6_addr *)&gid);
>> - ret = hns_roce_set_gid(hr_dev, i, index, &gid);
>> - if (ret)
>> - break;
>> - index++;
>> - ifa_list = ifa_list->ifa_next;
>> - }
>> - hns_roce_update_gids(hr_dev, i);
>> - }
>> }
>>
>> - return ret;
>> + return 0;
>> }
>>
>> static int hns_roce_query_device(struct ib_device *ib_dev,
>> @@ -444,31 +303,6 @@ static enum rdma_link_layer hns_roce_get_link_layer(struct ib_device *device,
>> static int hns_roce_query_gid(struct ib_device *ib_dev, u8 port_num, int index,
>> union ib_gid *gid)
>> {
>> - struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
>> - struct device *dev = &hr_dev->pdev->dev;
>> - u8 gid_idx = 0;
>> - u8 port;
>> -
>> - if (port_num < 1 || port_num > hr_dev->caps.num_ports ||
>> - index >= hr_dev->caps.gid_table_len[port_num - 1]) {
>> - dev_err(dev,
>> - "port_num %d index %d illegal! correct range: port_num 1~%d index 0~%d!\n",
>> - port_num, index, hr_dev->caps.num_ports,
>> - hr_dev->caps.gid_table_len[port_num - 1] - 1);
>> - return -EINVAL;
>> - }
>> -
>> - port = port_num - 1;
>> - gid_idx = hns_get_gid_index(hr_dev, port, index);
>> - if (gid_idx >= HNS_ROCE_MAX_GID_NUM) {
>> - dev_err(dev, "port_num %d index %d illegal! total gid num %d!\n",
>> - port_num, index, HNS_ROCE_MAX_GID_NUM);
>> - return -EINVAL;
>> - }
>> -
>> - memcpy(gid->raw, hr_dev->iboe.gid_table[gid_idx].raw,
>> - HNS_ROCE_GID_SIZE);
>> -
>> return 0;
>> }
>>
>> @@ -646,6 +480,8 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
>> ib_dev->get_link_layer = hns_roce_get_link_layer;
>> ib_dev->get_netdev = hns_roce_get_netdev;
>> ib_dev->query_gid = hns_roce_query_gid;
>> + ib_dev->add_gid = hns_roce_add_gid;
>> + ib_dev->del_gid = hns_roce_del_gid;
>> ib_dev->query_pkey = hns_roce_query_pkey;
>> ib_dev->alloc_ucontext = hns_roce_alloc_ucontext;
>> ib_dev->dealloc_ucontext = hns_roce_dealloc_ucontext;
>> @@ -688,32 +524,22 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
>> return ret;
>> }
>>
>> - ret = hns_roce_setup_mtu_gids(hr_dev);
>> + ret = hns_roce_setup_mtu_mac(hr_dev);
>> if (ret) {
>> - dev_err(dev, "roce_setup_mtu_gids failed!\n");
>> - goto error_failed_setup_mtu_gids;
>> + dev_err(dev, "setup_mtu_mac failed!\n");
>> + goto error_failed_setup_mtu_mac;
>> }
>>
>> iboe->nb.notifier_call = hns_roce_netdev_event;
>> ret = register_netdevice_notifier(&iboe->nb);
>> if (ret) {
>> dev_err(dev, "register_netdevice_notifier failed!\n");
>> - goto error_failed_setup_mtu_gids;
>> - }
>> -
>> - iboe->nb_inet.notifier_call = hns_roce_inet_event;
>> - ret = register_inetaddr_notifier(&iboe->nb_inet);
>> - if (ret) {
>> - dev_err(dev, "register inet addr notifier failed!\n");
>> - goto error_failed_register_inetaddr_notifier;
>> + goto error_failed_setup_mtu_mac;
>> }
>>
>> return 0;
>>
>> -error_failed_register_inetaddr_notifier:
>> - unregister_netdevice_notifier(&iboe->nb);
>> -
>> -error_failed_setup_mtu_gids:
>> +error_failed_setup_mtu_mac:
>> ib_unregister_device(ib_dev);
>>
>> return ret;
>>
>
> _______________________________________________
> linuxarm mailing list
> linuxarm-hv44wF8Li93QT0dZR+AlfA@public.gmane.org
> http://rnd-openeuler.huawei.com/mailman/listinfo/linuxarm
>
> .
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 1/2] mm: add locked parameter to get_user_pages()
From: Jesper Nilsson @ 2016-11-07 10:49 UTC (permalink / raw)
To: Lorenzo Stoakes
Cc: devel, Rik van Riel, linux-ia64, linux-cris-kernel, kvm,
linux-rdma, Linus Torvalds, Dave Hansen, Hugh Dickins,
linux-kernel, dri-devel, Michal Hocko, linux-mm, Paolo Bonzini,
Jan Kara, Andrew Morton, Mel Gorman, linux-media
In-Reply-To: <20161031100228.17917-2-lstoakes@gmail.com>
On Mon, Oct 31, 2016 at 10:02:27AM +0000, Lorenzo Stoakes wrote:
> This patch adds an int *locked parameter to get_user_pages() to allow
> VM_FAULT_RETRY faulting behaviour similar to get_user_pages_[un]locked().
>
> It additionally clears the way for get_user_pages_locked() to be removed as its
> sole remaining useful characteristic was to allow for VM_FAULT_RETRY behaviour
> when faulting in pages.
>
> It should not introduce any functional changes, however it does allow for
> subsequent changes to get_user_pages() callers to take advantage of
> VM_FAULT_RETRY.
For the cris-part:
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
> Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
/^JN - Jesper Nilsson
--
Jesper Nilsson -- jesper.nilsson@axis.com
^ permalink raw reply
* Re: [PATCH 1/2] mm: add locked parameter to get_user_pages()
From: Lorenzo Stoakes @ 2016-11-07 11:00 UTC (permalink / raw)
To: Jesper Nilsson
Cc: devel, Rik van Riel, linux-ia64, linux-cris-kernel, kvm,
linux-rdma, Linus Torvalds, Dave Hansen, Hugh Dickins,
linux-kernel, dri-devel, Michal Hocko, linux-mm, Paolo Bonzini,
Jan Kara, Andrew Morton, Mel Gorman, linux-media
In-Reply-To: <20161107104918.GQ30704@axis.com>
On Mon, Nov 07, 2016 at 11:49:18AM +0100, Jesper Nilsson wrote:
> For the cris-part:
> Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Thanks very much for that, however just to avoid any confusion, I realised this
series was not not the right way forward after discussion with Paolo and rather
it makes more sense to keep the API as it is and to update callers where it
makes sense to.
^ permalink raw reply
* RE: [PATCH for-next 01/11] IB/hns: Add the interface for querying QP1
From: Salil Mehta @ 2016-11-07 11:45 UTC (permalink / raw)
To: Anurup m, dledford@redhat.com
Cc: linux-rdma@vger.kernel.org, netdev@vger.kernel.org,
mehta.salil.lnk@gmail.com, linux-kernel@vger.kernel.org, Linuxarm
In-Reply-To: <582014FF.3070600@huawei.com>
> -----Original Message-----
> From: Anurup m
> Sent: Monday, November 07, 2016 5:46 AM
> To: Salil Mehta; dledford@redhat.com
> Cc: linux-rdma@vger.kernel.org; netdev@vger.kernel.org;
> mehta.salil.lnk@gmail.com; linux-kernel@vger.kernel.org; Linuxarm
> Subject: Re: [PATCH for-next 01/11] IB/hns: Add the interface for
> querying QP1
>
>
>
> On 11/4/2016 10:06 PM, Salil Mehta wrote:
> > From: Lijun Ou <oulijun@huawei.com>
> >
> > In old code, It only added the interface for querying non-specific
> > QP. This patch mainly adds an interface for querying QP1.
> >
> > Signed-off-by: Lijun Ou <oulijun@huawei.com>
> > Reviewed-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
> > Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> > ---
> > drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 87
> +++++++++++++++++++++++++++-
> > drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 6 +-
> > 2 files changed, 90 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> > index 71232e5..ca8b784 100644
> > --- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> > +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
> > @@ -2630,8 +2630,82 @@ static int hns_roce_v1_query_qpc(struct
> hns_roce_dev *hr_dev,
> > return ret;
> > }
> >
> > -int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr
> *qp_attr,
> > - int qp_attr_mask, struct ib_qp_init_attr
> *qp_init_attr)
> > +static int hns_roce_v1_q_sqp(struct ib_qp *ibqp, struct ib_qp_attr
> *qp_attr,
> > + int qp_attr_mask,
> > + struct ib_qp_init_attr *qp_init_attr)
> > +{
> > + struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> > + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> > + struct hns_roce_sqp_context *context;
> > + u32 addr;
> > +
> > + context = kzalloc(sizeof(*context), GFP_KERNEL);
> > + if (!context)
> > + return -ENOMEM;
> > +
> Do we really need dynamic alloc here as the size is fixed and this
> memory scope is
> only inside this function. I think better to use a static allocation.
Agreed. Somehow missed this in the internal review. Will change!
Thanks
Salil
>
> > + mutex_lock(&hr_qp->mutex);
> > +
> > + if (hr_qp->state == IB_QPS_RESET) {
> I think alloc can be moved after this check (if dynamic alloc is really
> needed).
> > + qp_attr->qp_state = IB_QPS_RESET;
> > + goto done;
> > + }
> > +
> > + addr = ROCEE_QP1C_CFG0_0_REG + hr_qp->port * sizeof(*context);
> > + context->qp1c_bytes_4 = roce_read(hr_dev, addr);
> > + context->sq_rq_bt_l = roce_read(hr_dev, addr + 1);
> > + context->qp1c_bytes_12 = roce_read(hr_dev, addr + 2);
> > + context->qp1c_bytes_16 = roce_read(hr_dev, addr + 3);
> > + context->qp1c_bytes_20 = roce_read(hr_dev, addr + 4);
> > + context->cur_rq_wqe_ba_l = roce_read(hr_dev, addr + 5);
> > + context->qp1c_bytes_28 = roce_read(hr_dev, addr + 6);
> > + context->qp1c_bytes_32 = roce_read(hr_dev, addr + 7);
> > + context->cur_sq_wqe_ba_l = roce_read(hr_dev, addr + 8);
> > + context->qp1c_bytes_40 = roce_read(hr_dev, addr + 9);
> > +
> > + hr_qp->state = roce_get_field(context->qp1c_bytes_4,
> > + QP1C_BYTES_4_QP_STATE_M,
> > + QP1C_BYTES_4_QP_STATE_S);
> > + qp_attr->qp_state = hr_qp->state;
> > + qp_attr->path_mtu = IB_MTU_256;
> > + qp_attr->path_mig_state = IB_MIG_ARMED;
> > + qp_attr->qkey = QKEY_VAL;
> > + qp_attr->rq_psn = 0;
> > + qp_attr->sq_psn = 0;
> > + qp_attr->dest_qp_num = 1;
> > + qp_attr->qp_access_flags = 6;
> > +
> > + qp_attr->pkey_index = roce_get_field(context->qp1c_bytes_20,
> > + QP1C_BYTES_20_PKEY_IDX_M,
> > + QP1C_BYTES_20_PKEY_IDX_S);
> > + qp_attr->port_num = hr_qp->port + 1;
> > + qp_attr->sq_draining = 0;
> > + qp_attr->max_rd_atomic = 0;
> > + qp_attr->max_dest_rd_atomic = 0;
> > + qp_attr->min_rnr_timer = 0;
> > + qp_attr->timeout = 0;
> > + qp_attr->retry_cnt = 0;
> > + qp_attr->rnr_retry = 0;
> > + qp_attr->alt_timeout = 0;
> > +
> > +done:
> > + qp_attr->cur_qp_state = qp_attr->qp_state;
> > + qp_attr->cap.max_recv_wr = hr_qp->rq.wqe_cnt;
> > + qp_attr->cap.max_recv_sge = hr_qp->rq.max_gs;
> > + qp_attr->cap.max_send_wr = hr_qp->sq.wqe_cnt;
> > + qp_attr->cap.max_send_sge = hr_qp->sq.max_gs;
> > + qp_attr->cap.max_inline_data = 0;
> > + qp_init_attr->cap = qp_attr->cap;
> > + qp_init_attr->create_flags = 0;
> > +
> > + mutex_unlock(&hr_qp->mutex);
> > + kfree(context);
> > +
> > + return 0;
> > +}
> > +
> > +static int hns_roce_v1_q_qp(struct ib_qp *ibqp, struct ib_qp_attr
> *qp_attr,
> > + int qp_attr_mask,
> > + struct ib_qp_init_attr *qp_init_attr)
> > {
> > struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> > struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> > @@ -2767,6 +2841,15 @@ int hns_roce_v1_query_qp(struct ib_qp *ibqp,
> struct ib_qp_attr *qp_attr,
> > return ret;
> > }
> >
> > +int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr
> *qp_attr,
> > + int qp_attr_mask, struct ib_qp_init_attr
> *qp_init_attr)
> > +{
> > + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
> > +
> > + return hr_qp->doorbell_qpn <= 1 ?
> > + hns_roce_v1_q_sqp(ibqp, qp_attr, qp_attr_mask,
> qp_init_attr) :
> > + hns_roce_v1_q_qp(ibqp, qp_attr, qp_attr_mask,
> qp_init_attr);
> > +}
> > static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev
> *hr_dev,
> > struct hns_roce_qp *hr_qp,
> > int is_user)
> > diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> > index 539b0a3b..2e1878b 100644
> > --- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> > +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
> > @@ -480,13 +480,17 @@ struct hns_roce_sqp_context {
> > u32 qp1c_bytes_12;
> > u32 qp1c_bytes_16;
> > u32 qp1c_bytes_20;
> > - u32 qp1c_bytes_28;
> > u32 cur_rq_wqe_ba_l;
> > + u32 qp1c_bytes_28;
> > u32 qp1c_bytes_32;
> > u32 cur_sq_wqe_ba_l;
> > u32 qp1c_bytes_40;
> > };
> >
> > +#define QP1C_BYTES_4_QP_STATE_S 0
> > +#define QP1C_BYTES_4_QP_STATE_M \
> > + (((1UL << 3) - 1) << QP1C_BYTES_4_QP_STATE_S)
> > +
> > #define QP1C_BYTES_4_SQ_WQE_SHIFT_S 8
> > #define QP1C_BYTES_4_SQ_WQE_SHIFT_M \
> > (((1UL << 4) - 1) << QP1C_BYTES_4_SQ_WQE_SHIFT_S)
> >
^ permalink raw reply
* Re: [PATCH rdma-rc 5/9] IB/mlx4: Handle well-known-gid in mad_demux processing
From: jackm @ 2016-11-07 14:42 UTC (permalink / raw)
To: Hal Rosenstock
Cc: Leon Romanovsky, dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Daniel Jurgens,
Leon Romanovsky
In-Reply-To: <d339ee5f-bf7e-dc35-3dd0-c6ff13a222b4-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On Sat, 5 Nov 2016 17:03:25 -0400
Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> On 11/5/2016 3:57 PM, Leon Romanovsky wrote:
> > From: Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> >
> > If OpenSM runs over ConnectX-3, and there are ConnectIB VFs active
> > on the network, the OpenSM will receive QP1 packets containing a GRH
> > where the destination GID is the "Well-Known GID" -- which is not a
> > GID in the HCA Port's GID Table.
> >
> > This GID must be tested-for separately -- and packets which contain
> > this destination GID should be routed to slave 0 (the PF).
> >
> > Fixes: 37bfc7c1e83f ('IB/mlx4: SR-IOV multiplex and demultiplex
> > MADs') Signed-off-by: Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> > Signed-off-by: Daniel Jurgens <danielj-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> > Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > ---
> > drivers/infiniband/hw/mlx4/mad.c | 16 ++++++++++++----
> > include/rdma/ib_verbs.h | 1 +
> > 2 files changed, 13 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/mlx4/mad.c
> > b/drivers/infiniband/hw/mlx4/mad.c index b8e9013..f0b8272 100644
> > --- a/drivers/infiniband/hw/mlx4/mad.c
> > +++ b/drivers/infiniband/hw/mlx4/mad.c
> > @@ -729,10 +729,18 @@ static int mlx4_ib_demux_mad(struct ib_device
> > *ibdev, u8 port,
> > /* If a grh is present, we demux according to it */
> > if (wc->wc_flags & IB_WC_GRH) {
> > - slave = mlx4_ib_find_real_gid(ibdev, port,
> > grh->dgid.global.interface_id);
> > - if (slave < 0) {
> > - mlx4_ib_warn(ibdev, "failed matching
> > grh\n");
> > - return -ENOENT;
> > + if (grh->dgid.global.interface_id ==
> > + cpu_to_be64(IB_SA_WELL_KNOWN_GUID) &&
> > + grh->dgid.global.subnet_prefix ==
> > + cpu_to_be64(IB_SA_WELL_KNOWN_GID_PREFIX)) {
> > + slave = 0;
> > + } else {
> > + slave = mlx4_ib_find_real_gid(ibdev, port,
> > +
> > grh->dgid.global.interface_id);
> > + if (slave < 0) {
> > + mlx4_ib_warn(ibdev, "failed
> > matching grh\n");
> > + return -ENOENT;
> > + }
> > }
> > }
> > /* Class-specific handling */
> > diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> > index 467a4b4..da5949f 100644
> > --- a/include/rdma/ib_verbs.h
> > +++ b/include/rdma/ib_verbs.h
> > @@ -100,6 +100,7 @@ enum rdma_node_type {
> >
> > enum {
> > /* set the local administered indication */
> > + IB_SA_WELL_KNOWN_GID_PREFIX = 0xfe80000000000000ull,
> > IB_SA_WELL_KNOWN_GUID = BIT_ULL(57) | 2,
>
> Isn't the SA well-known GID the concatenation of the subnet prefix
> (which isn't necessarily the default link local one) and the GUID
> 0x0200000000000002 ?
>
> -- Hal
PLEASE DO NOT APPLY THIS PATCH. Hal is correct, and the patch needs
to be fixed. We will submit an updated (i.e. fixed) version.
-Jack
>
> > };
> >
> >
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: rdma-core example spec file is broken
From: Jason Gunthorpe @ 2016-11-07 16:44 UTC (permalink / raw)
To: Alaa Hleihel
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, leonro-VPRAkNaXOzVWk0Htik3J/w,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <0e831337-e404-8973-b7fe-e176b3d688e8-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On Sun, Nov 06, 2016 at 02:01:20PM +0200, Alaa Hleihel wrote:
> Hi Jason,
>
> The example spec file is broken after commit ddd530be4622 ("Minor RPM spec file improvments").
>
> rpmbuild fails with the following error:
>
> Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.7l0yoN
> + umask 022
> + cd /tmp/xxx/BUILD
> + cd rdma-core-11
> /var/tmp/rpm-tmp.7l0yoN: line 31: syntax error near unexpected token `post'
> error: Bad exit status from /var/tmp/rpm-tmp.7l0yoN (%build)
>
> The issue is that the new "Requires" macros were added after the %prep and %build sections..
Ah, interesting, in my test scripts the common spec file is
superceeded by the version in redhat/ for systemd systems, which is
why I did not notice it.
Jason
>From f2388d9d110ab43295d50d7345d5dda289a87a41 Mon Sep 17 00:00:00 2001
From: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Date: Mon, 7 Nov 2016 09:37:38 -0700
Subject: [PATCH] Fix common rdma-core.spec for systemd systems
RPM needs the Requires lines for systemd to be earlier.
Signed-off-by: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
---
rdma-core.spec | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/rdma-core.spec b/rdma-core.spec
index 35186c3271b8..bea2a2267d1d 100644
--- a/rdma-core.spec
+++ b/rdma-core.spec
@@ -46,6 +46,13 @@ BuildRequires: make
%endif
%endif
+# Detect if systemd is supported on this system
+%if 0%{?_unitdir:1}
+Requires(post): systemd-units
+Requires(preun): systemd-units
+Requires(postun): systemd-units
+%endif
+
%description
Temporary packaging
@@ -59,9 +66,6 @@ This is a simple example without the split sub packages to get things started.
# Detect if systemd is supported on this system
%if 0%{?_unitdir:1}
%define my_unitdir %{_unitdir}
-Requires(post): systemd-units
-Requires(preun): systemd-units
-Requires(postun): systemd-units
%else
%define my_unitdir /tmp/
%endif
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core] redhat/spec: add back strict librdmacm Requires
From: Jarod Wilson @ 2016-11-07 17:23 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: Jarod Wilson, Jason Gunthorpe, Doug Ledford
In-Reply-To: <20161104004213.GA28485-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Per discussion on list with Doug and Jason, add back the strict librdmacm
Requires so that librdmacm and librdmacm-utils are as tightly coupled as
possible.
CC: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
CC: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Jarod Wilson <jarod-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
redhat/rdma-core.spec | 1 +
1 file changed, 1 insertion(+)
diff --git a/redhat/rdma-core.spec b/redhat/rdma-core.spec
index f65a519..c3d8189 100644
--- a/redhat/rdma-core.spec
+++ b/redhat/rdma-core.spec
@@ -170,6 +170,7 @@ librdmacm provides a userspace RDMA Communication Managment API.
%package -n librdmacm-utils
Summary: Examples for the librdmacm library
+Requires: librdmacm%{?_isa} = %{version}-%{release}
%description -n librdmacm-utils
Example test programs for the librdmacm library.
--
2.10.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: nvmet_rdma crash - DISCONNECT event with NULL queue
From: J Freyensee @ 2016-11-07 18:29 UTC (permalink / raw)
To: Sagi Grimberg, 'Christoph Hellwig', Steve Wise
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Minturn, Dave B
In-Reply-To: <bd82c206-668a-0794-2d51-3c48058350d3-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
On Sun, 2016-11-06 at 09:35 +0200, Sagi Grimberg wrote:
> >
> > Btw, I want to actually make the ctrlid global for the target
> > instead of
> > per-subsystem to ease a few things, and debuggability is just one
> > more
> > on the list.
How will that be NVMe-over-Fabrics spec compliant?
The way I interpret the spec, ctrlid (I'm assuming you mean cntlid) is
allocated on a NVM subsystem basis. For example, Figure 34 of the
Discovery Log Page entry and Figure 20 of the Connect Command implies
to me CNTLID values are allocated on a NVM Subsystem granular-level
when I see statements such as:
(Figure 20: Connect Command Data): "...If the NVM subsystem uses the
static controller model and the value is FFFEh for the Admin Queue,
then any available controller may be returned."
This implies to me cntlid are allocated on an NVM subsystem basis, not
an NVMe Target basis.
>
> Sounds good
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: nvmet_rdma crash - DISCONNECT event with NULL queue
From: 'Christoph Hellwig' @ 2016-11-07 18:41 UTC (permalink / raw)
To: J Freyensee
Cc: Sagi Grimberg, 'Christoph Hellwig', Steve Wise,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Minturn, Dave B
In-Reply-To: <1478543378.3350.17.camel-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
On Mon, Nov 07, 2016 at 10:29:38AM -0800, J Freyensee wrote:
> The way I interpret the spec, ctrlid (I'm assuming you mean cntlid) is
> allocated on a NVM subsystem basis. For example, Figure 34 of the
> Discovery Log Page entry and Figure 20 of the Connect Command implies
> to me CNTLID values are allocated on a NVM Subsystem granular-level
> when I see statements such as:
It is per-subsystem. But nothing in the spec prohibits and implementation
that has multiple subsystems to simply not allocate cntlids that
would conflict betweens it's subsystems.
And in fact there is a TP in the working group that would require
implementations not to reuse cntlids for it to work. We'll probably
hear more about that once it's published.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: nvmet_rdma crash - DISCONNECT event with NULL queue
From: J Freyensee @ 2016-11-07 18:50 UTC (permalink / raw)
To: 'Christoph Hellwig'
Cc: Sagi Grimberg, Steve Wise, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Minturn, Dave B
In-Reply-To: <20161107184126.GA4400-jcswGhMUV9g@public.gmane.org>
On Mon, 2016-11-07 at 19:41 +0100, 'Christoph Hellwig' wrote:
> On Mon, Nov 07, 2016 at 10:29:38AM -0800, J Freyensee wrote:
> >
> > The way I interpret the spec, ctrlid (I'm assuming you mean cntlid)
> > is
> > allocated on a NVM subsystem basis. For example, Figure 34 of the
> > Discovery Log Page entry and Figure 20 of the Connect Command
> > implies
> > to me CNTLID values are allocated on a NVM Subsystem granular-level
> > when I see statements such as:
>
> It is per-subsystem. But nothing in the spec prohibits and
> implementation
> that has multiple subsystems to simply not allocate cntlids that
> would conflict betweens it's subsystems.
OK, so basically the nvmet change would be to make sure unique cntlids
are used across all NVM subsystems within the NVMe Target then?
>
> And in fact there is a TP in the working group that would require
> implementations not to reuse cntlids for it to work. We'll probably
> hear more about that once it's published.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: nvmet_rdma crash - DISCONNECT event with NULL queue
From: 'Christoph Hellwig' @ 2016-11-07 18:51 UTC (permalink / raw)
To: J Freyensee
Cc: 'Christoph Hellwig', Sagi Grimberg, Steve Wise,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Minturn, Dave B
In-Reply-To: <1478544616.3350.29.camel-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
On Mon, Nov 07, 2016 at 10:50:16AM -0800, J Freyensee wrote:
> OK, so basically the nvmet change would be to make sure unique cntlids
> are used across all NVM subsystems within the NVMe Target then?
Yes.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] xprtrdma: Fix DMAR failure in frwr_op_map() after reconnect
From: Chuck Lever @ 2016-11-07 21:16 UTC (permalink / raw)
To: trond.myklebust-7I+n7zu2hftEKMMhf/gKZA,
anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-nfs-u79uwXL29TY76Z2rM5mHXA
When a LOCALINV WR is flushed, the frmr is marked STALE, then
frwr_op_unmap_sync DMA-unmaps the frmr's SGL. These STALE frmrs
are then recovered when frwr_op_map hunts for an INVALID frmr to
use.
All other cases that need frmr recovery leave that SGL DMA-mapped.
The FRMR recovery path unconditionally DMA-unmaps the frmr's SGL.
To avoid DMA unmapping the SGL twice for flushed LOCAL_INV WRs,
alter the recovery logic (rather than the hot frwr_op_unmap_sync
path) to distinguish among these cases. This solution also takes
care of the case where multiple LOCAL_INV WRs are issued for the
same rpcrdma_req, some complete successfully, but some are flushed.
Reported-by: Vasco Steinmetz <linux-W+c8CscfqXQmp8TqCH86vg@public.gmane.org>
Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Tested-by: Vasco Steinmetz <linux-W+c8CscfqXQmp8TqCH86vg@public.gmane.org>
---
I received a report of DMAR errors occurring on v4.8 and newer
NFS/RDMA clients. I was able to reproduce the problem, and came up
with this fix, against v4.9-rc. Please consider merging this into
the next v4.9-rc bug fix series.
Vasco tells me this does not apply 100% cleanly to v4.8, but was
tested there. I'm unsure whether to include a Cc: stable.
net/sunrpc/xprtrdma/frwr_ops.c | 37 ++++++++++++++++++++++---------------
net/sunrpc/xprtrdma/xprt_rdma.h | 3 ++-
2 files changed, 24 insertions(+), 16 deletions(-)
diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
index 2109495..26b26be 100644
--- a/net/sunrpc/xprtrdma/frwr_ops.c
+++ b/net/sunrpc/xprtrdma/frwr_ops.c
@@ -44,18 +44,20 @@
* being done.
*
* When the underlying transport disconnects, MRs are left in one of
- * three states:
+ * four states:
*
* INVALID: The MR was not in use before the QP entered ERROR state.
- * (Or, the LOCAL_INV WR has not completed or flushed yet).
- *
- * STALE: The MR was being registered or unregistered when the QP
- * entered ERROR state, and the pending WR was flushed.
*
* VALID: The MR was registered before the QP entered ERROR state.
*
- * When frwr_op_map encounters STALE and VALID MRs, they are recovered
- * with ib_dereg_mr and then are re-initialized. Beause MR recovery
+ * FLUSHED_FR: The MR was being registered when the QP entered ERROR
+ * state, and the pending WR was flushed.
+ *
+ * FLUSHED_LI: The MR was being invalidated when the QP entered ERROR
+ * state, and the pending WR was flushed.
+ *
+ * When frwr_op_map encounters FLUSHED and VALID MRs, they are recovered
+ * with ib_dereg_mr and then are re-initialized. Because MR recovery
* allocates fresh resources, it is deferred to a workqueue, and the
* recovered MRs are placed back on the rb_mws list when recovery is
* complete. frwr_op_map allocates another MR for the current RPC while
@@ -177,12 +179,15 @@
static void
frwr_op_recover_mr(struct rpcrdma_mw *mw)
{
+ enum rpcrdma_frmr_state state = mw->frmr.fr_state;
struct rpcrdma_xprt *r_xprt = mw->mw_xprt;
struct rpcrdma_ia *ia = &r_xprt->rx_ia;
int rc;
rc = __frwr_reset_mr(ia, mw);
- ib_dma_unmap_sg(ia->ri_device, mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ if (state != FRMR_FLUSHED_LI)
+ ib_dma_unmap_sg(ia->ri_device,
+ mw->mw_sg, mw->mw_nents, mw->mw_dir);
if (rc)
goto out_release;
@@ -262,10 +267,8 @@
}
static void
-__frwr_sendcompletion_flush(struct ib_wc *wc, struct rpcrdma_frmr *frmr,
- const char *wr)
+__frwr_sendcompletion_flush(struct ib_wc *wc, const char *wr)
{
- frmr->fr_state = FRMR_IS_STALE;
if (wc->status != IB_WC_WR_FLUSH_ERR)
pr_err("rpcrdma: %s: %s (%u/0x%x)\n",
wr, ib_wc_status_msg(wc->status),
@@ -288,7 +291,8 @@
if (wc->status != IB_WC_SUCCESS) {
cqe = wc->wr_cqe;
frmr = container_of(cqe, struct rpcrdma_frmr, fr_cqe);
- __frwr_sendcompletion_flush(wc, frmr, "fastreg");
+ frmr->fr_state = FRMR_FLUSHED_FR;
+ __frwr_sendcompletion_flush(wc, "fastreg");
}
}
@@ -308,7 +312,8 @@
if (wc->status != IB_WC_SUCCESS) {
cqe = wc->wr_cqe;
frmr = container_of(cqe, struct rpcrdma_frmr, fr_cqe);
- __frwr_sendcompletion_flush(wc, frmr, "localinv");
+ frmr->fr_state = FRMR_FLUSHED_LI;
+ __frwr_sendcompletion_flush(wc, "localinv");
}
}
@@ -328,8 +333,10 @@
/* WARNING: Only wr_cqe and status are reliable at this point */
cqe = wc->wr_cqe;
frmr = container_of(cqe, struct rpcrdma_frmr, fr_cqe);
- if (wc->status != IB_WC_SUCCESS)
- __frwr_sendcompletion_flush(wc, frmr, "localinv");
+ if (wc->status != IB_WC_SUCCESS) {
+ frmr->fr_state = FRMR_FLUSHED_LI;
+ __frwr_sendcompletion_flush(wc, "localinv");
+ }
complete(&frmr->fr_linv_done);
}
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index 0d35b76..6e1bba3 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -216,7 +216,8 @@ struct rpcrdma_rep {
enum rpcrdma_frmr_state {
FRMR_IS_INVALID, /* ready to be used */
FRMR_IS_VALID, /* in use */
- FRMR_IS_STALE, /* failed completion */
+ FRMR_FLUSHED_FR, /* flushed FASTREG WR */
+ FRMR_FLUSHED_LI, /* flushed LOCALINV WR */
};
struct rpcrdma_frmr {
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH rdma-core 1/7] libhns: Add initial main frame
From: Jason Gunthorpe @ 2016-11-07 23:15 UTC (permalink / raw)
To: oulijun
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linuxarm-hv44wF8Li93QT0dZR+AlfA
In-Reply-To: <5813F869.7010606-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
On Sat, Oct 29, 2016 at 09:16:25AM +0800, oulijun wrote:
> We hope that the only one userspace library file named
> libhns-rdmav2.so will be used for the different hardware
> version(hip06, hip07, ...), because there are only little change
> between their userspace drivers. So we need to distinguish hardware
> version.
I guess that makes sense, but you still need to be able to parse dt
compatible strings that are lists.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [RFC ABI V5 01/10] RDMA/core: Refactor IDR to be per-device
From: Jason Gunthorpe @ 2016-11-07 23:55 UTC (permalink / raw)
To: Hefty, Sean
Cc: Matan Barak, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Doug Ledford, Christoph Lameter, Liran Liss, Haggai Eran,
Majd Dibbiny, Tal Alon, Leon Romanovsky
In-Reply-To: <1828884A29C6694DAF28B7E6B8A82373AB0A445F-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
On Fri, Oct 28, 2016 at 10:53:13PM +0000, Hefty, Sean wrote:
> > The current code creates an IDR per type. Since types are currently
> > common for all vendors and known in advance, this was good enough.
> > However, the proposed ioctl based infrastructure allows each vendor
> > to declare only some of the common types and declare its own specific
> > types.
> >
> > Thus, we decided to implement IDR to be per device and refactor it to
> > use a new file.
>
> I think this needs to be more abstract. I would consider
> introducing the concept of an 'ioctl provider', with the idr per
> ioctl provider. You could then make each ib_device an ioctl
> provider. (Just embed the structure). I believe this will be
> necessary to support the rdma_cm, ib_cm, as well as devices that
> export different sets of ioctls, where an ib_device isn't
> necessarily available.
>
> Essentially, I would treat plugging into the uABI independent from
> plugging into the kernel verbs API. Otherwise, I think we'll end up
> with multiple ioctl 'frameworks'.
Matan,
I think you should change things so that all the *general* code uses
'urdma_' as a prefix instead of uverbs_. Only use uverbs_ on things
that truely only apply to uverbs. This will make things much
clearer how the code sharing is expected to work with rdma_cm
Sean is right, this shows why having the IDR be per device does not
work, rdma-cm really does need a per-file or global IDR - both
approaches should really be the same, and I think per-file has better
locking characteristics, so I'd recommend that.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] build: Fix build script to use correct cmake cmd
From: Jason Gunthorpe @ 2016-11-07 23:57 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Dennis Dalessandro,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20161025101200.GP25013-2ukJVAZIZ/Y@public.gmane.org>
On Tue, Oct 25, 2016 at 01:12:00PM +0300, Leon Romanovsky wrote:
> > stuff I have - eg should I make it pushable? It is easy to use, but
> > you need to have docker installed.
>
> I would be happy to get it and be more confident in my local tests.
You can test it out with this commit:
https://github.com/jgunthorpe/rdma-plumbing/commit/ef24b991c949ad4f50614bf6bf549e1cdf841358
It will need some tidying before it can merged, but let me know if it
is useful as-is.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [RFC ABI V5 07/10] IB/core: Support getting IOCTL header/SGEs from kernel space
From: Jason Gunthorpe @ 2016-11-08 0:43 UTC (permalink / raw)
To: Matan Barak
Cc: Leon Romanovsky, Christoph Hellwig, Matan Barak, linux-rdma,
Doug Ledford, Sean Hefty, Christoph Lameter, Liran Liss,
Haggai Eran, Majd Dibbiny, Tal Alon
In-Reply-To: <CAAKD3BB0k1UxV2qO3SqAD_t1vM2pcduOXiz8aJ5c+JXAmq_aWw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Sun, Oct 30, 2016 at 10:48:39AM +0200, Matan Barak wrote:
> On Fri, Oct 28, 2016 at 5:46 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> > On Fri, Oct 28, 2016 at 08:37:25AM -0700, Christoph Hellwig wrote:
> >> On Fri, Oct 28, 2016 at 06:33:06PM +0300, Leon Romanovsky wrote:
> >> > Just to summarize, to be sure that I understood you correctly.
> >> >
> >> > | write | -> | conversion logic | ---
> >> > | ioctl | ---------------------------
> >> >
> >> > Am I right?
> >>
> >> Yes, as long as the write and ioctl boxes do the copy_{from,to}_user.
> If we accept the limitations here (i.e - all commands attributes
> come either from kernel or from user, but you can't mix them -
> that's mean the write comparability layer either needs to copy all
> attributes or use a direct mapping for all of them), I could just
> either break ib_uverbs_cmd_verbs to a a few functions or just pass a
> callback of boxing the descriptors copy.
>From what I saw in the series, this looks easy enough to fix..
Just lightly refactor things so that the write() compat layer can call
into the ioctl processor with an already prepared tlv list in kernel
memory and form such a list on the stack when doing the compat stuff.
The bigger problem is the tlv list pointers themselves, they have to
point to user memory so the compat layer can only do so much of a
transformation.
I guess another flag in the copy_from_user wrapper would do the trick
if we need it.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCHv12 0/3] rdmacg: IB/core: rdma controller support
From: Liran Liss @ 2016-11-08 8:12 UTC (permalink / raw)
To: Parav Pandit
Cc: Leon Romanovsky, Tejun Heo,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma,
Li Zefan, Johannes Weiner, Doug Ledford, Christoph Hellwig,
Hefty, Sean, Jason Gunthorpe, Haggai Eran,
james.l.morris-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
Or Gerlitz, Matan Barak
In-Reply-To: <CAG53R5WdauHpML66g-O6zj+j_DUYWJMPjmL1xDaSxwDmPPYm2A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
> From: Parav Pandit [mailto:pandit.parav@gmail.com]
> >
> > Hmm..
> > I guess that you are right.
> >
> > So we can add another count for "HCA handles",
> I prefer this. This keeps it vendor agnostic and clean if we don't go percentage
> route.
OK; let's do it.
> Would indirection table also fall in this category?
>
No. It's just another HCA resource...
> > or alternatively, each provider will restrict the number of handles
> > per device to a reasonable small number (which won't be treated as one of the
> "HCA resources").
> This would require vendor drivers to get the understanding of cgroup object
> and pid and that breaks the modular approach. I like to avoid this.
>
> > Typically, a process shouldn't need to open more than a single handle...
> Right. well behaved application won't do multiple handles.
^ permalink raw reply
* Re: [PATCH rdma-rc 0/9] mlx4 fixes for 4.9
From: Leon Romanovsky @ 2016-11-08 8:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, Yuval Shaia
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1478375842-21513-1-git-send-email-leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 159 bytes --]
On Sat, Nov 05, 2016 at 09:57:13PM +0200, Leon Romanovsky wrote:
> Hi Doug,
Please drop this patch series, I'll respin it once I'll return to
office.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox