From: Leon Romanovsky <leon@kernel.org>
To: Devesh Sharma <devesh.sharma@broadcom.com>
Cc: linux-rdma@vger.kernel.org
Subject: Re: [PATCH V4 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports
Date: Wed, 2 Jun 2021 15:48:03 +0300 [thread overview]
Message-ID: <YLd+AzGd2NhnVas6@unreal> (raw)
In-Reply-To: <20210602042839.968833-2-devesh.sharma@broadcom.com>
On Wed, Jun 02, 2021 at 09:58:37AM +0530, Devesh Sharma wrote:
> Enabling Atomic operations for Gen P5 devices if the underlying
> platform supports global atomic ops.
>
> Fixes:7ff662b76167 ("Disable atomic capability on bnxt_re adapters")
^^^ space here
> Signed-off-by: Devesh Sharma <devesh.sharma@broadcom.com>
> ---
> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 4 ++++
> drivers/infiniband/hw/bnxt_re/main.c | 3 +++
> drivers/infiniband/hw/bnxt_re/qplib_res.c | 17 +++++++++++++++++
> drivers/infiniband/hw/bnxt_re/qplib_res.h | 1 +
> drivers/infiniband/hw/bnxt_re/qplib_sp.c | 13 ++++++++++++-
> drivers/infiniband/hw/bnxt_re/qplib_sp.h | 2 --
> 6 files changed, 37 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> index 537471ffaa79..a113d8d9e9ed 100644
> --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> @@ -163,6 +163,10 @@ int bnxt_re_query_device(struct ib_device *ibdev,
> ib_attr->max_qp_init_rd_atom = dev_attr->max_qp_init_rd_atom;
> ib_attr->atomic_cap = IB_ATOMIC_NONE;
> ib_attr->masked_atomic_cap = IB_ATOMIC_NONE;
> + if (dev_attr->is_atomic) {
> + ib_attr->atomic_cap = IB_ATOMIC_GLOB;
> + ib_attr->masked_atomic_cap = IB_ATOMIC_GLOB;
> + }
>
> ib_attr->max_ee_rd_atom = 0;
> ib_attr->max_res_rd_atom = 0;
> diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
> index b090dfa4f4cb..0de4e22f9750 100644
> --- a/drivers/infiniband/hw/bnxt_re/main.c
> +++ b/drivers/infiniband/hw/bnxt_re/main.c
> @@ -128,6 +128,9 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode)
> rdev->rcfw.res = &rdev->qplib_res;
>
> bnxt_re_set_drv_mode(rdev, wqe_mode);
> + if (bnxt_qplib_determine_atomics(en_dev->pdev))
> + ibdev_info(&rdev->ibdev,
> + "platform doesn't support global atomics.");
> return 0;
> }
>
> diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.c b/drivers/infiniband/hw/bnxt_re/qplib_res.c
> index 3ca47004b752..d722ca5cd464 100644
> --- a/drivers/infiniband/hw/bnxt_re/qplib_res.c
> +++ b/drivers/infiniband/hw/bnxt_re/qplib_res.c
> @@ -959,3 +959,20 @@ int bnxt_qplib_alloc_res(struct bnxt_qplib_res *res, struct pci_dev *pdev,
> bnxt_qplib_free_res(res);
> return rc;
> }
> +
> +bool bnxt_qplib_determine_atomics(struct pci_dev *dev)
> +{
> + int comp;
> + u16 ctl2;
> +
> + comp = pci_enable_atomic_ops_to_root(dev,
> + PCI_EXP_DEVCAP2_ATOMIC_COMP32);
> + if (comp)
> + return -ENOTSUPP;
> + comp = pci_enable_atomic_ops_to_root(dev,
> + PCI_EXP_DEVCAP2_ATOMIC_COMP64);
> + if (comp)
> + return -ENOTSUPP;
> + pcie_capability_read_word(dev, PCI_EXP_DEVCTL2, &ctl2);
> + return !(ctl2 & PCI_EXP_DEVCTL2_ATOMIC_REQ);
> +}
> diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h
> index 7a1ab38b95da..fb7fde4fed56 100644
> --- a/drivers/infiniband/hw/bnxt_re/qplib_res.h
> +++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h
> @@ -373,6 +373,7 @@ void bnxt_qplib_free_ctx(struct bnxt_qplib_res *res,
> int bnxt_qplib_alloc_ctx(struct bnxt_qplib_res *res,
> struct bnxt_qplib_ctx *ctx,
> bool virt_fn, bool is_p5);
> +bool bnxt_qplib_determine_atomics(struct pci_dev *dev);
>
> static inline void bnxt_qplib_hwq_incr_prod(struct bnxt_qplib_hwq *hwq, u32 cnt)
> {
> diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.c b/drivers/infiniband/hw/bnxt_re/qplib_sp.c
> index 049b3576302b..57407be16f27 100644
> --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.c
> +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.c
> @@ -54,6 +54,17 @@ const struct bnxt_qplib_gid bnxt_qplib_gid_zero = {{ 0, 0, 0, 0, 0, 0, 0, 0,
>
> /* Device */
>
> +static u8 bnxt_qplib_is_atomic_cap(struct bnxt_qplib_rcfw *rcfw)
> +{
> + u16 pcie_ctl2 = 0;
> +
> + if (!bnxt_qplib_is_chip_gen_p5(rcfw->res->cctx))
> + return false;
> +
> + pcie_capability_read_word(rcfw->pdev, PCI_EXP_DEVCTL2, &pcie_ctl2);
> + return (pcie_ctl2 & PCI_EXP_DEVCTL2_ATOMIC_REQ);
> +}
> +
> static void bnxt_qplib_query_version(struct bnxt_qplib_rcfw *rcfw,
> char *fw_ver)
> {
> @@ -162,7 +173,7 @@ int bnxt_qplib_get_dev_attr(struct bnxt_qplib_rcfw *rcfw,
> attr->tqm_alloc_reqs[i * 4 + 3] = *(++tqm_alloc);
> }
>
> - attr->is_atomic = false;
> + attr->is_atomic = bnxt_qplib_is_atomic_cap(rcfw);
> bail:
> bnxt_qplib_rcfw_free_sbuf(rcfw, sbuf);
> return rc;
> diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.h b/drivers/infiniband/hw/bnxt_re/qplib_sp.h
> index bc228340684f..260104783691 100644
> --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.h
> +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.h
> @@ -42,8 +42,6 @@
>
> #define BNXT_QPLIB_RESERVED_QP_WRS 128
>
> -#define PCI_EXP_DEVCTL2_ATOMIC_REQ 0x0040
> -
> struct bnxt_qplib_dev_attr {
> #define FW_VER_ARR_LEN 4
> u8 fw_ver[FW_VER_ARR_LEN];
> --
> 2.25.1
>
next prev parent reply other threads:[~2021-06-02 12:48 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-02 4:28 [PATCH V4 for-next 0/3] Broadcom's driver add global atomics Devesh Sharma
2021-06-02 4:28 ` [PATCH V4 for-next 1/3] RDMA/bnxt_re: Enable global atomic ops if platform supports Devesh Sharma
2021-06-02 12:48 ` Leon Romanovsky [this message]
2021-06-02 14:38 ` Devesh Sharma
2021-06-02 4:28 ` [PATCH V4 for-next 2/3] bnxt_re: Update maintainers list Devesh Sharma
2021-06-02 4:28 ` [PATCH V4 for-next 3/3] RDMA/bnxt_re: update ABI to pass wqe-mode to user space Devesh Sharma
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YLd+AzGd2NhnVas6@unreal \
--to=leon@kernel.org \
--cc=devesh.sharma@broadcom.com \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.