From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF9B9C433E6 for ; Wed, 10 Feb 2021 08:59:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0FDA564E2F for ; Wed, 10 Feb 2021 08:59:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FDA564E2F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50724 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9lLA-0002z2-To for qemu-devel@archiver.kernel.org; Wed, 10 Feb 2021 03:59:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l9lIi-0000xj-NN for qemu-devel@nongnu.org; Wed, 10 Feb 2021 03:56:46 -0500 Received: from mail-lf1-x12b.google.com ([2a00:1450:4864:20::12b]:42125) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l9lIY-0005D7-BP for qemu-devel@nongnu.org; Wed, 10 Feb 2021 03:56:44 -0500 Received: by mail-lf1-x12b.google.com with SMTP id f23so1647177lfk.9 for ; Wed, 10 Feb 2021 00:56:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0fpCbnRPEUtF++gy0X2pxm62cpd/qd1rpgmnQvaxZZQ=; b=Rml7Roq4OyzuPJxmFHITYEhSuUazvpSp5ZH9G02Ql1nCbFCWM2MK863tNXpiSVUdiu PalgXplpQY8Gq+7wRFaF0Pax7WD4/+2iW1Cu/BpUAnf/yXfO3PcB9Nz97kJNTrHN83ny R+9via2VGRwPMvqKNNT6YFM4FWuavm2S4oG1esNt/LoJJSfDe9KSL/6njv05tc8UG7Lo pSicTcGN4Mn8Rkt6pvgtlt2uYSODWpb/f+IRcnDS57g7sP3iRrOfD6pwgPmmRHsGvBuu O15gyDe9SbEKQtb+uThSiPizZSLQMxYwprjz0BVlstTLWffbetrKelYqVpi997XPfnd1 uECQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0fpCbnRPEUtF++gy0X2pxm62cpd/qd1rpgmnQvaxZZQ=; b=kLWePd2TWExqW843GCXBYbplA1Y4kW176qxgpKTU8kygpX43JmEyTjaWVFqQ03xyhL adfhvebJs9OaS8qHxYOWyZy+5+EM7zjebxHp3yw5RmKoz/FiqFzFwEIICllvFU9rKTX+ 0rc640efzoI6UAQG5c5kJBeI6IyxdrXseAnVX3pgeCLMFXPD4DTEbmkBDVz1s5Amh8j2 WNOO90WqjHa+6ZjAR4cvR/FydUfvsqIvrKaeijTOfnZoY1GCedJgaqG7K+yF4udiGwue O9xiFeM51uc18kRmOA5G0EMsE6aOrHavoGpFk418JHVySQbEki5rXbkexM18hUF1E2xq aWkQ== X-Gm-Message-State: AOAM5315b27cBSCNHiC8TJsDvCFZFdCcXV9Ok8DOXW5mmQVMZjnBRHQ/ /F2RbffPM+EtKdHf3ZUek8M0UVCaeBs3Ic0hK3k= X-Google-Smtp-Source: ABdhPJzaAu9bEN3/GjC32ugqdEuwtDh5gnxFB1my/pvJiQj6c3Dce5b+aixS00QmYdSzvRrDWJCdMlWSFpjkHkJ/CH4= X-Received: by 2002:a19:7d1:: with SMTP id 200mr1215671lfh.110.1612947389473; Wed, 10 Feb 2021 00:56:29 -0800 (PST) MIME-Version: 1.0 References: <20210122180029.575284-1-cohuck@redhat.com> In-Reply-To: <20210122180029.575284-1-cohuck@redhat.com> From: Yuval Shaia Date: Wed, 10 Feb 2021 10:56:18 +0200 Message-ID: Subject: Re: [PATCH RFC] pvrdma: wean code off pvrdma_ring.h kernel header To: Cornelia Huck Content-Type: multipart/alternative; boundary="00000000000030e71405baf7944f" Received-SPF: pass client-ip=2a00:1450:4864:20::12b; envelope-from=yuval.shaia.ml@gmail.com; helo=mail-lf1-x12b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , QEMU Developers , "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --00000000000030e71405baf7944f Content-Type: text/plain; charset="UTF-8" On Fri, 22 Jan 2021 at 20:00, Cornelia Huck wrote: > The pvrdma code relies on the pvrdma_ring.h kernel header for some > basic ring buffer handling. The content of that header isn't very > exciting, but contains some (q)atomic_*() invocations that (a) > cause manual massaging when doing a headers update, and (b) are > an indication that we probably should not be importing that header > at all. > > Let's reimplement the ring buffer handling directly in the pvrdma > code instead. This arguably also improves readability of the code. > > Importing the header can now be dropped. > > Signed-off-by: Cornelia Huck > --- > > Compile-tested only, needs both testing and more eyeballs :) > > --- > hw/rdma/vmw/pvrdma.h | 5 +- > hw/rdma/vmw/pvrdma_cmd.c | 6 +- > hw/rdma/vmw/pvrdma_dev_ring.c | 41 ++++--- > hw/rdma/vmw/pvrdma_dev_ring.h | 9 +- > hw/rdma/vmw/pvrdma_main.c | 4 +- > .../infiniband/hw/vmw_pvrdma/pvrdma_ring.h | 114 ------------------ > scripts/update-linux-headers.sh | 3 +- > 7 files changed, 38 insertions(+), 144 deletions(-) > delete mode 100644 > include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h > > diff --git a/hw/rdma/vmw/pvrdma.h b/hw/rdma/vmw/pvrdma.h > index 1d36a76f1e3b..d08965d3e2d5 100644 > --- a/hw/rdma/vmw/pvrdma.h > +++ b/hw/rdma/vmw/pvrdma.h > @@ -26,7 +26,6 @@ > #include "../rdma_backend_defs.h" > #include "../rdma_rm_defs.h" > > -#include "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h" > #include > "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h" > #include "pvrdma_dev_ring.h" > #include "qom/object.h" > @@ -64,10 +63,10 @@ typedef struct DSRInfo { > union pvrdma_cmd_req *req; > union pvrdma_cmd_resp *rsp; > > - struct pvrdma_ring *async_ring_state; > + PvrdmaRingState *async_ring_state; > PvrdmaRing async; > > - struct pvrdma_ring *cq_ring_state; > + PvrdmaRingState *cq_ring_state; > PvrdmaRing cq; > } DSRInfo; > > diff --git a/hw/rdma/vmw/pvrdma_cmd.c b/hw/rdma/vmw/pvrdma_cmd.c > index 692125ac2681..f59879e2574e 100644 > --- a/hw/rdma/vmw/pvrdma_cmd.c > +++ b/hw/rdma/vmw/pvrdma_cmd.c > @@ -262,7 +262,7 @@ static int create_cq_ring(PCIDevice *pci_dev , > PvrdmaRing **ring, > r = g_malloc(sizeof(*r)); > *ring = r; > > - r->ring_state = (struct pvrdma_ring *) > + r->ring_state = (PvrdmaRingState *) > rdma_pci_dma_map(pci_dev, tbl[0], TARGET_PAGE_SIZE); > > if (!r->ring_state) { > @@ -398,7 +398,7 @@ static int create_qp_rings(PCIDevice *pci_dev, > uint64_t pdir_dma, > *rings = sr; > > /* Create send ring */ > - sr->ring_state = (struct pvrdma_ring *) > + sr->ring_state = (PvrdmaRingState *) > rdma_pci_dma_map(pci_dev, tbl[0], TARGET_PAGE_SIZE); > if (!sr->ring_state) { > rdma_error_report("Failed to map to QP ring state"); > @@ -639,7 +639,7 @@ static int create_srq_ring(PCIDevice *pci_dev, > PvrdmaRing **ring, > r = g_malloc(sizeof(*r)); > *ring = r; > > - r->ring_state = (struct pvrdma_ring *) > + r->ring_state = (PvrdmaRingState *) > rdma_pci_dma_map(pci_dev, tbl[0], TARGET_PAGE_SIZE); > if (!r->ring_state) { > rdma_error_report("Failed to map tp SRQ ring state"); > diff --git a/hw/rdma/vmw/pvrdma_dev_ring.c b/hw/rdma/vmw/pvrdma_dev_ring.c > index f0bcde74b06a..074ac59b84db 100644 > --- a/hw/rdma/vmw/pvrdma_dev_ring.c > +++ b/hw/rdma/vmw/pvrdma_dev_ring.c > @@ -22,11 +22,10 @@ > #include "trace.h" > > #include "../rdma_utils.h" > -#include "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h" > #include "pvrdma_dev_ring.h" > > int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *dev, > - struct pvrdma_ring *ring_state, uint32_t max_elems, > + PvrdmaRingState *ring_state, uint32_t max_elems, > size_t elem_sz, dma_addr_t *tbl, uint32_t npages) > { > int i; > @@ -73,48 +72,54 @@ out: > > void *pvrdma_ring_next_elem_read(PvrdmaRing *ring) > { > - int e; > - unsigned int idx = 0, offset; > + unsigned int idx, offset; > + const uint32_t tail = qatomic_read(&ring->ring_state->prod_tail); > + const uint32_t head = qatomic_read(&ring->ring_state->cons_head); > > - e = pvrdma_idx_ring_has_data(ring->ring_state, ring->max_elems, &idx); > - if (e <= 0) { > + if (tail & ~((ring->max_elems << 1) - 1) || > + head & ~((ring->max_elems << 1) - 1) || > + tail == head) { > trace_pvrdma_ring_next_elem_read_no_data(ring->name); > return NULL; > } > > + idx = head & (ring->max_elems - 1); > offset = idx * ring->elem_sz; > return ring->pages[offset / TARGET_PAGE_SIZE] + (offset % > TARGET_PAGE_SIZE); > } > > void pvrdma_ring_read_inc(PvrdmaRing *ring) > { > - pvrdma_idx_ring_inc(&ring->ring_state->cons_head, ring->max_elems); > + uint32_t idx = qatomic_read(&ring->ring_state->cons_head); > + > + idx = (idx + 1) & ((ring->max_elems << 1) - 1); > + qatomic_set(&ring->ring_state->cons_head, idx); > } > > void *pvrdma_ring_next_elem_write(PvrdmaRing *ring) > { > - int idx; > - unsigned int offset, tail; > + unsigned int idx, offset; > + const uint32_t tail = qatomic_read(&ring->ring_state->prod_tail); > + const uint32_t head = qatomic_read(&ring->ring_state->cons_head); > > - idx = pvrdma_idx_ring_has_space(ring->ring_state, ring->max_elems, > &tail); > - if (idx <= 0) { > + if (tail & ~((ring->max_elems << 1) - 1) || > + head & ~((ring->max_elems << 1) - 1) || > + tail == (head ^ ring->max_elems)) { > rdma_error_report("CQ is full"); > return NULL; > } > > - idx = pvrdma_idx(&ring->ring_state->prod_tail, ring->max_elems); > - if (idx < 0 || tail != idx) { > - rdma_error_report("Invalid idx %d", idx); > - return NULL; > - } > - > + idx = tail & (ring->max_elems - 1); > offset = idx * ring->elem_sz; > return ring->pages[offset / TARGET_PAGE_SIZE] + (offset % > TARGET_PAGE_SIZE); > } > > void pvrdma_ring_write_inc(PvrdmaRing *ring) > { > - pvrdma_idx_ring_inc(&ring->ring_state->prod_tail, ring->max_elems); > + uint32_t idx = qatomic_read(&ring->ring_state->prod_tail); > + > + idx = (idx + 1) & ((ring->max_elems << 1) - 1); > + qatomic_set(&ring->ring_state->prod_tail, idx); > } > > void pvrdma_ring_free(PvrdmaRing *ring) > diff --git a/hw/rdma/vmw/pvrdma_dev_ring.h b/hw/rdma/vmw/pvrdma_dev_ring.h > index 5f2a0cf9b9fa..d231588ce004 100644 > --- a/hw/rdma/vmw/pvrdma_dev_ring.h > +++ b/hw/rdma/vmw/pvrdma_dev_ring.h > @@ -19,18 +19,23 @@ > > #define MAX_RING_NAME_SZ 32 > > +typedef struct PvrdmaRingState { > + int prod_tail; /* producer tail */ > + int cons_head; /* consumer head */ > +} PvrdmaRingState; > + > typedef struct PvrdmaRing { > char name[MAX_RING_NAME_SZ]; > PCIDevice *dev; > uint32_t max_elems; > size_t elem_sz; > - struct pvrdma_ring *ring_state; /* used only for unmap */ > + PvrdmaRingState *ring_state; /* used only for unmap */ > int npages; > void **pages; > } PvrdmaRing; > > int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *dev, > - struct pvrdma_ring *ring_state, uint32_t max_elems, > + PvrdmaRingState *ring_state, uint32_t max_elems, > size_t elem_sz, dma_addr_t *tbl, uint32_t npages); > void *pvrdma_ring_next_elem_read(PvrdmaRing *ring); > void pvrdma_ring_read_inc(PvrdmaRing *ring); > diff --git a/hw/rdma/vmw/pvrdma_main.c b/hw/rdma/vmw/pvrdma_main.c > index 85935703322f..84ae8024fcfd 100644 > --- a/hw/rdma/vmw/pvrdma_main.c > +++ b/hw/rdma/vmw/pvrdma_main.c > @@ -85,7 +85,7 @@ static void free_dev_ring(PCIDevice *pci_dev, PvrdmaRing > *ring, > rdma_pci_dma_unmap(pci_dev, ring_state, TARGET_PAGE_SIZE); > } > > -static int init_dev_ring(PvrdmaRing *ring, struct pvrdma_ring > **ring_state, > +static int init_dev_ring(PvrdmaRing *ring, PvrdmaRingState **ring_state, > const char *name, PCIDevice *pci_dev, > dma_addr_t dir_addr, uint32_t num_pages) > { > @@ -114,7 +114,7 @@ static int init_dev_ring(PvrdmaRing *ring, struct > pvrdma_ring **ring_state, > /* RX ring is the second */ > (*ring_state)++; > rc = pvrdma_ring_init(ring, name, pci_dev, > - (struct pvrdma_ring *)*ring_state, > + (PvrdmaRingState *)*ring_state, > (num_pages - 1) * TARGET_PAGE_SIZE / > sizeof(struct pvrdma_cqne), > sizeof(struct pvrdma_cqne), > diff --git > a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h > b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h > deleted file mode 100644 > index 7b4062a1a107..000000000000 > --- > a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h > +++ /dev/null > @@ -1,114 +0,0 @@ > -/* > - * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. > - * > - * This program is free software; you can redistribute it and/or > - * modify it under the terms of EITHER the GNU General Public License > - * version 2 as published by the Free Software Foundation or the BSD > - * 2-Clause License. This program is distributed in the hope that it > - * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED > - * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. > - * See the GNU General Public License version 2 for more details at > - * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. > - * > - * You should have received a copy of the GNU General Public License > - * along with this program available in the file COPYING in the main > - * directory of this source tree. > - * > - * The BSD 2-Clause License > - * > - * Redistribution and use in source and binary forms, with or > - * without modification, are permitted provided that the following > - * conditions are met: > - * > - * - Redistributions of source code must retain the above > - * copyright notice, this list of conditions and the following > - * disclaimer. > - * > - * - Redistributions in binary form must reproduce the above > - * copyright notice, this list of conditions and the following > - * disclaimer in the documentation and/or other materials > - * provided with the distribution. > - * > - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, > - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) > - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > - * OF THE POSSIBILITY OF SUCH DAMAGE. > - */ > - > -#ifndef __PVRDMA_RING_H__ > -#define __PVRDMA_RING_H__ > - > -#include "standard-headers/linux/types.h" > - > -#define PVRDMA_INVALID_IDX -1 /* Invalid index. */ > - > -struct pvrdma_ring { > - int prod_tail; /* Producer tail. */ > - int cons_head; /* Consumer head. */ > -}; > - > -struct pvrdma_ring_state { > - struct pvrdma_ring tx; /* Tx ring. */ > - struct pvrdma_ring rx; /* Rx ring. */ > -}; > - > -static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems) > -{ > - /* Generates fewer instructions than a less-than. */ > - return (idx & ~((max_elems << 1) - 1)) == 0; > -} > - > -static inline int32_t pvrdma_idx(int *var, uint32_t max_elems) > -{ > - const unsigned int idx = qatomic_read(var); > - > - if (pvrdma_idx_valid(idx, max_elems)) > - return idx & (max_elems - 1); > - return PVRDMA_INVALID_IDX; > -} > - > -static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems) > -{ > - uint32_t idx = qatomic_read(var) + 1; /* Increment. */ > - > - idx &= (max_elems << 1) - 1; /* Modulo size, flip gen. > */ > - qatomic_set(var, idx); > -} > - > -static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring > *r, > - uint32_t max_elems, uint32_t > *out_tail) > -{ > - const uint32_t tail = qatomic_read(&r->prod_tail); > - const uint32_t head = qatomic_read(&r->cons_head); > - > - if (pvrdma_idx_valid(tail, max_elems) && > - pvrdma_idx_valid(head, max_elems)) { > - *out_tail = tail & (max_elems - 1); > - return tail != (head ^ max_elems); > - } > - return PVRDMA_INVALID_IDX; > -} > - > -static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring > *r, > - uint32_t max_elems, uint32_t > *out_head) > -{ > - const uint32_t tail = qatomic_read(&r->prod_tail); > - const uint32_t head = qatomic_read(&r->cons_head); > - > - if (pvrdma_idx_valid(tail, max_elems) && > - pvrdma_idx_valid(head, max_elems)) { > - *out_head = head & (max_elems - 1); > - return tail != head; > - } > - return PVRDMA_INVALID_IDX; > -} > - > -#endif /* __PVRDMA_RING_H__ */ > diff --git a/scripts/update-linux-headers.sh > b/scripts/update-linux-headers.sh > index fa6f2b6272b7..1050e361694f 100755 > --- a/scripts/update-linux-headers.sh > +++ b/scripts/update-linux-headers.sh > @@ -215,8 +215,7 @@ sed -e '1h;2,$H;$!d;g' -e 's/[^};]*pvrdma[^(| > ]*([^)]*);//g' \ > "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h" > \ > "$tmp_pvrdma_verbs"; > > -for i in "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h" \ > - "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h" \ > +for i in "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h" \ > "$tmp_pvrdma_verbs"; do \ > cp_portable "$i" \ > > "$output/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/" > -- > 2.26.2 > > Thanks! I guess somewhere in the kernel there is such a clean and generic implementation of a ring and VM folks could utilize that instead of writing their own. Tested-by: Yuval Shaia Reviewed-by: Yuval Shaia --00000000000030e71405baf7944f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, 22 Jan 2021 at 20:00, Corneli= a Huck <cohuck@redhat.com> w= rote:
The pvrdma= code relies on the pvrdma_ring.h kernel header for some
basic ring buffer handling. The content of that header isn't very
exciting, but contains some (q)atomic_*() invocations that (a)
cause manual massaging when doing a headers update, and (b) are
an indication that we probably should not be importing that header
at all.

Let's reimplement the ring buffer handling directly in the pvrdma
code instead. This arguably also improves readability of the code.

Importing the header can now be dropped.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
---

Compile-tested only, needs both testing and more eyeballs :)

---
=C2=A0hw/rdma/vmw/pvrdma.h=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A05 +-
=C2=A0hw/rdma/vmw/pvrdma_cmd.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A06 +-
=C2=A0hw/rdma/vmw/pvrdma_dev_ring.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 41 ++++---
=C2=A0hw/rdma/vmw/pvrdma_dev_ring.h=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A09 +-
=C2=A0hw/rdma/vmw/pvrdma_main.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A04 +-
=C2=A0.../infiniband/hw/vmw_pvrdma/pvrdma_ring.h=C2=A0 =C2=A0 | 114 -------= -----------
=C2=A0scripts/update-linux-headers.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|=C2=A0 =C2=A03 +-
=C2=A07 files changed, 38 insertions(+), 144 deletions(-)
=C2=A0delete mode 100644 include/standard-headers/drivers/infiniband/hw/vmw= _pvrdma/pvrdma_ring.h

diff --git a/hw/rdma/vmw/pvrdma.h b/hw/rdma/vmw/pvrdma.h
index 1d36a76f1e3b..d08965d3e2d5 100644
--- a/hw/rdma/vmw/pvrdma.h
+++ b/hw/rdma/vmw/pvrdma.h
@@ -26,7 +26,6 @@
=C2=A0#include "../rdma_backend_defs.h"
=C2=A0#include "../rdma_rm_defs.h"

-#include "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ri= ng.h"
=C2=A0#include "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrd= ma_dev_api.h"
=C2=A0#include "pvrdma_dev_ring.h"
=C2=A0#include "qom/object.h"
@@ -64,10 +63,10 @@ typedef struct DSRInfo {
=C2=A0 =C2=A0 =C2=A0union pvrdma_cmd_req *req;
=C2=A0 =C2=A0 =C2=A0union pvrdma_cmd_resp *rsp;

-=C2=A0 =C2=A0 struct pvrdma_ring *async_ring_state;
+=C2=A0 =C2=A0 PvrdmaRingState *async_ring_state;
=C2=A0 =C2=A0 =C2=A0PvrdmaRing async;

-=C2=A0 =C2=A0 struct pvrdma_ring *cq_ring_state;
+=C2=A0 =C2=A0 PvrdmaRingState *cq_ring_state;
=C2=A0 =C2=A0 =C2=A0PvrdmaRing cq;
=C2=A0} DSRInfo;

diff --git a/hw/rdma/vmw/pvrdma_cmd.c b/hw/rdma/vmw/pvrdma_cmd.c
index 692125ac2681..f59879e2574e 100644
--- a/hw/rdma/vmw/pvrdma_cmd.c
+++ b/hw/rdma/vmw/pvrdma_cmd.c
@@ -262,7 +262,7 @@ static int create_cq_ring(PCIDevice *pci_dev , PvrdmaRi= ng **ring,
=C2=A0 =C2=A0 =C2=A0r =3D g_malloc(sizeof(*r));
=C2=A0 =C2=A0 =C2=A0*ring =3D r;

-=C2=A0 =C2=A0 r->ring_state =3D (struct pvrdma_ring *)
+=C2=A0 =C2=A0 r->ring_state =3D (PvrdmaRingState *)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_pci_dma_map(pci_dev, tbl[0], TARGET_= PAGE_SIZE);

=C2=A0 =C2=A0 =C2=A0if (!r->ring_state) {
@@ -398,7 +398,7 @@ static int create_qp_rings(PCIDevice *pci_dev, uint64_t= pdir_dma,
=C2=A0 =C2=A0 =C2=A0*rings =3D sr;

=C2=A0 =C2=A0 =C2=A0/* Create send ring */
-=C2=A0 =C2=A0 sr->ring_state =3D (struct pvrdma_ring *)
+=C2=A0 =C2=A0 sr->ring_state =3D (PvrdmaRingState *)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_pci_dma_map(pci_dev, tbl[0], TARGET_= PAGE_SIZE);
=C2=A0 =C2=A0 =C2=A0if (!sr->ring_state) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_error_report("Failed to map to = QP ring state");
@@ -639,7 +639,7 @@ static int create_srq_ring(PCIDevice *pci_dev, PvrdmaRi= ng **ring,
=C2=A0 =C2=A0 =C2=A0r =3D g_malloc(sizeof(*r));
=C2=A0 =C2=A0 =C2=A0*ring =3D r;

-=C2=A0 =C2=A0 r->ring_state =3D (struct pvrdma_ring *)
+=C2=A0 =C2=A0 r->ring_state =3D (PvrdmaRingState *)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_pci_dma_map(pci_dev, t= bl[0], TARGET_PAGE_SIZE);
=C2=A0 =C2=A0 =C2=A0if (!r->ring_state) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_error_report("Failed to map tp = SRQ ring state");
diff --git a/hw/rdma/vmw/pvrdma_dev_ring.c b/hw/rdma/vmw/pvrdma_dev_ring.c<= br> index f0bcde74b06a..074ac59b84db 100644
--- a/hw/rdma/vmw/pvrdma_dev_ring.c
+++ b/hw/rdma/vmw/pvrdma_dev_ring.c
@@ -22,11 +22,10 @@
=C2=A0#include "trace.h"

=C2=A0#include "../rdma_utils.h"
-#include "standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ri= ng.h"
=C2=A0#include "pvrdma_dev_ring.h"

=C2=A0int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *d= ev,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0struct pvrdma_ring *ring_state, uint32_t max_elems,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0PvrdmaRingState *ring_state, uint32_t max_elems,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 size_t elem_sz, dma_addr_t *tbl, uint32_t npages)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0int i;
@@ -73,48 +72,54 @@ out:

=C2=A0void *pvrdma_ring_next_elem_read(PvrdmaRing *ring)
=C2=A0{
-=C2=A0 =C2=A0 int e;
-=C2=A0 =C2=A0 unsigned int idx =3D 0, offset;
+=C2=A0 =C2=A0 unsigned int idx, offset;
+=C2=A0 =C2=A0 const uint32_t tail =3D qatomic_read(&ring->ring_stat= e->prod_tail);
+=C2=A0 =C2=A0 const uint32_t head =3D qatomic_read(&ring->ring_stat= e->cons_head);

-=C2=A0 =C2=A0 e =3D pvrdma_idx_ring_has_data(ring->ring_state, ring->= ;max_elems, &idx);
-=C2=A0 =C2=A0 if (e <=3D 0) {
+=C2=A0 =C2=A0 if (tail & ~((ring->max_elems << 1) - 1) ||
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 head & ~((ring->max_elems << 1) -= 1) ||
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 tail =3D=3D head) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0trace_pvrdma_ring_next_elem_read_no_data(= ring->name);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NULL;
=C2=A0 =C2=A0 =C2=A0}

+=C2=A0 =C2=A0 idx =3D head & (ring->max_elems - 1);
=C2=A0 =C2=A0 =C2=A0offset =3D idx * ring->elem_sz;
=C2=A0 =C2=A0 =C2=A0return ring->pages[offset / TARGET_PAGE_SIZE] + (off= set % TARGET_PAGE_SIZE);
=C2=A0}

=C2=A0void pvrdma_ring_read_inc(PvrdmaRing *ring)
=C2=A0{
-=C2=A0 =C2=A0 pvrdma_idx_ring_inc(&ring->ring_state->cons_head, = ring->max_elems);
+=C2=A0 =C2=A0 uint32_t idx =3D qatomic_read(&ring->ring_state->c= ons_head);
+
+=C2=A0 =C2=A0 idx =3D (idx + 1) & ((ring->max_elems << 1) - 1= );
+=C2=A0 =C2=A0 qatomic_set(&ring->ring_state->cons_head, idx); =C2=A0}

=C2=A0void *pvrdma_ring_next_elem_write(PvrdmaRing *ring)
=C2=A0{
-=C2=A0 =C2=A0 int idx;
-=C2=A0 =C2=A0 unsigned int offset, tail;
+=C2=A0 =C2=A0 unsigned int idx, offset;
+=C2=A0 =C2=A0 const uint32_t tail =3D qatomic_read(&ring->ring_stat= e->prod_tail);
+=C2=A0 =C2=A0 const uint32_t head =3D qatomic_read(&ring->ring_stat= e->cons_head);

-=C2=A0 =C2=A0 idx =3D pvrdma_idx_ring_has_space(ring->ring_state, ring-= >max_elems, &tail);
-=C2=A0 =C2=A0 if (idx <=3D 0) {
+=C2=A0 =C2=A0 if (tail & ~((ring->max_elems << 1) - 1) ||
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 head & ~((ring->max_elems << 1) -= 1) ||
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 tail =3D=3D (head ^ ring->max_elems)) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rdma_error_report("CQ is full")= ;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NULL;
=C2=A0 =C2=A0 =C2=A0}

-=C2=A0 =C2=A0 idx =3D pvrdma_idx(&ring->ring_state->prod_tail, r= ing->max_elems);
-=C2=A0 =C2=A0 if (idx < 0 || tail !=3D idx) {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 rdma_error_report("Invalid idx %d", = idx);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 return NULL;
-=C2=A0 =C2=A0 }
-
+=C2=A0 =C2=A0 idx =3D tail & (ring->max_elems - 1);
=C2=A0 =C2=A0 =C2=A0offset =3D idx * ring->elem_sz;
=C2=A0 =C2=A0 =C2=A0return ring->pages[offset / TARGET_PAGE_SIZE] + (off= set % TARGET_PAGE_SIZE);
=C2=A0}

=C2=A0void pvrdma_ring_write_inc(PvrdmaRing *ring)
=C2=A0{
-=C2=A0 =C2=A0 pvrdma_idx_ring_inc(&ring->ring_state->prod_tail, = ring->max_elems);
+=C2=A0 =C2=A0 uint32_t idx =3D qatomic_read(&ring->ring_state->p= rod_tail);
+
+=C2=A0 =C2=A0 idx =3D (idx + 1) & ((ring->max_elems << 1) - 1= );
+=C2=A0 =C2=A0 qatomic_set(&ring->ring_state->prod_tail, idx); =C2=A0}

=C2=A0void pvrdma_ring_free(PvrdmaRing *ring)
diff --git a/hw/rdma/vmw/pvrdma_dev_ring.h b/hw/rdma/vmw/pvrdma_dev_ring.h<= br> index 5f2a0cf9b9fa..d231588ce004 100644
--- a/hw/rdma/vmw/pvrdma_dev_ring.h
+++ b/hw/rdma/vmw/pvrdma_dev_ring.h
@@ -19,18 +19,23 @@

=C2=A0#define MAX_RING_NAME_SZ 32

+typedef struct PvrdmaRingState {
+=C2=A0 =C2=A0 int prod_tail; /* producer tail */
+=C2=A0 =C2=A0 int cons_head; /* consumer head */
+} PvrdmaRingState;
+
=C2=A0typedef struct PvrdmaRing {
=C2=A0 =C2=A0 =C2=A0char name[MAX_RING_NAME_SZ];
=C2=A0 =C2=A0 =C2=A0PCIDevice *dev;
=C2=A0 =C2=A0 =C2=A0uint32_t max_elems;
=C2=A0 =C2=A0 =C2=A0size_t elem_sz;
-=C2=A0 =C2=A0 struct pvrdma_ring *ring_state; /* used only for unmap */ +=C2=A0 =C2=A0 PvrdmaRingState *ring_state; /* used only for unmap */
=C2=A0 =C2=A0 =C2=A0int npages;
=C2=A0 =C2=A0 =C2=A0void **pages;
=C2=A0} PvrdmaRing;

=C2=A0int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *d= ev,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0struct pvrdma_ring *ring_state, uint32_t max_elems,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0PvrdmaRingState *ring_state, uint32_t max_elems,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 size_t elem_sz, dma_addr_t *tbl, uint32_t npages);
=C2=A0void *pvrdma_ring_next_elem_read(PvrdmaRing *ring);
=C2=A0void pvrdma_ring_read_inc(PvrdmaRing *ring);
diff --git a/hw/rdma/vmw/pvrdma_main.c b/hw/rdma/vmw/pvrdma_main.c
index 85935703322f..84ae8024fcfd 100644
--- a/hw/rdma/vmw/pvrdma_main.c
+++ b/hw/rdma/vmw/pvrdma_main.c
@@ -85,7 +85,7 @@ static void free_dev_ring(PCIDevice *pci_dev, PvrdmaRing = *ring,
=C2=A0 =C2=A0 =C2=A0rdma_pci_dma_unmap(pci_dev, ring_state, TARGET_PAGE_SIZ= E);
=C2=A0}

-static int init_dev_ring(PvrdmaRing *ring, struct pvrdma_ring **ring_state= ,
+static int init_dev_ring(PvrdmaRing *ring, PvrdmaRingState **ring_state, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 const char *name, PCIDevice *pci_dev,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 dma_addr_t dir_addr, uint32_t num_pages)
=C2=A0{
@@ -114,7 +114,7 @@ static int init_dev_ring(PvrdmaRing *ring, struct pvrdm= a_ring **ring_state,
=C2=A0 =C2=A0 =C2=A0/* RX ring is the second */
=C2=A0 =C2=A0 =C2=A0(*ring_state)++;
=C2=A0 =C2=A0 =C2=A0rc =3D pvrdma_ring_init(ring, name, pci_dev,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 (struct pvrdma_ring *)*ring_state,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 (PvrdmaRingState *)*ring_state,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0(num_pages - 1) * TARGET_PAGE_SIZE /
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0sizeof(struct pvrdma_cqne),
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0sizeof(struct pvrdma_cqne),
diff --git a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrd= ma_ring.h b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdm= a_ring.h
deleted file mode 100644
index 7b4062a1a107..000000000000
--- a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring= .h
+++ /dev/null
@@ -1,114 +0,0 @@
-/*
- * Copyright (c) 2012-2016 VMware, Inc.=C2=A0 All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of EITHER the GNU General Public License
- * version 2 as published by the Free Software Foundation or the BSD
- * 2-Clause License. This program is distributed in the hope that it
- * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
- * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
- * See the GNU General Public License version 2 for more details at
- * http://www.gnu.org/licenses/old-licenses= /gpl-2.0.en.html.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program available in the file COPYING in the main
- * directory of this source tree.
- *
- * The BSD 2-Clause License
- *
- *=C2=A0 =C2=A0 =C2=A0Redistribution and use in source and binary forms, w= ith or
- *=C2=A0 =C2=A0 =C2=A0without modification, are permitted provided that th= e following
- *=C2=A0 =C2=A0 =C2=A0conditions are met:
- *
- *=C2=A0 =C2=A0 =C2=A0 - Redistributions of source code must retain the ab= ove
- *=C2=A0 =C2=A0 =C2=A0 =C2=A0 copyright notice, this list of conditions an= d the following
- *=C2=A0 =C2=A0 =C2=A0 =C2=A0 disclaimer.
- *
- *=C2=A0 =C2=A0 =C2=A0 - Redistributions in binary form must reproduce the= above
- *=C2=A0 =C2=A0 =C2=A0 =C2=A0 copyright notice, this list of conditions an= d the following
- *=C2=A0 =C2=A0 =C2=A0 =C2=A0 disclaimer in the documentation and/or other= materials
- *=C2=A0 =C2=A0 =C2=A0 =C2=A0 provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT= NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
- * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
- * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
- * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
- * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
- * OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef __PVRDMA_RING_H__
-#define __PVRDMA_RING_H__
-
-#include "standard-headers/linux/types.h"
-
-#define PVRDMA_INVALID_IDX=C2=A0 =C2=A0 =C2=A0-1=C2=A0 =C2=A0 =C2=A0 /* In= valid index. */
-
-struct pvrdma_ring {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0int prod_tail;=C2=A0 /* Producer tail. */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0int cons_head;=C2=A0 /* Consumer head. */
-};
-
-struct pvrdma_ring_state {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct pvrdma_ring tx;=C2=A0 /* Tx ring. */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0struct pvrdma_ring rx;=C2=A0 /* Rx ring. */
-};
-
-static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems)
-{
-=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Generates fewer instructions than a less-tha= n. */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0return (idx & ~((max_elems << 1) - 1)= ) =3D=3D 0;
-}
-
-static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
-{
-=C2=A0 =C2=A0 =C2=A0 =C2=A0const unsigned int idx =3D qatomic_read(var); -
-=C2=A0 =C2=A0 =C2=A0 =C2=A0if (pvrdma_idx_valid(idx, max_elems))
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return idx & (m= ax_elems - 1);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0return PVRDMA_INVALID_IDX;
-}
-
-static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems)
-{
-=C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t idx =3D qatomic_read(var) + 1;=C2=A0 = =C2=A0/* Increment. */
-
-=C2=A0 =C2=A0 =C2=A0 =C2=A0idx &=3D (max_elems << 1) - 1;=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* Modulo size, flip gen. */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0qatomic_set(var, idx);
-}
-
-static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *= r,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0uint32_t max_elems, uint32_t *out_tail)
-{
-=C2=A0 =C2=A0 =C2=A0 =C2=A0const uint32_t tail =3D qatomic_read(&r->= ;prod_tail);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0const uint32_t head =3D qatomic_read(&r->= ;cons_head);
-
-=C2=A0 =C2=A0 =C2=A0 =C2=A0if (pvrdma_idx_valid(tail, max_elems) &&= ;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pvrdma_idx_valid(head, max_elems)= ) {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*out_tail =3D tail = & (max_elems - 1);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return tail !=3D (h= ead ^ max_elems);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0}
-=C2=A0 =C2=A0 =C2=A0 =C2=A0return PVRDMA_INVALID_IDX;
-}
-
-static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r= ,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 uint32_t max_elems, uint32_t *out_head)
-{
-=C2=A0 =C2=A0 =C2=A0 =C2=A0const uint32_t tail =3D qatomic_read(&r->= ;prod_tail);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0const uint32_t head =3D qatomic_read(&r->= ;cons_head);
-
-=C2=A0 =C2=A0 =C2=A0 =C2=A0if (pvrdma_idx_valid(tail, max_elems) &&= ;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pvrdma_idx_valid(head, max_elems)= ) {
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*out_head =3D head = & (max_elems - 1);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return tail !=3D he= ad;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0}
-=C2=A0 =C2=A0 =C2=A0 =C2=A0return PVRDMA_INVALID_IDX;
-}
-
-#endif /* __PVRDMA_RING_H__ */
diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers= .sh
index fa6f2b6272b7..1050e361694f 100755
--- a/scripts/update-linux-headers.sh
+++ b/scripts/update-linux-headers.sh
@@ -215,8 +215,7 @@ sed=C2=A0 -e '1h;2,$H;$!d;g'=C2=A0 -e 's/[^= };]*pvrdma[^(| ]*([^)]*);//g' \
=C2=A0 =C2=A0 =C2=A0"$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ve= rbs.h" > \
=C2=A0 =C2=A0 =C2=A0"$tmp_pvrdma_verbs";

-for i in "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h"= \
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"$linux/drivers/infiniband/hw/vmw_p= vrdma/pvrdma_dev_api.h" \
+for i in "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h&qu= ot; \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "$tmp_pvrdma_verbs"; do \
=C2=A0 =C2=A0 =C2=A0cp_portable "$i" \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "$output/include/standard-headers/d= rivers/infiniband/hw/vmw_pvrdma/"
--
2.26.2


Thanks!
I guess somewhere in= the kernel there is such a clean and generic implementation of a ring and = VM folks could utilize that instead of writing their own.

Tested-by:=C2=A0Yuval Shaia <yuval.shaia.ml@gmail.com>
Reviewed-by: Yuval Shaia <<= a href=3D"mailto:yuval.shaia.ml@gmail.com">yuval.shaia.ml@gmail.com>=
--00000000000030e71405baf7944f--