Re: [PATCH for-rc v2] IB/hfi1: Fix wrong mmu_node used for user SDMA packet after invalidate

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, May 19, 2023 at 12:32:16PM -0400, Dennis Dalessandro wrote:
> From: Brendan Cunningham <bcunningham@cornelisnetworks.com>
> 
> The hfi1 user SDMA pinned-page cache will leave a stale cache entry when
> the cache-entry's virtual address range is invalidated but that cache
> entry is in-use by an outstanding SDMA request.
> 
> Subsequent user SDMA requests with buffers in or spanning the virtual
> address range of the stale cache entry will result in packets
> constructed from the wrong memory, the physical pages pointed to by the
> stale cache entry.
> 
> To fix this, remove mmu_rb_node cache entries from the mmu_rb_handler
> cache independent of the cache entry's refcount. Add 'struct kref
> refcount' to struct mmu_rb_node and manage mmu_rb_node lifetime with
> kref_get() and kref_put().
> 
> mmu_rb_node.refcount makes sdma_mmu_node.refcount redundant. Remove
> 'atomic_t refcount' from struct sdma_mmu_node and change sdma_mmu_node
> code to use mmu_rb_node.refcount.
> 
> Move the mmu_rb_handler destructor call after a
> wait-for-SDMA-request-completion call so mmu_rb_nodes that need
> mmu_rb_handler's workqueue to queue themselves up for destruction from
> an interrupt context may do so.
> 
> Fixes: f48ad614c100 ("IB/hfi1: Move driver out of staging")
> Fixes: 00cbce5cbf88 ("IB/hfi1: Fix bugs with non-PAGE_SIZE-end multi-iovec user SDMA requests")
> 
> Reviewed-by: Dean Luick <dean.luick@cornelisnetworks.com>
> Signed-off-by: Brendan Cunningham <bcunningham@cornelisnetworks.com>
> Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
> ---
> changes since v1: Update Fixes SHA
> ---
>  drivers/infiniband/hw/hfi1/ipoib_tx.c   |    4 -
>  drivers/infiniband/hw/hfi1/mmu_rb.c     |  101 ++++++++++++++---------
>  drivers/infiniband/hw/hfi1/mmu_rb.h     |    3 +
>  drivers/infiniband/hw/hfi1/sdma.c       |   23 ++++-
>  drivers/infiniband/hw/hfi1/sdma.h       |   47 +++++++----
>  drivers/infiniband/hw/hfi1/sdma_txreq.h |    2 
>  drivers/infiniband/hw/hfi1/user_sdma.c  |  137 ++++++++++++-------------------
>  drivers/infiniband/hw/hfi1/user_sdma.h  |    1 
>  drivers/infiniband/hw/hfi1/vnic_sdma.c  |    4 -
>  9 files changed, 177 insertions(+), 145 deletions(-)

This is too big for -rc, but I took it to for-next

Thanks,
Jason