Re: [RESEND PATCH v5 1/2] RDMA/rxe: Support RDMA Atomic Write operation

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Jul 08, 2022 at 04:02:36AM +0000, yangx.jy@fujitsu.com wrote:
> +static enum resp_states atomic_write_reply(struct rxe_qp *qp,
> +					   struct rxe_pkt_info *pkt)
> +{
> +	u64 src, *dst;
> +	struct resp_res *res = qp->resp.res;
> +	struct rxe_mr *mr = qp->resp.mr;
> +	int payload = payload_size(pkt);
> +
> +	if (!res) {
> +		res = rxe_prepare_res(qp, pkt, RXE_ATOMIC_WRITE_MASK);
> +		qp->resp.res = res;
> +	}
> +
> +	if (!res->replay) {
> +#ifdef CONFIG_64BIT
> +		memcpy(&src, payload_addr(pkt), payload);
> +
> +		dst = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, payload);
> +		/* check vaddr is 8 bytes aligned. */
> +		if (!dst || (uintptr_t)dst & 7)
> +			return RESPST_ERR_MISALIGNED_ATOMIC;
> +
> +		/* Do atomic write after all prior operations have completed */
> +		smp_store_release(dst, src);

Someone needs to fix iova_to_vaddr to do the missing kmap, we can't
just assume you can cast a u64 pfn to a vaddr like this.

> +		/* decrease resp.resid to zero */
> +		qp->resp.resid -= sizeof(payload);
> +
> +		qp->resp.msn++;
> +
> +		/* next expected psn, read handles this separately */
> +		qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK;
> +		qp->resp.ack_psn = qp->resp.psn;
> +
> +		qp->resp.opcode = pkt->opcode;
> +		qp->resp.status = IB_WC_SUCCESS;
> +
> +		return RESPST_ACKNOWLEDGE;
> +#else
> +		pr_err("32-bit arch doesn't support 8-byte atomic write\n");
> +		return RESPST_ERR_UNSUPPORTED_OPCODE;

No print on receiving a remote packet

Jason