Re: help with "PSN sequence error" in Ubuntu 20.04 (using CX-4 or CX-6 mellanox cards)

[Leon Romanovsky <leon@kernel.org>]

On Mon, Aug 02, 2021 at 11:37:06AM -0400, Olga Kornievskaia wrote:
> Hi folks,
> 
> We are encountering an error condition (while doing NFSoRDMA) but the
> problem seems to be in the RDMA core itself. The problem is that the
> client at some point is ending in an RDMA NAK with "PNS Sequence
> error" but the network trace shows all the PSNs are accounted for
> (snippet at the bottom). It's as if the client lost its knowledge of
> the current PSN.
> 
> Questions:
> 1. Is PSN handling done by the hardware card itself (in firmware) and
> not in the kernel (making this a card/firmware specific problem)? I
> was trying to look thru the rdma core/mlx5 driver code to see what
> would generate a NAK with such error but wasn't able to find one. Only
> found counters for nak_seq_error which made me think this is a
> firmware problem.

The decisions what is valid or not are done in the FW, kernel doesn't
check anything. Although, the kernel sets/gets next_send_psn/next_recv_psn.

Thanks