Fwd: pyverbs test_resize_cq fails in some cases

[Bob Pearson <rpearsonhpe@gmail.com>]

-------- Forwarded Message --------
Subject: pyverbs test_resize_cq fails in some cases
Date: Thu, 23 Mar 2023 21:37:28 -0500
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Edward Srouji <edwards@nvidia.com>
CC: Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>

Edward,

The pyverbs test: test_resize_cq fails for the rxe driver in some cases.

The code is definitely racy:

def test_resize_cq(self):
        """
        Test resize CQ, start with specific value and then increase and decrease
        the CQ size. The test also check bad flow of decrease the CQ size when
        there are more completions on it than the new value.
        """
        self.create_players(CQUDResources, cq_depth=3)
        # Decrease the CQ size.
        new_cq_size = 1
        try:
            self.client.cq.resize(new_cq_size)
        except PyverbsRDMAError as ex:
            if ex.error_code == errno.EOPNOTSUPP:
                raise unittest.SkipTest('Resize CQ is not supported')
            raise ex
        self.assertTrue(self.client.cq.cqe >= new_cq_size,
                        f'The actual CQ size ({self.client.cq.cqe}) is less '
                        'than guaranteed ({new_cq_size})')

        # Increase the CQ size.
        new_cq_size = 7
        self.client.cq.resize(new_cq_size)
        self.assertTrue(self.client.cq.cqe >= new_cq_size,
                        f'The actual CQ size ({self.client.cq.cqe}) is less '
                        'than guaranteed ({new_cq_size})')

        # Fill the CQ entries except one for avoid cq_overrun warnings.
        send_wr, _ = u.get_send_elements(self.client, False)
        ah_client = u.get_global_ah(self.client, self.gid_index, self.ib_port)
        for i in range(self.client.cq.cqe - 1):
            u.send(self.client, send_wr, ah=ah_client)

	### This posts 6 send work requests but does not wait for them to complete
	### The following proceeds while the sends are executing in the background
	### and the test can fail. An easy fix is to add the line
        time.sleep(1/1000)  ### or maybe something a little larger but this works for me.
	### alternatively you could test the data at the destination.

        # Decrease the CQ size to less than the CQ unpolled entries.
        new_cq_size = 1
        with self.assertRaises(PyverbsRDMAError) as ex:
            self.client.cq.resize(new_cq_size)
        self.assertEqual(ex.exception.error_code, errno.EINVAL)

Bob