Re: [PATCH v11 00/25] mm/gup: track dma-pinned pages: FOLL_PIN

John Hubbard
2019-12-20T23:14:18Z

On 12/20/19 10:48 AM, Leon Romanovsky wrote:
...
>> test_query_qp (tests.test_qp.QPTest) ... ok
>> test_rdmacm_sync_traffic (tests.test_rdmacm.CMTestCase) ... skipped 'No devices with net interface'
>>
>> ======================================================================
>> FAIL: test_query_port (tests.test_device.DeviceTest)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/kernel_work/rdma-core/tests/test_device.py", line 129, in test_query_port
>> self.verify_port_attr(port_attr)
>> File "/kernel_work/rdma-core/tests/test_device.py", line 113, in verify_port_attr
>> assert 'Invalid' not in d.speed_to_str(attr.active_speed)
>> AssertionError
>
> I'm very curious how did you get this assert "d.speed_to_str" covers all
> known speeds according to the IBTA.
>

Hi Leon,

Short answer: I can make that one pass, with a small fix the the rdma-core test
suite:

commit a1b9fb0846e1b2356d7a16f4fbdd1960cf8dcbe5 (HEAD -> fix_speed_to_str)
Author: John Hubbard <jhubbard@nvidia.com>
Date: Fri Dec 20 15:07:47 2019 -0800

device: fix speed_to_str(), to handle disabled links

For disabled links, the raw speed token is 0. However,
speed_to_str() doesn't have that in the list. This leads
to an assertion when running tests (test_query_port) when
one link is down and other link(s) are up.

Fix this by returning '(Disabled/down)' for the zero speed
case.

diff --git a/pyverbs/device.pyx b/pyverbs/device.pyx
index 33d133fd..f8b7826b 100755
--- a/pyverbs/device.pyx
+++ b/pyverbs/device.pyx
@@ -923,8 +923,8 @@ def width_to_str(width):

def speed_to_str(speed):
- l = {1: '2.5 Gbps', 2: '5.0 Gbps', 4: '5.0 Gbps', 8: '10.0 Gbps',
- 16: '14.0 Gbps', 32: '25.0 Gbps', 64: '50.0 Gbps'}
+ l = {0: '(Disabled/down)', 1: '2.5 Gbps', 2: '5.0 Gbps', 4: '5.0 Gbps',
+ 8: '10.0 Gbps', 16: '14.0 Gbps', 32: '25.0 Gbps', 64: '50.0 Gbps'}
try:
return '{s} ({n})'.format(s=l[speed], n=speed)
except KeyError:

Longer answer:
==============

It looks like this test suite assumes that every link is connected! (Probably
in most test systems, they are.) But in my setup, the ConnectX cards each have
two slots, and I only have (and only need) one cable. So one link is up, and
the other is disabled.

This leads to the other problem, which is that if a link is disabled, the
test suite finds a "0" token for attr.active_speed. That token is not in the
approved list, and so d.speed_to_str() asserts.

With some diagnostics added, I can see it checking each link: one passes, and
the other asserts:

diff --git a/tests/test_device.py b/tests/test_device.py
index 524e0e89..7b33d7db 100644
--- a/tests/test_device.py
+++ b/tests/test_device.py
@@ -110,6 +110,12 @@ class DeviceTest(unittest.TestCase):
assert 'Invalid' not in d.translate_mtu(attr.max_mtu)
assert 'Invalid' not in d.translate_mtu(attr.active_mtu)
assert 'Invalid' not in d.width_to_str(attr.active_width)
+ print("")
+ print('Diagnostics ===========================================')
+ print('phys_state: ', d.phys_state_to_str(attr.phys_state))
+ print('active_width): ', d.width_to_str(attr.active_width))
+ print('active_speed: ', d.speed_to_str(attr.active_speed))
+ print('END of Diagnostics ====================================')
assert 'Invalid' not in d.speed_to_str(attr.active_speed)
assert 'Invalid' not in d.translate_link_layer(attr.link_layer)
assert attr.max_msg_sz > 0x1000

assert attr.max_msg_sz > 0x1000

...and the test run from that is:

# ./build/bin/run_tests.py --verbose tests.test_device.DeviceTest
test_dev_list (tests.test_device.DeviceTest) ... ok
test_open_dev (tests.test_device.DeviceTest) ... ok
test_query_device (tests.test_device.DeviceTest) ... ok
test_query_device_ex (tests.test_device.DeviceTest) ... ok
test_query_gid (tests.test_device.DeviceTest) ... ok
test_query_port (tests.test_device.DeviceTest) ...
Diagnostics ===========================================
phys_state: Link up (5)
active_width): 4X (2)
active_speed: 25.0 Gbps (32)
END of Diagnostics ====================================

Diagnostics ===========================================
phys_state: Disabled (3)
active_width): 4X (2)
active_speed: Invalid speed
END of Diagnostics ====================================
FAIL
test_query_port_bad_flow (tests.test_device.DeviceTest) ... ok

======================================================================
FAIL: test_query_port (tests.test_device.DeviceTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/kernel_work/rdma-core/tests/test_device.py", line 135, in test_query_port
self.verify_port_attr(port_attr)
File "/kernel_work/rdma-core/tests/test_device.py", line 119, in verify_port_attr
assert 'Invalid' not in d.speed_to_str(attr.active_speed)
AssertionError

----------------------------------------------------------------------
Ran 7 tests in 0.055s

FAILED (failures=1)

thanks,
--
John Hubbard
NVIDIA