Help with PCIe THP on ConnectX-7

Help with PCIe THP on ConnectX-7

[Bruce Merry]

Hello

I'm hoping someone from NVIDIA Networking can help with this; I wasn't
sure of the best way to get in contact with the engineers working on
mlx5 kernel code so thought I'd try here. I'm trying to write some
(userspace) code using ibv_reg_mr_ex to register a memory region using
TLP Processing Hints (THP) to improve performance on an Epyc Turin
(Zen 5) system. However, in the kernel mlx5_st_create is bailing out
because of this check:

       if (!MLX5_CAP_GEN(dev, mkey_pcie_tph))
                return NULL;

As far as I can tell, that is checking a capability bit returned by
the firmware.

Before I spend a lot more time debugging, can you tell me whether
ConnectX-7 supports this feature at all (I'm on the latest 28.48.1000
firmware)? I see it mentioned in release notes for ConnectX-8
firmware, but not in ConnectX-7 release notes.

If ConnectX-7 does support it, do you have any tips on what might
cause that capability bit to be false and how to determine what the
cause is? For example, could that happen if the motherboard BIOS
doesn't support TPH?

Thanks
Bruce
-- 
Dr Bruce Merry
bmerry <@> gmail <.> com
http://www.brucemerry.org.za/
http://blog.brucemerry.org.za/