Re: [RFC 05/29] nvkm/vgpu: populate GSP_VF_INFO when NVIDIA vGPU is enabled

Jason Gunthorpe
2024-09-26T22:52:55Z

On Sun, Sep 22, 2024 at 05:49:27AM -0700, Zhi Wang wrote:
> +void nvkm_vgpu_mgr_populate_gsp_vf_info(struct nvkm_device *device,
> + void *info)
> +{
> + struct pci_dev *pdev = nvkm_to_pdev(device);
> + GspSystemInfo *gsp_info = info;
> + GSP_VF_INFO *vf_info = &gsp_info->gspVFInfo;
> + u32 lo, hi;
> + u16 v;
> + int pos;
> +
> + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_SRIOV);
> +
> + pci_read_config_word(pdev, pos + PCI_SRIOV_TOTAL_VF, &v);
> + vf_info->totalVFs = v;
> +
> + pci_read_config_word(pdev, pos + PCI_SRIOV_VF_OFFSET, &v);
> + vf_info->firstVFOffset = v;
> +
> + pci_read_config_dword(pdev, pos + PCI_SRIOV_BAR, &lo);
> + vf_info->FirstVFBar0Address = lo & 0xFFFFFFF0;
> +
> + pci_read_config_dword(pdev, pos + PCI_SRIOV_BAR + 4, &lo);
> + pci_read_config_dword(pdev, pos + PCI_SRIOV_BAR + 8, &hi);
> +
> + vf_info->FirstVFBar1Address = (((u64)hi) << 32) + (lo & 0xFFFFFFF0);
> +
> + pci_read_config_dword(pdev, pos + PCI_SRIOV_BAR + 12, &lo);
> + pci_read_config_dword(pdev, pos + PCI_SRIOV_BAR + 16, &hi);
> +
> + vf_info->FirstVFBar2Address = (((u64)hi) << 32) + (lo & 0xFFFFFFF0);
> +
> +#define IS_BAR_64(i) (((i) & 0x00000006) == 0x00000004)

This should come from the PCI core not be re-read with pci_read_config
and hand rolled macros.

From a mlx perspective this is really weird, I'd expect the FW to be
able to read its own config space.

Jason