[bugzilla-daemon@kernel.org: [Bug 219765] New: Regression: VFIO NVIDIA GPU Passthrough Fails in Linux 6.13.0 (GPU Reset & Audio Controller Disappears)]

[bugzilla-daemon@kernel.org: [Bug 219765] New: Regression: VFIO NVIDIA GPU Passthrough Fails in Linux 6.13.0 (GPU Reset & Audio Controller Disappears)]

[Bjorn Helgaas]

#regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=219765

I don't see an obvious culprit between v6.12 and v6.13, suggested
bisection if possible.

Workaround: boot with pcie_port_pm=off

Given the workaround, maybe it's worth trying a revert of this:

  dc421bb3c0db ("PCI: Enable runtime PM of the host bridge")


----- Forwarded message from bugzilla-daemon@kernel.org -----

https://bugzilla.kernel.org/show_bug.cgi?id=219765

Created attachment 307599
  --> https://bugzilla.kernel.org/attachment.cgi?id=307599&action=edit
dmesg logs for the kernel in which gpu passthrough works

After upgrading from Linux 6.12.10 to Linux 6.13.0, VFIO GPU passthrough fails
for an NVIDIA GPU (AD107). The GPU is not passed through to the VM, and its
audio device (01:00.1) disappears from Virt-Manager. This issue does not occur
in Linux 6.12.10.

I have attached the logs.

Re: [bugzilla-daemon@kernel.org: [Bug 219765] New: Regression: VFIO NVIDIA GPU Passthrough Fails in Linux 6.13.0 (GPU Reset & Audio Controller Disappears)]

[Rafael J. Wysocki]

On Mon, Feb 10, 2025 at 6:07 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=219765
>
> I don't see an obvious culprit between v6.12 and v6.13, suggested
> bisection if possible.
>
> Workaround: boot with pcie_port_pm=off
>
> Given the workaround, maybe it's worth trying a revert of this:
>
>   dc421bb3c0db ("PCI: Enable runtime PM of the host bridge")

I would ask the reporter to revert it and see if that helps.

The

vfio-pci 0000:01:00.1: Unable to change power state from D0 to D3hot,
device inaccessible

message in the failing case is kind of unexpected.

Overall, it would be good to understand what's going on so we don't
miss an opportunity to fix something.

> ----- Forwarded message from bugzilla-daemon@kernel.org -----
>
> https://bugzilla.kernel.org/show_bug.cgi?id=219765
>
> Created attachment 307599
>   --> https://bugzilla.kernel.org/attachment.cgi?id=307599&action=edit
> dmesg logs for the kernel in which gpu passthrough works
>
> After upgrading from Linux 6.12.10 to Linux 6.13.0, VFIO GPU passthrough fails
> for an NVIDIA GPU (AD107). The GPU is not passed through to the VM, and its
> audio device (01:00.1) disappears from Virt-Manager. This issue does not occur
> in Linux 6.12.10.
>
> I have attached the logs.
>

Re: [bugzilla-daemon@kernel.org: [Bug 219765] New: Regression: VFIO NVIDIA GPU Passthrough Fails in Linux 6.13.0 (GPU Reset & Audio Controller Disappears)]

[Rafael J. Wysocki]

On Mon, Feb 10, 2025 at 6:33 PM Rafael J. Wysocki <rafael@kernel.org> wrote:
>
> On Mon, Feb 10, 2025 at 6:07 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=219765
> >
> > I don't see an obvious culprit between v6.12 and v6.13, suggested
> > bisection if possible.
> >
> > Workaround: boot with pcie_port_pm=off
> >
> > Given the workaround, maybe it's worth trying a revert of this:
> >
> >   dc421bb3c0db ("PCI: Enable runtime PM of the host bridge")
>
> I would ask the reporter to revert it and see if that helps.
>
> The
>
> vfio-pci 0000:01:00.1: Unable to change power state from D0 to D3hot,
> device inaccessible
>
> message in the failing case is kind of unexpected.

And this message is unlikely to be related to commit dc421bb3c0db
because it indicates a failure to change device power states.

Re: [bugzilla-daemon@kernel.org: [Bug 219765] New: Regression: VFIO NVIDIA GPU Passthrough Fails in Linux 6.13.0 (GPU Reset & Audio Controller Disappears)]

[Bjorn Helgaas]

[+cc Ilpo, Jonathan]

On Mon, Feb 10, 2025 at 11:07:15AM -0600, Bjorn Helgaas wrote:
> #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=219765
> 
> I don't see an obvious culprit between v6.12 and v6.13, suggested
> bisection if possible.
> 
> Workaround: boot with pcie_port_pm=off

Update: pcie_port_pm=off turned out *not* to be a workaround, possibly
a red herring due to configuration differences.

> Given the workaround, maybe it's worth trying a revert of this:
> 
>   dc421bb3c0db ("PCI: Enable runtime PM of the host bridge")

Reporter bisected to 665745f27487 ("PCI/bwctrl: Re-add BW notification
portdrv as PCIe BW controller").

Bisect log in bugzilla.  GPU passthrough works correctly after
reverting these commits:

  665745f27487 ("PCI/bwctrl: Re-add BW notification portdrv as PCIe BW controller")
  de9a6c8d5dbf ("PCI/bwctrl: Add pcie_set_target_speed() to set PCIe Link Speed")
  
> ----- Forwarded message from bugzilla-daemon@kernel.org -----
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=219765
> 
> Created attachment 307599
>   --> https://bugzilla.kernel.org/attachment.cgi?id=307599&action=edit
> dmesg logs for the kernel in which gpu passthrough works
> 
> After upgrading from Linux 6.12.10 to Linux 6.13.0, VFIO GPU passthrough fails
> for an NVIDIA GPU (AD107). The GPU is not passed through to the VM, and its
> audio device (01:00.1) disappears from Virt-Manager. This issue does not occur
> in Linux 6.12.10.
> 
> I have attached the logs.