Re: virtio-pci in qemu-system-arm is broken in 8.2

[Peter Maydell <peter.maydell@linaro.org>]

On Tue, 9 Jan 2024 at 16:24, Michael Tokarev <mjt@tls.msk.ru> wrote:
>
> 09.01.2024 16:52, Peter Maydell:
> ..
> > Oh, your kernel isn't an LPAE one (i.e. CONFIG_LPAE is not
> > set). That will obviously never be able to access registers
> > above the 4GB mark (though the kernel's error message in this
> > situation is a bit unhelpful and could perhaps be improved).
> > If I set CONFIG_LPAE on the non-working config it starts working.
> >
> > I think then the answer is:
> >   * if you want to use the (default) highmem setup, use an LPAE kernel
> >   * if you want to use a non-LPAE kernel, tell QEMU to avoid
> >     highmem using '-machine virt,highmem=off'
> >
> > It was just a bug that we were accidentally disabling highmem
> > for the 32-bit 'max' CPU before b8f7959f28c4f3.
>
> Wow wow wow.  So it's a pebkac bug, not qemu bug.. :)
> I didn't even know about LPAE before this email, and knew very
> little about arm stuff too.
>
> Thank you very much for your work and time!
>
> The diagnostics here is definitely.. "interesting", so to say, it's
> very "obvious" what the problem is.. from the kernel messages :)

Yeah. I don't personally have the time to try to follow up
on improving the kernel handling of this, but if anybody else
does, I think the problem is that the function __of_address_to_resource()
which fills in a 'struct resource' from a DTB entry silently
truncates over-large values when it fills in r->start and r->end
(which are of type phys_addr_t, and so only 32 bits on non-LPAE
kernels). So that function seems like a good place to put some

  if (taddr or taddr + size are above 4GB) {
      warn("resource foo is too high in the address space to use
            (did you forget to enable CONFIG_LPAE?)\n");
      return -EINVAL;
  }

type handling which would then catch this user error relatively
comprehensibly for most devices.

> And the author(s) of the tool used in debian to run arm guest (which
> triggered this issue with qemu 8.2) didn't know this thing either,
> obviously.  Also, it seems to be a little-known fact that qemu has
> highmem enabled for 32bit arm by default and that one - unlike e.g
> i386 - actually breaks system (with pci bus being unaccessible),
> while on i386 the only outcome is not all memory is available.
> That feels like unfortunate decision to enable highmem by default
> on arm in qemu 2.12.

Yeah, in retrospect the other option might have been better.
But at this late date it doesn't seem worth revisiting.

> > The linux-image-6.6.9-armmp-lpae_6.6.9-1_armhf.deb kernel
> > will probably work (though I haven't tested it).
>
> Yes, it works.  Also -machine virt,highmem=off works.  There's
> no bug per se.
>
>
> Thank you for all the findings, Peter!  Much appreciated!

No worries.

-- PMM