Re: [PATCH v2] kvm/arm64: fixed passthrough gpu into vm on arm64

[Marc Zyngier <maz@kernel.org>]

On Mon, 04 Apr 2022 18:02:02 +0100,
Jason Gunthorpe <jgg@ziepe.ca> wrote:
> 
> On Mon, Apr 04, 2022 at 03:47:11PM +0100, Marc Zyngier wrote:
> > > I'm guessing it will turn into a SBSA like thing where the ARM ARM is
> > > kind of vauge but a SOC has to implement Normal-NC in a certain way to
> > > be functional for the server market.
> > 
> > The main issue is that this equivalence isn't architected, so people
> > can build whatever they want. SBSA means nothing to KVM (or Linux at
> > large), and there is currently no way to describe which devices are
> > safe to map as Normal-NC vs Device.
> 
> And people have, we know of some ARM SOC's that don't work fully with
> NORMAL_NC for this usage. That is already a problem for baremetal
> Linux, let alone KVM..
> 
> That is why I likened it to SBSA - if you want to build a server SOC
> that works with existing server software, you have to support
> NORMAL_NC in this way. Even if it isn't architected.

I see it the other way around. If it isn't architected (and in this
case not even detectable in a scalable way), it simply isn't
supportable by SW.

> The KVM challenge, at least, is to support a CPU with working
> NORMAL_NC to create VM that emulates the same CPU with working
> NORMAL_NC.
> 
> I didn't quite understand your other remarks though - is there a
> problem here? It seems like yes from the other thread you pointed at?

The main issue is that we have no idea what the behaviour is on a
given implementation, and no way to even detect that for a given
device, NORMAL_NC is a memory type that won't cause any issue.

> I would think that KVM should mirror the process page table
> configuration into the KVM page table and make this into a userspace
> problem?

But what makes it safe to do this the first place? There are tons of
HW out there that will simply issue a SError if you generate an
unaligned access targeting the right device, and letting userspace
decide on this is simply not acceptable.

> That turns it into a VFIO problem to negotiate with userspace and set
> the proper pgprot. At least VFIO has a better chance than KVM to
> consult DT or something to learn about the specific device's
> properties.
> 
> I don't know how VFIO/qemu/etc can make this all work automatically
> correctly 100% of the time. It seems to me it is the same problem as
> just basic baremetal "WC" is troubled on ARM in general today. Maybe
> some tables and a command line option in qemu is the best we can hope
> for.

Having a firmware description of what can be mapped with what
attributes would be pretty useful indeed. Not sure how that scales,
but the platform definitely needs to advertise *something* so that we
can allow userspace to say something.

> 
> Long ago I asked that the ARM folks could come with some Linux
> definition of all the WC-like modes and some arch calls to indicate
> which one(s) should be used. Nobody seemed interested in doing that,
> so the above SOC was left non-working in mainline Linux..
> 
> > We either have to take userspace's word for it, or rely on some other
> > heuristics (do this for PCIe, but not anything else). None of which
> > are entirely safe. Not to mention that no currently available CPU
> > implements FEAT_DGH.
> 
> DHG is an optimization, not a functional requirement. Currently
> available CPUs use one of the more expensive barriers that are
> architected to include DHG behavior.
> 
> In any event, this is an important optimization. It is why ARMv9 is
> introducing a new instruction specifically to optmize it.

ARMv9? No, seems like it was introduced in the v8.7 time frame, and
allowed retroactively from v8.0. N2 has it, but A510 doesn't, while V1
(an ARMv8.3 part) has it. But at least it is slowly creeping into
implementations.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel