Re: [PATCH v2 1/1] KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory

From: Marc Zyngier <maz@kernel.org>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	ankita@nvidia.com,
	Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>,
	oliver.upton@linux.dev, suzuki.poulose@arm.com,
	yuzenghui@huawei.com, will@kernel.org, ardb@kernel.org,
	akpm@linux-foundation.org, gshan@redhat.com, aniketa@nvidia.com,
	cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com,
	vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com,
	jhubbard@nvidia.com, danw@nvidia.com, mochs@nvidia.com,
	kvmarm@lists.linux.dev, kvm@vger.kernel.org,
	lpieralisi@kernel.org, linux-kernel@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2 1/1] KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory
Date: Tue, 05 Dec 2023 17:01:28 +0000	[thread overview]
Message-ID: <86bkb4bn2v.wl-maz@kernel.org> (raw)
In-Reply-To: <20231205164318.GG2692119@nvidia.com>

On Tue, 05 Dec 2023 16:43:18 +0000,
Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> On Tue, Dec 05, 2023 at 04:22:33PM +0000, Catalin Marinas wrote:

> > That's an argument to restrict this feature to PCIe. It's really about
> > fewer arguments on the behaviour of other devices. Marc did raise
> > another issue with the GIC VCPU interface (does this even have a vma in
> > the host VMM?). That's a class of devices where the mapping is
> > context-switched, so the TLBI+DSB rules don't help.

There is no vma. The CPU interface is entirely under control of KVM.
Userspace only provides the IPA for the mapping.

> 
> I don't know anything about the GIC VCPU interface, to give any
> comment unfortunately. Since it seems there is something to fix here I
> would appreciate some background..
>  
> When you say it is context switched do you mean kvm does a register
> write on every vm entry to set the proper HW context for the vCPU?

The CPU interface is mapped in every guest S2 page tables as a per-CPU
device, and under complete control of the guest. There is no KVM
register write to that frame (unless we're proxying it, but that's for
another day).

> 
> We are worrying that register write will possibly not order after
> NORMAL_NC?

Guest maps the device as Normal-NC (because it now can), which means
that there is no control over the alignment or anything like that. The
accesses could also be reordered, and/or hit after a context switch to
another guest. Which is why KVM has so far used nGnRE as the mapping
type.

	M.

-- 
Without deviation from the norm, progress is not possible.