Re: [PATCH v3 1/1] KVM: arm64: Allow cacheable stage 2 mapping using VMA flags

[Catalin Marinas <catalin.marinas@arm.com>]

On Tue, Apr 22, 2025 at 08:35:56PM -0300, Jason Gunthorpe wrote:
> On Tue, Apr 22, 2025 at 02:28:18PM -0700, Oliver Upton wrote:
> > Wait, so userspace simultaneously doesn't know the cacheability at host
> > stage-1 but *does* for stage-2? 
> 
> No, it doesn't know either. The point is the VMM doesn't care about
> any of this. It just wants to connect KVM to VFIO and have the kernel
> internally negotiate the details of how that works.
> 
> There is zero value in the VMM being aware that KVM/VFIO is using
> cachable or non-cachable mappings because it will never touch this
> memory anyhow, and arguably it would be happier if it wasn't even in a
> VMA in the first place.

I think this was discussed at some point in the past - what about
ignoring the VMA altogether and using an fd-based approach? Keep the
current VFIO+vma ABI non-cacheable and introduce a new one for cacheable
I/O mappings, e.g. KVM_MEM_IOFD. Is there a way for VFIO to communicate
the attributes to KVM on the type of mapping in the absence of a VMA
(e.g. via the inode)? If not:

Stage 2 could default to non-cacheable and we can add a
KVM_MEMORY_ATTRIBUTE_IO_WB as an option to
ioctl(KVM_SET_MEMORY_ATTRIBUTES) (setup requiring two ioctls). The
latter could fail if S2FWB is not available. But I agree, it does mean
building knowledge of the device in the VMM (it might not be that bad).

-- 
Catalin