Re: [PATCH 0/8] *** RFC: ARM KVM dirty tracking device ***

[Oliver Upton <oliver.upton@linux.dev>]

On Thu, Sep 26, 2024 at 11:00:39AM +0100, David Woodhouse wrote:
> > Beyond that, I have some reservations about maintaining support for
> > features that cannot actually be tested outside of your own environment.
> 
> That's more about the hardware driver itself which will follow, than
> the core API posted here.
> 
> I understand the reservation, but I think it's fine. In general, Linux
> does support esoteric hardware that not everyone can test every
> time. We do sweeping changes across all Ethernet drivers, for example,
> and some of those barely even exist any more.

Of course, but I think it is also reasonable to say that ethernet
support in the kernel is rather mature with a good variety of hardware.
By comparison, what we have here is a brand new driver interface with
architecture / KVM code, which is pretty rare, with a single
implementation.

I'm perfectly happy to tinker on page tracking interface(s) in the
future w/o testing everything, but I must insist that we have *some*
way of testing the initial infrastructure before even considering taking
it.

> This particular device should be available on bare metal EC2 instances,
> of course, but perhaps we should also implement it in QEMU. That would
> actually be beneficial for our internal testing anyway, as it would
> allow us to catch regressions much earlier in our own development
> process.

QEMU would be interesting, but hardware is always welcome too ;-)

> > > When ARM architectural solution (FEAT_HDBSS feature) is available, we intend to
> > > use it via the same interface most likely with adaptations.
> > 
> > Will the PTA stuff eventually get retired once you get support for FEAT_HDBSS
> > in hardware?
> 
> I don't think there is a definitive answer to that which is ready to
> tape out, but it certainly seems possible that future generations will
> eventually move to FEAT_HDBSS, maybe even reaching production by the
> end of the decade, at the earliest? And then a decade or two later, the
> existing hardware generations might even get retired, yes¹.
> 
> ¹ #include <forward-looking statement.disclaimer>

Well, hopefully that means you folks will look after it then :)

> > I think the best way forward here is to implement the architecture, and
> > hopefully after that your legacy driver can be made to fit the
> > interface. The FVP implements FEAT_HDBSS, so there's some (slow)
> > reference hardware to test against.
> 
> Is there actually any documentation available about FEAT_HDBSS? We've
> been asking, but haven't received it. I can find one or two mentions
> e.g. https://arm.jonpalmisc.com/2023_09_sysreg/AArch64-hdbssbr_el2 but
> nothing particularly useful.

Annoyingly no, the Arm ARM tends to lag the architecture by quite a bit.
The sysreg XML (from which I think this website is derived) gets updated
much more frequently.

> The main reason for posting this series early is to make sure we do all
> we can to accommodate FEAT_HDBSS. It's not the *end* of the world if
> the kernel-internal API has to be tweaked slightly when FEAT_HDBSS
> actually becomes reality in future, but obviously we'd prefer to
> support it right from the start.

Jury is still out on how FEAT_HDBSS is gonna fit with this PTA stuff.
I'm guessing your hardware has some way of disambiguating dirtied
addresses by VMID.

The architected solution, OTOH, is tied to a particular stage-2 MMU
configuration. KVM proper might need to manage the dirty tracking
hardware in that case as it'll need to be context switched on the
vcpu_load() / vcpu_put() boundary.

-- 
Thanks,
Oliver