Re: [PATCH v2 02/13] KVM: arm64: Enable eager hugepage splitting if HDBSS is available

[Leonardo Bras <leo.bras@arm.com>]

On Tue, Jun 30, 2026 at 11:43:01AM -0700, Oliver Upton wrote:
> On Tue, Jun 30, 2026 at 06:09:56PM +0100, Leonardo Bras wrote:
> > On Tue, Jun 30, 2026 at 08:44:20AM -0700, Oliver Upton wrote:
> > > On Tue, Jun 30, 2026 at 01:58:48PM +0100, Leonardo Bras wrote:
> > > > On Mon, Jun 29, 2026 at 10:06:38AM -0700, Oliver Upton wrote:
> > > > > > But this raises a topic I would like to understand:
> > > > > > - Do we actually need this to be a block_size to assure correctness? or is 
> > > > > >   it just about efficiency?
> > > > > 
> > > > > What value is there in having a chunk size larger than the largest
> > > > > possible block mapping? The whole UAPI is deliberately tied up with page
> > > > > table geometry.
> > > > 
> > > > Not larger, possibly smallerv My concern was the difference in pages to 
> > > > split between 4k, 16k and 64k.
> > > 
> > > Ok, well in any case the upper bound is going to be the largest possible
> > > block mapping for a given page granule.
> > >
> > 
> > Sure, we can do this.
> 
> I'm describing the UAPI behavior of what we already have. Please do not
> change anything and continue to leave userspace in charge of eager
> splitting / chunk size.
> 
> > > > > 
> > > > > Overall, I'm not buying the argument for changing the behavior of
> > > > > KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE. There are very good reasons for
> > > > > *not* eagerly splitting the entire address space, especially if you know
> > > > > the working set of the VM is small.
> > > > > 
> > > > > You can still use HDBSS without eagerly splitting, so long as block
> > > > > mappings are {DBM, S2AP_W} = {0, 0} and leaf mappings (which have
> > > > > a writable PFN) are {1, 0}.
> > > > > 
> > > > 
> > > > Block mappings being read-only, and leaf mappings being writable-clean, 
> > > > then? Could you please ellaborate on why does not it need eager-split?
> > > 
> > > Read-only translations will continue to generate permission faults
> > > whereas writable-clean descriptors can be updated by hardware. You get
> > > the opportunity to split a block mapping lazily while preserving
> > > hardware dirty tracking for page mappings.
> > > 
> > 
> > So you suggest we only enable DBM bit after we split the block, that will 
> > happen only after a block is dirtied for the first time after dirty-log 
> > starts? 
> 
> If the VMM wants to do lazy splitting, yes. Otherwise this would happen
> as part of eager page splitting for block mappings that are known to be
> writable.
> 
> > > The approach I think we may need is:
> > > 
> > >  - Use a software bit in the PTE to stash whether or not a PFN is
> > >    'software-writable' when constructing the stage-2. By this I mean
> > >    we've already faulted it in for write from the primary MMU.
> > > 
> > >  - At the time of write protection, reap the hardware-writable state
> > >    from all PTEs but preserve the software-writable bit.
> > > 
> > >  - Whenever splitting a block mapping, set the DBM bit in the page-level
> > >    PTEs if the block was software-writable and HDBSS is present.
> > > 
> > > That way you'd have sufficient metadata in the PTE to safely set DBM.
> > 
> > I remember that, for some reason I can't recall, it would not be great to 
> > set DBM during dirty-log start, and instead we should have it since VM 
> > creation. Maybe it had to do with part of the pagetable using the old 
> > encoding (no DBM), and the other part using the new one.
> > 
> > IIRC, only blocks that are backed by writable memory (S1) were supposed to 
> > receive the DBM bit. We could use that info for deciding what to split, 
> > then.
> 
> That's why I'm saying to use a software bit. We can't leave DBM=1 on
> block mappings and set VTCR_EL2.HD=1 for obvious reasons.
>

I am failing to see the reason for above.

IIUC, one option would be to:
- When constructing S2
  - Set DBM on every writable block, put them in writable-clean
  - Enable VTCR_EL2.HD=1 (HAFDBS will update dirty-state in HW)
- When dirty-logging starts
  - Set VTCR_EL2.HDBSS=1 (any new dirtying gets logged to HDBSS)
    - On every vcpu
  - Write-protect everything (with HACDBS possibly)
  - Split everything

What am I missing?
(above case is considering manual protect = off, but the same idea goes 
for the manual_protect_and_init_set, but cleaning is on-demand)
 
> > Another option would be to split when we are collecting a dirty-entry from 
> > HDBSS, but for live migration that would mean we have to transfer the whole 
> > block (possibly a large LEVEL1 block), because we have no idea which part 
> > of it got dirty.
> 
> We need to do dirty tracking at page granularity. The only scenario
> where we have writable block mappings while dirty logging is enabled on
> a memslot is if the VMM enabled KVM_DIRTY_LOG_INITIALLY_SET.

Agree

Thanks!
Leo