Re: [REGRESSION] vhost_iotlb: bound map allocation in add_range breaks vhost-vdpa on hosts with fragmented memory

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Oct 28, 2026 at 03:46:05PM +0100, Dragos Tatulea wrote:
> On 28.10.26 14:05, Michael S. Tsirkin wrote:
> > On Wed, Oct 28, 2026 at 01:44:30PM +0100, Dragos Tatulea wrote:
> >> Hi,
> >>
> >> We've started seeing regressions for a while in our verification CI for
> >> vDPA Live Migration tests. Occasionally the FW returns a syndrome
> >> related to invalid mappings during device reset.
> >>
> >> We've narrowed this down to commit 1ed35ac7f3fe "vhost_iotlb: bound map
> >> allocation in add_range" which limits the iotlbs to 2K by default. On a
> >> host with fragmented memory you can get easily more than 2K entries.
> >> This results in truncated/incomplete iotlbs mapped to the device and when
> >> the rings happen to be outside of the first 2K mappings the FW will complain
> >> because it got addresses that are not mapped.
> >>
> >> I'm not sure how to fix this though:
> >>
> >> - LLM suggests setting the 2K limit only in VA mode and using 0 as
> >>   before in PA mode because PA mode is limited anyway. Does this
> >>   work?
> > 
> > Well no, we do need the limit in PA mode I think.
> > 
> > But we should maybe  bound from what's actually being mapped rather than
> > a fixed count. e.g. X entries per pinned page?
> You could have many 1 PAGE_SIZE mappings though. It wouldn't help
> then.

it still mostly is 1 entry=1 pinned page

sharing is possible but uncommon

> > Bounding by bytes instead of entry count is another option.How to get this limit from userspace though? Would RLIMIT_MEMLOCK
> be an option? Although `ulimit -l unlimited` would need to be
> capped to some value...

guest ram no?

> 
> Also, if the entry limit is still there some kind of precedence between
> memory and entry limit needs to be decided.

I'd steal a page from rlimit then use that for the entries, when out of
that - steal another page.

> 
> > Also 0 is now rejected.
> > 
> > 
> > 
> >> - Can we detect this and reject the device / stop the device at boot time?
> >>   Probably not as this doesn't cover Live Migration.
> > 
> > Exactly. Besides, failure is not at boot.
> > 
> >>
> >> Any thoughts?
> > 
> > Does boosting max_iotlb_entries help?
> > vhost_vdpa.max_iotlb_entries=65536? Or sized to guest RAM?
> If set as module paramer? Yes, it works. But the user should see some
> kind of error to know that something is off. Otherwise things will start
> failing silently and randomly...
> 
> Thanks,
> Dragos