Re: S390 testing for IOMMUFD

[Eric Farman <farman@linux.ibm.com>]

On Tue, 2022-11-08 at 15:34 -0400, Jason Gunthorpe wrote:
> On Tue, Nov 08, 2022 at 10:29:33AM -0500, Eric Farman wrote:
> > On Tue, 2022-11-08 at 10:37 -0400, Jason Gunthorpe wrote:
> > > On Tue, Nov 08, 2022 at 09:19:17AM -0500, Eric Farman wrote:
> > > > On Tue, 2022-11-08 at 09:54 -0400, Jason Gunthorpe wrote:
> > > > > On Tue, Nov 08, 2022 at 08:50:53AM -0500, Matthew Rosato
> > > > > wrote:
> > > > > 
> > > > > > FWIW, vfio-pci via s390 is working fine so far, though I'll
> > > > > > put
> > > > > > it
> > > > > > through more paces over the next few weeks and report if I
> > > > > > find
> > > > > > anything.
> > > > > 
> > > > > OK great
> > > > > 
> > > > > > As far as mdev drivers...  
> > > > > > 
> > > > > > -ccw: Sounds like Eric is already aware there is an issue
> > > > > > and
> > > > > > is
> > > > > > investigating (I see errors as well).
> > > > 
> > > > I -think- the problem for -ccw is that the new vfio_pin_pages
> > > > requires
> > > > the input addresses to be page-aligned, and while most of ours
> > > > are,
> > > > the
> > > > first one in any given transaction may not be. We never
> > > > bothered to
> > > > mask off the addresses since it was handled for us, and we
> > > > needed
> > > > to
> > > > keep the offsets anyway.
> > > > 
> > > > By happenstance, I had some code that would do the masking
> > > > ourselves
> > > > (for an unrelated reason); I'll see if I can get that fit on
> > > > top
> > > > and if
> > > > it helps matters. After coffee.
> > > 
> > > Oh, yes, that makes alot of sense.
> > > 
> > > Ah, if that is how VFIO worked we could match it like below:
> > 
> > That's a start. The pin appears to have worked, but the unpin fails
> > at
> > the bottom of iommufd_access_unpin_pages:
> > 
> > WARN_ON(!iopt_area_contig_done(&iter));
> 
> This seems like a different bug, probably a ccw driver bug. The
> WARN_ON is designed to detect cases where the driver is unpinning an
> IOVA range that is not exactly what it pinned. The pin side already
> does this validation, so if it fails it means pin/unpin did not have
> identical iova ranges. Some debugging prints should confirm this.
> 
> I looked at CCW and came up with the following two things, can you
> look at them and finish them off? It will probably help.

I happen to already have patch 1 in a series I've been working on in
parallel with the private/parent split. I haven't forgotten it. :)

Patch 2 doesn't address the above symptoms, but a lot of that code is
getting reworked by the aforementioned series so I didn't spend a lot
of time studying your suggestion. And as I type this I see you just
sent a new patch, let me go try that...