RE: [RFC PATCH v2 03/20] hw/arm/smmuv3-accel: Add initial infrastructure for smmuv3-accel device

[Shameerali Kolothum Thodi via <qemu-arm@nongnu.org>]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: Wednesday, March 19, 2025 5:01 PM
> To: Jason Gunthorpe <jgg@nvidia.com>; Donald Dutile
> <ddutile@redhat.com>
> Cc: Nicolin Chen <nicolinc@nvidia.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org;
> qemu-devel@nongnu.org; peter.maydell@linaro.org;
> berrange@redhat.com; nathanc@nvidia.com; mochs@nvidia.com;
> smostafa@google.com; Linuxarm <linuxarm@huawei.com>; Wangzhou (B)
> <wangzhou1@hisilicon.com>; jiangkunkun <jiangkunkun@huawei.com>;
> Jonathan Cameron <jonathan.cameron@huawei.com>;
> zhangfei.gao@linaro.org
> Subject: Re: [RFC PATCH v2 03/20] hw/arm/smmuv3-accel: Add initial
> infrastructure for smmuv3-accel device
> 
> Hi,
> 
> 
> On 3/19/25 1:23 AM, Jason Gunthorpe wrote:
> > On Tue, Mar 18, 2025 at 05:22:51PM -0400, Donald Dutile wrote:
> >
> >> I agree with Eric that 'accel' isn't needed -- this should be
> >> ascertained from the pSMMU that a physical device is attached to.
> > I seem to remember the point was made that we don't actually know if
> > accel is possible, or desired, especially in the case of hotplug.
> that's why I think it would be better if we could instantiate a single
> type of device that can do both accel and non accel mode.
> Maybe that would be at the price of always enforcing MSI resv regions on
> guest to assure MSI nesting is possible.
> 
> >
> > The accelerated mode has a number of limitations that the software
> > mode does not have. I think it does make sense that the user would
> > deliberately choose to use a more restrictive operating mode and then
> > would have to meet the requirements - eg by creating the required
> > number and configuration of vSMMUs.
> To avoid any misunderstanding I am not pushing for have a single vSMMU
> instance. I advocate for having several instances, each somehow
> specialized for VFIO devices or emulated devices. Maybe we can opt-in
> with accel=on but the default could be auto (the property can be
> AUTO_ON_OFF) where the code detects if a VFIO device is translated.In
> case incompatible devices are translated into a same vSMMU instance I
> guess it could be detected and will fail.
> 
> What I am pusshing for is to have a single type of QEMU device which can
> do both accel and non accel.
> > In general I advocate for having several vSMMU instances, each of them
> >
> >> Now... how does vfio(?; why not qemu?) layer determine that? --
> >> where are SMMUv3 'accel' features exposed either: a) in the device
> >> struct (for the smmuv3) or (b) somewhere under sysfs? ... I couldn't
> >> find anything under either on my g-h system, but would appreciate a
> >> ptr if there is.
> > I think it is not discoverable yet other thatn through
> > try-and-fail. Discoverability would probably be some bits in an
> > iommufd GET_INFO ioctl or something like that.
> yeah but at least we can easily detect if a VFIO device is beeing
> translated by a vSMMU instance in which case there is no other choice to
> turn accel on.

Not sure, how you can handle hotplug in such a case? For example if  the smmuv3
dev starts with an emulated device and later try plug a vfio dev? In case of "accel"
the feature bits(IIDR) is queried from the host SMMUv3 and is presented to
to the vSMMU(See patch  #16). We can't do this once Guest is booted.

Also Daniel previously commented on RFCv1 that he would like to have explicit
vSMMU<-->pSMMU association in Qemu command line. 
https://lore.kernel.org/qemu-devel/Z6TLSdwgajmHVmGH@redhat.com/

Though we are not there yet without a cold-plugged VFIO dev at the moment,
having auto detection of accel is not the right approach if we want an explicit
association in Qemu command line.

Thanks,
Shameer