RE: [PATCH 1/7] docs/migration: add uadk compression feature

[Shameerali Kolothum Thodi via <qemu-devel@nongnu.org>]

> -----Original Message-----
> From: Liu, Yuan1 <yuan1.liu@intel.com>
> Sent: Thursday, May 30, 2024 2:25 PM
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> peterx@redhat.com; farosas@suse.de
> Cc: qemu-devel@nongnu.org; Linuxarm <linuxarm@huawei.com>; linwenkai (C)
> <linwenkai6@hisilicon.com>; zhangfei.gao@linaro.org; huangchenghai
> <huangchenghai2@huawei.com>
> Subject: RE: [PATCH 1/7] docs/migration: add uadk compression feature
> 
> > -----Original Message-----
> > From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > Sent: Wednesday, May 29, 2024 5:44 PM
> > To: peterx@redhat.com; farosas@suse.de; Liu, Yuan1 <yuan1.liu@intel.com>
> > Cc: qemu-devel@nongnu.org; linuxarm@huawei.com;
> linwenkai6@hisilicon.com;
> > zhangfei.gao@linaro.org; huangchenghai2@huawei.com
> > Subject: [PATCH 1/7] docs/migration: add uadk compression feature

[...]

> > +Since UADK uses Shared Virtual Addressing(SVA) and device access virtual
> > memory
> > +directly it is possible that SMMUv3 may enounter page faults while
> > walking the
> > +IO page tables. This may impact the performance. In order to mitigate
> > this,
> > +please make sure to specify ``-mem-prealloc`` parameter to the
> > destination VM
> > +boot parameters.
> 
> Thank you so much for putting the IAA solution at the top and cc me.
> 
> I think migration performance will be better with '-mem-prealloc' option,
> but I am considering whether '-mem-prealloc' is a mandatory option, from my
> experience, SVA performance drops mainly caused by IOTLB flush and IO page
> fault,
> I had some discussions with Peter Xu about the IOTLB flush issue, and it has
> been improved.
> https://patchew.org/QEMU/PH7PR11MB5941F04FBFB964CB2C968866A33E2@
> PH7PR11MB5941.namprd11.prod.outlook.com/

Thanks for the link. Yes I have seen that discussion and this series is on top of  that
patch for avoiding the zero page read fault.

> 
> For IO page fault, the QPL(IAA userspace library) can process page fault
> request instead of IOMMU,

Sorry I didn't get this part completely. So if the page fault happens how the library
can handle it without IOMMU? Or you meant library will do memory perfecting before
to avoid the page fault?

 it means we can disable the I/O page fault feature
> on the IAA device, and let the device still use SVA technology to avoid memory
> copy.
> 
> I will provide the test results in my next version, do you have any ideas or
> suggestions about this, thanks.

I think our UADK test tool had an option to prefect the memory(write some random data
to memory) to avoid page fault penalty. I am not sure that is exposed through the API or not.
I will check with our UADK team.

Please do CC me when you post your next revision.

Thanks,
Shameer