Re: [Qemu-devel] [RFC PATCH 0/3] x86: Add support for guest DMA dirty page tracking

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Sun, Dec 13, 2015 at 01:28:09PM -0800, Alexander Duyck wrote:
> This patch set is meant to be the guest side code for a proof of concept
> involving leaving pass-through devices in the guest during the warm-up
> phase of guest live migration.  In order to accomplish this I have added a

What does that mean? 'warm-up-phase'? 

> new function called dma_mark_dirty that will mark the pages associated with
> the DMA transaction as dirty in the case of either an unmap or a
> sync_.*_for_cpu where the DMA direction is either DMA_FROM_DEVICE or
> DMA_BIDIRECTIONAL.  The pass-through device must still be removed before
> the stop-and-copy phase, however allowing the device to be present should
> significantly improve the performance of the guest during the warm-up
> period.

.. if the warm-up phase is short I presume? If the warm-up phase takes
a long time (busy guest that is of 1TB size) it wouldn't help much as the
tracking of these DMA's may be quite long?

> 
> This current implementation is very preliminary and there are number of
> items still missing.  Specifically in order to make this a more complete 
> solution we need to support:
> 1.  Notifying hypervisor that drivers are dirtying DMA pages received

.. And somehow giving the hypervisor the GPFN so it can retain the PFN in
the VT-d as long as possible.

> 2.  Bypassing page dirtying when it is not needed.

How would this work with with device doing DMA operations _after_ the migration?
That is the driver submits and DMA READ.. migrates away, device is unplugged,
VT-d context is torn down - device does the DMA READ gets an VT-d error...

and what then? How should the device on the other host replay the DMA READ?

> 
> The two mechanisms referenced above would likely require coordination with
> QEMU and as such are open to discussion.  I haven't attempted to address
> them as I am not sure there is a consensus as of yet.  My personal
> preference would be to add a vendor-specific configuration block to the
> emulated pci-bridge interfaces created by QEMU that would allow us to
> essentially extend shpc to support guest live migration with pass-through
> devices.

shpc?

> 
> The functionality in this patch set is currently disabled by default.  To
> enable it you can select "SWIOTLB page dirtying" from the "Processor type
> and features" menu.
> 
> ---
> 
> Alexander Duyck (3):
>       swiotlb: Fold static unmap and sync calls into calling functions
>       xen/swiotlb: Fold static unmap and sync calls into calling functions
>       x86: Create dma_mark_dirty to dirty pages used for DMA by VM guest
> 
> 
>  arch/arm/include/asm/dma-mapping.h       |    3 +
>  arch/arm64/include/asm/dma-mapping.h     |    5 +-
>  arch/ia64/include/asm/dma.h              |    1 
>  arch/mips/include/asm/dma-mapping.h      |    1 
>  arch/powerpc/include/asm/swiotlb.h       |    1 
>  arch/tile/include/asm/dma-mapping.h      |    1 
>  arch/unicore32/include/asm/dma-mapping.h |    1 
>  arch/x86/Kconfig                         |   11 ++++
>  arch/x86/include/asm/swiotlb.h           |   26 ++++++++
>  drivers/xen/swiotlb-xen.c                |   92 +++++++++++++-----------------
>  lib/swiotlb.c                            |   83 ++++++++++++---------------
>  11 files changed, 123 insertions(+), 102 deletions(-)
> 
> --