[PATCH v2 0/4] x86/HVM: batch the flushes of grant unmaps

[PATCH v2 0/4] x86/HVM: batch the flushes of grant unmaps

[Luca Di Maio]

Hi all,

GNTTABOP_unmap_grant_ref already batches the TLB flushes of a PV
caller: gnttab_unmap_grant_ref() unmaps up to 32 grants, flushes the
TLBs once with gnttab_flush_tlb(), and only then releases the pages.
gnttab_flush_tlb() does nothing for a translated caller, whose unmaps
go through the p2m, and the p2m code flushes the guest TLBs and the
IOTLB for every page instead. A PVH dom0 running netback pays both
flushes for every page a domU transmits.

This series gives translated callers the same batching, for both
flushes, before the pages are released. The IOTLB half has no PV
counterpart: a PV dom0's IOMMU maps all of RAM by default, so its
grant unmaps don't touch it. Patches 1 and 2 are preparatory, and
patches 3 and 4 have the details.

iperf3 between a PVH dom0 running netback and two PVH domUs:

  machine:  c7i.metal-24xl, Xeon Platinum 8488C, 96 threads,
            VT-d with shared EPT tables
  dom0:     96 pinned vCPUs, 4 GiB, Debian 13, Linux 6.12
  domUs:    8 vCPUs, 1 GiB, Linux 6.18, on a bridge
  Xen:      staging at 9a6cc49b44, built with GCC 14.2
  runs:     10 s, Gbit/s received, higher is better, 60 per row
            from 4 boots of each build, interleaved
  shown:    mean of the runs +/- their relative standard deviation

                   base          patched       %change
  domU to dom0
    1 stream       0.64 +/-1%    3.11 +/-2%      +386%
    2 streams      0.81 +/-6%    3.16 +/-2%      +291%
    4 streams      0.84 +/-4%    2.69 +/-7%      +221%
    8 streams      0.83 +/-2%    2.40 +/-5%      +190%
    16 streams     0.81 +/-2%    2.23 +/-2%      +175%
  domU to domU
    1 stream       0.62 +/-6%    2.29 +/-5%      +267%
    2 streams      0.73 +/-7%    2.26 +/-10%     +211%
    4 streams      0.75 +/-4%    1.86 +/-16%     +150%
    8 streams      0.73 +/-4%    1.73 +/-9%      +136%
    16 streams     0.71 +/-3%    1.62 +/-6%      +128%
  dom0 to domU
    1 stream       4.19 +/-6%    3.93 +/-8%      -6.2%
    2 streams      4.87 +/-9%    4.68 +/-11%     -4.0%
    4 streams      8.73 +/-21%   8.34 +/-19%     -4.5%
    8 streams     12.53 +/-17%  13.05 +/-17%     +4.1%
    16 streams    16.85 +/-12%  16.40 +/-16%     -2.7%

The table comes from one of 13 sessions on this instance type, which
ran 7,779 runs over 57 boots in total.

A domU transmits 2.2 to 4.8 times as much, and 2.2 to 4.0 times as
much per busy dom0 vCPU, where base never exceeds 0.84 Gbit/s whatever
the number of streams.

From dom0 to a domU, netback fills the domU's buffers with grant
copies and maps few of its packets, so the series barely runs there
and the differences stay within the margin of error.

Between boots of the same build, the single-stream mean varied no
more than the run-to-run spread predicts. A debug build ran the same
benchmark without assertion failures or IOMMU faults.

Changes in v2:
- Patch 1 is new: VT-d rounds flush ranges to an aligned power of two
  itself, instead of the grant unmap batch doing it (Teddy).
- iommu_iotlb_flush() checks iommu_dont_flush_iotlb and records the
  range as well as the flags of the flushes it skips, instead of its
  callers recording the flags in a uint8_t (Teddy, Jan).
- The grant unmap batch flushes the recorded range, instead of the span
  of the unmapped gfns (Teddy).
- The table's patched column is now re-run against v2

Thanks,
L

Luca Di Maio (4):
  VT-d: round IOTLB flush ranges up to an aligned power of two
  IOMMU: record the flushes skipped under iommu_dont_flush_iotlb
  x86/HVM: defer IOTLB flushes across a grant unmap batch
  x86/HVM: defer p2m TLB flushes across a grant unmap batch

 xen/arch/arm/include/asm/grant_table.h     |  3 ++
 xen/arch/x86/hvm/grant_table.c             | 55 +++++++++++++++++++++-
 xen/arch/x86/include/asm/grant_table.h     | 13 +++++
 xen/arch/x86/include/asm/hvm/grant_table.h |  5 ++
 xen/arch/x86/include/asm/hvm/vcpu.h        | 10 ++++
 xen/arch/x86/mm/p2m-ept.c                  |  2 +-
 xen/common/grant_table.c                   |  6 +++
 xen/drivers/passthrough/iommu.c            | 17 ++++++-
 xen/drivers/passthrough/vtd/iommu.c        |  4 ++
 xen/include/xen/iommu.h                    |  6 +++
 10 files changed, 117 insertions(+), 4 deletions(-)


base-commit: 41ddcfe808ec1228289ae60c30768d50a58c98fe
-- 
2.47.3

[PATCH v2 1/4] VT-d: round IOTLB flush ranges up to an aligned power of two

[Luca Di Maio]

Page-selective invalidation takes only aligned power-of-two ranges, so
iommu_flush_iotlb() flushes any other range as the whole domain. A
later patch flushes one range per group of grant unmaps, which rarely
has that shape.

Round the range up to the smallest aligned power of two covering it.
Orders the IOMMU doesn't support still get a domain-selective flush,
and device TLB invalidations use the same range.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Luca Di Maio <luca@edera.dev>

---
Changes in v2:
- New patch, taking the rounding out of the grant unmap batch (Teddy).
---
 xen/drivers/passthrough/vtd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index b41a1d41c8..faf8255f26 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -742,6 +742,10 @@ static int __must_check cf_check iommu_flush_iotlb(struct domain *d, dfn_t dfn,
     {
         ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
         ASSERT(flush_flags);
+
+        /* PSI takes only aligned powers of two: cover the range with one. */
+        page_count = 1UL << flsl(dfn_x(dfn) ^ (dfn_x(dfn) + page_count - 1));
+        dfn = _dfn(dfn_x(dfn) & ~(page_count - 1));
     }
 
     /*
-- 
2.47.3

[PATCH v2 2/4] IOMMU: record the flushes skipped under iommu_dont_flush_iotlb

[Luca Di Maio]

While iommu_dont_flush_iotlb is set, iommu_legacy_map(),
iommu_legacy_unmap() and the shared-EPT path of ept_set_entry() skip
their IOTLB flush and drop its range and flags, so a caller deferring
the flush has to work out what to flush on its own.

Check the flag in iommu_iotlb_flush() instead, which all three call,
and record what it skips in the per-CPU iommu_skipped_flush: the
smallest range covering the skipped flushes, and their flags. No other
caller of iommu_iotlb_flush() runs while the flag is set, and the code
setting it clears it before its own flush. The next patch uses the
record for grant unmaps.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Luca Di Maio <luca@edera.dev>

---
Changes in v2:
- Check the flag in iommu_iotlb_flush() rather than in its callers,
  which also drops the duplicated flags in ept_set_entry() (Teddy).
- Record the range of the skipped flushes as well as their flags
  (Teddy).
- Store the flags as unsigned int in struct iommu_skipped_flush, instead
  of a uint8_t checked by a BUILD_BUG_ON() (Teddy, Jan).
---
 xen/arch/x86/mm/p2m-ept.c       |  2 +-
 xen/drivers/passthrough/iommu.c | 17 +++++++++++++++--
 xen/include/xen/iommu.h         |  6 ++++++
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 204bdb054a..33c3d8fc58 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -976,7 +976,7 @@ out:
     if ( rc == 0 && p2m_is_hostp2m(p2m) &&
          need_modify_vtd_table )
     {
-        if ( iommu_use_hap_pt(d) && !this_cpu(iommu_dont_flush_iotlb) )
+        if ( iommu_use_hap_pt(d) )
             rc = iommu_iotlb_flush(d, _dfn(gfn), 1ul << order,
                                    (iommu_flags ? IOMMU_FLUSHF_added : 0) |
                                    (vtd_pte_present ? IOMMU_FLUSHF_modified
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 4b22ecfe65..8e97086601 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -64,6 +64,7 @@ bool __read_mostly iommu_hap_pt_share = true;
 bool __read_mostly iommu_debug;
 
 DEFINE_PER_CPU(bool, iommu_dont_flush_iotlb);
+DEFINE_PER_CPU(struct iommu_skipped_flush, iommu_skipped_flush);
 
 static int __init cf_check parse_iommu_param(const char *s)
 {
@@ -390,7 +391,7 @@ int iommu_legacy_map(struct domain *d, dfn_t dfn, mfn_t mfn,
     ASSERT(!(flags & IOMMUF_preempt));
     rc = iommu_map(d, dfn, mfn, page_count, flags, &flush_flags);
 
-    if ( !this_cpu(iommu_dont_flush_iotlb) && !rc )
+    if ( !rc )
         rc = iommu_iotlb_flush(d, dfn, page_count, flush_flags);
 
     return rc;
@@ -458,7 +459,7 @@ int iommu_legacy_unmap(struct domain *d, dfn_t dfn, unsigned long page_count)
     unsigned int flush_flags = 0;
     int rc = iommu_unmap(d, dfn, page_count, 0, &flush_flags);
 
-    if ( !this_cpu(iommu_dont_flush_iotlb) && !rc )
+    if ( !rc )
         rc = iommu_iotlb_flush(d, dfn, page_count, flush_flags);
 
     return rc;
@@ -488,6 +489,18 @@ int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned long page_count,
     if ( dfn_eq(dfn, INVALID_DFN) )
         return -EINVAL;
 
+    if ( this_cpu(iommu_dont_flush_iotlb) )
+    {
+        struct iommu_skipped_flush *skipped = &this_cpu(iommu_skipped_flush);
+
+        skipped->first = _dfn(min(dfn_x(skipped->first), dfn_x(dfn)));
+        skipped->last = _dfn(max(dfn_x(skipped->last),
+                                 dfn_x(dfn) + page_count - 1));
+        skipped->flags |= flush_flags;
+
+        return 0;
+    }
+
     rc = iommu_call(hd->platform_ops, iotlb_flush, d, dfn, page_count,
                     flush_flags);
     if ( unlikely(rc) )
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 37c4a1dc82..a2aeaf8d41 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -502,6 +502,12 @@ void iommu_dev_iotlb_flush_timeout(struct domain *d, struct pci_dev *pdev);
  */
 DECLARE_PER_CPU(bool, iommu_dont_flush_iotlb);
 
+struct iommu_skipped_flush {
+    dfn_t first, last;
+    unsigned int flags;
+};
+DECLARE_PER_CPU(struct iommu_skipped_flush, iommu_skipped_flush);
+
 bool arch_iommu_use_permitted(const struct domain *d);
 
 #ifdef CONFIG_X86
-- 
2.47.3

[PATCH v2 3/4] x86/HVM: defer IOTLB flushes across a grant unmap batch

[Luca Di Maio]

gnttab_unmap_grant_ref() flushes the TLBs of a PV caller once per group
of GNTTAB_UNMAP_BATCH_SIZE unmaps. A translated caller instead gets an
IOTLB flush per page from the p2m code, so netback in a PVH dom0 issues
one for every page a domU transmits.

Open a batch for translated callers before each group, and end it next
to gnttab_flush_tlb(), so before unmap_common_complete() releases the
pages. Ending the batch issues one flush covering the flushes the
unmaps skipped, as iommu_skipped_flush recorded them.

The batch lives in the vCPU, and iommu_dont_flush_iotlb isn't held
across the guest copies between the unmaps, as those can sleep on a
vm_event ring and change the p2m, e.g. by unsharing a page. If the
flush fails, the pages may still be reachable through the IOMMU, so
the group isn't completed and its pages leak.

PV callers, Arm, and other callers of replace_grant_host_mapping() are
unchanged.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Luca Di Maio <luca@edera.dev>

---
Changes in v2:
- Flush the range recorded in iommu_skipped_flush, rather than one
  derived from the unmapped gfns (Teddy).
- Leave rounding the range to an aligned power of two to VT-d, in
  patch 1 (Teddy).
---
 xen/arch/arm/include/asm/grant_table.h     |  3 ++
 xen/arch/x86/hvm/grant_table.c             | 49 +++++++++++++++++++++-
 xen/arch/x86/include/asm/grant_table.h     | 13 ++++++
 xen/arch/x86/include/asm/hvm/grant_table.h |  5 +++
 xen/arch/x86/include/asm/hvm/vcpu.h        |  9 ++++
 xen/common/grant_table.c                   |  6 +++
 6 files changed, 84 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/include/asm/grant_table.h b/xen/arch/arm/include/asm/grant_table.h
index 7661a95341..66f90083db 100644
--- a/xen/arch/arm/include/asm/grant_table.h
+++ b/xen/arch/arm/include/asm/grant_table.h
@@ -40,6 +40,9 @@ int create_grant_host_mapping(uint64_t gpaddr, mfn_t frame,
 int replace_grant_host_mapping(uint64_t gpaddr, mfn_t frame,
                                uint64_t new_gpaddr, unsigned int flags);
 
+static inline void gnttab_unmap_batch_begin(void) {}
+static inline int gnttab_unmap_batch_end(void) { return 0; }
+
 /*
  * The region used by Xen on the memory will never be mapped in DOM0
  * memory layout. Therefore it can be used for the grant table.
diff --git a/xen/arch/x86/hvm/grant_table.c b/xen/arch/x86/hvm/grant_table.c
index afe449d888..5ecba6db65 100644
--- a/xen/arch/x86/hvm/grant_table.c
+++ b/xen/arch/x86/hvm/grant_table.c
@@ -41,6 +41,8 @@ int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
     p2m_type_t type;
     mfn_t old_mfn;
     struct domain *d = current->domain;
+    struct hvm_grant_unmap_batch *batch = &current->arch.hvm.grant_unmap_batch;
+    int rc;
 
     if ( new_addr != 0 || (flags & GNTMAP_contains_pte) )
         return GNTST_general_error;
@@ -54,7 +56,29 @@ int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
                  type, mfn_x(old_mfn), mfn_x(frame));
         return GNTST_general_error;
     }
-    if ( p2m_remove_page(d, _gfn(gfn), frame, PAGE_ORDER_4K) )
+
+    /* Not across the guest copies, which can change the p2m or sleep. */
+    if ( batch->open )
+    {
+        this_cpu(iommu_skipped_flush) =
+            (struct iommu_skipped_flush){ .first = INVALID_DFN };
+        this_cpu(iommu_dont_flush_iotlb) = true;
+    }
+
+    rc = p2m_remove_page(d, _gfn(gfn), frame, PAGE_ORDER_4K);
+
+    if ( batch->open )
+    {
+        const struct iommu_skipped_flush *skipped =
+            &this_cpu(iommu_skipped_flush);
+
+        this_cpu(iommu_dont_flush_iotlb) = false;
+        batch->iommu_flags |= skipped->flags;
+        batch->dfn_first = min(batch->dfn_first, dfn_x(skipped->first));
+        batch->dfn_last = max(batch->dfn_last, dfn_x(skipped->last));
+    }
+
+    if ( rc )
     {
         put_gfn(d, gfn);
         return GNTST_general_error;
@@ -64,6 +88,29 @@ int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
     return GNTST_okay;
 }
 
+void grant_p2m_unmap_batch_begin(void)
+{
+    struct hvm_grant_unmap_batch *batch = &current->arch.hvm.grant_unmap_batch;
+
+    ASSERT(!batch->open);
+    *batch = (struct hvm_grant_unmap_batch){ .open = true, .dfn_first = ~0UL };
+}
+
+int grant_p2m_unmap_batch_end(void)
+{
+    struct domain *d = current->domain;
+    struct hvm_grant_unmap_batch *batch = &current->arch.hvm.grant_unmap_batch;
+
+    batch->open = false;
+
+    if ( !batch->iommu_flags )
+        return 0;
+
+    return iommu_iotlb_flush(d, _dfn(batch->dfn_first),
+                             batch->dfn_last - batch->dfn_first + 1,
+                             batch->iommu_flags);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/include/asm/grant_table.h b/xen/arch/x86/include/asm/grant_table.h
index e939c9bacb..3aae47092f 100644
--- a/xen/arch/x86/include/asm/grant_table.h
+++ b/xen/arch/x86/include/asm/grant_table.h
@@ -32,6 +32,19 @@ static inline int replace_grant_host_mapping(uint64_t addr, mfn_t frame,
     return replace_grant_pv_mapping(addr, frame, new_addr, flags);
 }
 
+static inline void gnttab_unmap_batch_begin(void)
+{
+    if ( paging_mode_external(current->domain) )
+        grant_p2m_unmap_batch_begin();
+}
+
+static inline int __must_check gnttab_unmap_batch_end(void)
+{
+    if ( paging_mode_external(current->domain) )
+        return grant_p2m_unmap_batch_end();
+    return 0;
+}
+
 #define gnttab_get_frame_gfn(gt, st, idx) ({                             \
     mfn_t mfn_ = (st) ? gnttab_status_mfn(gt, idx)                       \
                       : gnttab_shared_mfn(gt, idx);                      \
diff --git a/xen/arch/x86/include/asm/hvm/grant_table.h b/xen/arch/x86/include/asm/hvm/grant_table.h
index 01e23f79b8..9877262f51 100644
--- a/xen/arch/x86/include/asm/hvm/grant_table.h
+++ b/xen/arch/x86/include/asm/hvm/grant_table.h
@@ -19,6 +19,8 @@ int create_grant_p2m_mapping(uint64_t addr, mfn_t frame,
                              unsigned int cache_flags);
 int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
                               uint64_t new_addr, unsigned int flags);
+void grant_p2m_unmap_batch_begin(void);
+int grant_p2m_unmap_batch_end(void);
 
 #else
 
@@ -37,6 +39,9 @@ static inline int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
     return GNTST_general_error;
 }
 
+static inline void grant_p2m_unmap_batch_begin(void) {}
+static inline int grant_p2m_unmap_batch_end(void) { return 0; }
+
 #endif
 
 #endif /* __X86_HVM_GRANT_TABLE_H__ */
diff --git a/xen/arch/x86/include/asm/hvm/vcpu.h b/xen/arch/x86/include/asm/hvm/vcpu.h
index 876c230784..990411e2c7 100644
--- a/xen/arch/x86/include/asm/hvm/vcpu.h
+++ b/xen/arch/x86/include/asm/hvm/vcpu.h
@@ -108,6 +108,13 @@ struct altp2mvcpu {
 
 #define vcpu_altp2m(v) ((v)->arch.hvm.avcpu)
 
+/* Per vCPU, as the guest copies between the unmaps can sleep. */
+struct hvm_grant_unmap_batch {
+    bool          open;
+    unsigned int  iommu_flags;
+    unsigned long dfn_first, dfn_last;
+};
+
 struct hvm_vcpu {
     /* Guest control-register and EFER values, just as the guest sees them. */
     unsigned long       guest_cr[5];
@@ -172,6 +179,8 @@ struct hvm_vcpu {
     /* Pending hw/sw interrupt (.vector = -1 means nothing pending). */
     struct x86_event     inject_event;
 
+    struct hvm_grant_unmap_batch grant_unmap_batch;
+
 #ifdef CONFIG_VIRIDIAN
     struct viridian_vcpu *viridian;
 #endif
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index b15805f3f9..ade42b4cb4 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1722,6 +1722,7 @@ gnttab_unmap_grant_ref(
         c = min(count, (unsigned int)GNTTAB_UNMAP_BATCH_SIZE);
         partial_done = 0;
 
+        gnttab_unmap_batch_begin();
         for ( i = 0; i < c; i++ )
         {
             if ( unlikely(__copy_from_guest(&op, uop, 1)) )
@@ -1734,6 +1735,9 @@ gnttab_unmap_grant_ref(
         }
 
         gnttab_flush_tlb(current->domain);
+        /* After a failed flush the pages may still be reachable: leak them. */
+        if ( gnttab_unmap_batch_end() )
+            partial_done = 0;
 
         for ( i = 0; i < partial_done; i++ )
             unmap_common_complete(&common[i]);
@@ -1749,6 +1753,8 @@ gnttab_unmap_grant_ref(
 
 fault:
     gnttab_flush_tlb(current->domain);
+    if ( gnttab_unmap_batch_end() )
+        partial_done = 0;
 
     for ( i = 0; i < partial_done; i++ )
         unmap_common_complete(&common[i]);
-- 
2.47.3

[PATCH v2 4/4] x86/HVM: defer p2m TLB flushes across a grant unmap batch

[Luca Di Maio]

A grant unmap by a translated caller also flushes the guest TLBs:
p2m_remove_page() sets need_flush of the host p2m, and put_gfn()
flushes when it drops the last level of the p2m lock. Each flush sends
an IPI to every other CPU in d->dirty_cpumask and waits for all of
them, so a PVH dom0 with busy vCPUs pays a round of IPIs for every page
it unmaps.

Defer that flush to the end of the batch, still before the pages are
released. Under the gfn lock of get_gfn_query() only the removal can
set need_flush, as the query neither allocates nor unshares, so no
other change's flush is deferred. p2m_free_ptp() still flushes before
it frees a table.

The final flush reaches the CPUs in d->dirty_cpumask at that time. A
CPU that left the mask in between has flushed already, as
__context_switch() clears it from the mask only after write_ptbase()
has flushed the guest TLBs. With EPT, each change also marks every CPU
for an INVEPT before its next VM entry.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Luca Di Maio <luca@edera.dev>

---
Changes in v2:
- None.
---
 xen/arch/x86/hvm/grant_table.c      | 6 ++++++
 xen/arch/x86/include/asm/hvm/vcpu.h | 1 +
 2 files changed, 7 insertions(+)

diff --git a/xen/arch/x86/hvm/grant_table.c b/xen/arch/x86/hvm/grant_table.c
index 5ecba6db65..7a2201d926 100644
--- a/xen/arch/x86/hvm/grant_table.c
+++ b/xen/arch/x86/hvm/grant_table.c
@@ -41,6 +41,7 @@ int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
     p2m_type_t type;
     mfn_t old_mfn;
     struct domain *d = current->domain;
+    struct p2m_domain *p2m = p2m_get_hostp2m(d);
     struct hvm_grant_unmap_batch *batch = &current->arch.hvm.grant_unmap_batch;
     int rc;
 
@@ -76,6 +77,8 @@ int replace_grant_p2m_mapping(uint64_t addr, mfn_t frame,
         batch->iommu_flags |= skipped->flags;
         batch->dfn_first = min(batch->dfn_first, dfn_x(skipped->first));
         batch->dfn_last = max(batch->dfn_last, dfn_x(skipped->last));
+        batch->tlb_flush |= p2m->need_flush;
+        p2m->need_flush = false;
     }
 
     if ( rc )
@@ -99,9 +102,12 @@ void grant_p2m_unmap_batch_begin(void)
 int grant_p2m_unmap_batch_end(void)
 {
     struct domain *d = current->domain;
+    struct p2m_domain *p2m = p2m_get_hostp2m(d);
     struct hvm_grant_unmap_batch *batch = &current->arch.hvm.grant_unmap_batch;
 
     batch->open = false;
+    if ( batch->tlb_flush )
+        p2m->tlb_flush(p2m);
 
     if ( !batch->iommu_flags )
         return 0;
diff --git a/xen/arch/x86/include/asm/hvm/vcpu.h b/xen/arch/x86/include/asm/hvm/vcpu.h
index 990411e2c7..ff6856928d 100644
--- a/xen/arch/x86/include/asm/hvm/vcpu.h
+++ b/xen/arch/x86/include/asm/hvm/vcpu.h
@@ -111,6 +111,7 @@ struct altp2mvcpu {
 /* Per vCPU, as the guest copies between the unmaps can sleep. */
 struct hvm_grant_unmap_batch {
     bool          open;
+    bool          tlb_flush;
     unsigned int  iommu_flags;
     unsigned long dfn_first, dfn_last;
 };
-- 
2.47.3