[PATCH v10 15/41] KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion safety check

[Ackerley Tng via B4 Relay <devnull+ackerleytng.google.com@kernel.org>]

From: Ackerley Tng <ackerleytng@google.com>

A guest_memfd folio is safe for conversion if guest_memfd holds the last
references on it. Any other references on the folio may indicate another
user, and guest_memfd cannot convert it to private if there may be an
existing host user.

A folio will have extra refcounts if it is present in a per-CPU lru_add
fbatch. guest_memfd does not actually participate in LRU, but
freshly-allocated folios are still added to the lru_add fbatch for batch
LRU statistics processing.

This one known "usage" of the folio is handled by draining the lru_add
fbatch. After draining, if the refcount is still elevated, then there's
truly some other user of this page, and the page is not safe for
conversion.

If the page may be dma pinned, DMA is obviously using it and hence not safe
for conversions. If the page is still mapped after guest_memfd tried to
unmap it earlier in the conversion process, it is also obviously not safe
for conversion. Exit early to avoid unnecessary draining in these 2 cases.

Provide a drain status to only drain once ever while processing a batch of
folios.

Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Suggested-by: David Hildenbrand <david@kernel.org>
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
---
 mm/swap.c              |  2 ++
 virt/kvm/guest_memfd.c | 23 +++++++++++++++++++----
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/mm/swap.c b/mm/swap.c
index 8e965c8ce9aa9..9f511b97ab110 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -37,6 +37,7 @@
 #include <linux/page_idle.h>
 #include <linux/local_lock.h>
 #include <linux/buffer_head.h>
+#include <linux/kvm_types.h>
 
 #include "internal.h"
 
@@ -995,6 +996,7 @@ void lru_cache_drain_for_folio(const struct folio *folio,
 			*drained = LRU_CACHE_DRAINED_ALL;
 	}
 }
+EXPORT_SYMBOL_FOR_KVM(lru_cache_drain_for_folio);
 
 atomic_t lru_disable_count = ATOMIC_INIT(0);
 
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 896699afcad9d..030af0855f8b0 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -8,6 +8,7 @@
 #include <linux/mempolicy.h>
 #include <linux/pseudo_fs.h>
 #include <linux/pagemap.h>
+#include <linux/swap.h>
 
 #include "kvm_mm.h"
 #include "guest_memfd.h"
@@ -542,11 +543,26 @@ static int kvm_gmem_mas_preallocate(struct ma_state *mas, u64 attributes,
 	return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL);
 }
 
+static bool __folio_safe_for_conversion(struct folio *folio,
+					enum lru_cache_drained *drained)
+{
+	const int filemap_get_folios_refcount = 1;
+
+	if (folio_maybe_dma_pinned(folio) || folio_mapped(folio))
+		return false;
+
+	lru_cache_drain_for_folio(folio, filemap_get_folios_refcount,
+				  drained);
+
+	return folio_ref_count(folio) ==
+	       folio_nr_pages(folio) + filemap_get_folios_refcount;
+}
+
 static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
 					    size_t nr_pages, pgoff_t *err_index)
 {
+	enum lru_cache_drained drained = LRU_CACHE_NOT_DRAINED;
 	struct address_space *mapping = inode->i_mapping;
-	const int filemap_get_folios_refcount = 1;
 	pgoff_t last = start + nr_pages - 1;
 	struct folio_batch fbatch;
 	bool safe = true;
@@ -560,9 +576,8 @@ static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
 		for (i = 0; i < folio_batch_count(&fbatch); ++i) {
 			struct folio *folio = fbatch.folios[i];
 
-			if (folio_ref_count(folio) !=
-			    folio_nr_pages(folio) + filemap_get_folios_refcount) {
-				safe = false;
+			safe = __folio_safe_for_conversion(folio, &drained);
+			if (!safe) {
 				*err_index = max(start, folio->index);
 				break;
 			}

-- 
2.55.0.654.g21b8a5bc05-goog