[PATCH 0/2] support device init for balloon pages

[PATCH 0/2] support device init for balloon pages

[Michael S. Tsirkin]

Once a page gets out of balloon (on deflate, or on access for
reporting) the Linux guest inits it before giving it to app.
It's a waste - the page is already inited by the host.

Add balloon support for skipping that, that provides significant
performance gains:
https://lore.kernel.org/all/cover.1776808209.git.mst@redhat.com/



Michael S. Tsirkin (2):
  balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
  balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_DEFLATED

 device-types/balloon/description.tex        | 108 ++++++++++++++++++++
 device-types/balloon/device-conformance.tex |   2 +
 device-types/balloon/driver-conformance.tex |   2 +
 3 files changed, 112 insertions(+)

-- 
MST

[PATCH 1/2] balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED

[Michael S. Tsirkin]

When the host processes reported pages it often naturally zeroes
them (e.g. MADV_DONTNEED), but the guest has no way to know this
and redundantly re-initializes them. Add a feature flag so the
device can report which pages it initialized.

The device may not be able to initialize every page (e.g.
unaligned regions). Use a per-entry bitmap so the device can
report non-contiguous partial success.

If PAGE_POISON is negotiated, fill with poison_val; otherwise
fill with zeros.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/242
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 device-types/balloon/description.tex        | 60 +++++++++++++++++++++
 device-types/balloon/device-conformance.tex |  1 +
 device-types/balloon/driver-conformance.tex |  1 +
 3 files changed, 62 insertions(+)

diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
index e7b957c..144983b 100644
--- a/device-types/balloon/description.tex
+++ b/device-types/balloon/description.tex
@@ -49,6 +49,12 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
 \item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
     page reporting. A virtqueue for reporting free guest memory is present.
 
+\item[ VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED(6) ] The device initializes
+    reported pages. If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated,
+    reported pages are filled with \field{poison_val}; otherwise, they
+    are filled with zeros. A bitmap indicating which pages were
+    successfully initialized is returned to the driver.
+
 \end{description}
 
 \drivernormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon Device / Feature bits}
@@ -63,6 +69,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
 it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
 negotiates VIRTIO_BALLOON_F_PAGE_POISON.
 
+The driver MUST NOT negotiate VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
+unless it also negotiates VIRTIO_BALLOON_F_PAGE_REPORTING.
+
 \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon Device / Feature bits}
 If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
 bit, and if the driver did not accept this feature bit, the
@@ -632,3 +641,54 @@ \subsubsection{Free Page Reporting}\label{sec:Device Types / Memory Balloon Devi
 If the VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the device
 MUST NOT modify the the content of a reported page to a value other than
 \field{poison_val}.
+
+\subsubsection{Device Initialized Reported Pages}\label{sec:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+Device Initialized Reported Pages provides a mechanism for the device
+to initialize reported pages on behalf of the driver. When this feature
+is negotiated, the device fills reported pages with the appropriate
+initialization value and returns a bitmap indicating which pages were
+successfully initialized.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device fills
+pages with \field{poison_val}; otherwise, the device fills pages
+with zeros.
+
+When VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED is negotiated, the format
+of buffers placed in the reporting_vq is modified: the driver
+prepends a device-writable bitmap buffer before the page buffers.
+The bitmap contains one bit per page buffer, where bit N corresponds
+to the Nth page buffer in the descriptor chain. The device sets a
+bit to 1 if it successfully initialized the corresponding page, or
+0 if it did not.
+
+\drivernormative{\paragraph}{Device Initialized Reported Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED feature has been negotiated.
+
+The driver MUST prepend a device-writable bitmap buffer to each
+reporting_vq descriptor chain, before the page buffers. The bitmap
+buffer MUST be at least $\lceil N/8 \rceil$ bytes, where N is the
+number of page buffers in the descriptor chain.
+
+The driver MUST NOT treat a reported page as initialized unless the
+corresponding bit in the bitmap is set to 1.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the driver
+need not initialize reported pages with \field{poison_val} before
+reporting them.
+
+\devicenormative{\paragraph}{Device Initialized Reported Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED feature has been negotiated.
+
+The device MUST set bit N in the bitmap to 1 if it successfully
+initialized page buffer N, or to 0 otherwise.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device
+MUST fill each successfully initialized page with \field{poison_val}.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
+MUST fill each successfully initialized page with zeros.
diff --git a/device-types/balloon/device-conformance.tex b/device-types/balloon/device-conformance.tex
index ff9ac4f..1d4df8c 100644
--- a/device-types/balloon/device-conformance.tex
+++ b/device-types/balloon/device-conformance.tex
@@ -9,4 +9,5 @@
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Free Page Hinting}
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Page Poison}
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Free Page Reporting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
 \end{itemize}
diff --git a/device-types/balloon/driver-conformance.tex b/device-types/balloon/driver-conformance.tex
index f898d74..6e5ccf0 100644
--- a/device-types/balloon/driver-conformance.tex
+++ b/device-types/balloon/driver-conformance.tex
@@ -9,4 +9,5 @@
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Free Page Hinting}
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Page Poison}
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Free Page Reporting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
 \end{itemize}
-- 
MST

[PATCH 2/2] balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_DEFLATED

[Michael S. Tsirkin]

Same idea as DEVICE_INIT_REPORTED but for deflation.
The device has been holding these pages since inflate, so
it can guarantee initialization of all of them. Make this
all-or-nothing: the device must not offer the feature if it
cannot initialize every page.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/242
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 device-types/balloon/description.tex        | 48 +++++++++++++++++++++
 device-types/balloon/device-conformance.tex |  1 +
 device-types/balloon/driver-conformance.tex |  1 +
 3 files changed, 50 insertions(+)

diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
index 144983b..7470763 100644
--- a/device-types/balloon/description.tex
+++ b/device-types/balloon/description.tex
@@ -55,6 +55,12 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
     are filled with zeros. A bitmap indicating which pages were
     successfully initialized is returned to the driver.
 
+\item[ VIRTIO_BALLOON_F_DEVICE_INIT_DEFLATED(7) ] The device initializes
+    deflated pages before returning them to the driver. If
+    VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, deflated pages
+    are filled with \field{poison_val}; otherwise, they are filled
+    with zeros.
+
 \end{description}
 
 \drivernormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon Device / Feature bits}
@@ -692,3 +698,45 @@ \subsubsection{Device Initialized Reported Pages}\label{sec:Device Types / Memor
 
 If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
 MUST fill each successfully initialized page with zeros.
+
+\subsubsection{Device Initialized Deflated Pages}\label{sec:Device Types / Memory Balloon Device / Device Operation / Device Initialized Deflated Pages}
+
+Device Initialized Deflated Pages provides a mechanism for the device
+to initialize deflated pages before returning them to the driver. When
+this feature is negotiated, the device fills all deflated pages with
+the appropriate initialization value before acknowledging the deflate
+request.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device fills
+pages with \field{poison_val}; otherwise, the device fills pages
+with zeros.
+
+\drivernormative{\paragraph}{Device Initialized Deflated Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Deflated Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_DEFLATED feature has been negotiated.
+
+The driver MUST NOT assume deflated pages are initialized until
+the device has acknowledged the deflate request.
+
+The driver MAY use deflated pages without initializing them, as they
+are guaranteed to be initialized once the deflate request is
+acknowledged.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the driver
+need not initialize deflated pages with \field{poison_val},
+as the device has already done so.
+
+\devicenormative{\paragraph}{Device Initialized Deflated Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Deflated Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_DEFLATED feature has been negotiated.
+
+The device MUST fill all deflated pages with the initialization value
+before acknowledging the deflate request.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device
+MUST fill each deflated page with \field{poison_val}.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
+MUST fill each deflated page with zeros.
diff --git a/device-types/balloon/device-conformance.tex b/device-types/balloon/device-conformance.tex
index 1d4df8c..dfa00c2 100644
--- a/device-types/balloon/device-conformance.tex
+++ b/device-types/balloon/device-conformance.tex
@@ -10,4 +10,5 @@
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Page Poison}
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Free Page Reporting}
 \item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Deflated Pages}
 \end{itemize}
diff --git a/device-types/balloon/driver-conformance.tex b/device-types/balloon/driver-conformance.tex
index 6e5ccf0..c35f6fa 100644
--- a/device-types/balloon/driver-conformance.tex
+++ b/device-types/balloon/driver-conformance.tex
@@ -10,4 +10,5 @@
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Page Poison}
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Free Page Reporting}
 \item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device Operation / Device Initialized Deflated Pages}
 \end{itemize}
-- 
MST