[PATCH v3 0/2] balloon: DEVICE_INIT_REPORTED and DEVICE_INIT_ON_INFLATE

[PATCH v3 0/2] balloon: DEVICE_INIT_REPORTED and DEVICE_INIT_ON_INFLATE

[Michael S. Tsirkin]

Once a page gets out of balloon (on deflate, or on access for
reporting) the Linux guest inits it before giving it to app.
It's a waste - the page is already inited by the host.

Add balloon support for skipping that, that provides significant
performance gains.

This adds two new feature bits for the balloon device that allow the
device to signal that it has initialized (zeroed or poison-filled) pages
on behalf of the driver.

The variant with poison does not really seem useful, so I am still
trying to figure out whether we should simply outlaw it.


Old implementations (new ones will be posted ASAP):
QEMU: https://lore.kernel.org/all/cover.1777824548.git.mst@redhat.com
Linux: https://lore.kernel.org/all/cover.1777803445.git.mst@redhat.com

Tested with QEMU + Linux guest: 232ms -> 51ms (-78%) for 256MB
page allocation in a 2GB VM.

Changes since v2:
- I realized we can't really avoid a new feature bit:
  poison is a promise from driver to initialize pages.
- lots of other changes

Changes v1->v2:
- reporting can actually just utilize used length to see what
  was initialized! so we do not need a bitmap for that.
- qemu really needs a way to skip init for some pages.
  Add that to inflate path.
  Add security considerations (for coco).

Michael S. Tsirkin (2):
  balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
  balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE

 device-types/balloon/description.tex | 147 ++++++++++++++++++++++++++-
 1 file changed, 145 insertions(+), 2 deletions(-)

--
MST

[PATCH v3 1/2] balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED

[Michael S. Tsirkin]

Add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED (bit 6): the device
initializes reported pages and signals success via the used length
of each reporting_vq element.

Add Security Considerations section covering information leakage,
data integrity, and denial of service.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/244
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 device-types/balloon/description.tex | 85 +++++++++++++++++++++++++++-
 1 file changed, 83 insertions(+), 2 deletions(-)

diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
index a1d9603..ce08a4a 100644
--- a/device-types/balloon/description.tex
+++ b/device-types/balloon/description.tex
@@ -49,6 +49,18 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
 \item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
     page reporting. A virtqueue for reporting free guest memory is present.
 
+\item[ VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED(6) ] The device initializes
+    reported pages.
+
+\end{description}
+
+\subsubsection{Feature bit requirements}\label{sec:Device Types / Memory Balloon Device / Feature bits / Feature bit requirements}
+
+Some balloon feature bits require other balloon feature bits
+(see \ref{drivernormative:Basic Facilities of a Virtio Device / Feature Bits}):
+
+\begin{description}
+\item[VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED] Requires VIRTIO_BALLOON_F_PAGE_REPORTING.
 \end{description}
 
 \drivernormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon Device / Feature bits}
@@ -61,7 +73,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
 
 If the driver is expecting the pages to retain some initialized value,
 it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
-negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+negotiates VIRTIO_BALLOON_F_PAGE_POISON or
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED.
 
 \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon Device / Feature bits}
 If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
@@ -625,10 +638,78 @@ \subsubsection{Free Page Reporting}\label{sec:Device Types / Memory Balloon Devi
 Normative statements in this section apply if the
 VIRTIO_BALLOON_F_PAGE_REPORTING feature has been negotiated.
 
-If the VIRTIO_BALLOON_F_PAGE_POISON feature has not been negotiated, the
+If neither the VIRTIO_BALLOON_F_PAGE_POISON nor
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED feature has been negotiated, the
 device MAY modify the contents of any page supplied in a report request
 before acknowledging that request by using the reporting_vq descriptor.
 
 If the VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the device
 MUST NOT modify the the content of a reported page to a value other than
 \field{poison_val}.
+
+\subsubsection{Device Initialized Reported Pages}\label{sec:Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+When VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED is negotiated, the device
+initializes reported pages by writing into each byte of the page.
+
+\drivernormative{\paragraph}{Device Initialized Reported Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED feature has been negotiated.
+
+The driver MUST NOT treat a reported page as initialized unless
+the used length covers it.
+
+\devicenormative{\paragraph}{Device Initialized Reported Pages}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Reported Pages}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED feature has been negotiated.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device
+MUST fill each initialized page with \field{poison_val}.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
+MUST fill each initialized page with zeros.
+
+\subsection{Security Considerations}\label{sec:Device Types / Memory Balloon Device / Security Considerations}
+
+In environments where the device is not trusted, such as
+confidential computing, the balloon device presents several
+security concerns.
+
+\subsubsection{Information Leakage}\label{sec:Device Types / Memory Balloon Device / Security Considerations / Information Leakage}
+
+When the driver inflates the balloon, the addresses of the pages
+are communicated to the device. On some systems, the page contents
+themselves are also accessible to the device. These pages may
+contain sensitive data from previous use by the guest. In systems
+where the device is untrusted, the driver should sanitize pages
+before inflating them.
+
+Free page hinting and free page reporting similarly expose
+information about which pages are free, and in the case of
+reporting, may make page contents accessible to the device. Memory
+statistics reporting exposes guest memory usage patterns.
+
+\subsubsection{Data Integrity}\label{sec:Device Types / Memory Balloon Device / Security Considerations / Data Integrity}
+
+When the device is untrusted, the driver cannot rely on the device
+to correctly initialize pages. An untrusted device can report a
+non-zero used length for VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
+without actually initializing the corresponding pages. If the
+driver treats such pages as initialized (e.g., marking them as
+zeroed), this could lead to information leakage between guest
+processes or other security violations.
+
+Drivers operating in environments with untrusted devices are
+expected to avoid negotiating
+VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, and to initialize
+all pages themselves.
+
+\subsubsection{Denial of Service}\label{sec:Device Types / Memory Balloon Device / Security Considerations / Denial of Service}
+
+A malicious device could set \field{num_pages} to an
+unreasonably large value, causing the driver to surrender most
+of the guest's memory. Drivers should apply reasonable limits
+and not inflate the balloon to the point where the guest
+becomes unable to function.
-- 
MST

[PATCH v3 2/2] balloon: add VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE

[Michael S. Tsirkin]

Add VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE (bit 7): the device
initializes inflated pages and returns a per-page bitmap indicating
which pages were successfully initialized.

Update Security Considerations Data Integrity subsection to cover
both DEVICE_INIT_REPORTED and DEVICE_INIT_ON_INFLATE.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/242
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 device-types/balloon/description.tex | 66 +++++++++++++++++++++++++++-
 1 file changed, 64 insertions(+), 2 deletions(-)

diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
index ce08a4a..bce3b9e 100644
--- a/device-types/balloon/description.tex
+++ b/device-types/balloon/description.tex
@@ -52,6 +52,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device / Featu
 \item[ VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED(6) ] The device initializes
     reported pages.
 
+\item[ VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE(7) ] The device initializes
+    inflated pages.
+
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Memory Balloon Device / Feature bits / Feature bit requirements}
@@ -671,6 +674,64 @@ \subsubsection{Device Initialized Reported Pages}\label{sec:Device Types / Memor
 If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
 MUST fill each initialized page with zeros.
 
+\subsubsection{Device Initialized Pages on Inflate}\label{sec:Device Types / Memory Balloon Device / Device Operation / Device Initialized Pages on Inflate}
+
+When VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE is negotiated, the
+device initializes inflated pages and reports which pages were
+successfully initialized via a per-page bitmap.
+
+The driver appends a device-writable bitmap buffer to each inflate
+descriptor chain, after the PFN outbuf. The bitmap contains one
+bit per balloon page (4KB), where bit N corresponds to the Nth PFN
+in the inflate request (0-indexed). Bit N is stored as bit (N mod 8)
+of byte (N / 8), with bit 0 being the least significant bit.
+The device sets a bit to 1 if it successfully initialized the
+corresponding page, or 0 if it did not.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device fills
+pages with \field{poison_val}; otherwise, the device fills pages
+with zeros.
+
+When the driver later deflates these pages, it may skip
+initialization for pages that were marked as initialized in the
+inflate bitmap.
+
+\drivernormative{\paragraph}{Device Initialized Pages on Inflate}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Pages on Inflate}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE feature has been negotiated.
+
+The driver MUST append a device-writable bitmap buffer to each
+inflate descriptor chain. The bitmap buffer MUST be at least
+$\lceil N/8 \rceil$ bytes, where N is the number of PFNs in the
+inflate request.
+
+The driver MUST NOT treat an inflated page as initialized unless
+the corresponding bit in the bitmap is set to 1.
+
+The driver MAY skip initializing a deflated page if the
+corresponding inflate bitmap bit was set to 1. This overrides
+the VIRTIO_BALLOON_F_PAGE_POISON requirement to initialize
+deflated pages with \field{poison_val} for such pages.
+
+\devicenormative{\paragraph}{Device Initialized Pages on Inflate}{Device Types / Memory Balloon Device / Device Operation / Device Initialized Pages on Inflate}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE feature has been negotiated.
+
+The device MUST set bit N in the bitmap to 1 if it successfully
+initialized PFN N, or to 0 otherwise.
+
+The device MUST set all padding bits in the last byte of the bitmap
+to 0. The device MAY write more bytes than $\lceil N/8 \rceil$ but
+MUST set all extra bits to 0.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is also negotiated, the device
+MUST fill each successfully initialized page with \field{poison_val}.
+
+If VIRTIO_BALLOON_F_PAGE_POISON is not negotiated, the device
+MUST fill each successfully initialized page with zeros.
+
 \subsection{Security Considerations}\label{sec:Device Types / Memory Balloon Device / Security Considerations}
 
 In environments where the device is not trusted, such as
@@ -696,14 +757,15 @@ \subsubsection{Data Integrity}\label{sec:Device Types / Memory Balloon Device /
 When the device is untrusted, the driver cannot rely on the device
 to correctly initialize pages. An untrusted device can report a
 non-zero used length for VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
+or set bits in the VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE bitmap
 without actually initializing the corresponding pages. If the
 driver treats such pages as initialized (e.g., marking them as
 zeroed), this could lead to information leakage between guest
 processes or other security violations.
 
 Drivers operating in environments with untrusted devices are
-expected to avoid negotiating
-VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, and to initialize
+expected to avoid negotiating VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED
+or VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, and to initialize
 all pages themselves.
 
 \subsubsection{Denial of Service}\label{sec:Device Types / Memory Balloon Device / Security Considerations / Denial of Service}
-- 
MST