Re: [FYI] DAMON and virtio-balloon in DeepSeek's DSec paper

[Gao Xiang <xiang@kernel.org>]

Hi,

On Fri, Sep 25, 2026 at 04:15:04PM +0800, Lance Yang wrote:
> +Cc DeepSeek and Muchun
> 
> On Fri, Sep 25, 2026 at 09:04:41AM +0200, David Hildenbrand (Arm) wrote:
> >On 9/25/26 07:44, Lance Yang wrote:
> >> Hi all,
> >> 
> >> I was reading DeepSeek's new DSec paper[1] and found a nice use of DAMON
> >> and virtio-balloon:
> >> 
> >> With this kind of workload, an agent may read a file once and never touch
> >> it again, while those pages remain in the guest page cache. Without memory
> >> pressure in the guest, they can stay cached even though the host would
> >> like that memory back ...
> >> 
> >> The trick is DAMON + virtio-balloon free-page reporting :) DAMON reclaims
> >
> >Heh, I read "virtio-balloon" and thought "balloon inflation/deflation, what year
> >is it?!". Free-page reporting makes much more sense.
> >
> >> cold file pages from the guest page cache; buddy gets a chance to coalesce
> >> them into reportable blocks, and virtio-balloon passes those blocks to
> >> Firecracker. Firecracker can then drop the host backing with MADV_DONTNEED.
> >
> >When I was at RH we were looking at this issue as well. virtio-pmem was one way
> >of avoiding the page cache in VM entirely. But it has its own limitations.
> 
> YES, they enable virtio-pmem with DAX for the read-only EROFS base-image
> and toolkit layers, while using DAMON with balloon free-page reporting to
> reclaim cold file pages from the guest page cache on larger writable disks.
> 
> They also point out that the guest must allocate struct page metadata for
> the entire pmem-backed address range. So virtio-pmem is not free either :)
> 
> BTW, Muchun recently posted a pretty cool series for exactly that:
> 
> https://lore.kernel.org/linux-mm/20260903122128.12264-1-songmuchun@bytedance.com/
> 
> (It shares vmemmap backing until a DAX fault needs private metadata,
> avoiding the full per-PFN cost up front.)
> 
> I have a feeling the DeepSeek team will be watching this one closely :P

There are several points virtio-pmem RW from my own viewpoints:

 - It makes the write async I/O synchronously, note that write I/Os are
   not quite the same as read I/Os (read I/Os are mostly sync). Storage
   also support multi-queues which can better leverage that, and that is
   why sometimes brd block device is not good at high-performance nvme
   for example.

 - Note that sandbox usually has a memory limit, but dax RW makes the
   whole rootfs addressable, e.g. if you have 128GiB rootfs, which means
   you could fault 128GiB on the host, instead of the sandbox memory
   size, so it might cause some security concern (as long as users
   shouldn't expect the host memory can be used up to 128GiB + memsize).

 - It can cause sync 4K faults on the host in the worst case (maybe
   large folios on the host can improve a bit yet not quite), in
   constant to the guest memory + THP usage. I don't know how the
   reclaim overhead is measured currently, but it seems the dsec paper
   also mentioned in this case.

 - The guest workload will still use mmap() for many sandbox apps, so
   `struct page` optimization is just for the optimized case, but not
   for the worst cases, the malicious VM sandboxes can still take
   much more `struct page` in the guest.

There would be better to have some benchmark here for typical RL
training RW virtio-pmem: but block storage semantics cannot already
be replaced with the memory semantics.

I think virtio-pmem RO is useful simply because it can reuse the same
page cache among multiple sandboxes on the host, which can even
warm-up other sandbox workloads, although it still has some security
concern but I guess for RL training it doesn't matter and read is
almost synchronous unlike writes.

Thanks,
Gao Xiang 

> 
> >Thanks for sharing!
> 
> Cheers!