[Qemu-devel] Why only devdax guarantees guest data persistence ?

[Qemu-devel] Why only devdax guarantees guest data persistence ?

[bipin.tomar]

Text from "docs/nvdimm.txt" says:
Guest Data Persistence
----------------------

Though QEMU supports multiple types of vNVDIMM backends on Linux,
currently the only one that can guarantee the guest write persistence
is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
which all guest access do not involve any host-side kernel cache.

I think here "host-side kernel cache" imply "page cache". Why does fsdax NOT have the same persistence guarantees as devdax for vNVDIMM?
Both the modes avoid using page cache then why is devdax explicitly called out?
-BT

Fw: Why only devdax guarantees guest data persistence ?

[bipin.tomar-/E1597aS9LQAvxtiuMwx3w]

 I haven't heard from anyone on qemu-devel. There are a lot of vNVDIMM experts here and I'm hoping that someone here may throw some light.

-BT

   ----- Forwarded Message ----- From: bipin.tomar-/E1597aS9LQAvxtiuMwx3w@public.gmane.org <bipin.tomar-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>To: qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org <qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org>Sent: Friday, February 15, 2019, 12:09:31 PM ESTSubject: Why only devdax guarantees guest data persistence ?
 Text from "docs/nvdimm.txt" says:
Guest Data Persistence
----------------------

Though QEMU supports multiple types of vNVDIMM backends on Linux,
currently the only one that can guarantee the guest write persistence
is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
which all guest access do not involve any host-side kernel cache.

I think here "host-side kernel cache" imply "page cache". Why does fsdax NOT have the same persistence guarantees as devdax for vNVDIMM?
Both the modes avoid using page cache then why is devdax explicitly called out?
-BT

Re: Fw: Why only devdax guarantees guest data persistence ?

[Dan Williams]

On Tue, Feb 19, 2019 at 9:22 AM bipin.tomar@yahoo.com
<bipin.tomar@yahoo.com> wrote:
>
>  I haven't heard from anyone on qemu-devel. There are a lot of vNVDIMM experts here and I'm hoping that someone here may throw some light.

Answers inline below:

>    ----- Forwarded Message ----- From: bipin.tomar@yahoo.com <bipin.tomar@yahoo.com>To: qemu-devel@nongnu.org <qemu-devel@nongnu.org>Sent: Friday, February 15, 2019, 12:09:31 PM ESTSubject: Why only devdax guarantees guest data persistence ?
>  Text from "docs/nvdimm.txt" says:
> Guest Data Persistence
> ----------------------
>
> Though QEMU supports multiple types of vNVDIMM backends on Linux,
> currently the only one that can guarantee the guest write persistence
> is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
> which all guest access do not involve any host-side kernel cache.
>
> I think here "host-side kernel cache" imply "page cache". Why does fsdax NOT have the same persistence guarantees as devdax for vNVDIMM?
> Both the modes avoid using page cache then why is devdax explicitly called out?

As long as the mapping is established with the MAP_SYNC facility then
the persistence guarantees are the same as the host. You may be
thinking of the pre-MAP_SYNC days where there was no facility to
coordinate the synchronization of host metadata.

I will note that device-passthrough is disabled for fsdax because
there is no facility to coordinate host truncation events vs
guest-device-initiated DMA, but that's a separate issue from
persistence guarantees. VFIO will simply fail to register devices in
this case.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [Qemu-devel] Why only devdax guarantees guest data persistence ?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

On Fri, Feb 15, 2019 at 05:09:31PM +0000, bipin.tomar--- via Qemu-devel wrote:
> Text from "docs/nvdimm.txt" says:
> Guest Data Persistence
> ----------------------
> 
> Though QEMU supports multiple types of vNVDIMM backends on Linux,
> currently the only one that can guarantee the guest write persistence
> is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
> which all guest access do not involve any host-side kernel cache.
> 
> I think here "host-side kernel cache" imply "page cache". Why does fsdax NOT have the same persistence guarantees as devdax for vNVDIMM?
> Both the modes avoid using page cache then why is devdax explicitly called out?

File systems may require msync(2)/fsync(2) to guarantee persistence even
with DAX (just a cache flush instruction may not be enough!).  Emulated
NVDIMM devices lack an fsync interface so guests are unable to fsync the
host file system.

This is not an issue with devdax since there is no host file system.

virtio-pmem is an effort to add a paravirtualized fsync-style interface
and should solve this problem in the future:
https://lkml.org/lkml/2019/1/9/541

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] Why only devdax guarantees guest data persistence ?

[Dan Williams]

On Wed, Feb 20, 2019 at 7:37 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>
> On Fri, Feb 15, 2019 at 05:09:31PM +0000, bipin.tomar--- via Qemu-devel wrote:
> > Text from "docs/nvdimm.txt" says:
> > Guest Data Persistence
> > ----------------------
> >
> > Though QEMU supports multiple types of vNVDIMM backends on Linux,
> > currently the only one that can guarantee the guest write persistence
> > is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
> > which all guest access do not involve any host-side kernel cache.
> >
> > I think here "host-side kernel cache" imply "page cache". Why does fsdax NOT have the same persistence guarantees as devdax for vNVDIMM?
> > Both the modes avoid using page cache then why is devdax explicitly called out?
>
> File systems may require msync(2)/fsync(2) to guarantee persistence even
> with DAX (just a cache flush instruction may not be enough!).  Emulated
> NVDIMM devices lack an fsync interface so guests are unable to fsync the
> host file system.
>
> This is not an issue with devdax since there is no host file system.
>
> virtio-pmem is an effort to add a paravirtualized fsync-style interface
> and should solve this problem in the future:
> https://lkml.org/lkml/2019/1/9/541

A para-virtualized flush is only necessary if the host implements
caches and dirty-metadata that cannot be coordinated / flushed by
MAP_SYNC. virtio-pmem is not a solution to a problem, virtio-pmem is a
non-DAX alternative implementation of persistent memory that requires
the guest to coordinate metadata and host page cache flushing.

Re: [Qemu-devel] Why only devdax guarantees guest data persistence ?

[Pankaj Gupta]

> wrote:
> > Text from "docs/nvdimm.txt" says:
> > Guest Data Persistence
> > ----------------------
> > 
> > Though QEMU supports multiple types of vNVDIMM backends on Linux,
> > currently the only one that can guarantee the guest write persistence
> > is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to
> > which all guest access do not involve any host-side kernel cache.
> > 
> > I think here "host-side kernel cache" imply "page cache". Why does fsdax
> > NOT have the same persistence guarantees as devdax for vNVDIMM?
> > Both the modes avoid using page cache then why is devdax explicitly called
> > out?
> 
> File systems may require msync(2)/fsync(2) to guarantee persistence even
> with DAX (just a cache flush instruction may not be enough!).  Emulated
> NVDIMM devices lack an fsync interface so guests are unable to fsync the
> host file system.

Just want to add to what Stefan already said.
 
For emulated vNVDIMM on regular storage like SSD we require an additional 
fsync on host backing file to guarantee write persistence (i.e fsync host 
page cache pages) and consistent host metadata.

If backing file is real NVDIMM, write persistence is taken care by host NVDIMM
driver and MAP_SYNC support takes care of metadata consistency in-case of guest
crash. 
> 
> This is not an issue with devdax since there is no host file system.

Yes.

> 
> virtio-pmem is an effort to add a paravirtualized fsync-style interface
> and should solve this problem in the future:
> https://lkml.org/lkml/2019/1/9/541
> 
> Stefan
>