Re: [Virtio-fs] [PATCH 3/4] virtiofsd: use file-backend memory region for virtiofsd's cache area

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Liu Bo (bo.liu@linux.alibaba.com) wrote:
> On Mon, May 20, 2019 at 09:49:34AM -0400, Vivek Goyal wrote:
> > On Fri, May 17, 2019 at 07:28:22PM -0700, Liu Bo wrote:
> > 
> > [..]
> > > > > > > > That case happened with cache=none and the dax mount option.
> > > > > > > > 
> > > > > > > > The general problem is when FUSE_READ/FUSE_WRITE is sent and the buffer
> > > > > > > > is outside guest RAM.
> > > > > 
> > > > > Stefan,
> > > > > 
> > > > > Can this be emulated by sending a request to qemu? If virtiofsd can detect
> > > > > that source/destination of READ/WRITE is not guest RAM, can it forward
> > > > > message to qemu to do this operation (which has access to all the DAX
> > > > > windows)?
> > > > > 
> > > > > This probably will mean introducing new messages like
> > > > > setupmapping/removemapping messages between virtiofsd/qemu.  
> > > > 
> > > > Yes, interesting idea!
> > > > 
> > > > When virtiofsd is unable to map the virtqueue iovecs due to addresses
> > > > outside guest RAM, it could forward READ/WRITE requests to QEMU along
> > > > with the file descriptor.  It would be slow but fixes the problem.
> > > >
> > > 
> > > It is probably not easy to do.
> > > 
> > > Imagine the following case,
> > > // foo1 is on a dax virtiofs, foo2 is on a nondax virtiofs
> > > 
> > > p = mmap(foo1, ...);
> > > write(foo2, p, ...);
> > > 
> > > virtiofsd where foo2 is using needs to interpret gpa from virtiofs
> > > where foo1 exists along with fd being foo1, but a write fuse_req
> > > doesn't have foo1's fd.
> > > 
> > > And are you suggesting that qemu goes to read the data on gpa and
> > > returns via vhost-user message?  or let this virtiofsd (foo2) do mmap
> > > on foo1 directly?
> > 
> > Hi Liu Bo,
> > 
> > Not sure I understand the question. Assuming foo2 virtiofs instances
> > is mounted with cache=none, write(foo2), will trigger FUSE_WRITE
> > request to daemon with source address being in dax window of first
> > virtiofs instance. 
> > 
> > This request should be forwarded to qemu which can just read from
> > foo1 mmaped area in qemu address space and write to foo2. I am assuming
> > before FUSE_WRITE comes in, mapping for foo1 has already been
> > establied.
> 
> Bofore sending any message to qemu, fv_queue_thread() will fail at
> vu_queue_pop()->vu_queue_map_desc()->virtqueue_map_desc()->vu_gpa_to_va(),
> because virtiofsd (where foo2 is on) doesn't have the information of
> foo1's mapping within its address space.

The trick is to bounce the problem back to qemu; as soon as the mapping
fails I add a flag to say the mapping failed; then that causes the
request to be sent to qemu via a slave command - and qemu has all of
the mappings so can read/write to all of the fs's.

Dave

> It could be reproduced by fstests/generic/413.
> 
> thanks,
> -liubo
> 
> > 
> > IOW, foo1 is already mmaped in qemu address space so there is no
> > need to pass fd of foo1 in FUSE_WRITE(foo2) request. Did I miss
> > something?
> > 
> > Vivek
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK