Re: [PATCH RESEND v6 01/36] memory: alloc RAM from file at offset

[Igor Mammedov <imammedo@redhat.com>]

On Wed, 13 May 2020 16:08:06 -0400
Jag Raman <jag.raman@oracle.com> wrote:

> > On May 13, 2020, at 11:25 AM, Igor Mammedov <imammedo@redhat.com> wrote:
> > 
> > On Wed, 13 May 2020 09:40:42 +0100
> > Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >   
> >> On Tue, May 12, 2020 at 07:56:33AM -0400, Jag Raman wrote:  
> >>> 
> >>>   
> >>>> On May 12, 2020, at 4:48 AM, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >>>> 
> >>>> On Wed, Apr 22, 2020 at 09:13:36PM -0700, elena.ufimtseva@oracle.com wrote:    
> >>>>> From: Jagannathan Raman <jag.raman@oracle.com>
> >>>>> 
> >>>>> Allow RAM MemoryRegion to be created from an offset in a file, instead
> >>>>> of allocating at offset of 0 by default. This is needed to synchronize
> >>>>> RAM between QEMU & remote process.    
> >>>> 
> >>>> Can you elaborate on why remote processes require the RAM to be offset
> >>>> from zero ?    
> >>> 
> >>> Hi Daniel,
> >>> 
> >>> As it turns out, the RAM is scattered across the physical address space
> >>> (system_memory) of QEMU. Therefore, the system memory is composed
> >>> of multiple sections of RAM, and some sections start at a non-zero RAM
> >>> offset.
> >>> 
> >>> As a result, the remote process needs the ability to map these RAM
> >>> sections into system_memory.    
> >> 
> >> To explain a bit more, my understanding is that the offset is
> >> specifically for mmap(2). As Jag alluded to, multiple sections can use a
> >> single backing RAM file. These sections have different offsets in the
> >> file.
> >> 
> >> Jag, maybe you can include a concrete explanation like the following in
> >> the commit description:
> >> 
> >> Launch QEMU like this:
> >> 
> >>  qemu-system-x86_64 -mem-path /dev/shm -m 8G
> >> 
> >> There is only one RAM file descriptor:
> >> 
> >>  $ cat /proc/$(pidof qemu)/fd
> >>  ...
> >>  lrwx------. 1 stefanha stefanha 64 May 13 09:34 19 -> '/dev/shm/qemu_back_mem.pc.ram.7YAlqn (deleted)'
> >> 
> >> But the memory tree shows that single file is split into multiple ranges
> >> of guest physical address space:
> >> 
> >>  (qemu) info mtree
> >>  memory-region: system
> >>  0000000000000000-ffffffffffffffff (prio 0, i/o): system
> >>    0000000000000000-00000000bfffffff (prio 0, i/o): alias ram-below-4g @pc.ram 0000000000000000-00000000bfffffff
> >>    ...
> >>    0000000100000000-000000023fffffff (prio 0, i/o): alias ram-above-4g @pc.ram 00000000c0000000-00000001ffffffff
> >> 
> >> This means QEMU needs to send two regions to the remote device process.
> >> They both mmap the same file but from different starting file offsets.  
> > 
> > are we talking here about GPA offests her or about host offsets in mmaped host file?
> > If it's the later then above mtree doesn't show true picture (those entries are just aliases),
> > main guest RAM is allocated as a single continuous chunk (so far) which belongs
> > to a memory-backend.  
> 
> Thanks for the info about ‘mtree’ QMP option. We’ll use that to better explain the need for offset
> during memory allocation.
> 
> As we can observe from the ‘mtree’ output, different sections of system memory vector into the
> RAM. Since we have only one file descriptor available for all of RAM, we need to mmap() these
> sections at different offsets within the memory file.
> 
> Hey Igor, the offset passed into the mmap() syscall is the offset within host file. Thanks for
> pointing out that these are aliases. I believe the mmap() operation is equivalent to the “alias”
> operation within MemoryRegion framework. We are sending the ‘fd’, ‘offset’ within the fd and
> the size of these RAM regions to the remote device over the unix socket. Hopefully, this looks
> good to you.

could you point to a specific patch where on remote device that is being received and used?

> 
> Thanks!
> --
> Jag
> 
> >   
> >> Stefan  
> > 
> >   
>