[Qemu-devel] Suggestion on 'virtio-pmem' implementation

[Qemu-devel] Suggestion on 'virtio-pmem' implementation

[Pankaj Gupta]

Hi,


I am implementing 'virtio-pmem' as a mechanism to
flush guest writes with 'fake DAX' flushing interface.

Below is the high level details of components:

1] 'virtio-pmem' device expose guest physical address
  details(start, len).   

2] 'virtio-pmem' driver in guest discovers this 
    information and configures 'libnvdimm'. Guest 'pmem' 
    driver works on this memory range.

3] Guest 'pmem' driver uses 'virtio-pmem' PV driver to 
   send flush commands.


I need suggestion implementing part 1]

* When tried with 'hotplug_memory.base' address as guest physical 
  address, I am facing 'EPT_MISCONFIG' errors when pmem does mkfs. 
  After digging more it looks like address range I am using as guest
  physical address is either already mapped as MMIO or reserved. 
  Though Guest hot-plugs this physical address into its virtual 
  memory range when guest tries to read/write the memory KVM cannot 
  translate the address and throw 'EPT_MISCONFIG' error.

* While I am trying to get the appropriate guest physical address
  which is free, I could see memory 'pc_dimm_memory_plug' code 
  has a function 'pc_dimm_get_free_addr' which works with 'PC DIMM'
  class. As I am using 'VIRTIO', there is no way AFAIK this function 
  can be used by VIRTIO or my PV device code.


I need ideas to get the free guest physical address from my PV 
device code so that we can use this range in guest address space.

Find below pointer to previous discussion:

https://marc.info/?l=kvm&m=151629709903946&w=2

Thanks,
Pankaj  
  

Re: [Qemu-devel] Suggestion on 'virtio-pmem' implementation

[David Hildenbrand]

Hi Pankaj,

I have a prototype (new one for virtio-mem I was working on over the last weeks) for exactly what you need. I basically factored out the notion of a memory device. So also virtio devices can be memory devices and get recognized e.g. in formerly known pc_dimm_get_free_address(), so it works out nicely with ordinary memory hotplug and such. 

Guess your main problem right now is that you don‘t create a memory slot properly. But I also have code for that that you can built onto.

I‘m right now in Sri Lanka on vacation, I‘ll be back on 23. September and will send you the link to a branch with the prototype asap.

Thanks,

David / dhildenb / david@redhat.com

Von meinem iPhone gesendet

> Am 14.03.2018 um 10:36 schrieb Pankaj Gupta <pagupta@redhat.com>:
> 
> 
> 
> Hi,
> 
> 
> I am implementing 'virtio-pmem' as a mechanism to
> flush guest writes with 'fake DAX' flushing interface.
> 
> Below is the high level details of components:
> 
> 1] 'virtio-pmem' device expose guest physical address
>  details(start, len).   
> 
> 2] 'virtio-pmem' driver in guest discovers this 
>    information and configures 'libnvdimm'. Guest 'pmem' 
>    driver works on this memory range.
> 
> 3] Guest 'pmem' driver uses 'virtio-pmem' PV driver to 
>   send flush commands.
> 
> 
> I need suggestion implementing part 1]
> 
> * When tried with 'hotplug_memory.base' address as guest physical 
>  address, I am facing 'EPT_MISCONFIG' errors when pmem does mkfs. 
>  After digging more it looks like address range I am using as guest
>  physical address is either already mapped as MMIO or reserved. 
>  Though Guest hot-plugs this physical address into its virtual 
>  memory range when guest tries to read/write the memory KVM cannot 
>  translate the address and throw 'EPT_MISCONFIG' error.
> 
> * While I am trying to get the appropriate guest physical address
>  which is free, I could see memory 'pc_dimm_memory_plug' code 
>  has a function 'pc_dimm_get_free_addr' which works with 'PC DIMM'
>  class. As I am using 'VIRTIO', there is no way AFAIK this function 
>  can be used by VIRTIO or my PV device code.
> 
> 
> I need ideas to get the free guest physical address from my PV 
> device code so that we can use this range in guest address space.
> 
> Find below pointer to previous discussion:
> 
> https://marc.info/?l=kvm&m=151629709903946&w=2
> 
> Thanks,
> Pankaj  
> 

Re: [Qemu-devel] Suggestion on 'virtio-pmem' implementation

[Pankaj Gupta]

Hi David,

> 
> Hi Pankaj,
> 
> I have a prototype (new one for virtio-mem I was working on over the last
> weeks) for exactly what you need. I basically factored out the notion of a
> memory device. So also virtio devices can be memory devices and get
> recognized e.g. in formerly known pc_dimm_get_free_address(), so it works
> out nicely with ordinary memory hotplug and such.

Nice. This is very useful re-factoring.

> 
> Guess your main problem right now is that you don‘t create a memory slot
> properly. But I also have code for that that you can built onto.

sure.

> 
> I‘m right now in Sri Lanka on vacation, I‘ll be back on 23. September and
> will send you the link to a branch with the prototype asap.

Cool. Enjoy your vacation and thanks for the reply.

Best regards,
Pankaj

> 
> Thanks,
> 
> David / dhildenb / david@redhat.com
> 
> Von meinem iPhone gesendet
> 
> > Am 14.03.2018 um 10:36 schrieb Pankaj Gupta <pagupta@redhat.com>:
> > 
> > 
> > 
> > Hi,
> > 
> > 
> > I am implementing 'virtio-pmem' as a mechanism to
> > flush guest writes with 'fake DAX' flushing interface.
> > 
> > Below is the high level details of components:
> > 
> > 1] 'virtio-pmem' device expose guest physical address
> >  details(start, len).
> > 
> > 2] 'virtio-pmem' driver in guest discovers this
> >    information and configures 'libnvdimm'. Guest 'pmem'
> >    driver works on this memory range.
> > 
> > 3] Guest 'pmem' driver uses 'virtio-pmem' PV driver to
> >   send flush commands.
> > 
> > 
> > I need suggestion implementing part 1]
> > 
> > * When tried with 'hotplug_memory.base' address as guest physical
> >  address, I am facing 'EPT_MISCONFIG' errors when pmem does mkfs.
> >  After digging more it looks like address range I am using as guest
> >  physical address is either already mapped as MMIO or reserved.
> >  Though Guest hot-plugs this physical address into its virtual
> >  memory range when guest tries to read/write the memory KVM cannot
> >  translate the address and throw 'EPT_MISCONFIG' error.
> > 
> > * While I am trying to get the appropriate guest physical address
> >  which is free, I could see memory 'pc_dimm_memory_plug' code
> >  has a function 'pc_dimm_get_free_addr' which works with 'PC DIMM'
> >  class. As I am using 'VIRTIO', there is no way AFAIK this function
> >  can be used by VIRTIO or my PV device code.
> > 
> > 
> > I need ideas to get the free guest physical address from my PV
> > device code so that we can use this range in guest address space.
> > 
> > Find below pointer to previous discussion:
> > 
> > https://marc.info/?l=kvm&m=151629709903946&w=2
> > 
> > Thanks,
> > Pankaj
> > 
>