Re: [Qemu-devel] [PATCH] Inter-VM shared memory PCI device

[Jamie Lokier <jamie@shareable.org>]

Avi Kivity wrote:
> On 03/08/2010 03:03 PM, Paul Brook wrote:
> >>On 03/08/2010 12:53 AM, Paul Brook wrote:
> >>     
> >>>>Support an inter-vm shared memory device that maps a shared-memory
> >>>>object as a PCI device in the guest.  This patch also supports
> >>>>interrupts between guest by communicating over a unix domain socket.
> >>>>This patch applies to the qemu-kvm repository.
> >>>>         
> >>>No. All new devices should be fully qdev based.
> >>>
> >>>I suspect you've also ignored a load of coherency issues, especially when
> >>>not using KVM. As soon as you have shared memory in more than one host
> >>>thread/process you have to worry about memory barriers.
> >>>       
> >>Shouldn't it be sufficient to require the guest to issue barriers (and
> >>to ensure tcg honours the barriers, if someone wants this with tcg)?.
> >>     
> >In a cross environment that becomes extremely hairy.  For example the x86
> >architecture effectively has an implicit write barrier before every store, 
> >and
> >an implicit read barrier before every load.
> >   
> 
> Ah yes.  For cross tcg environments you can map the memory using mmio 
> callbacks instead of directly, and issue the appropriate barriers there.

That makes sense.  It will force an mmio callback for every access to
the shared memory, which is ok for correctness but vastly slower when
running in TCG compared with KVM.

But it's hard to see what else could be done - those implicit write
barries on x86 have to be emulated somehow.  For TCG without inter-vm
shared memory, those barriers aren't a problem.

Non-random-corruption guest behaviour is paramount, so I hope the
inter-vm device will add those mmio callbacks for the cross-arch case
before it sees much action.  (Strictly, it isn't cross-arch, but
host-has-more-relaxed-implicit-memory-model-than-guest.  I'm assuming
TCG doesn't reorder memory instructions).

-- Jamie