Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO

[Avi Kivity <avi@redhat.com>]

Andrea Arcangeli wrote:
> On Sun, Dec 14, 2008 at 07:01:38PM +0200, Avi Kivity wrote:
>   
>> Actually, with Xen, RAM may be unmapped due do Xen limitations when qemu 
>> runs on dom0 mode.  But I think map/unmap makes sense even disregarding 
>>     
>
> I realize xen 32bit has issues... Qemu/KVM 32bit also has the same
> issues but there's no point in 2009 (that's when this stuff could go
> productive) in trying to run guests with >2G of ram on a 32bit
> host. The issues emerges (I guess with xen too) in trying to run those
> obsolete hardware configurations. Even the atom and extremely low
> power athlon have 64bit capability, and on embedded that runs a real
> 32bit cpu I can't see how somebody would want to run a >2G guest.
>   

kvm and Xen actually have different issues for 32-bit.  For kvm, 
supporting >2G on 32-bits is possible but messy and pointless, so we 
chose not to do it.  For Xen, this is a critical performance issue as 
64-bit userspace in pv guests is quite slow.  So dom0 runs as a 32-bit guest

Newer Xen shouldn't have this problem though; it runs qemu in kernel 
mode in a dedicated 64-bit domain.

>> Xen:  if we add memory hotunplug, we need to make sure we don't unplug 
>> memory that has pending dma operations on it.  map/unmap gives us the 
>> opportunity to refcount memory slots.
>>     
>
> So memory hotunplug here is considered differently than the real
> memory hotplug emulation that simulates removing dimm on the
> hardware. This is just the xen trick to handle >4G guest on a 32bit
> address space? Well that's just the thing I'm not interested to
> support. When 64bit wasn't mainstream it made some sense, these days
> it's good enough if we can boot any guest OS (including 64bit ones) on
> a 32bit build, but trying to run guests OS with >2G of ram doesn't
> look useful.
>   

Leaving Xen aside, memory hotunplug requires that we tell the memory 
when it's used and when it isn't.

>> We can't get all dma to stop during hotunplug, since net rx operations are 
>> long-running (infinite if there is no activity on the link).
>>
>> IMO, we do want map/unmap, but this would be just a rename of can_dma and 
>> friends, and wouldn't have at this time any additional functionality.  
>> Bouncing has to happen where we have the ability to schedule the actual 
>> operation, and that's clearly not map/unmap.
>>     
>
> It would be a bit more than a rename, also keep in mind that in the
> longer term as said we need to build the iovec in the exec.c path,
> it's not enough to return a void *, I like to support a not 1:1 flat
> space to avoid wasting host virtual address space with guest memory
> holes. But that's about it, guest memory has to be always mapped, just
> not with a 1:1 mapping, and surely not with a per-page array that
> translates each page physical address to a host virtual address, but
> with ranges. So this map thing that returns a 'void *' won't be there
> for long even if I rename.
>   

If it returns an iovec, still that doesn't change how it works.  I like 
the symmetry of map()/unmap() and the lock/unlock semantics (like 
kmap_atomic/kunmap_atomic and a myriad other get/put pairs).

[There's actually a language that supports this idiom, but that's a 
different flamewar]

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.