Re: [PATCH 2 of 5] add can_dma/post_dma for direct IO

Avi Kivity
2008-12-14T19:59:21Z

Andrea Arcangeli wrote:
> On Sun, Dec 14, 2008 at 07:01:38PM +0200, Avi Kivity wrote:
>
>> Actually, with Xen, RAM may be unmapped due do Xen limitations when qemu
>> runs on dom0 mode. But I think map/unmap makes sense even disregarding
>>
>
> I realize xen 32bit has issues... Qemu/KVM 32bit also has the same
> issues but there's no point in 2009 (that's when this stuff could go
> productive) in trying to run guests with >2G of ram on a 32bit
> host. The issues emerges (I guess with xen too) in trying to run those
> obsolete hardware configurations. Even the atom and extremely low
> power athlon have 64bit capability, and on embedded that runs a real
> 32bit cpu I can't see how somebody would want to run a >2G guest.
>

kvm and Xen actually have different issues for 32-bit. For kvm,
supporting >2G on 32-bits is possible but messy and pointless, so we
chose not to do it. For Xen, this is a critical performance issue as
64-bit userspace in pv guests is quite slow. So dom0 runs as a 32-bit guest

Newer Xen shouldn't have this problem though; it runs qemu in kernel
mode in a dedicated 64-bit domain.

>> Xen: if we add memory hotunplug, we need to make sure we don't unplug
>> memory that has pending dma operations on it. map/unmap gives us the
>> opportunity to refcount memory slots.
>>
>
> So memory hotunplug here is considered differently than the real
> memory hotplug emulation that simulates removing dimm on the
> hardware. This is just the xen trick to handle >4G guest on a 32bit
> address space? Well that's just the thing I'm not interested to
> support. When 64bit wasn't mainstream it made some sense, these days
> it's good enough if we can boot any guest OS (including 64bit ones) on
> a 32bit build, but trying to run guests OS with >2G of ram doesn't
> look useful.
>

Leaving Xen aside, memory hotunplug requires that we tell the memory
when it's used and when it isn't.

>> We can't get all dma to stop during hotunplug, since net rx operations are
>> long-running (infinite if there is no activity on the link).
>>
>> IMO, we do want map/unmap, but this would be just a rename of can_dma and
>> friends, and wouldn't have at this time any additional functionality.
>> Bouncing has to happen where we have the ability to schedule the actual
>> operation, and that's clearly not map/unmap.
>>
>
> It would be a bit more than a rename, also keep in mind that in the
> longer term as said we need to build the iovec in the exec.c path,
> it's not enough to return a void *, I like to support a not 1:1 flat
> space to avoid wasting host virtual address space with guest memory
> holes. But that's about it, guest memory has to be always mapped, just
> not with a 1:1 mapping, and surely not with a per-page array that
> translates each page physical address to a host virtual address, but
> with ranges. So this map thing that returns a 'void *' won't be there
> for long even if I rename.
>

If it returns an iovec, still that doesn't change how it works. I like
the symmetry of map()/unmap() and the lock/unlock semantics (like
kmap_atomic/kunmap_atomic and a myriad other get/put pairs).

[There's actually a language that supports this idiom, but that's a
different flamewar]

--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.