Re: Question about DMA on 1:1 mapping dom0 of arm64

[Ian Campbell <ian.campbell@citrix.com>]

On Mon, 2015-04-20 at 10:58 +0100, Stefano Stabellini wrote:
> On Sat, 18 Apr 2015, Chen Baozi wrote:
> > On Fri, Apr 17, 2015 at 05:13:16PM +0100, Stefano Stabellini wrote:
> > > On Fri, 17 Apr 2015, Ian Campbell wrote:
> > > > On Fri, 2015-04-17 at 15:34 +0100, Stefano Stabellini wrote:
> > > > > > > If I set dom0_mem to a small value (e.g. 512M), which makes all physical memory
> > > > > > > of dom0 below 4G, everything goes fine.
> > > > > > 
> > > > > > So you are getting allocated memory below 4G?
> > > > > > 
> > > > > > You message on IRC suggested you weren't, did you hack around this?
> > > > > > 
> > > > > > I think we have two options, either xen_swiotlb_init allocates pages
> > > > > > below 4GB (e.g. __GFP_DMA) or we do something to allow xen_swiotlb_fixup
> > > > > > to actually work even on a 1:1 dom0.
> > > > > 
> > > > > I don't think that making xen_swiotlb_fixup work on ARM is a good idea:
> > > > > it would break the 1:1.
> > > > 
> > > > This would actually work though, I think, because this is the swiotlb so
> > > > we definitely have the opportunity to return the actual DMA address
> > > > whenever we use this buffer and the device will use it in the right
> > > > places for sure.
> > > 
> > > The code is pretty complex as is -- I would rather avoid adding more
> > > complexity to it.  For example we would need to bring back a mechanism
> > > to track dma address -> pseudo-physical address mappings on arm, even
> > > though it would be far simpler of course.
> > > 
> > > Also I think it makes sense to use the swiotlb buffer for its original
> > > purpose.
> > > 
> > > If we could introduce a mechanism to get a lower than 4G buffer in dom0,
> > > but matching the 1:1, I think it would make the maintenance much easier
> > > on the linux side.
> > 
> > +1
> > 
> > Actually, we have already had the mechanism on arm32 to populate at least
> > one bank of memory below 4G. Thus, the only thing we have to do on the
> > hypervisor side is to make arm32 and arm64 share the same process in
> > allocate_memory_11(), removing the 'lowmem = is_32bit_domain(d)' related
> > conditions. If this is acceptable, the only thing we need to do in Linux
> > kernel is to add the __GFP_DMA flag when allocating pages for xen_io_tlb_start
> > in xen_swiotlb_init.
> 
> Please send out the Linux patch using __GFP_DMA and I'll queue it up.

What happens with __GFP_DMA if no suitable memory is available (i.e. all
of RAM is >4GB)?

Ian.