Re: Assigning contiguous memory to a driver domain

[Rafal Wojtczuk <rafal@invisiblethingslab.com>]

On Wed, Sep 15, 2010 at 11:08:42AM +0200, Rafal Wojtczuk wrote:
> On Tue, Sep 14, 2010 at 10:36:56AM +0100, Jan Beulich wrote:
> > >>> On 14.09.10 at 11:24, Rafal Wojtczuk <rafal@invisiblethingslab.com> wrote:
> > > Hello,
> > > 
> > > Could someone guide me in the right direction with the topic of assigning
> > > contiguous memory to a domain.
> > > 
> > > I have an issue with a PV domain that is assigned a PCI device. Sometimes, 
> > > the 
> > > driver fails to load
> > > Sep 13 10:36:43 localhost kernel: [  103.651858] iwlagn 0000:00:01.0:
> > > firmware: requesting iwlwifi-4965-2.ucode
> > > Sep 13 10:36:43 localhost kernel: [  103.669105] iwlagn 0000:00:01.0: loaded
> > > firmware version 228.61.2.24
> > > Sep 13 10:36:43 localhost kernel: [  103.669263] iwlagn 0000:00:01.0: failed
> > > to allocate pci memory
> > > 
> > > The reason seems to be that the domain does not have enough contiguous
> > > memory, in mfn terms.
> > 
> > No, how (dis)contiguous the memory of a domain is doesn't matter
> > here. What matters is whether the domain can *make* the requested
> > memory contiguous, and that depends on how much contiguous
> > memory Xen has at the point of the allocation.
> 
> Ah, so you are saying that regardless of whether a domain has some
> contiguous memory, the driver will call xen_create_contiguous_region when allocating
> memory (via dma_alloc_coherent ?).
> 
> Slightly out-of-the-list-scope: is there a convention when a driver should
> allocate DMA-able memory ? Is it safe to assume that as soon as the driver
> has loaded, it will no longer need to call xen_create_contiguous_region
> anymore and we can use up all free Xen memory ? 

Hmm, at least in case of tg3 driver, if xen free memory = 0, after I have
done "ifconfig eth0 down" in the driver domain, the subsequent "ifconfig eth0 up" 
failed with "SIOCSIFFLAGS: Cannot allocate memory".

So, it looks like in order to make a PV driver domain work, there must be some Xen free memory 
all the time ? Moreover, this free Xen memory must be contiguous to some
extent; is there any way to assure this ?

Regards,
Rafal Wojtczuk