Re: Assigning contiguous memory to a driver domain

[Joanna Rutkowska <joanna@invisiblethingslab.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3190 bytes --]

On 09/15/10 11:50, Jan Beulich wrote:
>>>> On 15.09.10 at 11:39, Rafal Wojtczuk <rafal@invisiblethingslab.com> wrote:
>> On Wed, Sep 15, 2010 at 11:08:42AM +0200, Rafal Wojtczuk wrote:
>>> On Tue, Sep 14, 2010 at 10:36:56AM +0100, Jan Beulich wrote:
>>>>>>> On 14.09.10 at 11:24, Rafal Wojtczuk <rafal@invisiblethingslab.com> wrote:
>>>>> Hello,
>>>>>
>>>>> Could someone guide me in the right direction with the topic of assigning
>>>>> contiguous memory to a domain.
>>>>>
>>>>> I have an issue with a PV domain that is assigned a PCI device. Sometimes, 
>>
>>>>> the 
>>>>> driver fails to load
>>>>> Sep 13 10:36:43 localhost kernel: [  103.651858] iwlagn 0000:00:01.0:
>>>>> firmware: requesting iwlwifi-4965-2.ucode
>>>>> Sep 13 10:36:43 localhost kernel: [  103.669105] iwlagn 0000:00:01.0: 
>> loaded
>>>>> firmware version 228.61.2.24
>>>>> Sep 13 10:36:43 localhost kernel: [  103.669263] iwlagn 0000:00:01.0: 
>> failed
>>>>> to allocate pci memory
>>>>>
>>>>> The reason seems to be that the domain does not have enough contiguous
>>>>> memory, in mfn terms.
>>>>
>>>> No, how (dis)contiguous the memory of a domain is doesn't matter
>>>> here. What matters is whether the domain can *make* the requested
>>>> memory contiguous, and that depends on how much contiguous
>>>> memory Xen has at the point of the allocation.
>>>
>>> Ah, so you are saying that regardless of whether a domain has some
>>> contiguous memory, the driver will call xen_create_contiguous_region when 
>> allocating
>>> memory (via dma_alloc_coherent ?).
>>>
>>> Slightly out-of-the-list-scope: is there a convention when a driver should
>>> allocate DMA-able memory ? Is it safe to assume that as soon as the driver
>>> has loaded, it will no longer need to call xen_create_contiguous_region
>>> anymore and we can use up all free Xen memory ? 
>>
>> Hmm, at least in case of tg3 driver, if xen free memory = 0, after I have
>> done "ifconfig eth0 down" in the driver domain, the subsequent "ifconfig 
>> eth0 up" 
>> failed with "SIOCSIFFLAGS: Cannot allocate memory".
> 
> Sure - why would the driver waste resources when the device may
> not be used.
> 
>> So, it looks like in order to make a PV driver domain work, there must be 
>> some Xen free memory 
>> all the time ?
> 
> Potentially yes, but this really depends on how the respective
> driver is written.
> 
>> Moreover, this free Xen memory must be contiguous to some
>> extent; is there any way to assure this ?
> 
> No.
> 

Wait! Are you saying there is no *way* to guarantee proper operation of
a driver domain in Xen?

Sure, we can tune our memory balancer to always keep some 100MB (or
200MB, or maybe 500MB?) of xen free memory, and *hope* that it will
contain enough continues pages, in case some driver in some driver
domain calls dma_alloc_coherent(), so the call will succeed.

But this is not a good solution: not only because it's a waste of memory
(I'd rather use this memory for Dom0/storage domain page cache instead)
but also, and most importantly, because I don't want to build a system
based on *hope*!

Can we do something about it?

joanna.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 518 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel