Re: [Qemu-devel] Guest cannot handle a PCI BAR > 1GB

[Cam Macdonell <cam@cs.ualberta.ca>]

[-- Attachment #1: Type: text/plain, Size: 4887 bytes --]

On Sun, Sep 5, 2010 at 10:50 AM, Avi Kivity <avi@redhat.com> wrote:

>  On 09/04/2010 01:22 AM, Cam Macdonell wrote:
>
>> Hi,
>>
>> I'm trying to test 2 GB (and eventually larger) BARs with ivshmem and
>> I get an error in the guest that it is able to find a mem resource for
>> a BAR larger than 1GB.  I'm using 64-bit BARs.
>>
>> when running with 6GB of RAM and a 1GB BAR for ivshmem, it finds a
>> resource (and searches beyond 32-bit values to find it).  Here is a
>> log from printfs added to the loop that searches the resources from
>> find_resource() in kernel/resource.c:363.
>>
>>
> This is a kernel question, not a qemu issue.  Copying lkml.
>
>
>  trying 'tmp.start' 1000 to
>>         'tmp.end' fff
>> trying 'tmp.start' 9f400 to
>>         'tmp.end' 9f3ff
>> trying 'tmp.start' a0000 to
>>         'tmp.end' effff
>> trying 'tmp.start' 100000 to
>>         'tmp.end' fffff
>> trying 'tmp.start' dfffd000 to
>>         'tmp.end' dfffcfff
>> trying 'tmp.start' e0000000 to
>>         'tmp.end' efffffff
>> trying 'tmp.start' f2000000 to
>>         'tmp.end' f1ffffff
>> trying 'tmp.start' f2001000 to
>>         'tmp.end' f200ffff
>> trying 'tmp.start' f2020000 to
>>         'tmp.end' f201ffff
>> trying 'tmp.start' f2021000 to
>>         'tmp.end' f202ffff
>> trying 'tmp.start' f2040000 to
>>         'tmp.end' f203ffff
>> trying 'tmp.start' f2040100 to
>>         'tmp.end' febfffff
>> trying 'tmp.start' fec00400 to
>>         'tmp.end' fffbffff
>> trying 'tmp.start' 100000000 to
>>         'tmp.end' ffffffff
>> trying 'tmp.start' 1a0000000 to
>>         'tmp.end' ffffffffffffffff
>> pci 0000:00:04.0: BAR 2: assigned [mem 0x1c0000000-0x1ffffffff 64bit]
>> pci 0000:00:04.0: BAR 2: set to [mem 0x1c0000000-0x1ffffffff 64bit]
>> (PCI address [0x1c0000000-0x1ffffffff]
>>
>> and you can see the BAR is successfully assigned.
>>
>> However, with a 2GB BAR (below), the search fails, but it also never
>> searches beyong 32-bits.  Again, all that's changed is the size of the
>> ivshmem region.
>>
>> trying 'tmp.start' 1000 to
>>         'tmp.end' fff
>> trying 'tmp.start' 9f400 to
>>         'tmp.end' 9f3ff
>> trying 'tmp.start' a0000 to
>>         'tmp.end' effff
>> trying 'tmp.start' 100000 to
>>         'tmp.end' fffff
>> trying 'tmp.start' dfffd000 to
>>         'tmp.end' dfffcfff
>> pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)
>>
>> Is there a limit to PCI BAR sizes or resources?  Any pointers or
>> further debugging tips are greatly appreciated.
>>
>>
> What kernel version are you looking at?
>

latest kvm git, 2.6.36-rc2+


>
> Please add printks to the loop so we can see this->start and this->end.  It
> smells like a truncation issue.


Success with a 1GB BAR

this->start 1000, this->end 9f3ff
this->start 9f400, this->end 9ffff
this->start f0000, this->end fffff
this->start 100000, this->end dfffcfff
this->start dfffd000, this->end dfffffff
this->start f0000000, this->end f1ffffff
this->start f2000000, this->end f2000fff
this->start f2010000, this->end f201ffff
this->start f2020000, this->end f2020fff
this->start f2030000, this->end f203ffff
this->start f2040000, this->end f20400ff
this->start fec00000, this->end fec003ff
this->start fffc0000, this->end ffffffff
this->start 100000000, this->end 11fffffff
tmp.start 120000000, tmp.end ffffffffffffffff
pci 0000:00:04.0: BAR 2: assigned [mem 0x140000000-0x17fffffff 64bit]

and when it fails with a 2GB BAR, the following is printed

this->start 1000, this->end 9f3ff
this->start 9f400, this->end 9ffff
this->start f0000, this->end fffff
this->start 100000, this->end dfffcfff
this->start dfffd000, this->end dfffffff
pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)

I added a few more debug statements and found that in the failure case, the
function returns that it found a region (the last one printed before the
error).  I've added printfs for the two tests in the if that determine when
a region is found:

       if (tmp.start < tmp.end && tmp.end - tmp.start >= size - 1) {
            new->start = tmp.start;
            new->end = tmp.start + size - 1;
            printk(KERN_INFO "returning 0\n");
            return 0;
        }

this->start 1000, this->end 9f3ff
tmp.start 80000000, tmp.end fff
    true: ffffffff80000fff >= 7fffffff
this->start 9f400, this->end 9ffff
tmp.start 80000000, tmp.end 9f3ff
    true: ffffffff8009f3ff >= 7fffffff
this->start f0000, this->end fffff
tmp.start 80000000, tmp.end effff
    true: ffffffff800effff >= 7fffffff
this->start 100000, this->end dfffcfff
tmp.start 80000000, tmp.end fffff
    true: ffffffff800fffff >= 7fffffff
this->start dfffd000, this->end dfffffff
tmp.start 100000, tmp.end dfffcfff
    true: 100000 < dfffcfff
    true: dfefcfff >= 7fffffff
returning 0
pci 0000:00:04.0: BAR 2: can't assign mem (size 0x80000000)





>
>
> --
> error compiling committee.c: too many arguments to function
>
>

[-- Attachment #2: Type: text/html, Size: 7191 bytes --]