Re: [Qemu-devel] [RFC/PATCH] Fix guest OS panic when 64bit BAR is present

[Alexey Korolev <alexey.korolev@endace.com>]

On 26/01/12 01:51, Michael S. Tsirkin wrote:
> On Wed, Jan 25, 2012 at 06:46:03PM +1300, Alexey Korolev wrote:
>> Hi, 
>> In this post
>> http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg03171.html I've
>> mentioned about the issues when 64Bit PCI BAR is present and 32bit
>> address range is selected for it.
>> The issue affects all recent qemu releases and all
>> old and recent guest Linux kernel versions.
>>
>> We've done some investigations. Let me explain what happens.
>> Assume we have 64bit BAR with size 32MB mapped at [0xF0000000 -
>> 0xF2000000]
>>
>> When Linux guest starts it does PCI bus enumeration.
>> The OS enumerates 64BIT bars using the following procedure.
>> 1. Write all FF's to lower half of 64bit BAR
>> 2. Write address back to lower half of 64bit BAR
>> 3. Write all FF's to higher half of 64bit BAR
>> 4. Write address back to higher half of 64bit BAR
>>
>> Linux code is here: 
>> http://lxr.linux.no/#linux+v3.2.1/drivers/pci/probe.c#L149
>>
>> What does it mean for qemu?
>>
>> At step 1. qemu pci_default_write_config() recevies all FFs for lower
>> part of the 64bit BAR. Then it applies the mask and converts the value
>> to "All FF's - size + 1" (FE000000 if size is 32MB).
>> Then pci_bar_address() checks if BAR address is valid. Since it is a
>> 64bit bar it reads 0x00000000FE000000 - this address is valid. So qemu
>> updates topology and sends request to update mappings in KVM with new
>> range for the 64bit BAR FE000000 - 0xFFFFFFFF. This usually means kernel
>> panic on boot, if there is another mapping in the FE000000 - 0xFFFFFFFF
>> range, which is quite common.
>>
>>
>> The following patch fixes the issue. It affects 64bit PCI BAR's only. 
>> The idea of the patch is: we introduce the states for low and high BARs
>> whose can have 3 possible values: BAR_VALID, PCIBAR64_PARTIAL_SIZE_QUERY
>> - someone has requested size of one half of the 64bit PCI BAR,
>> PCIBAR64_PARTIAL_ADDR_PROGRAM - someone has sent a request to update the
>> address of one half of the 64bit PCI BAR. The state becomes BAR_VALID
>> when both halfs are in the same state. We ignore BAR value until both
>> states become BAR_VALID
>>
>> Note: Please use the latest Seabios version (commit
>> 139d5ac037de828f89c36e39c6dd15610650cede and later), as older versions
>> didn't initialize high part of 64bit BAR. 
>>
>> The patch is tested on Linux 2.6.18 - 3.1.0 and Windows 2008 Server
>>
>> Signed-off-by: Alexey Korolev <alexey.korolev@endace.com>
> Interesting. However, looking at guest code,
> I note that memory and io are disabled
> during BAR sizing unless mmio always on is set.
> pci_bar_address should return PCI_BAR_UNMAPPED
> in this case, and we should never map this BAR
> until it's enabled. What's going on?
>
>
Oh. Good point. You are right here. Linux developers
have added a protection starting 2.6.36 for lower part of PCI BAR.
So this issue affects all guest kernels before 2.6.36.
Sorry about confusion.

The code without protection is here:

http://lxr.linux.no/#linux+v2.6.35.9/drivers/pci/probe.c#L162


To solve this issue for older kernel versions the submitted patch is still relevant.