From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:36058) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RqFtN-0008Cg-3O for qemu-devel@nongnu.org; Wed, 25 Jan 2012 22:20:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RqFtL-0004Yp-Hs for qemu-devel@nongnu.org; Wed, 25 Jan 2012 22:20:53 -0500 Received: from usrksweb02.endace.com ([174.143.168.194]:57149) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RqFtL-0004Yl-9h for qemu-devel@nongnu.org; Wed, 25 Jan 2012 22:20:51 -0500 Message-ID: <4F20C67D.30307@endace.com> Date: Thu, 26 Jan 2012 16:20:29 +1300 From: Alexey Korolev MIME-Version: 1.0 References: <4F1F971B.4020309@endace.com> <20120125125131.GC22203@redhat.com> In-Reply-To: <20120125125131.GC22203@redhat.com> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC/PATCH] Fix guest OS panic when 64bit BAR is present List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: sfd@endace.com, Kevin O'Connor , "qemu-devel@nongnu.org" On 26/01/12 01:51, Michael S. Tsirkin wrote: > On Wed, Jan 25, 2012 at 06:46:03PM +1300, Alexey Korolev wrote: >> Hi, >> In this post >> http://lists.gnu.org/archive/html/qemu-devel/2011-12/msg03171.html I've >> mentioned about the issues when 64Bit PCI BAR is present and 32bit >> address range is selected for it. >> The issue affects all recent qemu releases and all >> old and recent guest Linux kernel versions. >> >> We've done some investigations. Let me explain what happens. >> Assume we have 64bit BAR with size 32MB mapped at [0xF0000000 - >> 0xF2000000] >> >> When Linux guest starts it does PCI bus enumeration. >> The OS enumerates 64BIT bars using the following procedure. >> 1. Write all FF's to lower half of 64bit BAR >> 2. Write address back to lower half of 64bit BAR >> 3. Write all FF's to higher half of 64bit BAR >> 4. Write address back to higher half of 64bit BAR >> >> Linux code is here: >> http://lxr.linux.no/#linux+v3.2.1/drivers/pci/probe.c#L149 >> >> What does it mean for qemu? >> >> At step 1. qemu pci_default_write_config() recevies all FFs for lower >> part of the 64bit BAR. Then it applies the mask and converts the value >> to "All FF's - size + 1" (FE000000 if size is 32MB). >> Then pci_bar_address() checks if BAR address is valid. Since it is a >> 64bit bar it reads 0x00000000FE000000 - this address is valid. So qemu >> updates topology and sends request to update mappings in KVM with new >> range for the 64bit BAR FE000000 - 0xFFFFFFFF. This usually means kernel >> panic on boot, if there is another mapping in the FE000000 - 0xFFFFFFFF >> range, which is quite common. >> >> >> The following patch fixes the issue. It affects 64bit PCI BAR's only. >> The idea of the patch is: we introduce the states for low and high BARs >> whose can have 3 possible values: BAR_VALID, PCIBAR64_PARTIAL_SIZE_QUERY >> - someone has requested size of one half of the 64bit PCI BAR, >> PCIBAR64_PARTIAL_ADDR_PROGRAM - someone has sent a request to update the >> address of one half of the 64bit PCI BAR. The state becomes BAR_VALID >> when both halfs are in the same state. We ignore BAR value until both >> states become BAR_VALID >> >> Note: Please use the latest Seabios version (commit >> 139d5ac037de828f89c36e39c6dd15610650cede and later), as older versions >> didn't initialize high part of 64bit BAR. >> >> The patch is tested on Linux 2.6.18 - 3.1.0 and Windows 2008 Server >> >> Signed-off-by: Alexey Korolev > Interesting. However, looking at guest code, > I note that memory and io are disabled > during BAR sizing unless mmio always on is set. > pci_bar_address should return PCI_BAR_UNMAPPED > in this case, and we should never map this BAR > until it's enabled. What's going on? > > Oh. Good point. You are right here. Linux developers have added a protection starting 2.6.36 for lower part of PCI BAR. So this issue affects all guest kernels before 2.6.36. Sorry about confusion. The code without protection is here: http://lxr.linux.no/#linux+v2.6.35.9/drivers/pci/probe.c#L162 To solve this issue for older kernel versions the submitted patch is still relevant.