From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ozlabs.org (ozlabs.org [103.22.144.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 886701A0758 for ; Mon, 27 Apr 2015 15:01:09 +1000 (AEST) Received: from e23smtp01.au.ibm.com (e23smtp01.au.ibm.com [202.81.31.143]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 34D7F14030F for ; Mon, 27 Apr 2015 15:01:09 +1000 (AEST) Received: from /spool/local by e23smtp01.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 27 Apr 2015 15:01:08 +1000 Received: from d23relay09.au.ibm.com (d23relay09.au.ibm.com [9.185.63.181]) by d23dlp01.au.ibm.com (Postfix) with ESMTP id 250822CE8050 for ; Mon, 27 Apr 2015 15:01:05 +1000 (EST) Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay09.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t3R50uAG35913898 for ; Mon, 27 Apr 2015 15:01:05 +1000 Received: from d23av02.au.ibm.com (localhost [127.0.0.1]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t3R50Vw7031380 for ; Mon, 27 Apr 2015 15:00:31 +1000 From: Nikunj A Dadhania To: Thomas Huth Subject: Re: [PATCH 1/2] pci-phb: check for the 32-bit overflow In-Reply-To: <20150424130630.00bb9412@thh440s> References: <1429700240-32373-1-git-send-email-nikunj@linux.vnet.ibm.com> <20150423154314.595b8439@thh440s> <871tjaxjwe.fsf@abhimanyu.in.ibm.com> <20150424125657.7e1d5140@thh440s> <20150424130630.00bb9412@thh440s> Date: Mon, 27 Apr 2015 10:30:14 +0530 Message-ID: <87pp6qqi75.fsf@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain Cc: aik@ozlabs.ru, linuxppc-dev@ozlabs.org, david@gibson.dropbear.id.au List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Thomas Huth writes: > On Fri, 24 Apr 2015 12:56:57 +0200 > Thomas Huth wrote: > >> On Fri, 24 Apr 2015 09:22:33 +0530 >> Nikunj A Dadhania wrote: >> >> > >> > Hi Thomas, >> > >> > Thomas Huth writes: >> > > Am Wed, 22 Apr 2015 16:27:19 +0530 >> > > schrieb Nikunj A Dadhania : >> > > >> > >> With the addition of 64-bit BARS and increase in the mmio address >> > >> space, the code was hitting this limit. The memory of pci devices >> > >> across the bridges were not accessible due to which the drivers >> > >> failed. >> > >> >> > >> Signed-off-by: Nikunj A Dadhania >> > >> --- >> > >> board-qemu/slof/pci-phb.fs | 3 ++- >> > >> 1 file changed, 2 insertions(+), 1 deletion(-) >> > >> >> > >> diff --git a/board-qemu/slof/pci-phb.fs b/board-qemu/slof/pci-phb.fs >> > >> index 529772f..e307d95 100644 >> > >> --- a/board-qemu/slof/pci-phb.fs >> > >> +++ b/board-qemu/slof/pci-phb.fs >> > >> @@ -258,7 +258,8 @@ setup-puid >> > >> decode-64 2 / dup >r \ Decode and calc size/2 >> > >> pci-next-mem @ + dup pci-max-mem ! \ and calc max mem address >> > > >> > > Could pci-max-mem overflow, too? >> > >> > Should not, its only the boundary that was an issue. >> > >> > Qemu sends base and size, base + size can be till uint32 max. So for >> > example base was 0xC000.0000 and size was 0x4000.0000, we add up base + >> > size and put pci-max-mmio as 0x1.0000.0000, which would get programmend >> > in the bridge bars: lower limit as 0xC000 and 0x0000 as upper >> > limit. And no mmio access were going across the bridge. >> > >> > In my testing, I have found one more issue with translate-my-address, >> > it does not take care of 64-bit addresses. I have a patch working for >> > SLOF, but its breaking the guest kernel booting. >> > >> > > >> > >> dup pci-next-mmio ! \ which is the same as MMIO base >> > >> - r> + pci-max-mmio ! \ calc max MMIO address >> > >> + r> + FFFFFFFF min pci-max-mmio ! \ calc max MMIO address and >> > >> + \ check the 32-bit boundary >> >> Ok, thanks a lot for the example! I think your patch likely works in >> practice, but after staring at the code for a while, I think the real >> bug is slightly different. If I get the code above right, pci-max-mmio >> is normally set to the first address that is _not_ part of the mmio >> window anymore, right. Now have a look at pci-bridge-set-mmio-base in >> pci-scan.fs: >> >> : pci-bridge-set-mmio-base ( addr -- ) >> pci-next-mmio @ 100000 #aligned \ read the current Value and align to 1MB boundary >> dup 100000 + pci-next-mmio ! \ and write back with 1MB for bridge >> 10 rshift \ mmio-base reg is only the upper 16 bits >> pci-max-mmio @ FFFF0000 and or \ and Insert mmio Limit (set it to max) >> swap 20 + rtas-config-l! \ and write it into the bridge >> ; >> >> Seems like the pci-max-mmio, i.e. the first address that is not in the >> window anymore, is programmed into the memory limit register here - but >> according to the pci-to-pci bridge specification, it should be the last >> address of the window instead. >> >> So I think the correct fix would be to decrease the pci-max-mmio >> value in pci-bridge-set-mmio-base by 1- before programming it into the >> limit register (note: in pci-bridge-set-mmio-limit you can find a "1-" >> already, so I think this also should be done in >> pci-bridge-set-mmio-base, too) >> >> So if you've got some spare minutes, could you please check whether that >> would fix the issue, too? > > By the way, if I'm right, pci-bridge-set-mem-base seems to suffer from > the same problem, too. Both have the same issue, so fixed like below diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs index 15d0c8e..a552a74 100644 --- a/slof/fs/pci-scan.fs +++ b/slof/fs/pci-scan.fs @@ -87,7 +87,7 @@ here 100 allot CONSTANT pci-device-vec pci-next-mmio @ 100000 #aligned \ read the current Value and align to 1MB boundary dup 100000 + pci-next-mmio ! \ and write back with 1MB for bridge 10 rshift \ mmio-base reg is only the upper 16 bits - pci-max-mmio @ FFFF0000 and or \ and Insert mmio Limit (set it to max) + pci-max-mmio @ 1- FFFF0000 and or \ and Insert mmio Limit (set it to max) swap 20 + rtas-config-l! \ and write it into the bridge ; @@ -116,7 +116,7 @@ here 100 allot CONSTANT pci-device-vec 2 pick 2C + rtas-config-l! \ | and set the Limit THEN \ FI 10 rshift \ keep upper 16 bits - pci-max-mem @ FFFF0000 and or \ and Insert mmem Limit (set it to max) + pci-max-mem @ 1- FFFF0000 and or \ and Insert mmem Limit (set it to max) swap 24 + rtas-config-l! \ and write it into the bridge ;