* [PATCHv5 4/4] arm64: Add build salt to the vDSO
From: Laura Abbott @ 2018-07-03 23:34 UTC (permalink / raw)
To: mjw, H . J . Lu, Masahiro Yamada, Catalin Marinas, Will Deacon
Cc: Laura Abbott, Andy Lutomirski, Linus Torvalds, X86 ML,
linux-kernel, Nick Clifton, Cary Coutant, linux-kbuild,
linuxppc-dev, Michael Ellerman, linux-arm-kernel
In-Reply-To: <20180703233430.14416-1-labbott@redhat.com>
The vDSO needs to have a unique build id in a similar manner
to the kernel and modules. Use the build salt macro.
Signed-off-by: Laura Abbott <labbott@redhat.com>
---
v5: I was previously focused on x86 only but since powerpc gave a patch,
I figured I would do arm64 since the changes were also fairly simple.
---
arch/arm64/kernel/vdso/note.S | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/kernel/vdso/note.S b/arch/arm64/kernel/vdso/note.S
index b82c85e5d972..2c429dfd3f45 100644
--- a/arch/arm64/kernel/vdso/note.S
+++ b/arch/arm64/kernel/vdso/note.S
@@ -22,7 +22,10 @@
#include <linux/uts.h>
#include <linux/version.h>
#include <linux/elfnote.h>
+#include <linux/build-salt.h>
ELFNOTE_START(Linux, 0, "a")
.long LINUX_VERSION_CODE
ELFNOTE_END
+
+BUILD_SALT;
--
2.17.1
^ permalink raw reply related
* Re: [PATCHv3 0/4] drivers/base: bugfix for supplier<-consumer ordering in device_kset
From: Pingfan Liu @ 2018-07-04 2:47 UTC (permalink / raw)
To: rjw
Cc: linux-kernel, Greg Kroah-Hartman, Grygorii Strashko,
Christoph Hellwig, Bjorn Helgaas, Dave Young, linux-pci,
linuxppc-dev
In-Reply-To: <4685360.VNmeYLh0dQ@aspire.rjw.lan>
On Tue, Jul 3, 2018 at 10:36 PM Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>
> On Tuesday, July 3, 2018 8:50:38 AM CEST Pingfan Liu wrote:
> > commit 52cdbdd49853 ("driver core: correct device's shutdown order")
> > places an assumption of supplier<-consumer order on the process of probe.
> > But it turns out to break down the parent <- child order in some scene.
> > E.g in pci, a bridge is enabled by pci core, and behind it, the devices
> > have been probed. Then comes the bridge's module, which enables extra
> > feature(such as hotplug) on this bridge.
>
> So what *exactly* does happen in that case?
>
I saw the shpc_probe() is called on the bridge, although the probing
failed on that bare-metal. But if it success, then it will enable the
hotplug feature on the bridge.
Thanks,
Pingfan
^ permalink raw reply
* Re: [PATCHv3 2/4] drivers/base: utilize device tree info to shutdown devices
From: Pingfan Liu @ 2018-07-04 3:10 UTC (permalink / raw)
To: lukas
Cc: linux-kernel, Greg Kroah-Hartman, Rafael J . Wysocki,
Grygorii Strashko, Christoph Hellwig, Bjorn Helgaas, Dave Young,
linux-pci, linuxppc-dev
In-Reply-To: <CAFgQCTt0M=+AQKXFXcOkxppjnDK9EAyNxHN=2F+n_NvyN=rQqg@mail.gmail.com>
On Tue, Jul 3, 2018 at 5:26 PM Pingfan Liu <kernelfans@gmail.com> wrote:
>
> On Tue, Jul 3, 2018 at 3:51 PM Lukas Wunner <lukas@wunner.de> wrote:
> >
> > On Tue, Jul 03, 2018 at 02:50:40PM +0800, Pingfan Liu wrote:
> > > commit 52cdbdd49853 ("driver core: correct device's shutdown order")
> > > places an assumption of supplier<-consumer order on the process of probe.
> > > But it turns out to break down the parent <- child order in some scene.
> > > E.g in pci, a bridge is enabled by pci core, and behind it, the devices
> > > have been probed. Then comes the bridge's module, which enables extra
> > > feature(such as hotplug) on this bridge. This will break the
> > > parent<-children order and cause failure when "kexec -e" in some scenario.
> > >
> > > The detailed description of the scenario:
> > > An IBM Power9 machine on which, two drivers portdrv_pci and shpchp(a mod)
> > > match the PCI_CLASS_BRIDGE_PCI, but neither of them success to probe due
> > > to some issue. For this case, the bridge is moved after its children in
> > > devices_kset. Then, when "kexec -e", a ata-disk behind the bridge can not
> > > write back buffer in flight due to the former shutdown of the bridge which
> > > clears the BusMaster bit.
> >
> > If you revert commit cc27b735ad3a ("PCI/portdrv: Turn off PCIe services
> > during shutdown"), does the issue go away?
>
> Yes, it is gone.
Have not figured out why the issue was gone. But I think it just cover
some fault.
re-fetch the boot log of mainline kernel without any patch, and filter
out the pci domain 0004
grep "devices_kset: Moving 0004:" newlog.txt
[ 2.114986] devices_kset: Moving 0004:00:00.0 to end of list <---
pcie port drive's probe, but it failed
[ 2.115192] devices_kset: Moving 0004:01:00.0 to end of list
[ 2.115591] devices_kset: Moving 0004:02:02.0 to end of list
[ 2.115923] devices_kset: Moving 0004:02:0a.0 to end of list
[ 2.116141] devices_kset: Moving 0004:02:0b.0 to end of list
[ 2.116358] devices_kset: Moving 0004:02:0c.0 to end of list
[ 3.181860] devices_kset: Moving 0004:03:00.0 to end of list <---
the ata disk controller which sits behind the bridge
[ 10.267081] devices_kset: Moving 0004:00:00.0 to end of list <---
shpc_probe() on this bridge, failed too.
Hence we have the bridge (parent) after the child in devices_kset.
Thanks,
Pingfan
^ permalink raw reply
* RE: [PATCH v11 00/26] Speculative page faults
From: Song, HaiyanX @ 2018-07-04 3:23 UTC (permalink / raw)
To: Laurent Dufour
Cc: akpm@linux-foundation.org, mhocko@kernel.org,
peterz@infradead.org, kirill@shutemov.name, ak@linux.intel.com,
dave@stgolabs.net, jack@suse.cz, Matthew Wilcox,
khandual@linux.vnet.ibm.com, aneesh.kumar@linux.vnet.ibm.com,
benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org,
Thomas Gleixner, Ingo Molnar, hpa@zytor.com, Will Deacon,
Sergey Senozhatsky, sergey.senozhatsky.work@gmail.com,
Andrea Arcangeli, Alexei Starovoitov, Wang, Kemi, Daniel Jordan,
David Rientjes, Jerome Glisse, Ganesh Mahendran, Minchan Kim,
Punit Agrawal, vinayak menon, Yang Shi,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
haren@linux.vnet.ibm.com, npiggin@gmail.com,
bsingharora@gmail.com, paulmck@linux.vnet.ibm.com, Tim Chen,
linuxppc-dev@lists.ozlabs.org, x86@kernel.org
In-Reply-To: <3849e991-1354-d836-94ac-077d29a0dee4@linux.vnet.ibm.com>
Hi Laurent,=0A=
=0A=
=0A=
For the test result on Intel 4s skylake platform (192 CPUs, 768G Memory), t=
he below test cases all were run 3 times.=0A=
I check the test results, only page_fault3_thread/enable THP have 6% stddev=
for head commit, other tests have lower stddev.=0A=
=0A=
And I did not find other high variation on test case result.=0A=
=0A=
a). Enable THP=0A=
testcase base stddev change head =
stddev metric=0A=
page_fault3/enable THP 10519 =B1 3% -20.5% 8368 =
=B16% will-it-scale.per_thread_ops=0A=
page_fault2/enalbe THP 8281 =B1 2% -18.8% 6728 =
will-it-scale.per_thread_ops=0A=
brk1/eanble THP 998475 -2.2% 976893 =
will-it-scale.per_process_ops=0A=
context_switch1/enable THP 223910 -1.3% 220930 =
will-it-scale.per_process_ops=0A=
context_switch1/enable THP 233722 -1.0% 231288 =
will-it-scale.per_thread_ops=0A=
=0A=
b). Disable THP=0A=
page_fault3/disable THP 10856 -23.1% 8344 =
will-it-scale.per_thread_ops=0A=
page_fault2/disable THP 8147 -18.8% 6613 =
will-it-scale.per_thread_ops=0A=
brk1/disable THP 957 -7.9% 881 =
will-it-scale.per_thread_ops=0A=
context_switch1/disable THP 237006 -2.2% 231907 =
will-it-scale.per_thread_ops=0A=
brk1/disable THP 997317 -2.0% 977778 =
will-it-scale.per_process_ops=0A=
page_fault3/disable THP 467454 -1.8% 459251 =
will-it-scale.per_process_ops=0A=
context_switch1/disable THP 224431 -1.3% 221567 =
will-it-scale.per_process_ops=0A=
=0A=
=0A=
Best regards,=0A=
Haiyan Song=0A=
________________________________________=0A=
From: Laurent Dufour [ldufour@linux.vnet.ibm.com]=0A=
Sent: Monday, July 02, 2018 4:59 PM=0A=
To: Song, HaiyanX=0A=
Cc: akpm@linux-foundation.org; mhocko@kernel.org; peterz@infradead.org; kir=
ill@shutemov.name; ak@linux.intel.com; dave@stgolabs.net; jack@suse.cz; Mat=
thew Wilcox; khandual@linux.vnet.ibm.com; aneesh.kumar@linux.vnet.ibm.com; =
benh@kernel.crashing.org; mpe@ellerman.id.au; paulus@samba.org; Thomas Glei=
xner; Ingo Molnar; hpa@zytor.com; Will Deacon; Sergey Senozhatsky; sergey.s=
enozhatsky.work@gmail.com; Andrea Arcangeli; Alexei Starovoitov; Wang, Kemi=
; Daniel Jordan; David Rientjes; Jerome Glisse; Ganesh Mahendran; Minchan K=
im; Punit Agrawal; vinayak menon; Yang Shi; linux-kernel@vger.kernel.org; l=
inux-mm@kvack.org; haren@linux.vnet.ibm.com; npiggin@gmail.com; bsingharora=
@gmail.com; paulmck@linux.vnet.ibm.com; Tim Chen; linuxppc-dev@lists.ozlabs=
.org; x86@kernel.org=0A=
Subject: Re: [PATCH v11 00/26] Speculative page faults=0A=
=0A=
On 11/06/2018 09:49, Song, HaiyanX wrote:=0A=
> Hi Laurent,=0A=
>=0A=
> Regression test for v11 patch serials have been run, some regression is f=
ound by LKP-tools (linux kernel performance)=0A=
> tested on Intel 4s skylake platform. This time only test the cases which =
have been run and found regressions on=0A=
> V9 patch serials.=0A=
>=0A=
> The regression result is sorted by the metric will-it-scale.per_thread_op=
s.=0A=
> branch: Laurent-Dufour/Speculative-page-faults/20180520-045126=0A=
> commit id:=0A=
> head commit : a7a8993bfe3ccb54ad468b9f1799649e4ad1ff12=0A=
> base commit : ba98a1cdad71d259a194461b3a61471b49b14df1=0A=
> Benchmark: will-it-scale=0A=
> Download link: https://github.com/antonblanchard/will-it-scale/tree/maste=
r=0A=
>=0A=
> Metrics:=0A=
> will-it-scale.per_process_ops=3Dprocesses/nr_cpu=0A=
> will-it-scale.per_thread_ops=3Dthreads/nr_cpu=0A=
> test box: lkp-skl-4sp1(nr_cpu=3D192,memory=3D768G)=0A=
> THP: enable / disable=0A=
> nr_task:100%=0A=
>=0A=
> 1. Regressions:=0A=
>=0A=
> a). Enable THP=0A=
> testcase base change head =
metric=0A=
> page_fault3/enable THP 10519 -20.5% 836 wi=
ll-it-scale.per_thread_ops=0A=
> page_fault2/enalbe THP 8281 -18.8% 6728 wi=
ll-it-scale.per_thread_ops=0A=
> brk1/eanble THP 998475 -2.2% 976893 wi=
ll-it-scale.per_process_ops=0A=
> context_switch1/enable THP 223910 -1.3% 220930 wi=
ll-it-scale.per_process_ops=0A=
> context_switch1/enable THP 233722 -1.0% 231288 wi=
ll-it-scale.per_thread_ops=0A=
>=0A=
> b). Disable THP=0A=
> page_fault3/disable THP 10856 -23.1% 8344 wi=
ll-it-scale.per_thread_ops=0A=
> page_fault2/disable THP 8147 -18.8% 6613 wi=
ll-it-scale.per_thread_ops=0A=
> brk1/disable THP 957 -7.9% 881 wi=
ll-it-scale.per_thread_ops=0A=
> context_switch1/disable THP 237006 -2.2% 231907 wi=
ll-it-scale.per_thread_ops=0A=
> brk1/disable THP 997317 -2.0% 977778 wi=
ll-it-scale.per_process_ops=0A=
> page_fault3/disable THP 467454 -1.8% 459251 wi=
ll-it-scale.per_process_ops=0A=
> context_switch1/disable THP 224431 -1.3% 221567 wi=
ll-it-scale.per_process_ops=0A=
>=0A=
> Notes: for the above values of test result, the higher is better.=0A=
=0A=
I tried the same tests on my PowerPC victim VM (1024 CPUs, 11TB) and I can'=
t=0A=
get reproducible results. The results have huge variation, even on the vani=
lla=0A=
kernel, and I can't state on any changes due to that.=0A=
=0A=
I tried on smaller node (80 CPUs, 32G), and the tests ran better, but I did=
n't=0A=
measure any changes between the vanilla and the SPF patched ones:=0A=
=0A=
test THP enabled 4.17.0-rc4-mm1 spf delta=0A=
page_fault3_threads 2697.7 2683.5 -0.53%=0A=
page_fault2_threads 170660.6 169574.1 -0.64%=0A=
context_switch1_threads 6915269.2 6877507.3 -0.55%=0A=
context_switch1_processes 6478076.2 6529493.5 0.79%=0A=
brk1 243391.2 238527.5 -2.00%=0A=
=0A=
Tests were run 10 times, no high variation detected.=0A=
=0A=
Did you see high variation on your side ? How many times the test were run =
to=0A=
compute the average values ?=0A=
=0A=
Thanks,=0A=
Laurent.=0A=
=0A=
=0A=
>=0A=
> 2. Improvement: not found improvement based on the selected test cases.=
=0A=
>=0A=
>=0A=
> Best regards=0A=
> Haiyan Song=0A=
> ________________________________________=0A=
> From: owner-linux-mm@kvack.org [owner-linux-mm@kvack.org] on behalf of La=
urent Dufour [ldufour@linux.vnet.ibm.com]=0A=
> Sent: Monday, May 28, 2018 4:54 PM=0A=
> To: Song, HaiyanX=0A=
> Cc: akpm@linux-foundation.org; mhocko@kernel.org; peterz@infradead.org; k=
irill@shutemov.name; ak@linux.intel.com; dave@stgolabs.net; jack@suse.cz; M=
atthew Wilcox; khandual@linux.vnet.ibm.com; aneesh.kumar@linux.vnet.ibm.com=
; benh@kernel.crashing.org; mpe@ellerman.id.au; paulus@samba.org; Thomas Gl=
eixner; Ingo Molnar; hpa@zytor.com; Will Deacon; Sergey Senozhatsky; sergey=
.senozhatsky.work@gmail.com; Andrea Arcangeli; Alexei Starovoitov; Wang, Ke=
mi; Daniel Jordan; David Rientjes; Jerome Glisse; Ganesh Mahendran; Minchan=
Kim; Punit Agrawal; vinayak menon; Yang Shi; linux-kernel@vger.kernel.org;=
linux-mm@kvack.org; haren@linux.vnet.ibm.com; npiggin@gmail.com; bsingharo=
ra@gmail.com; paulmck@linux.vnet.ibm.com; Tim Chen; linuxppc-dev@lists.ozla=
bs.org; x86@kernel.org=0A=
> Subject: Re: [PATCH v11 00/26] Speculative page faults=0A=
>=0A=
> On 28/05/2018 10:22, Haiyan Song wrote:=0A=
>> Hi Laurent,=0A=
>>=0A=
>> Yes, these tests are done on V9 patch.=0A=
>=0A=
> Do you plan to give this V11 a run ?=0A=
>=0A=
>>=0A=
>>=0A=
>> Best regards,=0A=
>> Haiyan Song=0A=
>>=0A=
>> On Mon, May 28, 2018 at 09:51:34AM +0200, Laurent Dufour wrote:=0A=
>>> On 28/05/2018 07:23, Song, HaiyanX wrote:=0A=
>>>>=0A=
>>>> Some regression and improvements is found by LKP-tools(linux kernel pe=
rformance) on V9 patch series=0A=
>>>> tested on Intel 4s Skylake platform.=0A=
>>>=0A=
>>> Hi,=0A=
>>>=0A=
>>> Thanks for reporting this benchmark results, but you mentioned the "V9 =
patch=0A=
>>> series" while responding to the v11 header series...=0A=
>>> Were these tests done on v9 or v11 ?=0A=
>>>=0A=
>>> Cheers,=0A=
>>> Laurent.=0A=
>>>=0A=
>>>>=0A=
>>>> The regression result is sorted by the metric will-it-scale.per_thread=
_ops.=0A=
>>>> Branch: Laurent-Dufour/Speculative-page-faults/20180316-151833 (V9 pat=
ch series)=0A=
>>>> Commit id:=0A=
>>>> base commit: d55f34411b1b126429a823d06c3124c16283231f=0A=
>>>> head commit: 0355322b3577eeab7669066df42c550a56801110=0A=
>>>> Benchmark suite: will-it-scale=0A=
>>>> Download link:=0A=
>>>> https://github.com/antonblanchard/will-it-scale/tree/master/tests=0A=
>>>> Metrics:=0A=
>>>> will-it-scale.per_process_ops=3Dprocesses/nr_cpu=0A=
>>>> will-it-scale.per_thread_ops=3Dthreads/nr_cpu=0A=
>>>> test box: lkp-skl-4sp1(nr_cpu=3D192,memory=3D768G)=0A=
>>>> THP: enable / disable=0A=
>>>> nr_task: 100%=0A=
>>>>=0A=
>>>> 1. Regressions:=0A=
>>>> a) THP enabled:=0A=
>>>> testcase base change head =
metric=0A=
>>>> page_fault3/ enable THP 10092 -17.5% 8323 =
will-it-scale.per_thread_ops=0A=
>>>> page_fault2/ enable THP 8300 -17.2% 6869 =
will-it-scale.per_thread_ops=0A=
>>>> brk1/ enable THP 957.67 -7.6% 885 =
will-it-scale.per_thread_ops=0A=
>>>> page_fault3/ enable THP 172821 -5.3% 163692 =
will-it-scale.per_process_ops=0A=
>>>> signal1/ enable THP 9125 -3.2% 8834 =
will-it-scale.per_process_ops=0A=
>>>>=0A=
>>>> b) THP disabled:=0A=
>>>> testcase base change head =
metric=0A=
>>>> page_fault3/ disable THP 10107 -19.1% 8180 =
will-it-scale.per_thread_ops=0A=
>>>> page_fault2/ disable THP 8432 -17.8% 6931 =
will-it-scale.per_thread_ops=0A=
>>>> context_switch1/ disable THP 215389 -6.8% 200776 =
will-it-scale.per_thread_ops=0A=
>>>> brk1/ disable THP 939.67 -6.6% 877.3=
3 will-it-scale.per_thread_ops=0A=
>>>> page_fault3/ disable THP 173145 -4.7% 165064 =
will-it-scale.per_process_ops=0A=
>>>> signal1/ disable THP 9162 -3.9% 8802 =
will-it-scale.per_process_ops=0A=
>>>>=0A=
>>>> 2. Improvements:=0A=
>>>> a) THP enabled:=0A=
>>>> testcase base change head =
metric=0A=
>>>> malloc1/ enable THP 66.33 +469.8% 383.6=
7 will-it-scale.per_thread_ops=0A=
>>>> writeseek3/ enable THP 2531 +4.5% 2646 =
will-it-scale.per_thread_ops=0A=
>>>> signal1/ enable THP 989.33 +2.8% 1016 =
will-it-scale.per_thread_ops=0A=
>>>>=0A=
>>>> b) THP disabled:=0A=
>>>> testcase base change head =
metric=0A=
>>>> malloc1/ disable THP 90.33 +417.3% 467.3=
3 will-it-scale.per_thread_ops=0A=
>>>> read2/ disable THP 58934 +39.2% 82060 =
will-it-scale.per_thread_ops=0A=
>>>> page_fault1/ disable THP 8607 +36.4% 11736 =
will-it-scale.per_thread_ops=0A=
>>>> read1/ disable THP 314063 +12.7% 353934 =
will-it-scale.per_thread_ops=0A=
>>>> writeseek3/ disable THP 2452 +12.5% 2759 =
will-it-scale.per_thread_ops=0A=
>>>> signal1/ disable THP 971.33 +5.5% 1024 =
will-it-scale.per_thread_ops=0A=
>>>>=0A=
>>>> Notes: for above values in column "change", the higher value means tha=
t the related testcase result=0A=
>>>> on head commit is better than that on base commit for this benchmark.=
=0A=
>>>>=0A=
>>>>=0A=
>>>> Best regards=0A=
>>>> Haiyan Song=0A=
>>>>=0A=
>>>> ________________________________________=0A=
>>>> From: owner-linux-mm@kvack.org [owner-linux-mm@kvack.org] on behalf of=
Laurent Dufour [ldufour@linux.vnet.ibm.com]=0A=
>>>> Sent: Thursday, May 17, 2018 7:06 PM=0A=
>>>> To: akpm@linux-foundation.org; mhocko@kernel.org; peterz@infradead.org=
; kirill@shutemov.name; ak@linux.intel.com; dave@stgolabs.net; jack@suse.cz=
; Matthew Wilcox; khandual@linux.vnet.ibm.com; aneesh.kumar@linux.vnet.ibm.=
com; benh@kernel.crashing.org; mpe@ellerman.id.au; paulus@samba.org; Thomas=
Gleixner; Ingo Molnar; hpa@zytor.com; Will Deacon; Sergey Senozhatsky; ser=
gey.senozhatsky.work@gmail.com; Andrea Arcangeli; Alexei Starovoitov; Wang,=
Kemi; Daniel Jordan; David Rientjes; Jerome Glisse; Ganesh Mahendran; Minc=
han Kim; Punit Agrawal; vinayak menon; Yang Shi=0A=
>>>> Cc: linux-kernel@vger.kernel.org; linux-mm@kvack.org; haren@linux.vnet=
.ibm.com; npiggin@gmail.com; bsingharora@gmail.com; paulmck@linux.vnet.ibm.=
com; Tim Chen; linuxppc-dev@lists.ozlabs.org; x86@kernel.org=0A=
>>>> Subject: [PATCH v11 00/26] Speculative page faults=0A=
>>>>=0A=
>>>> This is a port on kernel 4.17 of the work done by Peter Zijlstra to ha=
ndle=0A=
>>>> page fault without holding the mm semaphore [1].=0A=
>>>>=0A=
>>>> The idea is to try to handle user space page faults without holding th=
e=0A=
>>>> mmap_sem. This should allow better concurrency for massively threaded=
=0A=
>>>> process since the page fault handler will not wait for other threads m=
emory=0A=
>>>> layout change to be done, assuming that this change is done in another=
part=0A=
>>>> of the process's memory space. This type page fault is named speculati=
ve=0A=
>>>> page fault. If the speculative page fault fails because of a concurren=
cy is=0A=
>>>> detected or because underlying PMD or PTE tables are not yet allocatin=
g, it=0A=
>>>> is failing its processing and a classic page fault is then tried.=0A=
>>>>=0A=
>>>> The speculative page fault (SPF) has to look for the VMA matching the =
fault=0A=
>>>> address without holding the mmap_sem, this is done by introducing a rw=
lock=0A=
>>>> which protects the access to the mm_rb tree. Previously this was done =
using=0A=
>>>> SRCU but it was introducing a lot of scheduling to process the VMA's=
=0A=
>>>> freeing operation which was hitting the performance by 20% as reported=
by=0A=
>>>> Kemi Wang [2]. Using a rwlock to protect access to the mm_rb tree is=
=0A=
>>>> limiting the locking contention to these operations which are expected=
to=0A=
>>>> be in a O(log n) order. In addition to ensure that the VMA is not free=
d in=0A=
>>>> our back a reference count is added and 2 services (get_vma() and=0A=
>>>> put_vma()) are introduced to handle the reference count. Once a VMA is=
=0A=
>>>> fetched from the RB tree using get_vma(), it must be later freed using=
=0A=
>>>> put_vma(). I can't see anymore the overhead I got while will-it-scale=
=0A=
>>>> benchmark anymore.=0A=
>>>>=0A=
>>>> The VMA's attributes checked during the speculative page fault process=
ing=0A=
>>>> have to be protected against parallel changes. This is done by using a=
per=0A=
>>>> VMA sequence lock. This sequence lock allows the speculative page faul=
t=0A=
>>>> handler to fast check for parallel changes in progress and to abort th=
e=0A=
>>>> speculative page fault in that case.=0A=
>>>>=0A=
>>>> Once the VMA has been found, the speculative page fault handler would =
check=0A=
>>>> for the VMA's attributes to verify that the page fault has to be handl=
ed=0A=
>>>> correctly or not. Thus, the VMA is protected through a sequence lock w=
hich=0A=
>>>> allows fast detection of concurrent VMA changes. If such a change is=
=0A=
>>>> detected, the speculative page fault is aborted and a *classic* page f=
ault=0A=
>>>> is tried. VMA sequence lockings are added when VMA attributes which a=
re=0A=
>>>> checked during the page fault are modified.=0A=
>>>>=0A=
>>>> When the PTE is fetched, the VMA is checked to see if it has been chan=
ged,=0A=
>>>> so once the page table is locked, the VMA is valid, so any other chang=
es=0A=
>>>> leading to touching this PTE will need to lock the page table, so no=
=0A=
>>>> parallel change is possible at this time.=0A=
>>>>=0A=
>>>> The locking of the PTE is done with interrupts disabled, this allows=
=0A=
>>>> checking for the PMD to ensure that there is not an ongoing collapsing=
=0A=
>>>> operation. Since khugepaged is firstly set the PMD to pmd_none and the=
n is=0A=
>>>> waiting for the other CPU to have caught the IPI interrupt, if the pmd=
is=0A=
>>>> valid at the time the PTE is locked, we have the guarantee that the=0A=
>>>> collapsing operation will have to wait on the PTE lock to move forward=
.=0A=
>>>> This allows the SPF handler to map the PTE safely. If the PMD value is=
=0A=
>>>> different from the one recorded at the beginning of the SPF operation,=
the=0A=
>>>> classic page fault handler will be called to handle the operation whil=
e=0A=
>>>> holding the mmap_sem. As the PTE lock is done with the interrupts disa=
bled,=0A=
>>>> the lock is done using spin_trylock() to avoid dead lock when handling=
a=0A=
>>>> page fault while a TLB invalidate is requested by another CPU holding =
the=0A=
>>>> PTE.=0A=
>>>>=0A=
>>>> In pseudo code, this could be seen as:=0A=
>>>> speculative_page_fault()=0A=
>>>> {=0A=
>>>> vma =3D get_vma()=0A=
>>>> check vma sequence count=0A=
>>>> check vma's support=0A=
>>>> disable interrupt=0A=
>>>> check pgd,p4d,...,pte=0A=
>>>> save pmd and pte in vmf=0A=
>>>> save vma sequence counter in vmf=0A=
>>>> enable interrupt=0A=
>>>> check vma sequence count=0A=
>>>> handle_pte_fault(vma)=0A=
>>>> ..=0A=
>>>> page =3D alloc_page()=0A=
>>>> pte_map_lock()=0A=
>>>> disable interrupt=0A=
>>>> abort if sequence counter has chan=
ged=0A=
>>>> abort if pmd or pte has changed=0A=
>>>> pte map and lock=0A=
>>>> enable interrupt=0A=
>>>> if abort=0A=
>>>> free page=0A=
>>>> abort=0A=
>>>> ...=0A=
>>>> }=0A=
>>>>=0A=
>>>> arch_fault_handler()=0A=
>>>> {=0A=
>>>> if (speculative_page_fault(&vma))=0A=
>>>> goto done=0A=
>>>> again:=0A=
>>>> lock(mmap_sem)=0A=
>>>> vma =3D find_vma();=0A=
>>>> handle_pte_fault(vma);=0A=
>>>> if retry=0A=
>>>> unlock(mmap_sem)=0A=
>>>> goto again;=0A=
>>>> done:=0A=
>>>> handle fault error=0A=
>>>> }=0A=
>>>>=0A=
>>>> Support for THP is not done because when checking for the PMD, we can =
be=0A=
>>>> confused by an in progress collapsing operation done by khugepaged. Th=
e=0A=
>>>> issue is that pmd_none() could be true either if the PMD is not alread=
y=0A=
>>>> populated or if the underlying PTE are in the way to be collapsed. So =
we=0A=
>>>> cannot safely allocate a PMD if pmd_none() is true.=0A=
>>>>=0A=
>>>> This series add a new software performance event named 'speculative-fa=
ults'=0A=
>>>> or 'spf'. It counts the number of successful page fault event handled=
=0A=
>>>> speculatively. When recording 'faults,spf' events, the faults one is=
=0A=
>>>> counting the total number of page fault events while 'spf' is only cou=
nting=0A=
>>>> the part of the faults processed speculatively.=0A=
>>>>=0A=
>>>> There are some trace events introduced by this series. They allow=0A=
>>>> identifying why the page faults were not processed speculatively. This=
=0A=
>>>> doesn't take in account the faults generated by a monothreaded process=
=0A=
>>>> which directly processed while holding the mmap_sem. This trace events=
are=0A=
>>>> grouped in a system named 'pagefault', they are:=0A=
>>>> - pagefault:spf_vma_changed : if the VMA has been changed in our back=
=0A=
>>>> - pagefault:spf_vma_noanon : the vma->anon_vma field was not yet set.=
=0A=
>>>> - pagefault:spf_vma_notsup : the VMA's type is not supported=0A=
>>>> - pagefault:spf_vma_access : the VMA's access right are not respected=
=0A=
>>>> - pagefault:spf_pmd_changed : the upper PMD pointer has changed in ou=
r=0A=
>>>> back.=0A=
>>>>=0A=
>>>> To record all the related events, the easier is to run perf with the=
=0A=
>>>> following arguments :=0A=
>>>> $ perf stat -e 'faults,spf,pagefault:*' <command>=0A=
>>>>=0A=
>>>> There is also a dedicated vmstat counter showing the number of success=
ful=0A=
>>>> page fault handled speculatively. I can be seen this way:=0A=
>>>> $ grep speculative_pgfault /proc/vmstat=0A=
>>>>=0A=
>>>> This series builds on top of v4.16-mmotm-2018-04-13-17-28 and is funct=
ional=0A=
>>>> on x86, PowerPC and arm64.=0A=
>>>>=0A=
>>>> ---------------------=0A=
>>>> Real Workload results=0A=
>>>>=0A=
>>>> As mentioned in previous email, we did non official runs using a "popu=
lar=0A=
>>>> in memory multithreaded database product" on 176 cores SMT8 Power syst=
em=0A=
>>>> which showed a 30% improvements in the number of transaction processed=
per=0A=
>>>> second. This run has been done on the v6 series, but changes introduce=
d in=0A=
>>>> this new version should not impact the performance boost seen.=0A=
>>>>=0A=
>>>> Here are the perf data captured during 2 of these runs on top of the v=
8=0A=
>>>> series:=0A=
>>>> vanilla spf=0A=
>>>> faults 89.418 101.364 +13%=0A=
>>>> spf n/a 97.989=0A=
>>>>=0A=
>>>> With the SPF kernel, most of the page fault were processed in a specul=
ative=0A=
>>>> way.=0A=
>>>>=0A=
>>>> Ganesh Mahendran had backported the series on top of a 4.9 kernel and =
gave=0A=
>>>> it a try on an android device. He reported that the application launch=
time=0A=
>>>> was improved in average by 6%, and for large applications (~100 thread=
s) by=0A=
>>>> 20%.=0A=
>>>>=0A=
>>>> Here are the launch time Ganesh mesured on Android 8.0 on top of a Qco=
m=0A=
>>>> MSM845 (8 cores) with 6GB (the less is better):=0A=
>>>>=0A=
>>>> Application 4.9 4.9+spf delta=0A=
>>>> com.tencent.mm 416 389 -7%=0A=
>>>> com.eg.android.AlipayGphone 1135 986 -13%=0A=
>>>> com.tencent.mtt 455 454 0%=0A=
>>>> com.qqgame.hlddz 1497 1409 -6%=0A=
>>>> com.autonavi.minimap 711 701 -1%=0A=
>>>> com.tencent.tmgp.sgame 788 748 -5%=0A=
>>>> com.immomo.momo 501 487 -3%=0A=
>>>> com.tencent.peng 2145 2112 -2%=0A=
>>>> com.smile.gifmaker 491 461 -6%=0A=
>>>> com.baidu.BaiduMap 479 366 -23%=0A=
>>>> com.taobao.taobao 1341 1198 -11%=0A=
>>>> com.baidu.searchbox 333 314 -6%=0A=
>>>> com.tencent.mobileqq 394 384 -3%=0A=
>>>> com.sina.weibo 907 906 0%=0A=
>>>> com.youku.phone 816 731 -11%=0A=
>>>> com.happyelements.AndroidAnimal.qq 763 717 -6%=0A=
>>>> com.UCMobile 415 411 -1%=0A=
>>>> com.tencent.tmgp.ak 1464 1431 -2%=0A=
>>>> com.tencent.qqmusic 336 329 -2%=0A=
>>>> com.sankuai.meituan 1661 1302 -22%=0A=
>>>> com.netease.cloudmusic 1193 1200 1%=0A=
>>>> air.tv.douyu.android 4257 4152 -2%=0A=
>>>>=0A=
>>>> ------------------=0A=
>>>> Benchmarks results=0A=
>>>>=0A=
>>>> Base kernel is v4.17.0-rc4-mm1=0A=
>>>> SPF is BASE + this series=0A=
>>>>=0A=
>>>> Kernbench:=0A=
>>>> ----------=0A=
>>>> Here are the results on a 16 CPUs X86 guest using kernbench on a 4.15=
=0A=
>>>> kernel (kernel is build 5 times):=0A=
>>>>=0A=
>>>> Average Half load -j 8=0A=
>>>> Run (std deviation)=0A=
>>>> BASE SPF=0A=
>>>> Elapsed Time 1448.65 (5.72312) 1455.84 (4.84951) 0.50%=
=0A=
>>>> User Time 10135.4 (30.3699) 10148.8 (31.1252) 0.13%=
=0A=
>>>> System Time 900.47 (2.81131) 923.28 (7.52779) 2.53%=
=0A=
>>>> Percent CPU 761.4 (1.14018) 760.2 (0.447214) -0.16%=
=0A=
>>>> Context Switches 85380 (3419.52) 84748 (1904.44) -0.74%=
=0A=
>>>> Sleeps 105064 (1240.96) 105074 (337.612) 0.01%=
=0A=
>>>>=0A=
>>>> Average Optimal load -j 16=0A=
>>>> Run (std deviation)=0A=
>>>> BASE SPF=0A=
>>>> Elapsed Time 920.528 (10.1212) 927.404 (8.91789) 0.75%=
=0A=
>>>> User Time 11064.8 (981.142) 11085 (990.897) 0.18%=
=0A=
>>>> System Time 979.904 (84.0615) 1001.14 (82.5523) 2.17%=
=0A=
>>>> Percent CPU 1089.5 (345.894) 1086.1 (343.545) -0.31%=
=0A=
>>>> Context Switches 159488 (78156.4) 158223 (77472.1) -0.79%=
=0A=
>>>> Sleeps 110566 (5877.49) 110388 (5617.75) -0.16%=
=0A=
>>>>=0A=
>>>>=0A=
>>>> During a run on the SPF, perf events were captured:=0A=
>>>> Performance counter stats for '../kernbench -M':=0A=
>>>> 526743764 faults=0A=
>>>> 210 spf=0A=
>>>> 3 pagefault:spf_vma_changed=0A=
>>>> 0 pagefault:spf_vma_noanon=0A=
>>>> 2278 pagefault:spf_vma_notsup=0A=
>>>> 0 pagefault:spf_vma_access=0A=
>>>> 0 pagefault:spf_pmd_changed=0A=
>>>>=0A=
>>>> Very few speculative page faults were recorded as most of the processe=
s=0A=
>>>> involved are monothreaded (sounds that on this architecture some threa=
ds=0A=
>>>> were created during the kernel build processing).=0A=
>>>>=0A=
>>>> Here are the kerbench results on a 80 CPUs Power8 system:=0A=
>>>>=0A=
>>>> Average Half load -j 40=0A=
>>>> Run (std deviation)=0A=
>>>> BASE SPF=0A=
>>>> Elapsed Time 117.152 (0.774642) 117.166 (0.476057) 0.01%=
=0A=
>>>> User Time 4478.52 (24.7688) 4479.76 (9.08555) 0.03%=
=0A=
>>>> System Time 131.104 (0.720056) 134.04 (0.708414) 2.24%=
=0A=
>>>> Percent CPU 3934 (19.7104) 3937.2 (19.0184) 0.08%=
=0A=
>>>> Context Switches 92125.4 (576.787) 92581.6 (198.622) 0.50%=
=0A=
>>>> Sleeps 317923 (652.499) 318469 (1255.59) 0.17%=
=0A=
>>>>=0A=
>>>> Average Optimal load -j 80=0A=
>>>> Run (std deviation)=0A=
>>>> BASE SPF=0A=
>>>> Elapsed Time 107.73 (0.632416) 107.31 (0.584936) -0.39%=
=0A=
>>>> User Time 5869.86 (1466.72) 5871.71 (1467.27) 0.03%=
=0A=
>>>> System Time 153.728 (23.8573) 157.153 (24.3704) 2.23%=
=0A=
>>>> Percent CPU 5418.6 (1565.17) 5436.7 (1580.91) 0.33%=
=0A=
>>>> Context Switches 223861 (138865) 225032 (139632) 0.52%=
=0A=
>>>> Sleeps 330529 (13495.1) 332001 (14746.2) 0.45%=
=0A=
>>>>=0A=
>>>> During a run on the SPF, perf events were captured:=0A=
>>>> Performance counter stats for '../kernbench -M':=0A=
>>>> 116730856 faults=0A=
>>>> 0 spf=0A=
>>>> 3 pagefault:spf_vma_changed=0A=
>>>> 0 pagefault:spf_vma_noanon=0A=
>>>> 476 pagefault:spf_vma_notsup=0A=
>>>> 0 pagefault:spf_vma_access=0A=
>>>> 0 pagefault:spf_pmd_changed=0A=
>>>>=0A=
>>>> Most of the processes involved are monothreaded so SPF is not activate=
d but=0A=
>>>> there is no impact on the performance.=0A=
>>>>=0A=
>>>> Ebizzy:=0A=
>>>> -------=0A=
>>>> The test is counting the number of records per second it can manage, t=
he=0A=
>>>> higher is the best. I run it like this 'ebizzy -mTt <nrcpus>'. To get=
=0A=
>>>> consistent result I repeated the test 100 times and measure the averag=
e=0A=
>>>> result. The number is the record processes per second, the higher is t=
he=0A=
>>>> best.=0A=
>>>>=0A=
>>>> BASE SPF delta=0A=
>>>> 16 CPUs x86 VM 742.57 1490.24 100.69%=0A=
>>>> 80 CPUs P8 node 13105.4 24174.23 84.46%=0A=
>>>>=0A=
>>>> Here are the performance counter read during a run on a 16 CPUs x86 VM=
:=0A=
>>>> Performance counter stats for './ebizzy -mTt 16':=0A=
>>>> 1706379 faults=0A=
>>>> 1674599 spf=0A=
>>>> 30588 pagefault:spf_vma_changed=0A=
>>>> 0 pagefault:spf_vma_noanon=0A=
>>>> 363 pagefault:spf_vma_notsup=0A=
>>>> 0 pagefault:spf_vma_access=0A=
>>>> 0 pagefault:spf_pmd_changed=0A=
>>>>=0A=
>>>> And the ones captured during a run on a 80 CPUs Power node:=0A=
>>>> Performance counter stats for './ebizzy -mTt 80':=0A=
>>>> 1874773 faults=0A=
>>>> 1461153 spf=0A=
>>>> 413293 pagefault:spf_vma_changed=0A=
>>>> 0 pagefault:spf_vma_noanon=0A=
>>>> 200 pagefault:spf_vma_notsup=0A=
>>>> 0 pagefault:spf_vma_access=0A=
>>>> 0 pagefault:spf_pmd_changed=0A=
>>>>=0A=
>>>> In ebizzy's case most of the page fault were handled in a speculative =
way,=0A=
>>>> leading the ebizzy performance boost.=0A=
>>>>=0A=
>>>> ------------------=0A=
>>>> Changes since v10 (https://lkml.org/lkml/2018/4/17/572):=0A=
>>>> - Accounted for all review feedbacks from Punit Agrawal, Ganesh Mahen=
dran=0A=
>>>> and Minchan Kim, hopefully.=0A=
>>>> - Remove unneeded check on CONFIG_SPECULATIVE_PAGE_FAULT in=0A=
>>>> __do_page_fault().=0A=
>>>> - Loop in pte_spinlock() and pte_map_lock() when pte try lock fails=
=0A=
>>>> instead=0A=
>>>> of aborting the speculative page fault handling. Dropping the now=
=0A=
>>>> useless=0A=
>>>> trace event pagefault:spf_pte_lock.=0A=
>>>> - No more try to reuse the fetched VMA during the speculative page fa=
ult=0A=
>>>> handling when retrying is needed. This adds a lot of complexity and=
=0A=
>>>> additional tests done didn't show a significant performance improve=
ment.=0A=
>>>> - Convert IS_ENABLED(CONFIG_NUMA) back to #ifdef due to build error.=
=0A=
>>>>=0A=
>>>> [1] http://linux-kernel.2935.n7.nabble.com/RFC-PATCH-0-6-Another-go-at=
-speculative-page-faults-tt965642.html#none=0A=
>>>> [2] https://patchwork.kernel.org/patch/9999687/=0A=
>>>>=0A=
>>>>=0A=
>>>> Laurent Dufour (20):=0A=
>>>> mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT=0A=
>>>> x86/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT=0A=
>>>> powerpc/mm: set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT=0A=
>>>> mm: introduce pte_spinlock for FAULT_FLAG_SPECULATIVE=0A=
>>>> mm: make pte_unmap_same compatible with SPF=0A=
>>>> mm: introduce INIT_VMA()=0A=
>>>> mm: protect VMA modifications using VMA sequence count=0A=
>>>> mm: protect mremap() against SPF hanlder=0A=
>>>> mm: protect SPF handler against anon_vma changes=0A=
>>>> mm: cache some VMA fields in the vm_fault structure=0A=
>>>> mm/migrate: Pass vm_fault pointer to migrate_misplaced_page()=0A=
>>>> mm: introduce __lru_cache_add_active_or_unevictable=0A=
>>>> mm: introduce __vm_normal_page()=0A=
>>>> mm: introduce __page_add_new_anon_rmap()=0A=
>>>> mm: protect mm_rb tree with a rwlock=0A=
>>>> mm: adding speculative page fault failure trace events=0A=
>>>> perf: add a speculative page fault sw event=0A=
>>>> perf tools: add support for the SPF perf event=0A=
>>>> mm: add speculative page fault vmstats=0A=
>>>> powerpc/mm: add speculative page fault=0A=
>>>>=0A=
>>>> Mahendran Ganesh (2):=0A=
>>>> arm64/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT=0A=
>>>> arm64/mm: add speculative page fault=0A=
>>>>=0A=
>>>> Peter Zijlstra (4):=0A=
>>>> mm: prepare for FAULT_FLAG_SPECULATIVE=0A=
>>>> mm: VMA sequence count=0A=
>>>> mm: provide speculative fault infrastructure=0A=
>>>> x86/mm: add speculative pagefault handling=0A=
>>>>=0A=
>>>> arch/arm64/Kconfig | 1 +=0A=
>>>> arch/arm64/mm/fault.c | 12 +=0A=
>>>> arch/powerpc/Kconfig | 1 +=0A=
>>>> arch/powerpc/mm/fault.c | 16 +=0A=
>>>> arch/x86/Kconfig | 1 +=0A=
>>>> arch/x86/mm/fault.c | 27 +-=0A=
>>>> fs/exec.c | 2 +-=0A=
>>>> fs/proc/task_mmu.c | 5 +-=0A=
>>>> fs/userfaultfd.c | 17 +-=0A=
>>>> include/linux/hugetlb_inline.h | 2 +-=0A=
>>>> include/linux/migrate.h | 4 +-=0A=
>>>> include/linux/mm.h | 136 +++++++-=0A=
>>>> include/linux/mm_types.h | 7 +=0A=
>>>> include/linux/pagemap.h | 4 +-=0A=
>>>> include/linux/rmap.h | 12 +-=0A=
>>>> include/linux/swap.h | 10 +-=0A=
>>>> include/linux/vm_event_item.h | 3 +=0A=
>>>> include/trace/events/pagefault.h | 80 +++++=0A=
>>>> include/uapi/linux/perf_event.h | 1 +=0A=
>>>> kernel/fork.c | 5 +-=0A=
>>>> mm/Kconfig | 22 ++=0A=
>>>> mm/huge_memory.c | 6 +-=0A=
>>>> mm/hugetlb.c | 2 +=0A=
>>>> mm/init-mm.c | 3 +=0A=
>>>> mm/internal.h | 20 ++=0A=
>>>> mm/khugepaged.c | 5 +=0A=
>>>> mm/madvise.c | 6 +-=0A=
>>>> mm/memory.c | 612 +++++++++++++++++++++++++=
++++-----=0A=
>>>> mm/mempolicy.c | 51 ++-=0A=
>>>> mm/migrate.c | 6 +-=0A=
>>>> mm/mlock.c | 13 +-=0A=
>>>> mm/mmap.c | 229 ++++++++++---=0A=
>>>> mm/mprotect.c | 4 +-=0A=
>>>> mm/mremap.c | 13 +=0A=
>>>> mm/nommu.c | 2 +-=0A=
>>>> mm/rmap.c | 5 +-=0A=
>>>> mm/swap.c | 6 +-=0A=
>>>> mm/swap_state.c | 8 +-=0A=
>>>> mm/vmstat.c | 5 +-=0A=
>>>> tools/include/uapi/linux/perf_event.h | 1 +=0A=
>>>> tools/perf/util/evsel.c | 1 +=0A=
>>>> tools/perf/util/parse-events.c | 4 +=0A=
>>>> tools/perf/util/parse-events.l | 1 +=0A=
>>>> tools/perf/util/python.c | 1 +=0A=
>>>> 44 files changed, 1161 insertions(+), 211 deletions(-)=0A=
>>>> create mode 100644 include/trace/events/pagefault.h=0A=
>>>>=0A=
>>>> --=0A=
>>>> 2.7.4=0A=
>>>>=0A=
>>>>=0A=
>>>=0A=
>>=0A=
>=0A=
=0A=
^ permalink raw reply
* Re: [PATCHv5 4/4] arm64: Add build salt to the vDSO
From: Masahiro Yamada @ 2018-07-04 3:55 UTC (permalink / raw)
To: Laura Abbott
Cc: Mark Wielaard, H . J . Lu, Catalin Marinas, Will Deacon,
Andy Lutomirski, Linus Torvalds, X86 ML,
Linux Kernel Mailing List, Nick Clifton, Cary Coutant,
Linux Kbuild mailing list, linuxppc-dev, Michael Ellerman,
linux-arm-kernel
In-Reply-To: <20180703233430.14416-5-labbott@redhat.com>
Hi.
2018-07-04 8:34 GMT+09:00 Laura Abbott <labbott@redhat.com>:
>
> The vDSO needs to have a unique build id in a similar manner
> to the kernel and modules. Use the build salt macro.
>
> Signed-off-by: Laura Abbott <labbott@redhat.com>
> ---
> v5: I was previously focused on x86 only but since powerpc gave a patch,
> I figured I would do arm64 since the changes were also fairly simple.
> ---
> arch/arm64/kernel/vdso/note.S | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/arm64/kernel/vdso/note.S b/arch/arm64/kernel/vdso/note.S
> index b82c85e5d972..2c429dfd3f45 100644
> --- a/arch/arm64/kernel/vdso/note.S
> +++ b/arch/arm64/kernel/vdso/note.S
> @@ -22,7 +22,10 @@
> #include <linux/uts.h>
> #include <linux/version.h>
> #include <linux/elfnote.h>
> +#include <linux/build-salt.h>
>
> ELFNOTE_START(Linux, 0, "a")
> .long LINUX_VERSION_CODE
> ELFNOTE_END
> +
> +BUILD_SALT;
I think this works, but
I prefer no-semicolon in assembly files.
For coding consistency,
I want ';' as statement delimiter in .c files.
But, only new line after each statement in .S files.
For example, in arch/x86/xen/xen-head.S
I see no semicolon after ELFNOTE().
I found this:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0473k/dom1359731141352.html
It says ';' starts a comment line
although it is not the case of GAS.
Same for 3/4.
--
Best Regards
Masahiro Yamada
^ permalink raw reply
* Re: [PATCHv5 0/4] Salted build ids via ELF notes
From: Masahiro Yamada @ 2018-07-04 3:57 UTC (permalink / raw)
To: Laura Abbott
Cc: Andy Lutomirski, Mark Wielaard, H . J . Lu, Michael Ellerman,
Catalin Marinas, Will Deacon, Linus Torvalds, X86 ML,
Linux Kernel Mailing List, Nick Clifton, Cary Coutant,
Linux Kbuild mailing list, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180703232200.11315-1-labbott@redhat.com>
Hi.
2018-07-04 8:21 GMT+09:00 Laura Abbott <labbott@redhat.com>:
>
> Hi,
>
> This is v5 of the series to allow unique build ids in the kernel. As a
> reminder of the context:
> ""
> In Fedora, the debug information is packaged separately (foo-debuginfo) and
> can be installed separately. There's been a long standing issue where only one
> version of a debuginfo info package can be installed at a time. Mark Wielaard
> made an effort for Fedora 27 to allow parallel installation of debuginfo (see
> https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo for
> more details)
>
> Part of the requirement to allow this to work is that build ids are
> unique between builds. The existing upstream rpm implementation ensures
> this by re-calculating the build-id using the version and release as a
> seed. This doesn't work 100% for the kernel because of the vDSO which is
> its own binary and doesn't get updated. After poking holes in a few of my
> ideas, there was a discussion with some people from the binutils team about
> adding --build-id-salt to let ld do the calculation debugedit is doing. There
> was a counter proposal made to add in the salt while building. The
> easiest proposal was to add an item in the linker script vs. linking in
> an object since we need the salt to go in every module as well as the
> kernel and vmlinux.
> ""
I think this information is helpful to explain the background
of this work, but the cover letter cannot be committed in git.
Could you add this in 1/4 please?
If I read only the simple log in 1/4,
I would wonder why it is useful...
> v5 uses the approach suggested by Masahiro Yamada which uses the
> existing ELF note macro to more easily add the salt (vs previous
> approaches which tried to adjust via linker section).
>
> If arch maintainers are okay, I'd like acks for this so this can go
> through the kbuild tree.
>
> Thanks,
> Laura
>
> Laura Abbott (4):
> kbuild: Add build salt to the kernel and modules
> x86: Add build salt to the vDSO
> powerpc: Add build salt to the vDSO
> arm64: Add build salt to the vDSO
>
> arch/arm64/kernel/vdso/note.S | 3 +++
> arch/powerpc/kernel/vdso32/note.S | 3 +++
> arch/x86/entry/vdso/vdso-note.S | 3 +++
> arch/x86/entry/vdso/vdso32/note.S | 3 +++
> include/linux/build-salt.h | 20 ++++++++++++++++++++
> init/Kconfig | 9 +++++++++
> init/version.c | 3 +++
> scripts/mod/modpost.c | 3 +++
> 8 files changed, 47 insertions(+)
> create mode 100644 include/linux/build-salt.h
>
> --
> 2.17.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Best Regards
Masahiro Yamada
^ permalink raw reply
* Re: [PATCHv5 1/4] kbuild: Add build salt to the kernel and modules
From: Masahiro Yamada @ 2018-07-04 3:59 UTC (permalink / raw)
To: Laura Abbott
Cc: Andy Lutomirski, Mark Wielaard, H . J . Lu, Michael Ellerman,
Catalin Marinas, Will Deacon, Linus Torvalds, X86 ML,
Linux Kernel Mailing List, Nick Clifton, Cary Coutant,
Linux Kbuild mailing list, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180703233430.14416-2-labbott@redhat.com>
Hi.
Thanks for the update.
2018-07-04 8:34 GMT+09:00 Laura Abbott <labbott@redhat.com>:
>
> The build id generated from --build-id can be generated in several different
> ways, with the default being the sha1 on the output of the linked file. For
> distributions, it can be useful to make sure this ID is unique, even if the
> actual file contents don't change. The easiest way to do this is to insert
> a section with some data.
>
> Add an ELF note to both the kernel and module which contains some data based
> off of a config option.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Signed-off-by: Laura Abbott <labbott@redhat.com>
> ---
> v5: I used S-o-b here since the majority of the code was written
> already.
I think Suggested-by is good enough.
S-o-b is appended as a patch is passed from people to people.
Anyway, this looks good except one bike-shed.
> Please feel free to change the tag if you think it's not
> appropriate. I also tweaked this to take an ascii string instead of just
> a hex value since this makes things much easier on the distribution
> side.
> ---
> diff --git a/init/Kconfig b/init/Kconfig
> index 041f3a022122..8de789f40db9 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -107,6 +107,15 @@ config LOCALVERSION_AUTO
>
> which is done within the script "scripts/setlocalversion".)
>
> +config BUILD_SALT
> + string "Build ID Salt"
> + default "Linux"
How about empty string ""
for default?
--
Best Regards
Masahiro Yamada
^ permalink raw reply
* [PATCH v6 0/4] resource: Use list_head to link sibling resource
From: Baoquan He @ 2018-07-04 4:10 UTC (permalink / raw)
To: linux-kernel, akpm, robh+dt, dan.j.williams, nicolas.pitre, josh,
fengguang.wu, bp, andy.shevchenko
Cc: patrik.r.jakobsson, airlied, kys, haiyangz, sthemmin,
dmitry.torokhov, frowand.list, keith.busch, jonathan.derrick,
lorenzo.pieralisi, bhelgaas, tglx, brijesh.singh, jglisse,
thomas.lendacky, gregkh, baiyaowei, richard.weiyang, devel,
linux-input, linux-nvdimm, devicetree, linux-pci, ebiederm,
vgoyal, dyoung, yinghai, kexec, monstr, davem, chris, jcmvbkbc,
gustavo, maarten.lankhorst, seanpaul, linux-parisc, linuxppc-dev,
Baoquan He
This patchset is doing:
1) Replace struct resource's sibling list from singly linked list to
list_head. Clearing out those pointer operation within singly linked
list for better code readability.
2) Based on list_head replacement, add a new function
walk_system_ram_res_rev() which can does reversed iteration on
iomem_resource's siblings.
3) Change kexec_file loading to search system RAM top down for kernel
loadin, using walk_system_ram_res_rev().
Note:
This patchset only passed testing on x86_64 arch with network
enabling. The thing we need pay attetion to is that a root resource's
child member need be initialized specifically with LIST_HEAD_INIT() if
statically defined or INIT_LIST_HEAD() for dynamically definition. Here
Just like we do for iomem_resource/ioport_resource, or the change in
get_pci_domain_busn_res().
v5:
http://lkml.kernel.org/r/20180612032831.29747-1-bhe@redhat.com
v4:
http://lkml.kernel.org/r/20180507063224.24229-1-bhe@redhat.com
v3:
http://lkml.kernel.org/r/20180419001848.3041-1-bhe@redhat.com
v2:
http://lkml.kernel.org/r/20180408024724.16812-1-bhe@redhat.com
v1:
http://lkml.kernel.org/r/20180322033722.9279-1-bhe@redhat.com
Changelog:
v5->v6:
Fix code style problems in reparent_resources() and use existing
error codes, according to Andy's suggestion.
Fix bugs test robot reported.
v4->v5:
Add new patch 0001 to move duplicated reparent_resources() to
kernel/resource.c to make it be shared by different ARCH-es.
Fix several code bugs reported by test robot on ARCH powerpc and
microblaze.
v3->v4:
Fix several bugs test robot reported. Rewrite cover letter and patch
log according to reviewer's comment.
v2->v3:
Rename resource functions first_child() and sibling() to
resource_first_chils() and resource_sibling(). Dan suggested this.
Move resource_first_chils() and resource_sibling() to linux/ioport.h
and make them as inline function. Rob suggested this. Accordingly add
linux/list.h including in linux/ioport.h, please help review if this
bring efficiency degradation or code redundancy.
The change on struct resource {} bring two pointers of size increase,
mention this in git log to make it more specifically, Rob suggested
this.
v1->v2:
Use list_head instead to link resource siblings. This is suggested by
Andrew.
Rewrite walk_system_ram_res_rev() after list_head is taken to link
resouce siblings.
Baoquan He (4):
resource: Move reparent_resources() to kernel/resource.c and make it
public
resource: Use list_head to link sibling resource
resource: add walk_system_ram_res_rev()
kexec_file: Load kernel at top of system RAM if required
arch/arm/plat-samsung/pm-check.c | 6 +-
arch/microblaze/pci/pci-common.c | 41 +----
arch/powerpc/kernel/pci-common.c | 39 +----
arch/sparc/kernel/ioport.c | 2 +-
arch/xtensa/include/asm/pci-bridge.h | 4 +-
drivers/eisa/eisa-bus.c | 2 +
drivers/gpu/drm/drm_memory.c | 3 +-
drivers/gpu/drm/gma500/gtt.c | 5 +-
drivers/hv/vmbus_drv.c | 52 +++---
drivers/input/joystick/iforce/iforce-main.c | 4 +-
drivers/nvdimm/namespace_devs.c | 6 +-
drivers/nvdimm/nd.h | 5 +-
drivers/of/address.c | 4 +-
drivers/parisc/lba_pci.c | 4 +-
drivers/pci/controller/vmd.c | 8 +-
drivers/pci/probe.c | 2 +
drivers/pci/setup-bus.c | 2 +-
include/linux/ioport.h | 21 ++-
kernel/kexec_file.c | 2 +
kernel/resource.c | 263 ++++++++++++++++++----------
20 files changed, 248 insertions(+), 227 deletions(-)
--
2.13.6
^ permalink raw reply
* [PATCH v6 1/4] resource: Move reparent_resources() to kernel/resource.c and make it public
From: Baoquan He @ 2018-07-04 4:10 UTC (permalink / raw)
To: linux-kernel, akpm, robh+dt, dan.j.williams, nicolas.pitre, josh,
fengguang.wu, bp, andy.shevchenko
Cc: patrik.r.jakobsson, airlied, kys, haiyangz, sthemmin,
dmitry.torokhov, frowand.list, keith.busch, jonathan.derrick,
lorenzo.pieralisi, bhelgaas, tglx, brijesh.singh, jglisse,
thomas.lendacky, gregkh, baiyaowei, richard.weiyang, devel,
linux-input, linux-nvdimm, devicetree, linux-pci, ebiederm,
vgoyal, dyoung, yinghai, kexec, monstr, davem, chris, jcmvbkbc,
gustavo, maarten.lankhorst, seanpaul, linux-parisc, linuxppc-dev,
Baoquan He
In-Reply-To: <20180704041038.8190-1-bhe@redhat.com>
reparent_resources() is duplicated in arch/microblaze/pci/pci-common.c
and arch/powerpc/kernel/pci-common.c, so move it to kernel/resource.c
so that it's shared.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/microblaze/pci/pci-common.c | 37 -------------------------------------
arch/powerpc/kernel/pci-common.c | 35 -----------------------------------
include/linux/ioport.h | 1 +
kernel/resource.c | 39 +++++++++++++++++++++++++++++++++++++++
4 files changed, 40 insertions(+), 72 deletions(-)
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index f34346d56095..7899bafab064 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -619,43 +619,6 @@ int pcibios_add_device(struct pci_dev *dev)
EXPORT_SYMBOL(pcibios_add_device);
/*
- * Reparent resource children of pr that conflict with res
- * under res, and make res replace those children.
- */
-static int __init reparent_resources(struct resource *parent,
- struct resource *res)
-{
- struct resource *p, **pp;
- struct resource **firstpp = NULL;
-
- for (pp = &parent->child; (p = *pp) != NULL; pp = &p->sibling) {
- if (p->end < res->start)
- continue;
- if (res->end < p->start)
- break;
- if (p->start < res->start || p->end > res->end)
- return -1; /* not completely contained */
- if (firstpp == NULL)
- firstpp = pp;
- }
- if (firstpp == NULL)
- return -1; /* didn't find any conflicting entries? */
- res->parent = parent;
- res->child = *firstpp;
- res->sibling = *pp;
- *firstpp = res;
- *pp = NULL;
- for (p = res->child; p != NULL; p = p->sibling) {
- p->parent = res;
- pr_debug("PCI: Reparented %s [%llx..%llx] under %s\n",
- p->name,
- (unsigned long long)p->start,
- (unsigned long long)p->end, res->name);
- }
- return 0;
-}
-
-/*
* Handle resources of PCI devices. If the world were perfect, we could
* just allocate all the resource regions and do nothing more. It isn't.
* On the other hand, we cannot just re-allocate all devices, as it would
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index fe9733ffffaa..926035bb378d 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -1088,41 +1088,6 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
EXPORT_SYMBOL(pcibios_align_resource);
/*
- * Reparent resource children of pr that conflict with res
- * under res, and make res replace those children.
- */
-static int reparent_resources(struct resource *parent,
- struct resource *res)
-{
- struct resource *p, **pp;
- struct resource **firstpp = NULL;
-
- for (pp = &parent->child; (p = *pp) != NULL; pp = &p->sibling) {
- if (p->end < res->start)
- continue;
- if (res->end < p->start)
- break;
- if (p->start < res->start || p->end > res->end)
- return -1; /* not completely contained */
- if (firstpp == NULL)
- firstpp = pp;
- }
- if (firstpp == NULL)
- return -1; /* didn't find any conflicting entries? */
- res->parent = parent;
- res->child = *firstpp;
- res->sibling = *pp;
- *firstpp = res;
- *pp = NULL;
- for (p = res->child; p != NULL; p = p->sibling) {
- p->parent = res;
- pr_debug("PCI: Reparented %s %pR under %s\n",
- p->name, p, res->name);
- }
- return 0;
-}
-
-/*
* Handle resources of PCI devices. If the world were perfect, we could
* just allocate all the resource regions and do nothing more. It isn't.
* On the other hand, we cannot just re-allocate all devices, as it would
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index da0ebaec25f0..dfdcd0bfe54e 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -192,6 +192,7 @@ extern int allocate_resource(struct resource *root, struct resource *new,
struct resource *lookup_resource(struct resource *root, resource_size_t start);
int adjust_resource(struct resource *res, resource_size_t start,
resource_size_t size);
+int reparent_resources(struct resource *parent, struct resource *res);
resource_size_t resource_alignment(struct resource *res);
static inline resource_size_t resource_size(const struct resource *res)
{
diff --git a/kernel/resource.c b/kernel/resource.c
index 30e1bc68503b..d1cbf4b50e17 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -983,6 +983,45 @@ int adjust_resource(struct resource *res, resource_size_t start,
}
EXPORT_SYMBOL(adjust_resource);
+/*
+ * reparent_resources - reparent resource children of parent that res covers
+ * @parent: parent resource descriptor
+ * @res: resource descriptor desired by caller
+ *
+ * Reparent resource children of 'parent' that conflict with 'res'
+ * under 'res', and make 'res' replace those children.
+ */
+int reparent_resources(struct resource *parent, struct resource *res)
+{
+ struct resource *p, **pp;
+ struct resource **firstpp = NULL;
+
+ for (pp = &parent->child; (p = *pp) != NULL; pp = &p->sibling) {
+ if (p->end < res->start)
+ continue;
+ if (res->end < p->start)
+ break;
+ if (p->start < res->start || p->end > res->end)
+ return -ENOTSUPP; /* not completely contained */
+ if (firstpp == NULL)
+ firstpp = pp;
+ }
+ if (firstpp == NULL)
+ return -ECANCELED; /* didn't find any conflicting entries? */
+ res->parent = parent;
+ res->child = *firstpp;
+ res->sibling = *pp;
+ *firstpp = res;
+ *pp = NULL;
+ for (p = res->child; p != NULL; p = p->sibling) {
+ p->parent = res;
+ pr_debug("PCI: Reparented %s %pR under %s\n",
+ p->name, p, res->name);
+ }
+ return 0;
+}
+EXPORT_SYMBOL(reparent_resources);
+
static void __init __reserve_region_with_split(struct resource *root,
resource_size_t start, resource_size_t end,
const char *name)
--
2.13.6
^ permalink raw reply related
* [PATCH v6 2/4] resource: Use list_head to link sibling resource
From: Baoquan He @ 2018-07-04 4:10 UTC (permalink / raw)
To: linux-kernel, akpm, robh+dt, dan.j.williams, nicolas.pitre, josh,
fengguang.wu, bp, andy.shevchenko
Cc: patrik.r.jakobsson, airlied, kys, haiyangz, sthemmin,
dmitry.torokhov, frowand.list, keith.busch, jonathan.derrick,
lorenzo.pieralisi, bhelgaas, tglx, brijesh.singh, jglisse,
thomas.lendacky, gregkh, baiyaowei, richard.weiyang, devel,
linux-input, linux-nvdimm, devicetree, linux-pci, ebiederm,
vgoyal, dyoung, yinghai, kexec, monstr, davem, chris, jcmvbkbc,
gustavo, maarten.lankhorst, seanpaul, linux-parisc, linuxppc-dev,
Baoquan He
In-Reply-To: <20180704041038.8190-1-bhe@redhat.com>
The struct resource uses singly linked list to link siblings, implemented
by pointer operation. Replace it with list_head for better code readability.
Based on this list_head replacement, it will be very easy to do reverse
iteration on iomem_resource's sibling list in later patch.
Besides, type of member variables of struct resource, sibling and child, are
changed from 'struct resource *' to 'struct list_head'. This brings two
pointers of size increase.
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Baoquan He <bhe@redhat.com>
Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
Cc: David Airlie <airlied@linux.ie>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Frank Rowand <frowand.list@gmail.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Jonathan Derrick <jonathan.derrick@intel.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Yaowei Bai <baiyaowei@cmss.chinamobile.com>
Cc: Wei Yang <richard.weiyang@gmail.com>
Cc: devel@linuxdriverproject.org
Cc: linux-input@vger.kernel.org
Cc: linux-nvdimm@lists.01.org
Cc: devicetree@vger.kernel.org
Cc: linux-pci@vger.kernel.org
---
arch/arm/plat-samsung/pm-check.c | 6 +-
arch/microblaze/pci/pci-common.c | 4 +-
arch/powerpc/kernel/pci-common.c | 4 +-
arch/sparc/kernel/ioport.c | 2 +-
arch/xtensa/include/asm/pci-bridge.h | 4 +-
drivers/eisa/eisa-bus.c | 2 +
drivers/gpu/drm/drm_memory.c | 3 +-
drivers/gpu/drm/gma500/gtt.c | 5 +-
drivers/hv/vmbus_drv.c | 52 +++----
drivers/input/joystick/iforce/iforce-main.c | 4 +-
drivers/nvdimm/namespace_devs.c | 6 +-
drivers/nvdimm/nd.h | 5 +-
drivers/of/address.c | 4 +-
drivers/parisc/lba_pci.c | 4 +-
drivers/pci/controller/vmd.c | 8 +-
drivers/pci/probe.c | 2 +
drivers/pci/setup-bus.c | 2 +-
include/linux/ioport.h | 17 ++-
kernel/resource.c | 208 ++++++++++++++--------------
19 files changed, 175 insertions(+), 167 deletions(-)
diff --git a/arch/arm/plat-samsung/pm-check.c b/arch/arm/plat-samsung/pm-check.c
index cd2c02c68bc3..5494355b1c49 100644
--- a/arch/arm/plat-samsung/pm-check.c
+++ b/arch/arm/plat-samsung/pm-check.c
@@ -46,8 +46,8 @@ typedef u32 *(run_fn_t)(struct resource *ptr, u32 *arg);
static void s3c_pm_run_res(struct resource *ptr, run_fn_t fn, u32 *arg)
{
while (ptr != NULL) {
- if (ptr->child != NULL)
- s3c_pm_run_res(ptr->child, fn, arg);
+ if (!list_empty(&ptr->child))
+ s3c_pm_run_res(resource_first_child(&ptr->child), fn, arg);
if ((ptr->flags & IORESOURCE_SYSTEM_RAM)
== IORESOURCE_SYSTEM_RAM) {
@@ -57,7 +57,7 @@ static void s3c_pm_run_res(struct resource *ptr, run_fn_t fn, u32 *arg)
arg = (fn)(ptr, arg);
}
- ptr = ptr->sibling;
+ ptr = resource_sibling(ptr);
}
}
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index 7899bafab064..2bf73e27e231 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -533,7 +533,9 @@ void pci_process_bridge_OF_ranges(struct pci_controller *hose,
res->flags = range.flags;
res->start = range.cpu_addr;
res->end = range.cpu_addr + range.size - 1;
- res->parent = res->child = res->sibling = NULL;
+ res->parent = NULL;
+ INIT_LIST_HEAD(&res->child);
+ INIT_LIST_HEAD(&res->sibling);
}
}
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 926035bb378d..28fbe83c9daf 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -761,7 +761,9 @@ void pci_process_bridge_OF_ranges(struct pci_controller *hose,
res->flags = range.flags;
res->start = range.cpu_addr;
res->end = range.cpu_addr + range.size - 1;
- res->parent = res->child = res->sibling = NULL;
+ res->parent = NULL;
+ INIT_LIST_HEAD(&res->child);
+ INIT_LIST_HEAD(&res->sibling);
}
}
}
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index cca9134cfa7d..99efe4e98b16 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -669,7 +669,7 @@ static int sparc_io_proc_show(struct seq_file *m, void *v)
struct resource *root = m->private, *r;
const char *nm;
- for (r = root->child; r != NULL; r = r->sibling) {
+ list_for_each_entry(r, &root->child, sibling) {
if ((nm = r->name) == NULL) nm = "???";
seq_printf(m, "%016llx-%016llx: %s\n",
(unsigned long long)r->start,
diff --git a/arch/xtensa/include/asm/pci-bridge.h b/arch/xtensa/include/asm/pci-bridge.h
index 0b68c76ec1e6..f487b06817df 100644
--- a/arch/xtensa/include/asm/pci-bridge.h
+++ b/arch/xtensa/include/asm/pci-bridge.h
@@ -71,8 +71,8 @@ static inline void pcibios_init_resource(struct resource *res,
res->flags = flags;
res->name = name;
res->parent = NULL;
- res->sibling = NULL;
- res->child = NULL;
+ INIT_LIST_HEAD(&res->child);
+ INIT_LIST_HEAD(&res->sibling);
}
diff --git a/drivers/eisa/eisa-bus.c b/drivers/eisa/eisa-bus.c
index 1e8062f6dbfc..dba78f75fd06 100644
--- a/drivers/eisa/eisa-bus.c
+++ b/drivers/eisa/eisa-bus.c
@@ -408,6 +408,8 @@ static struct resource eisa_root_res = {
.start = 0,
.end = 0xffffffff,
.flags = IORESOURCE_IO,
+ .sibling = LIST_HEAD_INIT(eisa_root_res.sibling),
+ .child = LIST_HEAD_INIT(eisa_root_res.child),
};
static int eisa_bus_count;
diff --git a/drivers/gpu/drm/drm_memory.c b/drivers/gpu/drm/drm_memory.c
index d69e4fc1ee77..33baa7fa5e41 100644
--- a/drivers/gpu/drm/drm_memory.c
+++ b/drivers/gpu/drm/drm_memory.c
@@ -155,9 +155,8 @@ u64 drm_get_max_iomem(void)
struct resource *tmp;
resource_size_t max_iomem = 0;
- for (tmp = iomem_resource.child; tmp; tmp = tmp->sibling) {
+ list_for_each_entry(tmp, &iomem_resource.child, sibling)
max_iomem = max(max_iomem, tmp->end);
- }
return max_iomem;
}
diff --git a/drivers/gpu/drm/gma500/gtt.c b/drivers/gpu/drm/gma500/gtt.c
index 3949b0990916..addd3bc009af 100644
--- a/drivers/gpu/drm/gma500/gtt.c
+++ b/drivers/gpu/drm/gma500/gtt.c
@@ -565,7 +565,7 @@ int psb_gtt_init(struct drm_device *dev, int resume)
int psb_gtt_restore(struct drm_device *dev)
{
struct drm_psb_private *dev_priv = dev->dev_private;
- struct resource *r = dev_priv->gtt_mem->child;
+ struct resource *r;
struct gtt_range *range;
unsigned int restored = 0, total = 0, size = 0;
@@ -573,14 +573,13 @@ int psb_gtt_restore(struct drm_device *dev)
mutex_lock(&dev_priv->gtt_mutex);
psb_gtt_init(dev, 1);
- while (r != NULL) {
+ list_for_each_entry(r, &dev_priv->gtt_mem->child, sibling) {
range = container_of(r, struct gtt_range, resource);
if (range->pages) {
psb_gtt_insert(dev, range, 1);
size += range->resource.end - range->resource.start;
restored++;
}
- r = r->sibling;
total++;
}
mutex_unlock(&dev_priv->gtt_mutex);
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index b10fe26c4891..d87ec5a1bc4c 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1412,9 +1412,8 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx)
{
resource_size_t start = 0;
resource_size_t end = 0;
- struct resource *new_res;
+ struct resource *new_res, *tmp;
struct resource **old_res = &hyperv_mmio;
- struct resource **prev_res = NULL;
switch (res->type) {
@@ -1461,44 +1460,36 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx)
/*
* If two ranges are adjacent, merge them.
*/
- do {
- if (!*old_res) {
- *old_res = new_res;
- break;
- }
-
- if (((*old_res)->end + 1) == new_res->start) {
- (*old_res)->end = new_res->end;
+ if (!*old_res) {
+ *old_res = new_res;
+ return AE_OK;
+ }
+ tmp = *old_res;
+ list_for_each_entry_from(tmp, &tmp->parent->child, sibling) {
+ if ((tmp->end + 1) == new_res->start) {
+ tmp->end = new_res->end;
kfree(new_res);
break;
}
- if ((*old_res)->start == new_res->end + 1) {
- (*old_res)->start = new_res->start;
+ if (tmp->start == new_res->end + 1) {
+ tmp->start = new_res->start;
kfree(new_res);
break;
}
- if ((*old_res)->start > new_res->end) {
- new_res->sibling = *old_res;
- if (prev_res)
- (*prev_res)->sibling = new_res;
- *old_res = new_res;
+ if (tmp->start > new_res->end) {
+ list_add(&new_res->sibling, tmp->sibling.prev);
break;
}
-
- prev_res = old_res;
- old_res = &(*old_res)->sibling;
-
- } while (1);
+ }
return AE_OK;
}
static int vmbus_acpi_remove(struct acpi_device *device)
{
- struct resource *cur_res;
- struct resource *next_res;
+ struct resource *res;
if (hyperv_mmio) {
if (fb_mmio) {
@@ -1507,10 +1498,9 @@ static int vmbus_acpi_remove(struct acpi_device *device)
fb_mmio = NULL;
}
- for (cur_res = hyperv_mmio; cur_res; cur_res = next_res) {
- next_res = cur_res->sibling;
- kfree(cur_res);
- }
+ res = hyperv_mmio;
+ list_for_each_entry_from(res, &res->parent->child, sibling)
+ kfree(res);
}
return 0;
@@ -1596,7 +1586,8 @@ int vmbus_allocate_mmio(struct resource **new, struct hv_device *device_obj,
}
}
- for (iter = hyperv_mmio; iter; iter = iter->sibling) {
+ iter = hyperv_mmio;
+ list_for_each_entry_from(iter, &iter->parent->child, sibling) {
if ((iter->start >= max) || (iter->end <= min))
continue;
@@ -1639,7 +1630,8 @@ void vmbus_free_mmio(resource_size_t start, resource_size_t size)
struct resource *iter;
down(&hyperv_mmio_lock);
- for (iter = hyperv_mmio; iter; iter = iter->sibling) {
+ iter = hyperv_mmio;
+ list_for_each_entry_from(iter, &iter->parent->child, sibling) {
if ((iter->start >= start + size) || (iter->end <= start))
continue;
diff --git a/drivers/input/joystick/iforce/iforce-main.c b/drivers/input/joystick/iforce/iforce-main.c
index daeeb4c7e3b0..5c0be27b33ff 100644
--- a/drivers/input/joystick/iforce/iforce-main.c
+++ b/drivers/input/joystick/iforce/iforce-main.c
@@ -305,8 +305,8 @@ int iforce_init_device(struct iforce *iforce)
iforce->device_memory.end = 200;
iforce->device_memory.flags = IORESOURCE_MEM;
iforce->device_memory.parent = NULL;
- iforce->device_memory.child = NULL;
- iforce->device_memory.sibling = NULL;
+ INIT_LIST_HEAD(&iforce->device_memory.child);
+ INIT_LIST_HEAD(&iforce->device_memory.sibling);
/*
* Wait until device ready - until it sends its first response.
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 28afdd668905..f53d410d9981 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -637,7 +637,7 @@ static resource_size_t scan_allocate(struct nd_region *nd_region,
retry:
first = 0;
for_each_dpa_resource(ndd, res) {
- struct resource *next = res->sibling, *new_res = NULL;
+ struct resource *next = resource_sibling(res), *new_res = NULL;
resource_size_t allocate, available = 0;
enum alloc_loc loc = ALLOC_ERR;
const char *action;
@@ -763,7 +763,7 @@ static resource_size_t scan_allocate(struct nd_region *nd_region,
* an initial "pmem-reserve pass". Only do an initial BLK allocation
* when none of the DPA space is reserved.
*/
- if ((is_pmem || !ndd->dpa.child) && n == to_allocate)
+ if ((is_pmem || list_empty(&ndd->dpa.child)) && n == to_allocate)
return init_dpa_allocation(label_id, nd_region, nd_mapping, n);
return n;
}
@@ -779,7 +779,7 @@ static int merge_dpa(struct nd_region *nd_region,
retry:
for_each_dpa_resource(ndd, res) {
int rc;
- struct resource *next = res->sibling;
+ struct resource *next = resource_sibling(res);
resource_size_t end = res->start + resource_size(res);
if (!next || strcmp(res->name, label_id->id) != 0
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 32e0364b48b9..da7da15e03e7 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -102,11 +102,10 @@ unsigned sizeof_namespace_label(struct nvdimm_drvdata *ndd);
(unsigned long long) (res ? res->start : 0), ##arg)
#define for_each_dpa_resource(ndd, res) \
- for (res = (ndd)->dpa.child; res; res = res->sibling)
+ list_for_each_entry(res, &(ndd)->dpa.child, sibling)
#define for_each_dpa_resource_safe(ndd, res, next) \
- for (res = (ndd)->dpa.child, next = res ? res->sibling : NULL; \
- res; res = next, next = next ? next->sibling : NULL)
+ list_for_each_entry_safe(res, next, &(ndd)->dpa.child, sibling)
struct nd_percpu_lane {
int count;
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 53349912ac75..e2e25719ab52 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -330,7 +330,9 @@ int of_pci_range_to_resource(struct of_pci_range *range,
{
int err;
res->flags = range->flags;
- res->parent = res->child = res->sibling = NULL;
+ res->parent = NULL;
+ INIT_LIST_HEAD(&res->child);
+ INIT_LIST_HEAD(&res->sibling);
res->name = np->full_name;
if (res->flags & IORESOURCE_IO) {
diff --git a/drivers/parisc/lba_pci.c b/drivers/parisc/lba_pci.c
index 69bd98421eb1..7482bdfd1959 100644
--- a/drivers/parisc/lba_pci.c
+++ b/drivers/parisc/lba_pci.c
@@ -170,8 +170,8 @@ lba_dump_res(struct resource *r, int d)
for (i = d; i ; --i) printk(" ");
printk(KERN_DEBUG "%p [%lx,%lx]/%lx\n", r,
(long)r->start, (long)r->end, r->flags);
- lba_dump_res(r->child, d+2);
- lba_dump_res(r->sibling, d);
+ lba_dump_res(resource_first_child(&r->child), d+2);
+ lba_dump_res(resource_sibling(r), d);
}
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 942b64fc7f1f..e3ace20345c7 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -542,14 +542,14 @@ static struct pci_ops vmd_ops = {
static void vmd_attach_resources(struct vmd_dev *vmd)
{
- vmd->dev->resource[VMD_MEMBAR1].child = &vmd->resources[1];
- vmd->dev->resource[VMD_MEMBAR2].child = &vmd->resources[2];
+ list_add(&vmd->resources[1].sibling, &vmd->dev->resource[VMD_MEMBAR1].child);
+ list_add(&vmd->resources[2].sibling, &vmd->dev->resource[VMD_MEMBAR2].child);
}
static void vmd_detach_resources(struct vmd_dev *vmd)
{
- vmd->dev->resource[VMD_MEMBAR1].child = NULL;
- vmd->dev->resource[VMD_MEMBAR2].child = NULL;
+ INIT_LIST_HEAD(&vmd->dev->resource[VMD_MEMBAR1].child);
+ INIT_LIST_HEAD(&vmd->dev->resource[VMD_MEMBAR2].child);
}
/*
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index ac876e32de4b..9624dd1dfd49 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -59,6 +59,8 @@ static struct resource *get_pci_domain_busn_res(int domain_nr)
r->res.start = 0;
r->res.end = 0xff;
r->res.flags = IORESOURCE_BUS | IORESOURCE_PCI_FIXED;
+ INIT_LIST_HEAD(&r->res.child);
+ INIT_LIST_HEAD(&r->res.sibling);
list_add_tail(&r->list, &pci_domain_busn_res_list);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 79b1824e83b4..8e685af8938d 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2107,7 +2107,7 @@ int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
continue;
/* Ignore BARs which are still in use */
- if (res->child)
+ if (!list_empty(&res->child))
continue;
ret = add_to_list(&saved, bridge, res, 0, 0);
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index dfdcd0bfe54e..b7456ae889dd 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -12,6 +12,7 @@
#ifndef __ASSEMBLY__
#include <linux/compiler.h>
#include <linux/types.h>
+#include <linux/list.h>
/*
* Resources are tree-like, allowing
* nesting etc..
@@ -22,7 +23,8 @@ struct resource {
const char *name;
unsigned long flags;
unsigned long desc;
- struct resource *parent, *sibling, *child;
+ struct list_head child, sibling;
+ struct resource *parent;
};
/*
@@ -216,7 +218,6 @@ static inline bool resource_contains(struct resource *r1, struct resource *r2)
return r1->start <= r2->start && r1->end >= r2->end;
}
-
/* Convenience shorthand with allocation */
#define request_region(start,n,name) __request_region(&ioport_resource, (start), (n), (name), 0)
#define request_muxed_region(start,n,name) __request_region(&ioport_resource, (start), (n), (name), IORESOURCE_MUXED)
@@ -287,6 +288,18 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)
return (r1->start <= r2->end && r1->end >= r2->start);
}
+static inline struct resource *resource_sibling(struct resource *res)
+{
+ if (res->parent && !list_is_last(&res->sibling, &res->parent->child))
+ return list_next_entry(res, sibling);
+ return NULL;
+}
+
+static inline struct resource *resource_first_child(struct list_head *head)
+{
+ return list_first_entry_or_null(head, struct resource, sibling);
+}
+
#endif /* __ASSEMBLY__ */
#endif /* _LINUX_IOPORT_H */
diff --git a/kernel/resource.c b/kernel/resource.c
index d1cbf4b50e17..6d647a3824b1 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -31,6 +31,8 @@ struct resource ioport_resource = {
.start = 0,
.end = IO_SPACE_LIMIT,
.flags = IORESOURCE_IO,
+ .sibling = LIST_HEAD_INIT(ioport_resource.sibling),
+ .child = LIST_HEAD_INIT(ioport_resource.child),
};
EXPORT_SYMBOL(ioport_resource);
@@ -39,6 +41,8 @@ struct resource iomem_resource = {
.start = 0,
.end = -1,
.flags = IORESOURCE_MEM,
+ .sibling = LIST_HEAD_INIT(iomem_resource.sibling),
+ .child = LIST_HEAD_INIT(iomem_resource.child),
};
EXPORT_SYMBOL(iomem_resource);
@@ -57,20 +61,20 @@ static DEFINE_RWLOCK(resource_lock);
* by boot mem after the system is up. So for reusing the resource entry
* we need to remember the resource.
*/
-static struct resource *bootmem_resource_free;
+static struct list_head bootmem_resource_free = LIST_HEAD_INIT(bootmem_resource_free);
static DEFINE_SPINLOCK(bootmem_resource_lock);
static struct resource *next_resource(struct resource *p, bool sibling_only)
{
/* Caller wants to traverse through siblings only */
if (sibling_only)
- return p->sibling;
+ return resource_sibling(p);
- if (p->child)
- return p->child;
- while (!p->sibling && p->parent)
+ if (!list_empty(&p->child))
+ return resource_first_child(&p->child);
+ while (!resource_sibling(p) && p->parent)
p = p->parent;
- return p->sibling;
+ return resource_sibling(p);
}
static void *r_next(struct seq_file *m, void *v, loff_t *pos)
@@ -90,7 +94,7 @@ static void *r_start(struct seq_file *m, loff_t *pos)
struct resource *p = PDE_DATA(file_inode(m->file));
loff_t l = 0;
read_lock(&resource_lock);
- for (p = p->child; p && l < *pos; p = r_next(m, p, &l))
+ for (p = resource_first_child(&p->child); p && l < *pos; p = r_next(m, p, &l))
;
return p;
}
@@ -153,8 +157,7 @@ static void free_resource(struct resource *res)
if (!PageSlab(virt_to_head_page(res))) {
spin_lock(&bootmem_resource_lock);
- res->sibling = bootmem_resource_free;
- bootmem_resource_free = res;
+ list_add(&res->sibling, &bootmem_resource_free);
spin_unlock(&bootmem_resource_lock);
} else {
kfree(res);
@@ -166,10 +169,9 @@ static struct resource *alloc_resource(gfp_t flags)
struct resource *res = NULL;
spin_lock(&bootmem_resource_lock);
- if (bootmem_resource_free) {
- res = bootmem_resource_free;
- bootmem_resource_free = res->sibling;
- }
+ res = resource_first_child(&bootmem_resource_free);
+ if (res)
+ list_del(&res->sibling);
spin_unlock(&bootmem_resource_lock);
if (res)
@@ -177,6 +179,8 @@ static struct resource *alloc_resource(gfp_t flags)
else
res = kzalloc(sizeof(struct resource), flags);
+ INIT_LIST_HEAD(&res->child);
+ INIT_LIST_HEAD(&res->sibling);
return res;
}
@@ -185,7 +189,7 @@ static struct resource * __request_resource(struct resource *root, struct resour
{
resource_size_t start = new->start;
resource_size_t end = new->end;
- struct resource *tmp, **p;
+ struct resource *tmp;
if (end < start)
return root;
@@ -193,64 +197,62 @@ static struct resource * __request_resource(struct resource *root, struct resour
return root;
if (end > root->end)
return root;
- p = &root->child;
- for (;;) {
- tmp = *p;
- if (!tmp || tmp->start > end) {
- new->sibling = tmp;
- *p = new;
+
+ if (list_empty(&root->child)) {
+ list_add(&new->sibling, &root->child);
+ new->parent = root;
+ INIT_LIST_HEAD(&new->child);
+ return NULL;
+ }
+
+ list_for_each_entry(tmp, &root->child, sibling) {
+ if (tmp->start > end) {
+ list_add(&new->sibling, tmp->sibling.prev);
new->parent = root;
+ INIT_LIST_HEAD(&new->child);
return NULL;
}
- p = &tmp->sibling;
if (tmp->end < start)
continue;
return tmp;
}
+
+ list_add_tail(&new->sibling, &root->child);
+ new->parent = root;
+ INIT_LIST_HEAD(&new->child);
+ return NULL;
}
static int __release_resource(struct resource *old, bool release_child)
{
- struct resource *tmp, **p, *chd;
+ struct resource *tmp, *next, *chd;
- p = &old->parent->child;
- for (;;) {
- tmp = *p;
- if (!tmp)
- break;
+ list_for_each_entry_safe(tmp, next, &old->parent->child, sibling) {
if (tmp == old) {
- if (release_child || !(tmp->child)) {
- *p = tmp->sibling;
+ if (release_child || list_empty(&tmp->child)) {
+ list_del(&tmp->sibling);
} else {
- for (chd = tmp->child;; chd = chd->sibling) {
+ list_for_each_entry(chd, &tmp->child, sibling)
chd->parent = tmp->parent;
- if (!(chd->sibling))
- break;
- }
- *p = tmp->child;
- chd->sibling = tmp->sibling;
+ list_splice(&tmp->child, tmp->sibling.prev);
+ list_del(&tmp->sibling);
}
+
old->parent = NULL;
return 0;
}
- p = &tmp->sibling;
}
return -EINVAL;
}
static void __release_child_resources(struct resource *r)
{
- struct resource *tmp, *p;
+ struct resource *tmp, *next;
resource_size_t size;
- p = r->child;
- r->child = NULL;
- while (p) {
- tmp = p;
- p = p->sibling;
-
+ list_for_each_entry_safe(tmp, next, &r->child, sibling) {
tmp->parent = NULL;
- tmp->sibling = NULL;
+ list_del_init(&tmp->sibling);
__release_child_resources(tmp);
printk(KERN_DEBUG "release child resource %pR\n", tmp);
@@ -259,6 +261,8 @@ static void __release_child_resources(struct resource *r)
tmp->start = 0;
tmp->end = size - 1;
}
+
+ INIT_LIST_HEAD(&tmp->child);
}
void release_child_resources(struct resource *r)
@@ -343,7 +347,8 @@ static int find_next_iomem_res(struct resource *res, unsigned long desc,
read_lock(&resource_lock);
- for (p = iomem_resource.child; p; p = next_resource(p, sibling_only)) {
+ for (p = resource_first_child(&iomem_resource.child); p;
+ p = next_resource(p, sibling_only)) {
if ((p->flags & res->flags) != res->flags)
continue;
if ((desc != IORES_DESC_NONE) && (desc != p->desc))
@@ -532,7 +537,7 @@ int region_intersects(resource_size_t start, size_t size, unsigned long flags,
struct resource *p;
read_lock(&resource_lock);
- for (p = iomem_resource.child; p ; p = p->sibling) {
+ list_for_each_entry(p, &iomem_resource.child, sibling) {
bool is_type = (((p->flags & flags) == flags) &&
((desc == IORES_DESC_NONE) ||
(desc == p->desc)));
@@ -586,7 +591,7 @@ static int __find_resource(struct resource *root, struct resource *old,
resource_size_t size,
struct resource_constraint *constraint)
{
- struct resource *this = root->child;
+ struct resource *this = resource_first_child(&root->child);
struct resource tmp = *new, avail, alloc;
tmp.start = root->start;
@@ -596,7 +601,7 @@ static int __find_resource(struct resource *root, struct resource *old,
*/
if (this && this->start == root->start) {
tmp.start = (this == old) ? old->start : this->end + 1;
- this = this->sibling;
+ this = resource_sibling(this);
}
for(;;) {
if (this)
@@ -632,7 +637,7 @@ next: if (!this || this->end == root->end)
if (this != old)
tmp.start = this->end + 1;
- this = this->sibling;
+ this = resource_sibling(this);
}
return -EBUSY;
}
@@ -676,7 +681,7 @@ static int reallocate_resource(struct resource *root, struct resource *old,
goto out;
}
- if (old->child) {
+ if (!list_empty(&old->child)) {
err = -EBUSY;
goto out;
}
@@ -757,7 +762,7 @@ struct resource *lookup_resource(struct resource *root, resource_size_t start)
struct resource *res;
read_lock(&resource_lock);
- for (res = root->child; res; res = res->sibling) {
+ list_for_each_entry(res, &root->child, sibling) {
if (res->start == start)
break;
}
@@ -790,32 +795,27 @@ static struct resource * __insert_resource(struct resource *parent, struct resou
break;
}
- for (next = first; ; next = next->sibling) {
+ for (next = first; ; next = resource_sibling(next)) {
/* Partial overlap? Bad, and unfixable */
if (next->start < new->start || next->end > new->end)
return next;
- if (!next->sibling)
+ if (!resource_sibling(next))
break;
- if (next->sibling->start > new->end)
+ if (resource_sibling(next)->start > new->end)
break;
}
-
new->parent = parent;
- new->sibling = next->sibling;
- new->child = first;
+ list_add(&new->sibling, &next->sibling);
+ INIT_LIST_HEAD(&new->child);
- next->sibling = NULL;
- for (next = first; next; next = next->sibling)
+ /*
+ * From first to next, they all fall into new's region, so change them
+ * as new's children.
+ */
+ list_cut_position(&new->child, first->sibling.prev, &next->sibling);
+ list_for_each_entry(next, &new->child, sibling)
next->parent = new;
- if (parent->child == first) {
- parent->child = new;
- } else {
- next = parent->child;
- while (next->sibling != first)
- next = next->sibling;
- next->sibling = new;
- }
return NULL;
}
@@ -937,19 +937,17 @@ static int __adjust_resource(struct resource *res, resource_size_t start,
if ((start < parent->start) || (end > parent->end))
goto out;
- if (res->sibling && (res->sibling->start <= end))
+ if (resource_sibling(res) && (resource_sibling(res)->start <= end))
goto out;
- tmp = parent->child;
- if (tmp != res) {
- while (tmp->sibling != res)
- tmp = tmp->sibling;
+ if (res->sibling.prev != &parent->child) {
+ tmp = list_prev_entry(res, sibling);
if (start <= tmp->end)
goto out;
}
skip:
- for (tmp = res->child; tmp; tmp = tmp->sibling)
+ list_for_each_entry(tmp, &res->child, sibling)
if ((tmp->start < start) || (tmp->end > end))
goto out;
@@ -993,28 +991,31 @@ EXPORT_SYMBOL(adjust_resource);
*/
int reparent_resources(struct resource *parent, struct resource *res)
{
- struct resource *p, **pp;
- struct resource **firstpp = NULL;
+ struct resource *p, *first = NULL;
- for (pp = &parent->child; (p = *pp) != NULL; pp = &p->sibling) {
+ list_for_each_entry(p, &parent->child, sibling) {
if (p->end < res->start)
continue;
if (res->end < p->start)
break;
if (p->start < res->start || p->end > res->end)
return -ENOTSUPP; /* not completely contained */
- if (firstpp == NULL)
- firstpp = pp;
+ if (first == NULL)
+ first = p;
}
- if (firstpp == NULL)
+ if (first == NULL)
return -ECANCELED; /* didn't find any conflicting entries? */
res->parent = parent;
- res->child = *firstpp;
- res->sibling = *pp;
- *firstpp = res;
- *pp = NULL;
- for (p = res->child; p != NULL; p = p->sibling) {
- p->parent = res;
+ list_add(&res->sibling, p->sibling.prev);
+ INIT_LIST_HEAD(&res->child);
+
+ /*
+ * From first to p's previous sibling, they all fall into
+ * res's region, change them as res's children.
+ */
+ list_cut_position(&res->child, first->sibling.prev, res->sibling.prev);
+ list_for_each_entry(p, &res->child, sibling) {
+ p->parent = res;
pr_debug("PCI: Reparented %s %pR under %s\n",
p->name, p, res->name);
}
@@ -1213,34 +1214,32 @@ EXPORT_SYMBOL(__request_region);
void __release_region(struct resource *parent, resource_size_t start,
resource_size_t n)
{
- struct resource **p;
+ struct resource *res;
resource_size_t end;
- p = &parent->child;
+ res = resource_first_child(&parent->child);
end = start + n - 1;
write_lock(&resource_lock);
for (;;) {
- struct resource *res = *p;
-
if (!res)
break;
if (res->start <= start && res->end >= end) {
if (!(res->flags & IORESOURCE_BUSY)) {
- p = &res->child;
+ res = resource_first_child(&res->child);
continue;
}
if (res->start != start || res->end != end)
break;
- *p = res->sibling;
+ list_del(&res->sibling);
write_unlock(&resource_lock);
if (res->flags & IORESOURCE_MUXED)
wake_up(&muxed_resource_wait);
free_resource(res);
return;
}
- p = &res->sibling;
+ res = resource_sibling(res);
}
write_unlock(&resource_lock);
@@ -1275,9 +1274,7 @@ EXPORT_SYMBOL(__release_region);
int release_mem_region_adjustable(struct resource *parent,
resource_size_t start, resource_size_t size)
{
- struct resource **p;
- struct resource *res;
- struct resource *new_res;
+ struct resource *res, *new_res;
resource_size_t end;
int ret = -EINVAL;
@@ -1288,16 +1285,16 @@ int release_mem_region_adjustable(struct resource *parent,
/* The alloc_resource() result gets checked later */
new_res = alloc_resource(GFP_KERNEL);
- p = &parent->child;
+ res = resource_first_child(&parent->child);
write_lock(&resource_lock);
- while ((res = *p)) {
+ while ((res)) {
if (res->start >= end)
break;
/* look for the next resource if it does not fit into */
if (res->start > start || res->end < end) {
- p = &res->sibling;
+ res = resource_sibling(res);
continue;
}
@@ -1305,14 +1302,14 @@ int release_mem_region_adjustable(struct resource *parent,
break;
if (!(res->flags & IORESOURCE_BUSY)) {
- p = &res->child;
+ res = resource_first_child(&res->child);
continue;
}
/* found the target resource; let's adjust accordingly */
if (res->start == start && res->end == end) {
/* free the whole entry */
- *p = res->sibling;
+ list_del(&res->sibling);
free_resource(res);
ret = 0;
} else if (res->start == start && res->end != end) {
@@ -1335,14 +1332,13 @@ int release_mem_region_adjustable(struct resource *parent,
new_res->flags = res->flags;
new_res->desc = res->desc;
new_res->parent = res->parent;
- new_res->sibling = res->sibling;
- new_res->child = NULL;
+ INIT_LIST_HEAD(&new_res->child);
ret = __adjust_resource(res, res->start,
start - res->start);
if (ret)
break;
- res->sibling = new_res;
+ list_add(&new_res->sibling, &res->sibling);
new_res = NULL;
}
@@ -1523,7 +1519,7 @@ static int __init reserve_setup(char *str)
res->end = io_start + io_num - 1;
res->flags |= IORESOURCE_BUSY;
res->desc = IORES_DESC_NONE;
- res->child = NULL;
+ INIT_LIST_HEAD(&res->child);
if (request_resource(parent, res) == 0)
reserved = x+1;
}
@@ -1543,7 +1539,7 @@ int iomem_map_sanity_check(resource_size_t addr, unsigned long size)
loff_t l;
read_lock(&resource_lock);
- for (p = p->child; p ; p = r_next(NULL, p, &l)) {
+ for (p = resource_first_child(&p->child); p; p = r_next(NULL, p, &l)) {
/*
* We can probably skip the resources without
* IORESOURCE_IO attribute?
@@ -1599,7 +1595,7 @@ bool iomem_is_exclusive(u64 addr)
addr = addr & PAGE_MASK;
read_lock(&resource_lock);
- for (p = p->child; p ; p = r_next(NULL, p, &l)) {
+ for (p = resource_first_child(&p->child); p; p = r_next(NULL, p, &l)) {
/*
* We can probably skip the resources without
* IORESOURCE_IO attribute?
--
2.13.6
^ permalink raw reply related
* [PATCH v6 3/4] resource: add walk_system_ram_res_rev()
From: Baoquan He @ 2018-07-04 4:10 UTC (permalink / raw)
To: linux-kernel, akpm, robh+dt, dan.j.williams, nicolas.pitre, josh,
fengguang.wu, bp, andy.shevchenko
Cc: patrik.r.jakobsson, airlied, kys, haiyangz, sthemmin,
dmitry.torokhov, frowand.list, keith.busch, jonathan.derrick,
lorenzo.pieralisi, bhelgaas, tglx, brijesh.singh, jglisse,
thomas.lendacky, gregkh, baiyaowei, richard.weiyang, devel,
linux-input, linux-nvdimm, devicetree, linux-pci, ebiederm,
vgoyal, dyoung, yinghai, kexec, monstr, davem, chris, jcmvbkbc,
gustavo, maarten.lankhorst, seanpaul, linux-parisc, linuxppc-dev,
Baoquan He
In-Reply-To: <20180704041038.8190-1-bhe@redhat.com>
This function, being a variant of walk_system_ram_res() introduced in
commit 8c86e70acead ("resource: provide new functions to walk through
resources"), walks through a list of all the resources of System RAM
in reversed order, i.e., from higher to lower.
It will be used in kexec_file code.
Signed-off-by: Baoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Wei Yang <richard.weiyang@gmail.com>
---
include/linux/ioport.h | 3 +++
kernel/resource.c | 40 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index b7456ae889dd..066cc263e2cc 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -279,6 +279,9 @@ extern int
walk_system_ram_res(u64 start, u64 end, void *arg,
int (*func)(struct resource *, void *));
extern int
+walk_system_ram_res_rev(u64 start, u64 end, void *arg,
+ int (*func)(struct resource *, void *));
+extern int
walk_iomem_res_desc(unsigned long desc, unsigned long flags, u64 start, u64 end,
void *arg, int (*func)(struct resource *, void *));
diff --git a/kernel/resource.c b/kernel/resource.c
index 6d647a3824b1..4c5fbef4ea24 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -23,6 +23,8 @@
#include <linux/pfn.h>
#include <linux/mm.h>
#include <linux/resource_ext.h>
+#include <linux/string.h>
+#include <linux/vmalloc.h>
#include <asm/io.h>
@@ -443,6 +445,44 @@ int walk_system_ram_res(u64 start, u64 end, void *arg,
}
/*
+ * This function, being a variant of walk_system_ram_res(), calls the @func
+ * callback against all memory ranges of type System RAM which are marked as
+ * IORESOURCE_SYSTEM_RAM and IORESOUCE_BUSY in reversed order, i.e., from
+ * higher to lower.
+ */
+int walk_system_ram_res_rev(u64 start, u64 end, void *arg,
+ int (*func)(struct resource *, void *))
+{
+ unsigned long flags;
+ struct resource *res;
+ int ret = -1;
+
+ flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+
+ read_lock(&resource_lock);
+ list_for_each_entry_reverse(res, &iomem_resource.child, sibling) {
+ if (start >= end)
+ break;
+ if ((res->flags & flags) != flags)
+ continue;
+ if (res->desc != IORES_DESC_NONE)
+ continue;
+ if (res->end < start)
+ break;
+
+ if ((res->end >= start) && (res->start < end)) {
+ ret = (*func)(res, arg);
+ if (ret)
+ break;
+ }
+ end = res->start - 1;
+
+ }
+ read_unlock(&resource_lock);
+ return ret;
+}
+
+/*
* This function calls the @func callback against all memory ranges, which
* are ranges marked as IORESOURCE_MEM and IORESOUCE_BUSY.
*/
--
2.13.6
^ permalink raw reply related
* [PATCH v6 4/4] kexec_file: Load kernel at top of system RAM if required
From: Baoquan He @ 2018-07-04 4:10 UTC (permalink / raw)
To: linux-kernel, akpm, robh+dt, dan.j.williams, nicolas.pitre, josh,
fengguang.wu, bp, andy.shevchenko
Cc: patrik.r.jakobsson, airlied, kys, haiyangz, sthemmin,
dmitry.torokhov, frowand.list, keith.busch, jonathan.derrick,
lorenzo.pieralisi, bhelgaas, tglx, brijesh.singh, jglisse,
thomas.lendacky, gregkh, baiyaowei, richard.weiyang, devel,
linux-input, linux-nvdimm, devicetree, linux-pci, ebiederm,
vgoyal, dyoung, yinghai, kexec, monstr, davem, chris, jcmvbkbc,
gustavo, maarten.lankhorst, seanpaul, linux-parisc, linuxppc-dev,
Baoquan He
In-Reply-To: <20180704041038.8190-1-bhe@redhat.com>
For kexec_file loading, if kexec_buf.top_down is 'true', the memory which
is used to load kernel/initrd/purgatory is supposed to be allocated from
top to down. This is what we have been doing all along in the old kexec
loading interface and the kexec loading is still default setting in some
distributions. However, the current kexec_file loading interface doesn't
do likt this. The function arch_kexec_walk_mem() it calls ignores checking
kexec_buf.top_down, but calls walk_system_ram_res() directly to go through
all resources of System RAM from bottom to up, to try to find memory region
which can contain the specific kexec buffer, then call locate_mem_hole_callback()
to allocate memory in that found memory region from top to down. This brings
confusion especially when KASLR is widely supported , users have to make clear
why kexec/kdump kernel loading position is different between these two
interfaces in order to exclude unnecessary noises. Hence these two interfaces
need be unified on behaviour.
Here add checking if kexec_buf.top_down is 'true' in arch_kexec_walk_mem(),
if yes, call the newly added walk_system_ram_res_rev() to find memory region
from top to down to load kernel.
Signed-off-by: Baoquan He <bhe@redhat.com>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: kexec@lists.infradead.org
---
kernel/kexec_file.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index c6a3b6851372..75226c1d08ce 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -518,6 +518,8 @@ int __weak arch_kexec_walk_mem(struct kexec_buf *kbuf,
IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY,
crashk_res.start, crashk_res.end,
kbuf, func);
+ else if (kbuf->top_down)
+ return walk_system_ram_res_rev(0, ULONG_MAX, kbuf, func);
else
return walk_system_ram_res(0, ULONG_MAX, kbuf, func);
}
--
2.13.6
^ permalink raw reply related
* Re: [PATCH v5 1/4] resource: Move reparent_resources() to kernel/resource.c and make it public
From: Baoquan He @ 2018-07-04 4:18 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Linux Kernel Mailing List, Andrew Morton, Rob Herring,
Dan Williams, Nicolas Pitre, Josh Triplett, kbuild test robot,
Borislav Petkov, Patrik Jakobsson, David Airlie, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Dmitry Torokhov, Frank Rowand,
Keith Busch, Jon Derrick, Lorenzo Pieralisi, Bjorn Helgaas,
Thomas Gleixner, brijesh.singh, Jérôme Glisse,
Tom Lendacky, Greg Kroah-Hartman, baiyaowei, richard.weiyang,
devel, linux-input, linux-nvdimm, devicetree, linux-pci,
Eric Biederman, Vivek Goyal, Dave Young, Yinghai Lu, kexec,
Michal Simek, David S. Miller, Chris Zankel, Max Filippov,
Gustavo Padovan, Maarten Lankhorst, Sean Paul, linux-parisc,
open list:LINUX FOR POWERPC PA SEMI PWRFICIENT,
Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman
In-Reply-To: <CAHp75VeEjhcYF4Fy4O_srdv_nEGGjGwAHhZufscB_tiLQVQ=FQ@mail.gmail.com>
On 07/03/18 at 11:57pm, Andy Shevchenko wrote:
> On Tue, Jul 3, 2018 at 5:55 PM, Baoquan He <bhe@redhat.com> wrote:
> > On 06/12/18 at 05:24pm, Andy Shevchenko wrote:
> >> On Tue, Jun 12, 2018 at 5:20 PM, Andy Shevchenko
> >> <andy.shevchenko@gmail.com> wrote:
>
> >> > I briefly looked at the code and error codes we have, so, my proposal
> >> > is one of the following
> >>
> >> > - use -ECANCELED (not the best choice for first occurrence here,
> >> > though I can't find better)
> >>
> >> Actually -ENOTSUPP might suit the first case (although the actual
> >> would be something like -EOVERLAP, which we don't have)
> >
> > Sorry for late reply, and many thanks for your great suggestion.
> >
>
> > I am fine to use -ENOTSUPP as the first returned value, and -ECANCELED
> > for the 2nd one.
>
> I have no strong opinion, but I like (slightly better) this approach ^^^
Done, post v6 in this way, many thanks.
>
> > Or define an enum as you suggested inside the function
> > or in header file.
>
> >
> > Or use -EBUSY for the first case because existing resource is
> > overlapping but not fully contained by 'res'; and -EINVAL for
> > the 2nd case since didn't find any one resources which is contained by
> > 'res', means we passed in a invalid resource.
> >
> > All is fine to me, I can repost with each of them.
>
> >> > - use positive integers (or enum), like
> >> > #define RES_REPARENTED 0
> >> > #define RES_OVERLAPPED 1
> >> > #define RES_NOCONFLICT 2
>
> --
> With Best Regards,
> Andy Shevchenko
^ permalink raw reply
* Re: [PATCHv3 3/4] drivers/base: clean up the usage of devices_kset_move_last()
From: Pingfan Liu @ 2018-07-04 4:40 UTC (permalink / raw)
To: rjw
Cc: Grygorii Strashko, linux-kernel, Greg Kroah-Hartman,
Christoph Hellwig, Bjorn Helgaas, Dave Young, linux-pci,
linuxppc-dev
In-Reply-To: <3375966.ydPZj5TMVj@aspire.rjw.lan>
On Tue, Jul 3, 2018 at 10:28 PM Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>
> On Tuesday, July 3, 2018 8:50:41 AM CEST Pingfan Liu wrote:
> > Clean up the referring to the code in commit 52cdbdd49853 ("driver core:
> > correct device's shutdown order"). So later we can revert it safely.
> >
> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > Cc: Grygorii Strashko <grygorii.strashko@ti.com>
> > Cc: Christoph Hellwig <hch@infradead.org>
> > Cc: Bjorn Helgaas <helgaas@kernel.org>
> > Cc: Dave Young <dyoung@redhat.com>
> > Cc: linux-pci@vger.kernel.org
> > Cc: linuxppc-dev@lists.ozlabs.org
> > Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> > ---
> > drivers/base/core.c | 7 -------
> > 1 file changed, 7 deletions(-)
> >
> > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > index 684b994..db3deb8 100644
> > --- a/drivers/base/core.c
> > +++ b/drivers/base/core.c
> > @@ -127,13 +127,6 @@ static int device_reorder_to_tail(struct device *dev, void *not_used)
> > {
> > struct device_link *link;
> >
> > - /*
> > - * Devices that have not been registered yet will be put to the ends
> > - * of the lists during the registration, so skip them here.
> > - */
> > - if (device_is_registered(dev))
> > - devices_kset_move_last(dev);
> > -
> > if (device_pm_initialized(dev))
> > device_pm_move_last(dev);
>
> You can't do this.
>
> If you do it, that will break power management in some situations.
>
Could you shed light on it? I had a quick browsing of pm code, but it
is a big function, and I got lost in it.
If the above code causes failure, then does it imply that the seq in
devices_kset should be the same as dpm_list? But in device_shutdown(),
it only intersect with pm by
pm_runtime_get_noresume(dev) and pm_runtime_barrier(dev). How do these
function affect the seq in dpm_list? Need your help to see how to
handle this issue.
Thanks and regards,
Pingfan
^ permalink raw reply
* [PATCH kernel v3 1/2] vfio/spapr: Use IOMMU pageshift rather than pagesize
From: Alexey Kardashevskiy @ 2018-07-04 5:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, Alex Williamson,
Michael Ellerman, Paul Mackerras
In-Reply-To: <20180704050052.20045-1-aik@ozlabs.ru>
The size is always equal to 1 page so let's use this. Later on this will
be used for other checks which use page shifts to check the granularity
of access.
This should cause no behavioral change.
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Acked-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
As Alex suggested, this should go via the ppc tree which the next patch
is going to (which is ppc-kvm, probably?).
---
drivers/vfio/vfio_iommu_spapr_tce.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 759a5bd..2da5f05 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -457,13 +457,13 @@ static void tce_iommu_unuse_page(struct tce_container *container,
}
static int tce_iommu_prereg_ua_to_hpa(struct tce_container *container,
- unsigned long tce, unsigned long size,
+ unsigned long tce, unsigned long shift,
unsigned long *phpa, struct mm_iommu_table_group_mem_t **pmem)
{
long ret = 0;
struct mm_iommu_table_group_mem_t *mem;
- mem = mm_iommu_lookup(container->mm, tce, size);
+ mem = mm_iommu_lookup(container->mm, tce, 1ULL << shift);
if (!mem)
return -EINVAL;
@@ -487,7 +487,7 @@ static void tce_iommu_unuse_page_v2(struct tce_container *container,
if (!pua)
return;
- ret = tce_iommu_prereg_ua_to_hpa(container, *pua, IOMMU_PAGE_SIZE(tbl),
+ ret = tce_iommu_prereg_ua_to_hpa(container, *pua, tbl->it_page_shift,
&hpa, &mem);
if (ret)
pr_debug("%s: tce %lx at #%lx was not cached, ret=%d\n",
@@ -611,7 +611,7 @@ static long tce_iommu_build_v2(struct tce_container *container,
entry + i);
ret = tce_iommu_prereg_ua_to_hpa(container,
- tce, IOMMU_PAGE_SIZE(tbl), &hpa, &mem);
+ tce, tbl->it_page_shift, &hpa, &mem);
if (ret)
break;
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 0/2] KVM: PPC: Check if IOMMU page is contained in the pinned physical page
From: Alexey Kardashevskiy @ 2018-07-04 5:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, Alex Williamson,
Michael Ellerman, Paul Mackerras
This is to improve page boundaries checking and should probably
be cc:stable. I came accross this while debugging nvlink2 passthrough
but the lack of checking might be exploited by the existing userspace.
Changes:
v3:
* enforced huge pages not to cross preregistered chunk boundaries
v2:
* 2/2: explicitly check for compound pages before calling compound_order()
This is based on sha1
021c917 Linus Torvalds "Linux 4.18-rc3".
Please comment. Thanks.
Alexey Kardashevskiy (2):
vfio/spapr: Use IOMMU pageshift rather than pagesize
KVM: PPC: Check if IOMMU page is contained in the pinned physical page
arch/powerpc/include/asm/mmu_context.h | 4 ++--
arch/powerpc/kvm/book3s_64_vio.c | 2 +-
arch/powerpc/kvm/book3s_64_vio_hv.c | 6 +++--
arch/powerpc/mm/mmu_context_iommu.c | 41 +++++++++++++++++++++++++++++++---
drivers/vfio/vfio_iommu_spapr_tce.c | 10 ++++-----
5 files changed, 50 insertions(+), 13 deletions(-)
--
2.11.0
^ permalink raw reply
* [PATCH kernel v3 2/2] KVM: PPC: Check if IOMMU page is contained in the pinned physical page
From: Alexey Kardashevskiy @ 2018-07-04 5:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, Alex Williamson,
Michael Ellerman, Paul Mackerras
In-Reply-To: <20180704050052.20045-1-aik@ozlabs.ru>
A VM which has:
- a DMA capable device passed through to it (eg. network card);
- running a malicious kernel that ignores H_PUT_TCE failure;
- capability of using IOMMU pages bigger that physical pages
can create an IOMMU mapping that exposes (for example) 16MB of
the host physical memory to the device when only 64K was allocated to the VM.
The remaining 16MB - 64K will be some other content of host memory, possibly
including pages of the VM, but also pages of host kernel memory, host
programs or other VMs.
The attacking VM does not control the location of the page it can map,
and is only allowed to map as many pages as it has pages of RAM.
We already have a check in drivers/vfio/vfio_iommu_spapr_tce.c that
an IOMMU page is contained in the physical page so the PCI hardware won't
get access to unassigned host memory; however this check is missing in
the KVM fastpath (H_PUT_TCE accelerated code). We were lucky so far and
did not hit this yet as the very first time when the mapping happens
we do not have tbl::it_userspace allocated yet and fall back to
the userspace which in turn calls VFIO IOMMU driver, this fails and
the guest does not retry,
This stores the smallest preregistered page size in the preregistered
region descriptor and changes the mm_iommu_xxx API to check this against
the IOMMU page size. This only allows huge pages use if the entire
preregistered block is backed with huge pages which are completely
contained the preregistered chunk; otherwise this defaults to PAGE_SIZE.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v3:
* fixed upper limit for the page size
* added checks that we don't register parts of a huge page
v2:
* explicitely check for compound pages before calling compound_order()
---
The bug is: run QEMU _without_ hugepages (no -mempath) and tell it to
advertise 16MB pages to the guest; a typical pseries guest will use 16MB
for IOMMU pages without checking the mmu pagesize and this will fail
at https://git.qemu.org/?p=qemu.git;a=blob;f=hw/vfio/common.c;h=fb396cf00ac40eb35967a04c9cc798ca896eed57;hb=refs/heads/master#l256
With the change, mapping will fail in KVM and the guest will print:
mlx5_core 0000:00:00.0: ibm,create-pe-dma-window(2027) 0 8000000 20000000 18 1f returned 0 (liobn = 0x80000001 starting addr = 8000000 0)
mlx5_core 0000:00:00.0: created tce table LIOBN 0x80000001 for /pci@800000020000000/ethernet@0
mlx5_core 0000:00:00.0: failed to map direct window for /pci@800000020000000/ethernet@0: -1
---
arch/powerpc/include/asm/mmu_context.h | 4 ++--
arch/powerpc/kvm/book3s_64_vio.c | 2 +-
arch/powerpc/kvm/book3s_64_vio_hv.c | 6 +++--
arch/powerpc/mm/mmu_context_iommu.c | 41 +++++++++++++++++++++++++++++++---
drivers/vfio/vfio_iommu_spapr_tce.c | 2 +-
5 files changed, 46 insertions(+), 9 deletions(-)
diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h
index 896efa5..79d570c 100644
--- a/arch/powerpc/include/asm/mmu_context.h
+++ b/arch/powerpc/include/asm/mmu_context.h
@@ -35,9 +35,9 @@ extern struct mm_iommu_table_group_mem_t *mm_iommu_lookup_rm(
extern struct mm_iommu_table_group_mem_t *mm_iommu_find(struct mm_struct *mm,
unsigned long ua, unsigned long entries);
extern long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem,
- unsigned long ua, unsigned long *hpa);
+ unsigned long ua, unsigned int pageshift, unsigned long *hpa);
extern long mm_iommu_ua_to_hpa_rm(struct mm_iommu_table_group_mem_t *mem,
- unsigned long ua, unsigned long *hpa);
+ unsigned long ua, unsigned int pageshift, unsigned long *hpa);
extern long mm_iommu_mapped_inc(struct mm_iommu_table_group_mem_t *mem);
extern void mm_iommu_mapped_dec(struct mm_iommu_table_group_mem_t *mem);
#endif
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index d066e37..8c456fa 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -449,7 +449,7 @@ long kvmppc_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
/* This only handles v2 IOMMU type, v1 is handled via ioctl() */
return H_TOO_HARD;
- if (WARN_ON_ONCE(mm_iommu_ua_to_hpa(mem, ua, &hpa)))
+ if (WARN_ON_ONCE(mm_iommu_ua_to_hpa(mem, ua, tbl->it_page_shift, &hpa)))
return H_HARDWARE;
if (mm_iommu_mapped_inc(mem))
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 925fc31..5b298f5 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -279,7 +279,8 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
if (!mem)
return H_TOO_HARD;
- if (WARN_ON_ONCE_RM(mm_iommu_ua_to_hpa_rm(mem, ua, &hpa)))
+ if (WARN_ON_ONCE_RM(mm_iommu_ua_to_hpa_rm(mem, ua, tbl->it_page_shift,
+ &hpa)))
return H_HARDWARE;
pua = (void *) vmalloc_to_phys(pua);
@@ -469,7 +470,8 @@ long kvmppc_rm_h_put_tce_indirect(struct kvm_vcpu *vcpu,
mem = mm_iommu_lookup_rm(vcpu->kvm->mm, ua, IOMMU_PAGE_SIZE_4K);
if (mem)
- prereg = mm_iommu_ua_to_hpa_rm(mem, ua, &tces) == 0;
+ prereg = mm_iommu_ua_to_hpa_rm(mem, ua,
+ IOMMU_PAGE_SHIFT_4K, &tces) == 0;
}
if (!prereg) {
diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c
index abb4364..842dfce 100644
--- a/arch/powerpc/mm/mmu_context_iommu.c
+++ b/arch/powerpc/mm/mmu_context_iommu.c
@@ -27,6 +27,7 @@ struct mm_iommu_table_group_mem_t {
struct rcu_head rcu;
unsigned long used;
atomic64_t mapped;
+ unsigned int pageshift;
u64 ua; /* userspace address */
u64 entries; /* number of entries in hpas[] */
u64 *hpas; /* vmalloc'ed */
@@ -125,7 +126,8 @@ long mm_iommu_get(struct mm_struct *mm, unsigned long ua, unsigned long entries,
{
struct mm_iommu_table_group_mem_t *mem;
long i, j, ret = 0, locked_entries = 0;
- struct page *page = NULL;
+ unsigned int pageshift;
+ struct page *page = NULL, *head = NULL;
mutex_lock(&mem_list_mutex);
@@ -159,6 +161,7 @@ long mm_iommu_get(struct mm_struct *mm, unsigned long ua, unsigned long entries,
goto unlock_exit;
}
+ mem->pageshift = 64;
mem->hpas = vzalloc(array_size(entries, sizeof(mem->hpas[0])));
if (!mem->hpas) {
kfree(mem);
@@ -199,9 +202,35 @@ long mm_iommu_get(struct mm_struct *mm, unsigned long ua, unsigned long entries,
}
}
populate:
+ pageshift = PAGE_SHIFT;
+ if (PageCompound(page)) {
+ /* Make sure huge page is contained completely */
+ struct page *tmphead = compound_head(page);
+ unsigned int n = compound_order(tmphead);
+
+ if (!head) {
+ /* Is it a head of a huge page? */
+ if (page == tmphead) {
+ head = tmphead;
+ pageshift += n;
+ }
+ } else if (head == tmphead) {
+ /* Still same huge page, good */
+ pageshift += n;
+
+ /* End of the huge page */
+ if (page - head == (1UL << n) - 1)
+ head = NULL;
+ }
+ }
+ mem->pageshift = min(mem->pageshift, pageshift);
mem->hpas[i] = page_to_pfn(page) << PAGE_SHIFT;
}
+ /* We have an incomplete huge page, default to PAGE_SHIFT */
+ if (head)
+ mem->pageshift = PAGE_SHIFT;
+
atomic64_set(&mem->mapped, 1);
mem->used = 1;
mem->ua = ua;
@@ -349,7 +378,7 @@ struct mm_iommu_table_group_mem_t *mm_iommu_find(struct mm_struct *mm,
EXPORT_SYMBOL_GPL(mm_iommu_find);
long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem,
- unsigned long ua, unsigned long *hpa)
+ unsigned long ua, unsigned int pageshift, unsigned long *hpa)
{
const long entry = (ua - mem->ua) >> PAGE_SHIFT;
u64 *va = &mem->hpas[entry];
@@ -357,6 +386,9 @@ long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem,
if (entry >= mem->entries)
return -EFAULT;
+ if (pageshift > mem->pageshift)
+ return -EFAULT;
+
*hpa = *va | (ua & ~PAGE_MASK);
return 0;
@@ -364,7 +396,7 @@ long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem,
EXPORT_SYMBOL_GPL(mm_iommu_ua_to_hpa);
long mm_iommu_ua_to_hpa_rm(struct mm_iommu_table_group_mem_t *mem,
- unsigned long ua, unsigned long *hpa)
+ unsigned long ua, unsigned int pageshift, unsigned long *hpa)
{
const long entry = (ua - mem->ua) >> PAGE_SHIFT;
void *va = &mem->hpas[entry];
@@ -373,6 +405,9 @@ long mm_iommu_ua_to_hpa_rm(struct mm_iommu_table_group_mem_t *mem,
if (entry >= mem->entries)
return -EFAULT;
+ if (pageshift > mem->pageshift)
+ return -EFAULT;
+
pa = (void *) vmalloc_to_phys(va);
if (!pa)
return -EFAULT;
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 2da5f05..7cd63b0 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -467,7 +467,7 @@ static int tce_iommu_prereg_ua_to_hpa(struct tce_container *container,
if (!mem)
return -EINVAL;
- ret = mm_iommu_ua_to_hpa(mem, tce, phpa);
+ ret = mm_iommu_ua_to_hpa(mem, tce, shift, phpa);
if (ret)
return -EINVAL;
--
2.11.0
^ permalink raw reply related
* Re: [PATCH 2/3] powerpc/powernv: DMA operations for discontiguous allocation
From: Russell Currey @ 2018-07-04 6:12 UTC (permalink / raw)
To: linuxppc-dev; +Cc: benh, alistair, aik, tpearson
In-Reply-To: <20180629073437.4060-3-ruscur@russell.cc>
[-- Attachment #1: Type: text/plain, Size: 1121 bytes --]
On Fri, 2018-06-29 at 17:34 +1000, Russell Currey wrote:
<snip>
> + /*
> + * The TCE isn't being used, so let's try and
> allocate it.
> + * Bits 0 and 1 are read/write, and we use bit 2 as
> a "lock"
> + * bit. This is to prevent any race where the value
> is set in
> + * the TCE table but the invalidate/mb() hasn't
> finished yet.
> + */
> + entry = cpu_to_be64((addr - offset) | 7);
> + ret = cmpxchg(&pe->tces[i], tce, entry);
> + if (ret != tce) {
> + /* conflict, start looking again just in
> case */
> + i--;
> + continue;
> + }
> + pnv_pci_phb3_tce_invalidate(pe, 0, 0, addr - offset,
> 1);
This is wrong and won't work outside of PHB3, will make a generic
handler
> + mb();
> + /* clear the lock bit now that we know it's active
> */
> + ret = cmpxchg(&pe->tces[i], entry, cpu_to_be64((addr
> - offset) | 3));
> + if (ret != entry) {
> + /* conflict, start looking again just in
> case */
> + i--;
> + continue;
> + }
> +
> + return (i << phb->ioda.max_tce_order) | offset;
> + }
> + /* If we get here, the table must be full, so error out. */
> + return -1ULL;
> +}
> +
[-- Attachment #2: qtpass.desktop --]
[-- Type: application/x-desktop, Size: 911 bytes --]
^ permalink raw reply
* [PATCH kernel v3 1/6] powerpc/powernv: Remove useless wrapper
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
This gets rid of a useless wrapper around
pnv_pci_ioda2_table_free_pages().
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index cc5942d..02275a0 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2199,11 +2199,6 @@ static void pnv_ioda2_tce_free(struct iommu_table *tbl, long index,
pnv_pci_ioda2_tce_invalidate(tbl, index, npages, false);
}
-static void pnv_ioda2_table_free(struct iommu_table *tbl)
-{
- pnv_pci_ioda2_table_free_pages(tbl);
-}
-
static struct iommu_table_ops pnv_ioda2_iommu_ops = {
.set = pnv_ioda2_tce_build,
#ifdef CONFIG_IOMMU_API
@@ -2212,7 +2207,7 @@ static struct iommu_table_ops pnv_ioda2_iommu_ops = {
#endif
.clear = pnv_ioda2_tce_free,
.get = pnv_tce_get,
- .free = pnv_ioda2_table_free,
+ .free = pnv_pci_ioda2_table_free_pages,
};
static int pnv_pci_ioda_dev_dma_weight(struct pci_dev *dev, void *data)
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 4/6] powerpc/powernv: Add indirect levels to it_userspace
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
We want to support sparse memory and therefore huge chunks of DMA windows
do not need to be mapped. If a DMA window big enough to require 2 or more
indirect levels, and a DMA window is used to map all RAM (which is
a default case for 64bit window), we can actually save some memory by
not allocation TCE for regions which we are not going to map anyway.
The hardware tables alreary support indirect levels but we also keep
host-physical-to-userspace translation array which is allocated by
vmalloc() and is a flat array which might use quite some memory.
This converts it_userspace from vmalloc'ed array to a multi level table.
As the format becomes platform dependend, this replaces the direct access
to it_usespace with a iommu_table_ops::useraddrptr hook which returns
a pointer to the userspace copy of a TCE; future extension will return
NULL if the level was not allocated.
This should not change non-KVM handling of TCE tables and it_userspace
will not be allocated for non-KVM tables.
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v2:
* fixed compile error by ditching one inline helper
---
arch/powerpc/include/asm/iommu.h | 6 +--
arch/powerpc/platforms/powernv/pci.h | 3 +-
arch/powerpc/kvm/book3s_64_vio_hv.c | 8 ----
arch/powerpc/platforms/powernv/pci-ioda-tce.c | 65 +++++++++++++++++++++------
arch/powerpc/platforms/powernv/pci-ioda.c | 23 +++++++---
drivers/vfio/vfio_iommu_spapr_tce.c | 46 -------------------
6 files changed, 73 insertions(+), 78 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 803ac70..4bdcf22 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -69,6 +69,8 @@ struct iommu_table_ops {
long index,
unsigned long *hpa,
enum dma_data_direction *direction);
+
+ __be64 *(*useraddrptr)(struct iommu_table *tbl, long index);
#endif
void (*clear)(struct iommu_table *tbl,
long index, long npages);
@@ -123,9 +125,7 @@ struct iommu_table {
};
#define IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry) \
- ((tbl)->it_userspace ? \
- &((tbl)->it_userspace[(entry) - (tbl)->it_offset]) : \
- NULL)
+ ((tbl)->it_ops->useraddrptr((tbl), (entry)))
/* Pure 2^n version of get_order */
static inline __attribute_const__
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index fa90f60..2962f6d 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -267,11 +267,12 @@ extern int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
extern void pnv_tce_free(struct iommu_table *tbl, long index, long npages);
extern int pnv_tce_xchg(struct iommu_table *tbl, long index,
unsigned long *hpa, enum dma_data_direction *direction);
+extern __be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index);
extern unsigned long pnv_tce_get(struct iommu_table *tbl, long index);
extern long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
__u32 page_shift, __u64 window_size, __u32 levels,
- struct iommu_table *tbl);
+ bool alloc_userspace_copy, struct iommu_table *tbl);
extern void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl);
extern long pnv_pci_link_table_and_group(int node, int num,
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 841aef7..8cc1caf 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -206,10 +206,6 @@ static long kvmppc_rm_tce_iommu_mapped_dec(struct kvm *kvm,
/* it_userspace allocation might be delayed */
return H_TOO_HARD;
- pua = (void *) vmalloc_to_phys(pua);
- if (WARN_ON_ONCE_RM(!pua))
- return H_HARDWARE;
-
mem = mm_iommu_lookup_rm(kvm->mm, be64_to_cpu(*pua), pgsize);
if (!mem)
return H_TOO_HARD;
@@ -283,10 +279,6 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
&hpa)))
return H_HARDWARE;
- pua = (void *) vmalloc_to_phys(pua);
- if (WARN_ON_ONCE_RM(!pua))
- return H_HARDWARE;
-
if (WARN_ON_ONCE_RM(mm_iommu_mapped_inc(mem)))
return H_CLOSED;
diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
index 700ceb1..f14b282 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda-tce.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
@@ -31,9 +31,9 @@ void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
tbl->it_type = TCE_PCI;
}
-static __be64 *pnv_tce(struct iommu_table *tbl, long idx)
+static __be64 *pnv_tce(struct iommu_table *tbl, bool user, long idx)
{
- __be64 *tmp = ((__be64 *)tbl->it_base);
+ __be64 *tmp = user ? tbl->it_userspace : (__be64 *) tbl->it_base;
int level = tbl->it_indirect_levels;
const long shift = ilog2(tbl->it_level_size);
unsigned long mask = (tbl->it_level_size - 1) << (level * shift);
@@ -67,7 +67,7 @@ int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
((rpn + i) << tbl->it_page_shift);
unsigned long idx = index - tbl->it_offset + i;
- *(pnv_tce(tbl, idx)) = cpu_to_be64(newtce);
+ *(pnv_tce(tbl, false, idx)) = cpu_to_be64(newtce);
}
return 0;
@@ -86,12 +86,21 @@ int pnv_tce_xchg(struct iommu_table *tbl, long index,
if (newtce & TCE_PCI_WRITE)
newtce |= TCE_PCI_READ;
- oldtce = be64_to_cpu(xchg(pnv_tce(tbl, idx), cpu_to_be64(newtce)));
+ oldtce = be64_to_cpu(xchg(pnv_tce(tbl, false, idx),
+ cpu_to_be64(newtce)));
*hpa = oldtce & ~(TCE_PCI_READ | TCE_PCI_WRITE);
*direction = iommu_tce_direction(oldtce);
return 0;
}
+
+__be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index)
+{
+ if (WARN_ON_ONCE(!tbl->it_userspace))
+ return NULL;
+
+ return pnv_tce(tbl, true, index - tbl->it_offset);
+}
#endif
void pnv_tce_free(struct iommu_table *tbl, long index, long npages)
@@ -101,13 +110,15 @@ void pnv_tce_free(struct iommu_table *tbl, long index, long npages)
for (i = 0; i < npages; i++) {
unsigned long idx = index - tbl->it_offset + i;
- *(pnv_tce(tbl, idx)) = cpu_to_be64(0);
+ *(pnv_tce(tbl, false, idx)) = cpu_to_be64(0);
}
}
unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
{
- return be64_to_cpu(*(pnv_tce(tbl, index - tbl->it_offset)));
+ __be64 *ptce = pnv_tce(tbl, false, index - tbl->it_offset);
+
+ return be64_to_cpu(*ptce);
}
static void pnv_pci_ioda2_table_do_free_pages(__be64 *addr,
@@ -144,6 +155,10 @@ void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl)
pnv_pci_ioda2_table_do_free_pages((__be64 *)tbl->it_base, size,
tbl->it_indirect_levels);
+ if (tbl->it_userspace) {
+ pnv_pci_ioda2_table_do_free_pages(tbl->it_userspace, size,
+ tbl->it_indirect_levels);
+ }
}
static __be64 *pnv_pci_ioda2_table_do_alloc_pages(int nid, unsigned int shift,
@@ -191,10 +206,11 @@ static __be64 *pnv_pci_ioda2_table_do_alloc_pages(int nid, unsigned int shift,
long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
__u32 page_shift, __u64 window_size, __u32 levels,
- struct iommu_table *tbl)
+ bool alloc_userspace_copy, struct iommu_table *tbl)
{
- void *addr;
+ void *addr, *uas = NULL;
unsigned long offset = 0, level_shift, total_allocated = 0;
+ unsigned long total_allocated_uas = 0;
const unsigned int window_shift = ilog2(window_size);
unsigned int entries_shift = window_shift - page_shift;
unsigned int table_shift = max_t(unsigned int, entries_shift + 3,
@@ -228,10 +244,20 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
* we did not allocate as much as we wanted,
* release partially allocated table.
*/
- if (offset < tce_table_size) {
- pnv_pci_ioda2_table_do_free_pages(addr,
- 1ULL << (level_shift - 3), levels - 1);
- return -ENOMEM;
+ if (offset < tce_table_size)
+ goto free_tces_exit;
+
+ /* Allocate userspace view of the TCE table */
+ if (alloc_userspace_copy) {
+ offset = 0;
+ uas = pnv_pci_ioda2_table_do_alloc_pages(nid, level_shift,
+ levels, tce_table_size, &offset,
+ &total_allocated_uas);
+ if (!uas)
+ goto free_tces_exit;
+ if (offset < tce_table_size ||
+ total_allocated_uas != total_allocated)
+ goto free_uas_exit;
}
/* Setup linux iommu table */
@@ -240,11 +266,22 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
tbl->it_level_size = 1ULL << (level_shift - 3);
tbl->it_indirect_levels = levels - 1;
tbl->it_allocated_size = total_allocated;
+ tbl->it_userspace = uas;
- pr_devel("Created TCE table: ws=%08llx ts=%lx @%08llx\n",
- window_size, tce_table_size, bus_offset);
+ pr_debug("Created TCE table: ws=%08llx ts=%lx @%08llx base=%lx uas=%p levels=%d\n",
+ window_size, tce_table_size, bus_offset, tbl->it_base,
+ tbl->it_userspace, levels);
return 0;
+
+free_uas_exit:
+ pnv_pci_ioda2_table_do_free_pages(uas,
+ 1ULL << (level_shift - 3), levels - 1);
+free_tces_exit:
+ pnv_pci_ioda2_table_do_free_pages(addr,
+ 1ULL << (level_shift - 3), levels - 1);
+
+ return -ENOMEM;
}
static void pnv_iommu_table_group_link_free(struct rcu_head *head)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 3fc91d4..20a9d59 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2036,6 +2036,7 @@ static struct iommu_table_ops pnv_ioda1_iommu_ops = {
#ifdef CONFIG_IOMMU_API
.exchange = pnv_ioda1_tce_xchg,
.exchange_rm = pnv_ioda1_tce_xchg_rm,
+ .useraddrptr = pnv_tce_useraddrptr,
#endif
.clear = pnv_ioda1_tce_free,
.get = pnv_tce_get,
@@ -2200,6 +2201,7 @@ static struct iommu_table_ops pnv_ioda2_iommu_ops = {
#ifdef CONFIG_IOMMU_API
.exchange = pnv_ioda2_tce_xchg,
.exchange_rm = pnv_ioda2_tce_xchg_rm,
+ .useraddrptr = pnv_tce_useraddrptr,
#endif
.clear = pnv_ioda2_tce_free,
.get = pnv_tce_get,
@@ -2455,7 +2457,7 @@ void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable)
static long pnv_pci_ioda2_create_table(struct iommu_table_group *table_group,
int num, __u32 page_shift, __u64 window_size, __u32 levels,
- struct iommu_table **ptbl)
+ bool alloc_userspace_copy, struct iommu_table **ptbl)
{
struct pnv_ioda_pe *pe = container_of(table_group, struct pnv_ioda_pe,
table_group);
@@ -2472,7 +2474,7 @@ static long pnv_pci_ioda2_create_table(struct iommu_table_group *table_group,
ret = pnv_pci_ioda2_table_alloc_pages(nid,
bus_offset, page_shift, window_size,
- levels, tbl);
+ levels, alloc_userspace_copy, tbl);
if (ret) {
iommu_tce_table_put(tbl);
return ret;
@@ -2505,7 +2507,7 @@ static long pnv_pci_ioda2_setup_default_config(struct pnv_ioda_pe *pe)
rc = pnv_pci_ioda2_create_table(&pe->table_group, 0,
IOMMU_PAGE_SHIFT_4K,
window_size,
- POWERNV_IOMMU_DEFAULT_LEVELS, &tbl);
+ POWERNV_IOMMU_DEFAULT_LEVELS, false, &tbl);
if (rc) {
pe_err(pe, "Failed to create 32-bit TCE table, err %ld",
rc);
@@ -2592,7 +2594,16 @@ static unsigned long pnv_pci_ioda2_get_table_size(__u32 page_shift,
tce_table_size, direct_table_size);
}
- return bytes;
+ return bytes + bytes; /* one for HW table, one for userspace copy */
+}
+
+static long pnv_pci_ioda2_create_table_userspace(
+ struct iommu_table_group *table_group,
+ int num, __u32 page_shift, __u64 window_size, __u32 levels,
+ struct iommu_table **ptbl)
+{
+ return pnv_pci_ioda2_create_table(table_group,
+ num, page_shift, window_size, levels, true, ptbl);
}
static void pnv_ioda2_take_ownership(struct iommu_table_group *table_group)
@@ -2621,7 +2632,7 @@ static void pnv_ioda2_release_ownership(struct iommu_table_group *table_group)
static struct iommu_table_group_ops pnv_pci_ioda2_ops = {
.get_table_size = pnv_pci_ioda2_get_table_size,
- .create_table = pnv_pci_ioda2_create_table,
+ .create_table = pnv_pci_ioda2_create_table_userspace,
.set_window = pnv_pci_ioda2_set_window,
.unset_window = pnv_pci_ioda2_unset_window,
.take_ownership = pnv_ioda2_take_ownership,
@@ -2726,7 +2737,7 @@ static void pnv_ioda2_npu_take_ownership(struct iommu_table_group *table_group)
static struct iommu_table_group_ops pnv_pci_ioda2_npu_ops = {
.get_table_size = pnv_pci_ioda2_get_table_size,
- .create_table = pnv_pci_ioda2_create_table,
+ .create_table = pnv_pci_ioda2_create_table_userspace,
.set_window = pnv_pci_ioda2_npu_set_window,
.unset_window = pnv_pci_ioda2_npu_unset_window,
.take_ownership = pnv_ioda2_npu_take_ownership,
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 17a418c..e99971b 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -211,44 +211,6 @@ static long tce_iommu_register_pages(struct tce_container *container,
return 0;
}
-static long tce_iommu_userspace_view_alloc(struct iommu_table *tbl,
- struct mm_struct *mm)
-{
- unsigned long cb = _ALIGN_UP(sizeof(tbl->it_userspace[0]) *
- tbl->it_size, PAGE_SIZE);
- unsigned long *uas;
- long ret;
-
- BUG_ON(tbl->it_userspace);
-
- ret = try_increment_locked_vm(mm, cb >> PAGE_SHIFT);
- if (ret)
- return ret;
-
- uas = vzalloc(cb);
- if (!uas) {
- decrement_locked_vm(mm, cb >> PAGE_SHIFT);
- return -ENOMEM;
- }
- tbl->it_userspace = (__be64 *) uas;
-
- return 0;
-}
-
-static void tce_iommu_userspace_view_free(struct iommu_table *tbl,
- struct mm_struct *mm)
-{
- unsigned long cb = _ALIGN_UP(sizeof(tbl->it_userspace[0]) *
- tbl->it_size, PAGE_SIZE);
-
- if (!tbl->it_userspace)
- return;
-
- vfree(tbl->it_userspace);
- tbl->it_userspace = NULL;
- decrement_locked_vm(mm, cb >> PAGE_SHIFT);
-}
-
static bool tce_page_is_contained(struct page *page, unsigned page_shift)
{
/*
@@ -599,12 +561,6 @@ static long tce_iommu_build_v2(struct tce_container *container,
unsigned long hpa;
enum dma_data_direction dirtmp;
- if (!tbl->it_userspace) {
- ret = tce_iommu_userspace_view_alloc(tbl, container->mm);
- if (ret)
- return ret;
- }
-
for (i = 0; i < pages; ++i) {
struct mm_iommu_table_group_mem_t *mem = NULL;
__be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry + i);
@@ -685,7 +641,6 @@ static void tce_iommu_free_table(struct tce_container *container,
{
unsigned long pages = tbl->it_allocated_size >> PAGE_SHIFT;
- tce_iommu_userspace_view_free(tbl, container->mm);
iommu_tce_table_put(tbl);
decrement_locked_vm(container->mm, pages);
}
@@ -1200,7 +1155,6 @@ static void tce_iommu_release_ownership(struct tce_container *container,
continue;
tce_iommu_clear(container, tbl, tbl->it_offset, tbl->it_size);
- tce_iommu_userspace_view_free(tbl, container->mm);
if (tbl->it_map)
iommu_release_ownership(tbl);
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 5/6] powerpc/powernv: Rework TCE level allocation
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
This moves actual pages allocation to a separate function which is going
to be reused later in on-demand TCE allocation.
While we are at it, remove unnecessary level size round up as the caller
does this already.
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/platforms/powernv/pci-ioda-tce.c | 30 +++++++++++++++++----------
1 file changed, 19 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
index f14b282..36c2eb0 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda-tce.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
@@ -31,6 +31,23 @@ void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
tbl->it_type = TCE_PCI;
}
+static __be64 *pnv_alloc_tce_level(int nid, unsigned int shift)
+{
+ struct page *tce_mem = NULL;
+ __be64 *addr;
+
+ tce_mem = alloc_pages_node(nid, GFP_KERNEL, shift - PAGE_SHIFT);
+ if (!tce_mem) {
+ pr_err("Failed to allocate a TCE memory, level shift=%d\n",
+ shift);
+ return NULL;
+ }
+ addr = page_address(tce_mem);
+ memset(addr, 0, 1UL << shift);
+
+ return addr;
+}
+
static __be64 *pnv_tce(struct iommu_table *tbl, bool user, long idx)
{
__be64 *tmp = user ? tbl->it_userspace : (__be64 *) tbl->it_base;
@@ -165,21 +182,12 @@ static __be64 *pnv_pci_ioda2_table_do_alloc_pages(int nid, unsigned int shift,
unsigned int levels, unsigned long limit,
unsigned long *current_offset, unsigned long *total_allocated)
{
- struct page *tce_mem = NULL;
__be64 *addr, *tmp;
- unsigned int order = max_t(unsigned int, shift, PAGE_SHIFT) -
- PAGE_SHIFT;
- unsigned long allocated = 1UL << (order + PAGE_SHIFT);
+ unsigned long allocated = 1UL << shift;
unsigned int entries = 1UL << (shift - 3);
long i;
- tce_mem = alloc_pages_node(nid, GFP_KERNEL, order);
- if (!tce_mem) {
- pr_err("Failed to allocate a TCE memory, order=%d\n", order);
- return NULL;
- }
- addr = page_address(tce_mem);
- memset(addr, 0, allocated);
+ addr = pnv_alloc_tce_level(nid, shift);
*total_allocated += allocated;
--levels;
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 6/6] powerpc/powernv/ioda: Allocate indirect TCE levels on demand
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
At the moment we allocate the entire TCE table, twice (hardware part and
userspace translation cache). This normally works as we normally have
contigous memory and the guest will map entire RAM for 64bit DMA.
However if we have sparse RAM (one example is a memory device), then
we will allocate TCEs which will never be used as the guest only maps
actual memory for DMA. If it is a single level TCE table, there is nothing
we can really do but if it a multilevel table, we can skip allocating
TCEs we know we won't need.
This adds ability to allocate only first level, saving memory.
This changes iommu_table::free() to avoid allocating of an extra level;
iommu_table::set() will do this when needed.
This adds @alloc parameter to iommu_table::exchange() to tell the callback
if it can allocate an extra level; the flag is set to "false" for
the realmode KVM handlers of H_PUT_TCE hcalls and the callback returns
H_TOO_HARD.
This still requires the entire table to be counted in mm::locked_vm.
To be conservative, this only does on-demand allocation when
the usespace cache table is requested which is the case of VFIO.
The example math for a system replicating a powernv setup with NVLink2
in a guest:
16GB RAM mapped at 0x0
128GB GPU RAM window (16GB of actual RAM) mapped at 0x244000000000
the table to cover that all with 64K pages takes:
(((0x244000000000 + 0x2000000000) >> 16)*8)>>20 = 4556MB
If we allocate only necessary TCE levels, we will only need:
(((0x400000000 + 0x400000000) >> 16)*8)>>20 = 4MB (plus some for indirect
levels).
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
Changes:
v2:
* fixed bug in cleanup path which forced the entire table to be
allocated right before destroying
* added memory allocation error handling pnv_tce()
---
arch/powerpc/include/asm/iommu.h | 7 ++-
arch/powerpc/platforms/powernv/pci.h | 6 ++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 4 +-
arch/powerpc/platforms/powernv/pci-ioda-tce.c | 73 +++++++++++++++++++++------
arch/powerpc/platforms/powernv/pci-ioda.c | 8 +--
drivers/vfio/vfio_iommu_spapr_tce.c | 2 +-
6 files changed, 73 insertions(+), 27 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 4bdcf22..daa3ee5 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -70,7 +70,7 @@ struct iommu_table_ops {
unsigned long *hpa,
enum dma_data_direction *direction);
- __be64 *(*useraddrptr)(struct iommu_table *tbl, long index);
+ __be64 *(*useraddrptr)(struct iommu_table *tbl, long index, bool alloc);
#endif
void (*clear)(struct iommu_table *tbl,
long index, long npages);
@@ -122,10 +122,13 @@ struct iommu_table {
__be64 *it_userspace; /* userspace view of the table */
struct iommu_table_ops *it_ops;
struct kref it_kref;
+ int it_nid;
};
+#define IOMMU_TABLE_USERSPACE_ENTRY_RM(tbl, entry) \
+ ((tbl)->it_ops->useraddrptr((tbl), (entry), false))
#define IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry) \
- ((tbl)->it_ops->useraddrptr((tbl), (entry)))
+ ((tbl)->it_ops->useraddrptr((tbl), (entry), true))
/* Pure 2^n version of get_order */
static inline __attribute_const__
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 2962f6d..0020937 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -266,8 +266,10 @@ extern int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
unsigned long attrs);
extern void pnv_tce_free(struct iommu_table *tbl, long index, long npages);
extern int pnv_tce_xchg(struct iommu_table *tbl, long index,
- unsigned long *hpa, enum dma_data_direction *direction);
-extern __be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index);
+ unsigned long *hpa, enum dma_data_direction *direction,
+ bool alloc);
+extern __be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index,
+ bool alloc);
extern unsigned long pnv_tce_get(struct iommu_table *tbl, long index);
extern long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 8cc1caf..efb90d8 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -200,7 +200,7 @@ static long kvmppc_rm_tce_iommu_mapped_dec(struct kvm *kvm,
{
struct mm_iommu_table_group_mem_t *mem = NULL;
const unsigned long pgsize = 1ULL << tbl->it_page_shift;
- __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY_RM(tbl, entry);
if (!pua)
/* it_userspace allocation might be delayed */
@@ -264,7 +264,7 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
{
long ret;
unsigned long hpa = 0;
- __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY_RM(tbl, entry);
struct mm_iommu_table_group_mem_t *mem;
if (!pua)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
index 36c2eb0..fe96910 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda-tce.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
@@ -48,7 +48,7 @@ static __be64 *pnv_alloc_tce_level(int nid, unsigned int shift)
return addr;
}
-static __be64 *pnv_tce(struct iommu_table *tbl, bool user, long idx)
+static __be64 *pnv_tce(struct iommu_table *tbl, bool user, long idx, bool alloc)
{
__be64 *tmp = user ? tbl->it_userspace : (__be64 *) tbl->it_base;
int level = tbl->it_indirect_levels;
@@ -57,7 +57,23 @@ static __be64 *pnv_tce(struct iommu_table *tbl, bool user, long idx)
while (level) {
int n = (idx & mask) >> (level * shift);
- unsigned long tce = be64_to_cpu(tmp[n]);
+ unsigned long tce;
+
+ if (tmp[n] == 0) {
+ __be64 *tmp2;
+
+ if (!alloc)
+ return NULL;
+
+ tmp2 = pnv_alloc_tce_level(tbl->it_nid,
+ ilog2(tbl->it_level_size) + 3);
+ if (!tmp2)
+ return NULL;
+
+ tmp[n] = cpu_to_be64(__pa(tmp2) |
+ TCE_PCI_READ | TCE_PCI_WRITE);
+ }
+ tce = be64_to_cpu(tmp[n]);
tmp = __va(tce & ~(TCE_PCI_READ | TCE_PCI_WRITE));
idx &= ~mask;
@@ -84,7 +100,7 @@ int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
((rpn + i) << tbl->it_page_shift);
unsigned long idx = index - tbl->it_offset + i;
- *(pnv_tce(tbl, false, idx)) = cpu_to_be64(newtce);
+ *(pnv_tce(tbl, false, idx, true)) = cpu_to_be64(newtce);
}
return 0;
@@ -92,31 +108,46 @@ int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
#ifdef CONFIG_IOMMU_API
int pnv_tce_xchg(struct iommu_table *tbl, long index,
- unsigned long *hpa, enum dma_data_direction *direction)
+ unsigned long *hpa, enum dma_data_direction *direction,
+ bool alloc)
{
u64 proto_tce = iommu_direction_to_tce_perm(*direction);
unsigned long newtce = *hpa | proto_tce, oldtce;
unsigned long idx = index - tbl->it_offset;
+ __be64 *ptce = NULL;
BUG_ON(*hpa & ~IOMMU_PAGE_MASK(tbl));
+ if (*direction == DMA_NONE) {
+ ptce = pnv_tce(tbl, false, idx, false);
+ if (!ptce) {
+ *hpa = 0;
+ return 0;
+ }
+ }
+
+ if (!ptce) {
+ ptce = pnv_tce(tbl, false, idx, alloc);
+ if (!ptce)
+ return alloc ? H_HARDWARE : H_TOO_HARD;
+ }
+
if (newtce & TCE_PCI_WRITE)
newtce |= TCE_PCI_READ;
- oldtce = be64_to_cpu(xchg(pnv_tce(tbl, false, idx),
- cpu_to_be64(newtce)));
+ oldtce = be64_to_cpu(xchg(ptce, cpu_to_be64(newtce)));
*hpa = oldtce & ~(TCE_PCI_READ | TCE_PCI_WRITE);
*direction = iommu_tce_direction(oldtce);
return 0;
}
-__be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index)
+__be64 *pnv_tce_useraddrptr(struct iommu_table *tbl, long index, bool alloc)
{
if (WARN_ON_ONCE(!tbl->it_userspace))
return NULL;
- return pnv_tce(tbl, true, index - tbl->it_offset);
+ return pnv_tce(tbl, true, index - tbl->it_offset, alloc);
}
#endif
@@ -126,14 +157,19 @@ void pnv_tce_free(struct iommu_table *tbl, long index, long npages)
for (i = 0; i < npages; i++) {
unsigned long idx = index - tbl->it_offset + i;
+ __be64 *ptce = pnv_tce(tbl, false, idx, false);
- *(pnv_tce(tbl, false, idx)) = cpu_to_be64(0);
+ if (ptce)
+ *ptce = cpu_to_be64(0);
}
}
unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
{
- __be64 *ptce = pnv_tce(tbl, false, index - tbl->it_offset);
+ __be64 *ptce = pnv_tce(tbl, false, index - tbl->it_offset, false);
+
+ if (!ptce)
+ return 0;
return be64_to_cpu(*ptce);
}
@@ -224,6 +260,7 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
unsigned int table_shift = max_t(unsigned int, entries_shift + 3,
PAGE_SHIFT);
const unsigned long tce_table_size = 1UL << table_shift;
+ unsigned int tmplevels = levels;
if (!levels || (levels > POWERNV_IOMMU_MAX_LEVELS))
return -EINVAL;
@@ -231,6 +268,9 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
if (!is_power_of_2(window_size))
return -EINVAL;
+ if (alloc_userspace_copy && (window_size > (1ULL << 32)))
+ tmplevels = 1;
+
/* Adjust direct table size from window_size and levels */
entries_shift = (entries_shift + levels - 1) / levels;
level_shift = entries_shift + 3;
@@ -241,7 +281,7 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
/* Allocate TCE table */
addr = pnv_pci_ioda2_table_do_alloc_pages(nid, level_shift,
- levels, tce_table_size, &offset, &total_allocated);
+ tmplevels, tce_table_size, &offset, &total_allocated);
/* addr==NULL means that the first level allocation failed */
if (!addr)
@@ -252,7 +292,7 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
* we did not allocate as much as we wanted,
* release partially allocated table.
*/
- if (offset < tce_table_size)
+ if (tmplevels == levels && offset < tce_table_size)
goto free_tces_exit;
/* Allocate userspace view of the TCE table */
@@ -263,8 +303,8 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
&total_allocated_uas);
if (!uas)
goto free_tces_exit;
- if (offset < tce_table_size ||
- total_allocated_uas != total_allocated)
+ if (tmplevels == levels && (offset < tce_table_size ||
+ total_allocated_uas != total_allocated))
goto free_uas_exit;
}
@@ -275,10 +315,11 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
tbl->it_indirect_levels = levels - 1;
tbl->it_allocated_size = total_allocated;
tbl->it_userspace = uas;
+ tbl->it_nid = nid;
- pr_debug("Created TCE table: ws=%08llx ts=%lx @%08llx base=%lx uas=%p levels=%d\n",
+ pr_debug("Created TCE table: ws=%08llx ts=%lx @%08llx base=%lx uas=%p levels=%d/%d\n",
window_size, tce_table_size, bus_offset, tbl->it_base,
- tbl->it_userspace, levels);
+ tbl->it_userspace, tmplevels, levels);
return 0;
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 20a9d59..7e4e9e8 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2003,7 +2003,7 @@ static int pnv_ioda1_tce_build(struct iommu_table *tbl, long index,
static int pnv_ioda1_tce_xchg(struct iommu_table *tbl, long index,
unsigned long *hpa, enum dma_data_direction *direction)
{
- long ret = pnv_tce_xchg(tbl, index, hpa, direction);
+ long ret = pnv_tce_xchg(tbl, index, hpa, direction, true);
if (!ret)
pnv_pci_p7ioc_tce_invalidate(tbl, index, 1, false);
@@ -2014,7 +2014,7 @@ static int pnv_ioda1_tce_xchg(struct iommu_table *tbl, long index,
static int pnv_ioda1_tce_xchg_rm(struct iommu_table *tbl, long index,
unsigned long *hpa, enum dma_data_direction *direction)
{
- long ret = pnv_tce_xchg(tbl, index, hpa, direction);
+ long ret = pnv_tce_xchg(tbl, index, hpa, direction, false);
if (!ret)
pnv_pci_p7ioc_tce_invalidate(tbl, index, 1, true);
@@ -2168,7 +2168,7 @@ static int pnv_ioda2_tce_build(struct iommu_table *tbl, long index,
static int pnv_ioda2_tce_xchg(struct iommu_table *tbl, long index,
unsigned long *hpa, enum dma_data_direction *direction)
{
- long ret = pnv_tce_xchg(tbl, index, hpa, direction);
+ long ret = pnv_tce_xchg(tbl, index, hpa, direction, true);
if (!ret)
pnv_pci_ioda2_tce_invalidate(tbl, index, 1, false);
@@ -2179,7 +2179,7 @@ static int pnv_ioda2_tce_xchg(struct iommu_table *tbl, long index,
static int pnv_ioda2_tce_xchg_rm(struct iommu_table *tbl, long index,
unsigned long *hpa, enum dma_data_direction *direction)
{
- long ret = pnv_tce_xchg(tbl, index, hpa, direction);
+ long ret = pnv_tce_xchg(tbl, index, hpa, direction, false);
if (!ret)
pnv_pci_ioda2_tce_invalidate(tbl, index, 1, true);
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index e99971b..96721b1 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -631,7 +631,7 @@ static long tce_iommu_create_table(struct tce_container *container,
page_shift, window_size, levels, ptbl);
WARN_ON(!ret && !(*ptbl)->it_ops->free);
- WARN_ON(!ret && ((*ptbl)->it_allocated_size != table_size));
+ WARN_ON(!ret && ((*ptbl)->it_allocated_size > table_size));
return ret;
}
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 0/6] powerpc/powernv/iommu: Optimize memory use
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
This patchset aims to reduce actual memory use for guests with
sparse memory. The pseries guest uses dynamic DMA windows to map
the entire guest RAM but it only actually maps onlined memory
which may be not be contiguous. I hit this when tried passing
through NVLink2-connected GPU RAM of NVIDIA V100 and trying to
map this RAM at the same offset as in the real hardware
forced me to rework I handle these windows.
This moves userspace-to-host-physical translation table
(iommu_table::it_userspace) from VFIO TCE IOMMU subdriver to
the platform code and reuses the already existing multilevel
TCE table code which we have for the hardware tables.
At last in 6/6 I switch to on-demand allocation so we do not
allocate huge chunks of the table if we do not have to;
there is some math in 6/6.
Changes:
v3:
* rebased on v4.18-rc3 and fixed compile error in 6/6
v2:
* bugfix and error handling in 6/6
This is based on sha1
021c917 Linus Torvalds "Linux 4.18-rc3".
Please comment. Thanks.
Alexey Kardashevskiy (6):
powerpc/powernv: Remove useless wrapper
powerpc/powernv: Move TCE manupulation code to its own file
KVM: PPC: Make iommu_table::it_userspace big endian
powerpc/powernv: Add indirect levels to it_userspace
powerpc/powernv: Rework TCE level allocation
powerpc/powernv/ioda: Allocate indirect TCE levels on demand
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/include/asm/iommu.h | 11 +-
arch/powerpc/platforms/powernv/pci.h | 44 ++-
arch/powerpc/kvm/book3s_64_vio.c | 11 +-
arch/powerpc/kvm/book3s_64_vio_hv.c | 18 +-
arch/powerpc/platforms/powernv/pci-ioda-tce.c | 399 ++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/pci-ioda.c | 184 ++----------
arch/powerpc/platforms/powernv/pci.c | 158 ----------
drivers/vfio/vfio_iommu_spapr_tce.c | 65 +----
9 files changed, 478 insertions(+), 414 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/pci-ioda-tce.c
--
2.11.0
^ permalink raw reply
* [PATCH kernel v3 2/6] powerpc/powernv: Move TCE manupulation code to its own file
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
Right now we have allocation code in pci-ioda.c and traversing code in
pci.c, let's keep them toghether. However both files are big enough
already so let's move this business to a new file.
While we at it, move the code which links IOMMU table groups to
IOMMU tables as it is not specific to any PNV PHB model.
These puts exported symbols from the new file together.
This fixes several warnings from checkpatch.pl like this:
"WARNING: Prefer 'unsigned int' to bare use of 'unsigned'".
As this is almost cut-n-paste, there should be no behavioral change.
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/pci.h | 41 ++--
arch/powerpc/platforms/powernv/pci-ioda-tce.c | 313 ++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/pci-ioda.c | 146 ------------
arch/powerpc/platforms/powernv/pci.c | 158 -------------
5 files changed, 340 insertions(+), 320 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/pci-ioda-tce.c
diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
index 703a350..b540ce8e 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -6,7 +6,7 @@ obj-y += opal-msglog.o opal-hmi.o opal-power.o opal-irqchip.o
obj-y += opal-kmsg.o opal-powercap.o opal-psr.o opal-sensor-groups.o
obj-$(CONFIG_SMP) += smp.o subcore.o subcore-asm.o
-obj-$(CONFIG_PCI) += pci.o pci-ioda.o npu-dma.o
+obj-$(CONFIG_PCI) += pci.o pci-ioda.o npu-dma.o pci-ioda-tce.o
obj-$(CONFIG_CXL_BASE) += pci-cxl.o
obj-$(CONFIG_EEH) += eeh-powernv.o
obj-$(CONFIG_PPC_SCOM) += opal-xscom.o
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index eada4b6..fa90f60 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -201,13 +201,6 @@ struct pnv_phb {
};
extern struct pci_ops pnv_pci_ops;
-extern int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
- unsigned long uaddr, enum dma_data_direction direction,
- unsigned long attrs);
-extern void pnv_tce_free(struct iommu_table *tbl, long index, long npages);
-extern int pnv_tce_xchg(struct iommu_table *tbl, long index,
- unsigned long *hpa, enum dma_data_direction *direction);
-extern unsigned long pnv_tce_get(struct iommu_table *tbl, long index);
void pnv_pci_dump_phb_diag_data(struct pci_controller *hose,
unsigned char *log_buff);
@@ -217,14 +210,6 @@ int pnv_pci_cfg_write(struct pci_dn *pdn,
int where, int size, u32 val);
extern struct iommu_table *pnv_pci_table_alloc(int nid);
-extern long pnv_pci_link_table_and_group(int node, int num,
- struct iommu_table *tbl,
- struct iommu_table_group *table_group);
-extern void pnv_pci_unlink_table_and_group(struct iommu_table *tbl,
- struct iommu_table_group *table_group);
-extern void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
- void *tce_mem, u64 tce_size,
- u64 dma_offset, unsigned page_shift);
extern void pnv_pci_init_ioda_hub(struct device_node *np);
extern void pnv_pci_init_ioda2_phb(struct device_node *np);
extern void pnv_pci_init_npu_phb(struct device_node *np);
@@ -272,4 +257,30 @@ extern void pnv_cxl_cx4_teardown_msi_irqs(struct pci_dev *pdev);
/* phb ops (cxl switches these when enabling the kernel api on the phb) */
extern const struct pci_controller_ops pnv_cxl_cx4_ioda_controller_ops;
+/* pci-ioda-tce.c */
+#define POWERNV_IOMMU_DEFAULT_LEVELS 1
+#define POWERNV_IOMMU_MAX_LEVELS 5
+
+extern int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
+ unsigned long uaddr, enum dma_data_direction direction,
+ unsigned long attrs);
+extern void pnv_tce_free(struct iommu_table *tbl, long index, long npages);
+extern int pnv_tce_xchg(struct iommu_table *tbl, long index,
+ unsigned long *hpa, enum dma_data_direction *direction);
+extern unsigned long pnv_tce_get(struct iommu_table *tbl, long index);
+
+extern long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
+ __u32 page_shift, __u64 window_size, __u32 levels,
+ struct iommu_table *tbl);
+extern void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl);
+
+extern long pnv_pci_link_table_and_group(int node, int num,
+ struct iommu_table *tbl,
+ struct iommu_table_group *table_group);
+extern void pnv_pci_unlink_table_and_group(struct iommu_table *tbl,
+ struct iommu_table_group *table_group);
+extern void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
+ void *tce_mem, u64 tce_size,
+ u64 dma_offset, unsigned int page_shift);
+
#endif /* __POWERNV_PCI_H */
diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
new file mode 100644
index 0000000..700ceb1
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/pci-ioda-tce.c
@@ -0,0 +1,313 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * TCE helpers for IODA PCI/PCIe on PowerNV platforms
+ *
+ * Copyright 2018 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/iommu.h>
+
+#include <asm/iommu.h>
+#include <asm/tce.h>
+#include "pci.h"
+
+void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
+ void *tce_mem, u64 tce_size,
+ u64 dma_offset, unsigned int page_shift)
+{
+ tbl->it_blocksize = 16;
+ tbl->it_base = (unsigned long)tce_mem;
+ tbl->it_page_shift = page_shift;
+ tbl->it_offset = dma_offset >> tbl->it_page_shift;
+ tbl->it_index = 0;
+ tbl->it_size = tce_size >> 3;
+ tbl->it_busno = 0;
+ tbl->it_type = TCE_PCI;
+}
+
+static __be64 *pnv_tce(struct iommu_table *tbl, long idx)
+{
+ __be64 *tmp = ((__be64 *)tbl->it_base);
+ int level = tbl->it_indirect_levels;
+ const long shift = ilog2(tbl->it_level_size);
+ unsigned long mask = (tbl->it_level_size - 1) << (level * shift);
+
+ while (level) {
+ int n = (idx & mask) >> (level * shift);
+ unsigned long tce = be64_to_cpu(tmp[n]);
+
+ tmp = __va(tce & ~(TCE_PCI_READ | TCE_PCI_WRITE));
+ idx &= ~mask;
+ mask >>= shift;
+ --level;
+ }
+
+ return tmp + idx;
+}
+
+int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
+ unsigned long uaddr, enum dma_data_direction direction,
+ unsigned long attrs)
+{
+ u64 proto_tce = iommu_direction_to_tce_perm(direction);
+ u64 rpn = __pa(uaddr) >> tbl->it_page_shift;
+ long i;
+
+ if (proto_tce & TCE_PCI_WRITE)
+ proto_tce |= TCE_PCI_READ;
+
+ for (i = 0; i < npages; i++) {
+ unsigned long newtce = proto_tce |
+ ((rpn + i) << tbl->it_page_shift);
+ unsigned long idx = index - tbl->it_offset + i;
+
+ *(pnv_tce(tbl, idx)) = cpu_to_be64(newtce);
+ }
+
+ return 0;
+}
+
+#ifdef CONFIG_IOMMU_API
+int pnv_tce_xchg(struct iommu_table *tbl, long index,
+ unsigned long *hpa, enum dma_data_direction *direction)
+{
+ u64 proto_tce = iommu_direction_to_tce_perm(*direction);
+ unsigned long newtce = *hpa | proto_tce, oldtce;
+ unsigned long idx = index - tbl->it_offset;
+
+ BUG_ON(*hpa & ~IOMMU_PAGE_MASK(tbl));
+
+ if (newtce & TCE_PCI_WRITE)
+ newtce |= TCE_PCI_READ;
+
+ oldtce = be64_to_cpu(xchg(pnv_tce(tbl, idx), cpu_to_be64(newtce)));
+ *hpa = oldtce & ~(TCE_PCI_READ | TCE_PCI_WRITE);
+ *direction = iommu_tce_direction(oldtce);
+
+ return 0;
+}
+#endif
+
+void pnv_tce_free(struct iommu_table *tbl, long index, long npages)
+{
+ long i;
+
+ for (i = 0; i < npages; i++) {
+ unsigned long idx = index - tbl->it_offset + i;
+
+ *(pnv_tce(tbl, idx)) = cpu_to_be64(0);
+ }
+}
+
+unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
+{
+ return be64_to_cpu(*(pnv_tce(tbl, index - tbl->it_offset)));
+}
+
+static void pnv_pci_ioda2_table_do_free_pages(__be64 *addr,
+ unsigned long size, unsigned int levels)
+{
+ const unsigned long addr_ul = (unsigned long) addr &
+ ~(TCE_PCI_READ | TCE_PCI_WRITE);
+
+ if (levels) {
+ long i;
+ u64 *tmp = (u64 *) addr_ul;
+
+ for (i = 0; i < size; ++i) {
+ unsigned long hpa = be64_to_cpu(tmp[i]);
+
+ if (!(hpa & (TCE_PCI_READ | TCE_PCI_WRITE)))
+ continue;
+
+ pnv_pci_ioda2_table_do_free_pages(__va(hpa), size,
+ levels - 1);
+ }
+ }
+
+ free_pages(addr_ul, get_order(size << 3));
+}
+
+void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl)
+{
+ const unsigned long size = tbl->it_indirect_levels ?
+ tbl->it_level_size : tbl->it_size;
+
+ if (!tbl->it_size)
+ return;
+
+ pnv_pci_ioda2_table_do_free_pages((__be64 *)tbl->it_base, size,
+ tbl->it_indirect_levels);
+}
+
+static __be64 *pnv_pci_ioda2_table_do_alloc_pages(int nid, unsigned int shift,
+ unsigned int levels, unsigned long limit,
+ unsigned long *current_offset, unsigned long *total_allocated)
+{
+ struct page *tce_mem = NULL;
+ __be64 *addr, *tmp;
+ unsigned int order = max_t(unsigned int, shift, PAGE_SHIFT) -
+ PAGE_SHIFT;
+ unsigned long allocated = 1UL << (order + PAGE_SHIFT);
+ unsigned int entries = 1UL << (shift - 3);
+ long i;
+
+ tce_mem = alloc_pages_node(nid, GFP_KERNEL, order);
+ if (!tce_mem) {
+ pr_err("Failed to allocate a TCE memory, order=%d\n", order);
+ return NULL;
+ }
+ addr = page_address(tce_mem);
+ memset(addr, 0, allocated);
+ *total_allocated += allocated;
+
+ --levels;
+ if (!levels) {
+ *current_offset += allocated;
+ return addr;
+ }
+
+ for (i = 0; i < entries; ++i) {
+ tmp = pnv_pci_ioda2_table_do_alloc_pages(nid, shift,
+ levels, limit, current_offset, total_allocated);
+ if (!tmp)
+ break;
+
+ addr[i] = cpu_to_be64(__pa(tmp) |
+ TCE_PCI_READ | TCE_PCI_WRITE);
+
+ if (*current_offset >= limit)
+ break;
+ }
+
+ return addr;
+}
+
+long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
+ __u32 page_shift, __u64 window_size, __u32 levels,
+ struct iommu_table *tbl)
+{
+ void *addr;
+ unsigned long offset = 0, level_shift, total_allocated = 0;
+ const unsigned int window_shift = ilog2(window_size);
+ unsigned int entries_shift = window_shift - page_shift;
+ unsigned int table_shift = max_t(unsigned int, entries_shift + 3,
+ PAGE_SHIFT);
+ const unsigned long tce_table_size = 1UL << table_shift;
+
+ if (!levels || (levels > POWERNV_IOMMU_MAX_LEVELS))
+ return -EINVAL;
+
+ if (!is_power_of_2(window_size))
+ return -EINVAL;
+
+ /* Adjust direct table size from window_size and levels */
+ entries_shift = (entries_shift + levels - 1) / levels;
+ level_shift = entries_shift + 3;
+ level_shift = max_t(unsigned int, level_shift, PAGE_SHIFT);
+
+ if ((level_shift - 3) * levels + page_shift >= 55)
+ return -EINVAL;
+
+ /* Allocate TCE table */
+ addr = pnv_pci_ioda2_table_do_alloc_pages(nid, level_shift,
+ levels, tce_table_size, &offset, &total_allocated);
+
+ /* addr==NULL means that the first level allocation failed */
+ if (!addr)
+ return -ENOMEM;
+
+ /*
+ * First level was allocated but some lower level failed as
+ * we did not allocate as much as we wanted,
+ * release partially allocated table.
+ */
+ if (offset < tce_table_size) {
+ pnv_pci_ioda2_table_do_free_pages(addr,
+ 1ULL << (level_shift - 3), levels - 1);
+ return -ENOMEM;
+ }
+
+ /* Setup linux iommu table */
+ pnv_pci_setup_iommu_table(tbl, addr, tce_table_size, bus_offset,
+ page_shift);
+ tbl->it_level_size = 1ULL << (level_shift - 3);
+ tbl->it_indirect_levels = levels - 1;
+ tbl->it_allocated_size = total_allocated;
+
+ pr_devel("Created TCE table: ws=%08llx ts=%lx @%08llx\n",
+ window_size, tce_table_size, bus_offset);
+
+ return 0;
+}
+
+static void pnv_iommu_table_group_link_free(struct rcu_head *head)
+{
+ struct iommu_table_group_link *tgl = container_of(head,
+ struct iommu_table_group_link, rcu);
+
+ kfree(tgl);
+}
+
+void pnv_pci_unlink_table_and_group(struct iommu_table *tbl,
+ struct iommu_table_group *table_group)
+{
+ long i;
+ bool found;
+ struct iommu_table_group_link *tgl;
+
+ if (!tbl || !table_group)
+ return;
+
+ /* Remove link to a group from table's list of attached groups */
+ found = false;
+ list_for_each_entry_rcu(tgl, &tbl->it_group_list, next) {
+ if (tgl->table_group == table_group) {
+ list_del_rcu(&tgl->next);
+ call_rcu(&tgl->rcu, pnv_iommu_table_group_link_free);
+ found = true;
+ break;
+ }
+ }
+ if (WARN_ON(!found))
+ return;
+
+ /* Clean a pointer to iommu_table in iommu_table_group::tables[] */
+ found = false;
+ for (i = 0; i < IOMMU_TABLE_GROUP_MAX_TABLES; ++i) {
+ if (table_group->tables[i] == tbl) {
+ table_group->tables[i] = NULL;
+ found = true;
+ break;
+ }
+ }
+ WARN_ON(!found);
+}
+
+long pnv_pci_link_table_and_group(int node, int num,
+ struct iommu_table *tbl,
+ struct iommu_table_group *table_group)
+{
+ struct iommu_table_group_link *tgl = NULL;
+
+ if (WARN_ON(!tbl || !table_group))
+ return -EINVAL;
+
+ tgl = kzalloc_node(sizeof(struct iommu_table_group_link), GFP_KERNEL,
+ node);
+ if (!tgl)
+ return -ENOMEM;
+
+ tgl->table_group = table_group;
+ list_add_rcu(&tgl->next, &tbl->it_group_list);
+
+ table_group->tables[num] = tbl;
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 02275a0..3fc91d4 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -51,12 +51,8 @@
#define PNV_IODA1_M64_SEGS 8 /* Segments per M64 BAR */
#define PNV_IODA1_DMA32_SEGSIZE 0x10000000
-#define POWERNV_IOMMU_DEFAULT_LEVELS 1
-#define POWERNV_IOMMU_MAX_LEVELS 5
-
static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU_NVLINK",
"NPU_OCAPI" };
-static void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl);
void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
const char *fmt, ...)
@@ -2457,10 +2453,6 @@ void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable)
pe->tce_bypass_enabled = enable;
}
-static long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
- __u32 page_shift, __u64 window_size, __u32 levels,
- struct iommu_table *tbl);
-
static long pnv_pci_ioda2_create_table(struct iommu_table_group *table_group,
int num, __u32 page_shift, __u64 window_size, __u32 levels,
struct iommu_table **ptbl)
@@ -2768,144 +2760,6 @@ static void pnv_pci_ioda_setup_iommu_api(void)
static void pnv_pci_ioda_setup_iommu_api(void) { };
#endif
-static __be64 *pnv_pci_ioda2_table_do_alloc_pages(int nid, unsigned shift,
- unsigned levels, unsigned long limit,
- unsigned long *current_offset, unsigned long *total_allocated)
-{
- struct page *tce_mem = NULL;
- __be64 *addr, *tmp;
- unsigned order = max_t(unsigned, shift, PAGE_SHIFT) - PAGE_SHIFT;
- unsigned long allocated = 1UL << (order + PAGE_SHIFT);
- unsigned entries = 1UL << (shift - 3);
- long i;
-
- tce_mem = alloc_pages_node(nid, GFP_KERNEL, order);
- if (!tce_mem) {
- pr_err("Failed to allocate a TCE memory, order=%d\n", order);
- return NULL;
- }
- addr = page_address(tce_mem);
- memset(addr, 0, allocated);
- *total_allocated += allocated;
-
- --levels;
- if (!levels) {
- *current_offset += allocated;
- return addr;
- }
-
- for (i = 0; i < entries; ++i) {
- tmp = pnv_pci_ioda2_table_do_alloc_pages(nid, shift,
- levels, limit, current_offset, total_allocated);
- if (!tmp)
- break;
-
- addr[i] = cpu_to_be64(__pa(tmp) |
- TCE_PCI_READ | TCE_PCI_WRITE);
-
- if (*current_offset >= limit)
- break;
- }
-
- return addr;
-}
-
-static void pnv_pci_ioda2_table_do_free_pages(__be64 *addr,
- unsigned long size, unsigned level);
-
-static long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset,
- __u32 page_shift, __u64 window_size, __u32 levels,
- struct iommu_table *tbl)
-{
- void *addr;
- unsigned long offset = 0, level_shift, total_allocated = 0;
- const unsigned window_shift = ilog2(window_size);
- unsigned entries_shift = window_shift - page_shift;
- unsigned table_shift = max_t(unsigned, entries_shift + 3, PAGE_SHIFT);
- const unsigned long tce_table_size = 1UL << table_shift;
-
- if (!levels || (levels > POWERNV_IOMMU_MAX_LEVELS))
- return -EINVAL;
-
- if (!is_power_of_2(window_size))
- return -EINVAL;
-
- /* Adjust direct table size from window_size and levels */
- entries_shift = (entries_shift + levels - 1) / levels;
- level_shift = entries_shift + 3;
- level_shift = max_t(unsigned, level_shift, PAGE_SHIFT);
-
- if ((level_shift - 3) * levels + page_shift >= 55)
- return -EINVAL;
-
- /* Allocate TCE table */
- addr = pnv_pci_ioda2_table_do_alloc_pages(nid, level_shift,
- levels, tce_table_size, &offset, &total_allocated);
-
- /* addr==NULL means that the first level allocation failed */
- if (!addr)
- return -ENOMEM;
-
- /*
- * First level was allocated but some lower level failed as
- * we did not allocate as much as we wanted,
- * release partially allocated table.
- */
- if (offset < tce_table_size) {
- pnv_pci_ioda2_table_do_free_pages(addr,
- 1ULL << (level_shift - 3), levels - 1);
- return -ENOMEM;
- }
-
- /* Setup linux iommu table */
- pnv_pci_setup_iommu_table(tbl, addr, tce_table_size, bus_offset,
- page_shift);
- tbl->it_level_size = 1ULL << (level_shift - 3);
- tbl->it_indirect_levels = levels - 1;
- tbl->it_allocated_size = total_allocated;
-
- pr_devel("Created TCE table: ws=%08llx ts=%lx @%08llx\n",
- window_size, tce_table_size, bus_offset);
-
- return 0;
-}
-
-static void pnv_pci_ioda2_table_do_free_pages(__be64 *addr,
- unsigned long size, unsigned level)
-{
- const unsigned long addr_ul = (unsigned long) addr &
- ~(TCE_PCI_READ | TCE_PCI_WRITE);
-
- if (level) {
- long i;
- u64 *tmp = (u64 *) addr_ul;
-
- for (i = 0; i < size; ++i) {
- unsigned long hpa = be64_to_cpu(tmp[i]);
-
- if (!(hpa & (TCE_PCI_READ | TCE_PCI_WRITE)))
- continue;
-
- pnv_pci_ioda2_table_do_free_pages(__va(hpa), size,
- level - 1);
- }
- }
-
- free_pages(addr_ul, get_order(size << 3));
-}
-
-static void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl)
-{
- const unsigned long size = tbl->it_indirect_levels ?
- tbl->it_level_size : tbl->it_size;
-
- if (!tbl->it_size)
- return;
-
- pnv_pci_ioda2_table_do_free_pages((__be64 *)tbl->it_base, size,
- tbl->it_indirect_levels);
-}
-
static unsigned long pnv_ioda_parse_tce_sizes(struct pnv_phb *phb)
{
struct pci_controller *hose = phb->hose;
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index b265ecc..13aef23 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -802,85 +802,6 @@ struct pci_ops pnv_pci_ops = {
.write = pnv_pci_write_config,
};
-static __be64 *pnv_tce(struct iommu_table *tbl, long idx)
-{
- __be64 *tmp = ((__be64 *)tbl->it_base);
- int level = tbl->it_indirect_levels;
- const long shift = ilog2(tbl->it_level_size);
- unsigned long mask = (tbl->it_level_size - 1) << (level * shift);
-
- while (level) {
- int n = (idx & mask) >> (level * shift);
- unsigned long tce = be64_to_cpu(tmp[n]);
-
- tmp = __va(tce & ~(TCE_PCI_READ | TCE_PCI_WRITE));
- idx &= ~mask;
- mask >>= shift;
- --level;
- }
-
- return tmp + idx;
-}
-
-int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
- unsigned long uaddr, enum dma_data_direction direction,
- unsigned long attrs)
-{
- u64 proto_tce = iommu_direction_to_tce_perm(direction);
- u64 rpn = __pa(uaddr) >> tbl->it_page_shift;
- long i;
-
- if (proto_tce & TCE_PCI_WRITE)
- proto_tce |= TCE_PCI_READ;
-
- for (i = 0; i < npages; i++) {
- unsigned long newtce = proto_tce |
- ((rpn + i) << tbl->it_page_shift);
- unsigned long idx = index - tbl->it_offset + i;
-
- *(pnv_tce(tbl, idx)) = cpu_to_be64(newtce);
- }
-
- return 0;
-}
-
-#ifdef CONFIG_IOMMU_API
-int pnv_tce_xchg(struct iommu_table *tbl, long index,
- unsigned long *hpa, enum dma_data_direction *direction)
-{
- u64 proto_tce = iommu_direction_to_tce_perm(*direction);
- unsigned long newtce = *hpa | proto_tce, oldtce;
- unsigned long idx = index - tbl->it_offset;
-
- BUG_ON(*hpa & ~IOMMU_PAGE_MASK(tbl));
-
- if (newtce & TCE_PCI_WRITE)
- newtce |= TCE_PCI_READ;
-
- oldtce = be64_to_cpu(xchg(pnv_tce(tbl, idx), cpu_to_be64(newtce)));
- *hpa = oldtce & ~(TCE_PCI_READ | TCE_PCI_WRITE);
- *direction = iommu_tce_direction(oldtce);
-
- return 0;
-}
-#endif
-
-void pnv_tce_free(struct iommu_table *tbl, long index, long npages)
-{
- long i;
-
- for (i = 0; i < npages; i++) {
- unsigned long idx = index - tbl->it_offset + i;
-
- *(pnv_tce(tbl, idx)) = cpu_to_be64(0);
- }
-}
-
-unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
-{
- return be64_to_cpu(*(pnv_tce(tbl, index - tbl->it_offset)));
-}
-
struct iommu_table *pnv_pci_table_alloc(int nid)
{
struct iommu_table *tbl;
@@ -895,85 +816,6 @@ struct iommu_table *pnv_pci_table_alloc(int nid)
return tbl;
}
-long pnv_pci_link_table_and_group(int node, int num,
- struct iommu_table *tbl,
- struct iommu_table_group *table_group)
-{
- struct iommu_table_group_link *tgl = NULL;
-
- if (WARN_ON(!tbl || !table_group))
- return -EINVAL;
-
- tgl = kzalloc_node(sizeof(struct iommu_table_group_link), GFP_KERNEL,
- node);
- if (!tgl)
- return -ENOMEM;
-
- tgl->table_group = table_group;
- list_add_rcu(&tgl->next, &tbl->it_group_list);
-
- table_group->tables[num] = tbl;
-
- return 0;
-}
-
-static void pnv_iommu_table_group_link_free(struct rcu_head *head)
-{
- struct iommu_table_group_link *tgl = container_of(head,
- struct iommu_table_group_link, rcu);
-
- kfree(tgl);
-}
-
-void pnv_pci_unlink_table_and_group(struct iommu_table *tbl,
- struct iommu_table_group *table_group)
-{
- long i;
- bool found;
- struct iommu_table_group_link *tgl;
-
- if (!tbl || !table_group)
- return;
-
- /* Remove link to a group from table's list of attached groups */
- found = false;
- list_for_each_entry_rcu(tgl, &tbl->it_group_list, next) {
- if (tgl->table_group == table_group) {
- list_del_rcu(&tgl->next);
- call_rcu(&tgl->rcu, pnv_iommu_table_group_link_free);
- found = true;
- break;
- }
- }
- if (WARN_ON(!found))
- return;
-
- /* Clean a pointer to iommu_table in iommu_table_group::tables[] */
- found = false;
- for (i = 0; i < IOMMU_TABLE_GROUP_MAX_TABLES; ++i) {
- if (table_group->tables[i] == tbl) {
- table_group->tables[i] = NULL;
- found = true;
- break;
- }
- }
- WARN_ON(!found);
-}
-
-void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
- void *tce_mem, u64 tce_size,
- u64 dma_offset, unsigned page_shift)
-{
- tbl->it_blocksize = 16;
- tbl->it_base = (unsigned long)tce_mem;
- tbl->it_page_shift = page_shift;
- tbl->it_offset = dma_offset >> tbl->it_page_shift;
- tbl->it_index = 0;
- tbl->it_size = tce_size >> 3;
- tbl->it_busno = 0;
- tbl->it_type = TCE_PCI;
-}
-
void pnv_pci_dma_dev_setup(struct pci_dev *pdev)
{
struct pci_controller *hose = pci_bus_to_host(pdev->bus);
--
2.11.0
^ permalink raw reply related
* [PATCH kernel v3 3/6] KVM: PPC: Make iommu_table::it_userspace big endian
From: Alexey Kardashevskiy @ 2018-07-04 6:13 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, David Gibson, kvm-ppc, kvm, Alex Williamson,
Benjamin Herrenschmidt, Michael Ellerman, Russell Currey
In-Reply-To: <20180704061349.20742-1-aik@ozlabs.ru>
We are going to reuse multilevel TCE code for the userspace copy of
the TCE table and since it is big endian, let's make the copy big endian
too.
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/include/asm/iommu.h | 2 +-
arch/powerpc/kvm/book3s_64_vio.c | 11 ++++++-----
arch/powerpc/kvm/book3s_64_vio_hv.c | 10 +++++-----
drivers/vfio/vfio_iommu_spapr_tce.c | 19 +++++++++----------
4 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 20febe0..803ac70 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -117,7 +117,7 @@ struct iommu_table {
unsigned long *it_map; /* A simple allocation bitmap for now */
unsigned long it_page_shift;/* table iommu page size */
struct list_head it_group_list;/* List of iommu_table_group_link */
- unsigned long *it_userspace; /* userspace view of the table */
+ __be64 *it_userspace; /* userspace view of the table */
struct iommu_table_ops *it_ops;
struct kref it_kref;
};
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index 8167ce8..6f34edd 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -377,19 +377,19 @@ static long kvmppc_tce_iommu_mapped_dec(struct kvm *kvm,
{
struct mm_iommu_table_group_mem_t *mem = NULL;
const unsigned long pgsize = 1ULL << tbl->it_page_shift;
- unsigned long *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
if (!pua)
/* it_userspace allocation might be delayed */
return H_TOO_HARD;
- mem = mm_iommu_lookup(kvm->mm, *pua, pgsize);
+ mem = mm_iommu_lookup(kvm->mm, be64_to_cpu(*pua), pgsize);
if (!mem)
return H_TOO_HARD;
mm_iommu_mapped_dec(mem);
- *pua = 0;
+ *pua = cpu_to_be64(0);
return H_SUCCESS;
}
@@ -436,7 +436,8 @@ long kvmppc_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
enum dma_data_direction dir)
{
long ret;
- unsigned long hpa, *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ unsigned long hpa;
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
struct mm_iommu_table_group_mem_t *mem;
if (!pua)
@@ -463,7 +464,7 @@ long kvmppc_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
if (dir != DMA_NONE)
kvmppc_tce_iommu_mapped_dec(kvm, tbl, entry);
- *pua = ua;
+ *pua = cpu_to_be64(ua);
return 0;
}
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 5b298f5..841aef7 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -200,7 +200,7 @@ static long kvmppc_rm_tce_iommu_mapped_dec(struct kvm *kvm,
{
struct mm_iommu_table_group_mem_t *mem = NULL;
const unsigned long pgsize = 1ULL << tbl->it_page_shift;
- unsigned long *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
if (!pua)
/* it_userspace allocation might be delayed */
@@ -210,13 +210,13 @@ static long kvmppc_rm_tce_iommu_mapped_dec(struct kvm *kvm,
if (WARN_ON_ONCE_RM(!pua))
return H_HARDWARE;
- mem = mm_iommu_lookup_rm(kvm->mm, *pua, pgsize);
+ mem = mm_iommu_lookup_rm(kvm->mm, be64_to_cpu(*pua), pgsize);
if (!mem)
return H_TOO_HARD;
mm_iommu_mapped_dec(mem);
- *pua = 0;
+ *pua = cpu_to_be64(0);
return H_SUCCESS;
}
@@ -268,7 +268,7 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
{
long ret;
unsigned long hpa = 0;
- unsigned long *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
struct mm_iommu_table_group_mem_t *mem;
if (!pua)
@@ -303,7 +303,7 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl,
if (dir != DMA_NONE)
kvmppc_rm_tce_iommu_mapped_dec(kvm, tbl, entry);
- *pua = ua;
+ *pua = cpu_to_be64(ua);
return 0;
}
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 7cd63b0..17a418c 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -230,7 +230,7 @@ static long tce_iommu_userspace_view_alloc(struct iommu_table *tbl,
decrement_locked_vm(mm, cb >> PAGE_SHIFT);
return -ENOMEM;
}
- tbl->it_userspace = uas;
+ tbl->it_userspace = (__be64 *) uas;
return 0;
}
@@ -482,20 +482,20 @@ static void tce_iommu_unuse_page_v2(struct tce_container *container,
struct mm_iommu_table_group_mem_t *mem = NULL;
int ret;
unsigned long hpa = 0;
- unsigned long *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry);
if (!pua)
return;
- ret = tce_iommu_prereg_ua_to_hpa(container, *pua, tbl->it_page_shift,
- &hpa, &mem);
+ ret = tce_iommu_prereg_ua_to_hpa(container, be64_to_cpu(*pua),
+ tbl->it_page_shift, &hpa, &mem);
if (ret)
- pr_debug("%s: tce %lx at #%lx was not cached, ret=%d\n",
- __func__, *pua, entry, ret);
+ pr_debug("%s: tce %llx at #%lx was not cached, ret=%d\n",
+ __func__, be64_to_cpu(*pua), entry, ret);
if (mem)
mm_iommu_mapped_dec(mem);
- *pua = 0;
+ *pua = cpu_to_be64(0);
}
static int tce_iommu_clear(struct tce_container *container,
@@ -607,8 +607,7 @@ static long tce_iommu_build_v2(struct tce_container *container,
for (i = 0; i < pages; ++i) {
struct mm_iommu_table_group_mem_t *mem = NULL;
- unsigned long *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl,
- entry + i);
+ __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY(tbl, entry + i);
ret = tce_iommu_prereg_ua_to_hpa(container,
tce, tbl->it_page_shift, &hpa, &mem);
@@ -642,7 +641,7 @@ static long tce_iommu_build_v2(struct tce_container *container,
if (dirtmp != DMA_NONE)
tce_iommu_unuse_page_v2(container, tbl, entry + i);
- *pua = tce;
+ *pua = cpu_to_be64(tce);
tce += IOMMU_PAGE_SIZE(tbl);
}
--
2.11.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox