* Re: RFC on writel and writel_relaxed
From: Jason Gunthorpe @ 2018-03-28 16:57 UTC (permalink / raw)
To: Will Deacon
Cc: Benjamin Herrenschmidt, Arnd Bergmann, Sinan Kaya, David Laight,
Oliver, open list:LINUX FOR POWERPC (32-BIT AND 64-BIT),
linux-rdma@vger.kernel.org, Paul E. McKenney, Peter Zijlstra,
Ingo Molnar, Jonathan Corbet
In-Reply-To: <20180328101345.GA30850@arm.com>
On Wed, Mar 28, 2018 at 11:13:45AM +0100, Will Deacon wrote:
> On Wed, Mar 28, 2018 at 09:01:27PM +1100, Benjamin Herrenschmidt wrote:
> > On Wed, 2018-03-28 at 11:55 +0200, Arnd Bergmann wrote:
> > > > powerpc and ARM can't quite make them synchronous I think, but at least
> > > > they should have the same semantics as writel.
> > >
> > > One thing that ARM does IIRC is that it only guarantees to order writel() within
> > > one device, and the memory mapped PCI I/O space window almost certainly
> > > counts as a separate device to the CPU.
> >
> > That sounds bogus.
>
> To elaborate, if you do the following on arm:
>
> writel(DEVICE_FOO);
> writel(DEVICE_BAR);
>
> we generally cannot guarantee in which order those accesses will hit the
> devices even if we add every barrier under the sun. You'd need something
> in between, specific to DEVICE_FOO (probably a read-back) to really push
> the first write out. This doesn't sound like it would be that uncommon to
> me.
The PCI posted write does not require the above to execute 'in order'
only that any bus segment shared by the two devices have the writes
issued in CPU order. ie at a shared PCI root port for instance.
If I recall this is very similar to the ordering that ARM's on-chip
AXI interconnect is supposed to provide.. So I'd be very surprised if
a modern ARM64 has an meaningful difference from x86 here.
When talking about ordering between the devices, the relevant question
is what happens if the writel(DEVICE_BAR) triggers DEVICE_BAR to DMA
from the DEVICE_FOO. 'ordered' means that in this case
writel(DEVICE_FOO) must be presented to FOO before anything generated
by BAR.
Jason
^ permalink raw reply
* Re: [PATCH v9 08/24] mm: Protect VMA modifications using VMA sequence count
From: Laurent Dufour @ 2018-03-28 16:57 UTC (permalink / raw)
To: David Rientjes
Cc: paulmck, peterz, akpm, kirill, ak, mhocko, dave, jack,
Matthew Wilcox, benh, mpe, paulus, Thomas Gleixner, Ingo Molnar,
hpa, Will Deacon, Sergey Senozhatsky, Andrea Arcangeli,
Alexei Starovoitov, kemi.wang, sergey.senozhatsky.work,
Daniel Jordan, linux-kernel, linux-mm, haren, khandual, npiggin,
bsingharora, Tim Chen, linuxppc-dev, x86
In-Reply-To: <alpine.DEB.2.20.1803271441290.38095@chino.kir.corp.google.com>
On 27/03/2018 23:45, David Rientjes wrote:
> On Tue, 13 Mar 2018, Laurent Dufour wrote:
>
>> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
>> index 65ae54659833..a2d9c87b7b0b 100644
>> --- a/fs/proc/task_mmu.c
>> +++ b/fs/proc/task_mmu.c
>> @@ -1136,8 +1136,11 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf,
>> goto out_mm;
>> }
>> for (vma = mm->mmap; vma; vma = vma->vm_next) {
>> - vma->vm_flags &= ~VM_SOFTDIRTY;
>> + vm_write_begin(vma);
>> + WRITE_ONCE(vma->vm_flags,
>> + vma->vm_flags & ~VM_SOFTDIRTY);
>> vma_set_page_prot(vma);
>> + vm_write_end(vma);
>> }
>> downgrade_write(&mm->mmap_sem);
>> break;
>> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
>> index cec550c8468f..b8212ba17695 100644
>> --- a/fs/userfaultfd.c
>> +++ b/fs/userfaultfd.c
>> @@ -659,8 +659,11 @@ int dup_userfaultfd(struct vm_area_struct *vma, struct list_head *fcs)
>>
>> octx = vma->vm_userfaultfd_ctx.ctx;
>> if (!octx || !(octx->features & UFFD_FEATURE_EVENT_FORK)) {
>> + vm_write_begin(vma);
>> vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
>> - vma->vm_flags &= ~(VM_UFFD_WP | VM_UFFD_MISSING);
>> + WRITE_ONCE(vma->vm_flags,
>> + vma->vm_flags & ~(VM_UFFD_WP | VM_UFFD_MISSING));
>> + vm_write_end(vma);
>> return 0;
>> }
>>
>
> In several locations in this patch vm_write_begin(vma) ->
> vm_write_end(vma) is nesting things other than vma->vm_flags,
> vma->vm_policy, etc. I think it's better to do vm_write_end(vma) as soon
> as the members that the seqcount protects are modified. In other words,
> this isn't offering protection for vma->vm_userfaultfd_ctx. There are
> several examples of this in the patch.
That's true in this particular case, and I could change that to not include the
change to vm_userfaultfd_ctx.
This being said, I don't think this will have a major impact, but I'll make a
close review on this patch to be sure there is too large protected part of code.
>> @@ -885,8 +888,10 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
>> vma = prev;
>> else
>> prev = vma;
>> - vma->vm_flags = new_flags;
>> + vm_write_begin(vma);
>> + WRITE_ONCE(vma->vm_flags, new_flags);
>> vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
>> + vm_write_end(vma);
>> }
>> up_write(&mm->mmap_sem);
>> mmput(mm);
>> @@ -1434,8 +1439,10 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
>> * the next vma was merged into the current one and
>> * the current one has not been updated yet.
>> */
>> - vma->vm_flags = new_flags;
>> + vm_write_begin(vma);
>> + WRITE_ONCE(vma->vm_flags, new_flags);
>> vma->vm_userfaultfd_ctx.ctx = ctx;
>> + vm_write_end(vma);
>>
>> skip:
>> prev = vma;
>> @@ -1592,8 +1599,10 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
>> * the next vma was merged into the current one and
>> * the current one has not been updated yet.
>> */
>> - vma->vm_flags = new_flags;
>> + vm_write_begin(vma);
>> + WRITE_ONCE(vma->vm_flags, new_flags);
>> vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
>> + vm_write_end(vma);
>>
>> skip:
>> prev = vma;
>> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
>> index b7e2268dfc9a..32314e9e48dd 100644
>> --- a/mm/khugepaged.c
>> +++ b/mm/khugepaged.c
>> @@ -1006,6 +1006,7 @@ static void collapse_huge_page(struct mm_struct *mm,
>> if (mm_find_pmd(mm, address) != pmd)
>> goto out;
>>
>> + vm_write_begin(vma);
>> anon_vma_lock_write(vma->anon_vma);
>>
>> pte = pte_offset_map(pmd, address);
>> @@ -1041,6 +1042,7 @@ static void collapse_huge_page(struct mm_struct *mm,
>> pmd_populate(mm, pmd, pmd_pgtable(_pmd));
>> spin_unlock(pmd_ptl);
>> anon_vma_unlock_write(vma->anon_vma);
>> + vm_write_end(vma);
>> result = SCAN_FAIL;
>> goto out;
>> }
>> @@ -1075,6 +1077,7 @@ static void collapse_huge_page(struct mm_struct *mm,
>> set_pmd_at(mm, address, pmd, _pmd);
>> update_mmu_cache_pmd(vma, address, pmd);
>> spin_unlock(pmd_ptl);
>> + vm_write_end(vma);
>>
>> *hpage = NULL;
>>
>> diff --git a/mm/madvise.c b/mm/madvise.c
>> index 4d3c922ea1a1..e328f7ab5942 100644
>> --- a/mm/madvise.c
>> +++ b/mm/madvise.c
>> @@ -184,7 +184,9 @@ static long madvise_behavior(struct vm_area_struct *vma,
>> /*
>> * vm_flags is protected by the mmap_sem held in write mode.
>> */
>> - vma->vm_flags = new_flags;
>> + vm_write_begin(vma);
>> + WRITE_ONCE(vma->vm_flags, new_flags);
>> + vm_write_end(vma);
>> out:
>> return error;
>> }
>> @@ -450,9 +452,11 @@ static void madvise_free_page_range(struct mmu_gather *tlb,
>> .private = tlb,
>> };
>>
>> + vm_write_begin(vma);
>> tlb_start_vma(tlb, vma);
>> walk_page_range(addr, end, &free_walk);
>> tlb_end_vma(tlb, vma);
>> + vm_write_end(vma);
>> }
>>
>> static int madvise_free_single_vma(struct vm_area_struct *vma,
>> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
>> index e0e706f0b34e..2632c6f93b63 100644
>> --- a/mm/mempolicy.c
>> +++ b/mm/mempolicy.c
>> @@ -380,8 +380,11 @@ void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new)
>> struct vm_area_struct *vma;
>>
>> down_write(&mm->mmap_sem);
>> - for (vma = mm->mmap; vma; vma = vma->vm_next)
>> + for (vma = mm->mmap; vma; vma = vma->vm_next) {
>> + vm_write_begin(vma);
>> mpol_rebind_policy(vma->vm_policy, new);
>> + vm_write_end(vma);
>> + }
>> up_write(&mm->mmap_sem);
>> }
>>
>> @@ -554,9 +557,11 @@ unsigned long change_prot_numa(struct vm_area_struct *vma,
>> {
>> int nr_updated;
>>
>> + vm_write_begin(vma);
>> nr_updated = change_protection(vma, addr, end, PAGE_NONE, 0, 1);
>> if (nr_updated)
>> count_vm_numa_events(NUMA_PTE_UPDATES, nr_updated);
>> + vm_write_end(vma);
>>
>> return nr_updated;
>> }
>> @@ -657,6 +662,7 @@ static int vma_replace_policy(struct vm_area_struct *vma,
>> if (IS_ERR(new))
>> return PTR_ERR(new);
>>
>> + vm_write_begin(vma);
>> if (vma->vm_ops && vma->vm_ops->set_policy) {
>> err = vma->vm_ops->set_policy(vma, new);
>> if (err)
>> @@ -664,11 +670,17 @@ static int vma_replace_policy(struct vm_area_struct *vma,
>> }
>>
>> old = vma->vm_policy;
>> - vma->vm_policy = new; /* protected by mmap_sem */
>> + /*
>> + * The speculative page fault handler access this field without
>> + * hodling the mmap_sem.
>> + */
>
> "The speculative page fault handler accesses this field without holding
> vma->vm_mm->mmap_sem"
Oops :/
>
>> + WRITE_ONCE(vma->vm_policy, new);
>> + vm_write_end(vma);
>> mpol_put(old);
>>
>> return 0;
>> err_out:
>> + vm_write_end(vma);
>> mpol_put(new);
>> return err;
>> }
>
> Wait, doesn't vma_dup_policy() also need to protect dst->vm_policy?
Indeed this is not necessary because vma_dup_policy() is called when dst is not
yet linked in the RB tree, so it can't be found by the speculative page fault
handler. This is not the case of vma_replace_policy, and this is why the
protection are needed here.
>
> diff --git a/mm/mempolicy.c b/mm/mempolicy.c
> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -2121,7 +2121,9 @@ int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst)
>
> if (IS_ERR(pol))
> return PTR_ERR(pol);
> - dst->vm_policy = pol;
> + vm_write_begin(dst);
> + WRITE_ONCE(dst->vm_policy, pol);
> + vm_write_end(dst);
> return 0;
> }
^ permalink raw reply
* Re: [PATCH] Extract initrd free logic from arch-specific code.
From: Kees Cook @ 2018-03-28 16:55 UTC (permalink / raw)
To: Shea Levy
Cc: linux-riscv, LKML, Christoph Hellwig, Richard Henderson,
Ivan Kokshaysky, Matt Turner, Vineet Gupta, Russell King,
Catalin Marinas, Will Deacon, Mark Salter, Aurelien Jacquiot,
Mikael Starvik, Jesper Nilsson, Yoshinori Sato, Richard Kuo,
Tony Luck, Fenghua Yu, Geert Uytterhoeven, James Hogan,
Michal Simek, Ralf Baechle, David Howells, Ley Foon Tan,
Jonas Bonn, Stefan Kristiansson, Stafford Horne,
James E.J. Bottomley, Helge Deller, Benjamin Herrenschmidt,
Paul Mackerras, Michael Ellerman, Palmer Dabbelt, Albert Ou,
Martin Schwidefsky, Heiko Carstens, Chen Liqin, Lennox Wu,
Rich Felker, David S. Miller, Jeff Dike, Richard Weinberger,
Guan Xuetao, Thomas Gleixner, Ingo Molnar, H. Peter Anvin, X86 ML,
Chris Zankel, Max Filippov, Kate Stewart, Greg Kroah-Hartman,
Philippe Ombredanne, Eugeniy Paltsev, Al Viro, Vladimir Murzin,
Linus Walleij, Michal Hocko, Andrew Morton, Sudip Mukherjee,
Marc Zyngier, Rob Herring, Vlastimil Babka, Balbir Singh,
Christophe Leroy, Joe Perches, Oliver O'Halloran,
Dan Williams, Wei Yang, Christian König, Arnd Bergmann,
Deepa Dinamani, Daniel Thompson, Rob Landley, Florian Fainelli,
linux-alpha, linux-snps-arc, linux-arm-kernel,
adi-buildroot-devel, linux-c6x-dev, linux-cris-kernel,
uclinux-h8-devel, linux-hexagon, linux-ia64, linux-m68k,
linux-metag, Linux MIPS Mailing List, linux-am33-list, nios2-dev,
openrisc, linux-parisc, PowerPC, linux-s390, linux-sh, sparclinux,
user-mode-linux-devel, user-mode-linux-user, linux-xtensa
In-Reply-To: <20180328152714.6103-1-shea@shealevy.com>
On Wed, Mar 28, 2018 at 8:26 AM, Shea Levy <shea@shealevy.com> wrote:
> Now only those architectures that have custom initrd free requirements
> need to define free_initrd_mem.
>
> Signed-off-by: Shea Levy <shea@shealevy.com>
Yay consolidation! :)
> --- a/usr/Kconfig
> +++ b/usr/Kconfig
> @@ -233,3 +233,7 @@ config INITRAMFS_COMPRESSION
> default ".lzma" if RD_LZMA
> default ".bz2" if RD_BZIP2
> default ""
> +
> +config HAVE_ARCH_FREE_INITRD_MEM
> + bool
> + default n
If you keep the Kconfig, you can leave off "default n", and I'd
suggest adding a help section just to describe what the per-arch
responsibilities are when select-ing the config. (See
HAVE_ARCH_SECCOMP_FILTER for an example.)
-Kees
--
Kees Cook
Pixel Security
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Nicholas Piggin @ 2018-03-28 16:23 UTC (permalink / raw)
To: David Miller
Cc: benh, paulmck, arnd, linux-rdma, linuxppc-dev, linus971,
will.deacon, alexander.duyck, okaya, jgg, David.Laight, oohall,
netdev, alexander.h.duyck, torvalds
In-Reply-To: <20180328.115509.481837809903086401.davem@davemloft.net>
On Wed, 28 Mar 2018 11:55:09 -0400 (EDT)
David Miller <davem@davemloft.net> wrote:
> From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Date: Thu, 29 Mar 2018 02:13:16 +1100
>
> > Let's fix all archs, it's way easier than fixing all drivers. Half of
> > the archs are unused or dead anyway.
>
> Agreed.
While we're making decrees here, can we do something about mmiowb?
The semantics are basically indecipherable.
This is a variation on the mandatory write barrier that causes writes to weakly
ordered I/O regions to be partially ordered. Its effects may go beyond the
CPU->Hardware interface and actually affect the hardware at some level.
How can a driver writer possibly get that right?
IIRC it was added for some big ia64 system that was really expensive
to implement the proper wmb() semantics on. So wmb() semantics were
quietly downgraded, then the subsequently broken drivers they cared
about were fixed by adding the stronger mmiowb().
What should have happened was wmb and writel remained correct, sane, and
expensive, and they add an mmio_wmb() to order MMIO stores made by the
writel_relaxed accessors, then use that to speed up the few drivers they
care about.
Now that ia64 doesn't matter too much, can we deprecate mmiowb and just
make wmb ordering talk about stores to the device, not to some
intermediate stage of the interconnect where it can be subsequently
reordered wrt the device? Drivers can be converted back to using wmb
or writel gradually.
Thanks,
Nick
^ permalink raw reply
* RE: RFC on writel and writel_relaxed
From: David Laight @ 2018-03-28 16:16 UTC (permalink / raw)
To: 'Benjamin Herrenschmidt', Will Deacon
Cc: Linus Torvalds, Alexander Duyck, Sinan Kaya, Arnd Bergmann,
Jason Gunthorpe, Oliver,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT),
linux-rdma@vger.kernel.org, Paul E. McKenney,
netdev@vger.kernel.org
In-Reply-To: <1522249950.21446.23.camel@kernel.crashing.org>
RnJvbTogQmVuamFtaW4gSGVycmVuc2NobWlkdA0KPiBTZW50OiAyOCBNYXJjaCAyMDE4IDE2OjEz
DQouLi4NCj4gPiBJJ3ZlIGFsd2F5cyB3b25kZXJlZCBleGFjdGx5IHdoYXQgdGhlIHR3aTtpc3lu
YyB3ZXJlIGZvciAtIGFsd2F5cyBzZWVtZWQNCj4gPiB2ZXJ5IGhlYXZ5IGhhbmRlZCBmb3IgbW9z
dCBtbWlvIHJlYWRzLg0KPiA+IFBhcnRpY3VsYXJseSBpZiB5b3UgYXJlIGRvaW5nIG1taW8gcmVh
ZHMgZnJvbSBhIGRhdGEgZmlmby4NCj4gDQo+IElmIHlvdSBkbyB0aGF0IHlvdSBzaG91bGQgdXNl
IHRoZSAicyIgdmVyc2lvbiBvZiB0aGUgYWNjZXNzb3JzLiBUaG9zZQ0KPiB3aWxsIG9ubHkgZG8g
dGhlIGFib3ZlIHRyaWNrIGF0IHRoZSBlbmQgb2YgdGhlIGFjY2VzcyBzZXJpZXMuIEFsc28gYQ0K
PiBGSUZPIG5lZWRzIHNwZWNpYWwgY2FyZSBhYm91dCBlbmRpYW5uZXNzIGFueXdheSwgc28geW91
IHNob3VsZCB1c2UNCj4gdGhvc2UgYWNjZXNzb3JzIHJlZ2FyZGxlc3MuIChIaW50OiB5b3UgbmV2
ZXIgZW5kaWFuIHN3YXAgYSBGSUZPIGV2ZW4gb24NCj4gQkUgb24gYSBMRSBkZXZpY2UsIHVubGVz
cyBzb21ldGhpbmcncyBiZWVuIHdpcmVkIHZlcnkgYmFkbHkgaW4gSFcpLg0KDQpUaGF0IHdhcyBh
Y3R1YWxseSBhIDY0IGJpdCB3aWRlIGZpZm8gY29ubmVjdGVkIHRvIGEgMTZiaXQgd2lkZSBQSU8g
aW50ZXJmYWNlLg0KUmVhZGluZyB0aGUgaGlnaCBhZGRyZXNzICdjbG9ja2VkJyB0aGUgZmlmby4N
ClNvIHRoZSBmaXJzdCAzIHJlYWRzIGNvdWxkIGhhcHBlbiBpbiBhbnkgb3JkZXIsIGJ1dCB0aGUg
NHRoIGhhZCB0byBiZSBsYXN0Lg0KVGhpcyBpcyBhIHNtYWxsIHBwYyBhbmQgd2Ugc2hvdmVsIGEg
bG90IG9mIGRhdGEgdGhyb3VnaCB0aGF0IGZpZm8uDQoNCldoZXRoZXIgaXQgbmVlZGVkIGJ5dGVz
d2FwcGluZyBkZXBlbmRlZCBjb21wbGV0ZWx5IG9uIGhvdyBvdXIgaGFyZHdhcmUgcGVvcGxlDQpo
YWQgYnVpbHQgdGhlIHBjYiAobm90IG1hZGUgZWFzeSBieSBzb21lIGRvY3MgdXNpbmcgdGhlIGli
bSBiaXQgbnVtYmVyaW5nKS4NCkluIGZhY3QgaXQgZGlkbid0Li4uLg0KDQpXaGlsZSB0aGF0IGRy
aXZlciBvbmx5IGhhZCB0byBydW4gb24gYSB2ZXJ5IHNwZWNpZmljIHNtYWxsIHBwYywgZ2VuZXJp
YyBkcml2ZXJzDQptaWdodCBoYXZlIHNpbWlsYXIgaXNzdWVzLg0KDQpJIHN1c3BlY3QgdGhhdCB3
cml0ZWwoKSBpcyBhbHdheXMgKG9yIHNob3VsZCBhbHdheXMgYmUpOg0KCWJhcnJpZXJfYmVmb3Jl
X3dyaXRlbCgpDQoJd3JpdGVsX3JlbGF4ZWQoKQ0KCWJhcnJpZXJfYWZ0ZXJfd3JpdGVsKCkNClNv
IGlmIGEgZHJpdmVyIG5lZWRzIHRvIGRvIG11bHRpcGxlIHdyaXRlcyAod2l0aG91dCBzdHJvbmcg
b3JkZXJpbmcpDQppdCBzaG91bGQgYmUgYWJsZSB0byByZXBlYXQgdGhlIHdyaXRlbF9yZWxheGVk
KCkgd2l0aCBvbmx5IG9uZSBzZXQNCm9mIGJhcnJpZXJzLg0KU2ltaWxhcmx5IGZvciByZWFkbCgp
Lg0KSW4gYWRkaXRpb24gYSBsZXNzZXIgYmFycmllciBpcyBwcm9iYWJseSBlbm91Z2ggYmV0d2Vl
biBhIHJlYWRsX3JlbGF4ZWQoKQ0KYW5kIGEgd3JpdGVsX3JlbGF4ZWQoKSB0aGF0IGlzIGNvbmRp
dGlvbmFsIG9uIHRoZSByZWFkIHZhbHVlLg0KDQoJRGF2aWQNCg0K
^ permalink raw reply
* Re: [PATCH] Extract initrd free logic from arch-specific code.
From: Shea Levy @ 2018-03-28 16:04 UTC (permalink / raw)
To: Rob Landley, linux-riscv, linux-kernel
Cc: Christoph Hellwig, Richard Henderson, Ivan Kokshaysky,
Matt Turner, Vineet Gupta, Russell King, Catalin Marinas,
Will Deacon, Mark Salter, Aurelien Jacquiot, Mikael Starvik,
Jesper Nilsson, Yoshinori Sato, Richard Kuo, Tony Luck,
Fenghua Yu, Geert Uytterhoeven, James Hogan, Michal Simek,
Ralf Baechle, David Howells, Ley Foon Tan, Jonas Bonn,
Stefan Kristiansson, Stafford Horne, James E.J. Bottomley,
Helge Deller, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Palmer Dabbelt, Albert Ou, Martin Schwidefsky,
Heiko Carstens, Chen Liqin, Lennox Wu, Rich Felker,
David S. Miller, Jeff Dike, Richard Weinberger, Guan Xuetao,
Thomas Gleixner, Ingo Molnar, H. Peter Anvin, x86, Chris Zankel,
Max Filippov, Kate Stewart, Greg Kroah-Hartman,
Philippe Ombredanne, Eugeniy Paltsev, Al Viro, Vladimir Murzin,
Linus Walleij, Michal Hocko, Andrew Morton, Sudip Mukherjee,
Marc Zyngier, Rob Herring, Kees Cook, Vlastimil Babka,
Balbir Singh, Christophe Leroy, Joe Perches,
Oliver O'Halloran, Dan Williams, Wei Yang,
Christian König, Arnd Bergmann, Deepa Dinamani,
Daniel Thompson, Florian Fainelli, linux-alpha, linux-snps-arc,
linux-arm-kernel, adi-buildroot-devel, linux-c6x-dev,
linux-cris-kernel, uclinux-h8-devel, linux-hexagon, linux-ia64,
linux-m68k, linux-metag, linux-mips, linux-am33-list, nios2-dev,
openrisc, linux-parisc, linuxppc-dev, linux-s390, linux-sh,
sparclinux, user-mode-linux-devel, user-mode-linux-user,
linux-xtensa
In-Reply-To: <05620fee-e8b5-0668-77b8-da073dc78c40@landley.net>
[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]
Hi Rob,
Rob Landley <rob@landley.net> writes:
> On 03/28/2018 10:26 AM, Shea Levy wrote:
>> Now only those architectures that have custom initrd free requirements
>> need to define free_initrd_mem.
> ...
>> --- a/arch/arc/mm/init.c
>> +++ b/arch/arc/mm/init.c
>> @@ -229,10 +229,3 @@ void __ref free_initmem(void)
>> {
>> free_initmem_default(-1);
>> }
>> -
>> -#ifdef CONFIG_BLK_DEV_INITRD
>> -void __init free_initrd_mem(unsigned long start, unsigned long end)
>> -{
>> - free_reserved_area((void *)start, (void *)end, -1, "initrd");
>> -}
>> -#endif
>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
>> index 3f972e83909b..19d1c5594e2d 100644
>> --- a/arch/arm/Kconfig
>> +++ b/arch/arm/Kconfig
>> @@ -47,6 +47,7 @@ config ARM
>> select HARDIRQS_SW_RESEND
>> select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
>> select HAVE_ARCH_BITREVERSE if (CPU_32v7M || CPU_32v7) && !CPU_32v6
>> + select HAVE_ARCH_FREE_INITRD_MEM
>> select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
>> select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
>> select HAVE_ARCH_MMAP_RND_BITS if MMU
>
> Isn't this why weak symbols were invented?
>
This approach was suggested by Christoph Hellwig upthread, and seems to
have some precedent elsewhere (e.g. strncasecmp), but I agree weak
symbols seem appropriate here. I'm happy to implement either approach!
>
> Confused,
>
> Rob
Thanks,
Shea
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: [PATCH] Extract initrd free logic from arch-specific code.
From: Rob Landley @ 2018-03-28 15:58 UTC (permalink / raw)
To: Shea Levy, linux-riscv, linux-kernel
Cc: Christoph Hellwig, Richard Henderson, Ivan Kokshaysky,
Matt Turner, Vineet Gupta, Russell King, Catalin Marinas,
Will Deacon, Mark Salter, Aurelien Jacquiot, Mikael Starvik,
Jesper Nilsson, Yoshinori Sato, Richard Kuo, Tony Luck,
Fenghua Yu, Geert Uytterhoeven, James Hogan, Michal Simek,
Ralf Baechle, David Howells, Ley Foon Tan, Jonas Bonn,
Stefan Kristiansson, Stafford Horne, James E.J. Bottomley,
Helge Deller, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Palmer Dabbelt, Albert Ou, Martin Schwidefsky,
Heiko Carstens, Chen Liqin, Lennox Wu, Rich Felker,
David S. Miller, Jeff Dike, Richard Weinberger, Guan Xuetao,
Thomas Gleixner, Ingo Molnar, H. Peter Anvin, x86, Chris Zankel,
Max Filippov, Kate Stewart, Greg Kroah-Hartman,
Philippe Ombredanne, Eugeniy Paltsev, Al Viro, Vladimir Murzin,
Linus Walleij, Michal Hocko, Andrew Morton, Sudip Mukherjee,
Marc Zyngier, Rob Herring, Kees Cook, Vlastimil Babka,
Balbir Singh, Christophe Leroy, Joe Perches,
Oliver O'Halloran, Dan Williams, Wei Yang,
Christian König, Arnd Bergmann, Deepa Dinamani,
Daniel Thompson, Florian Fainelli, linux-alpha, linux-snps-arc,
linux-arm-kernel, adi-buildroot-devel, linux-c6x-dev,
linux-cris-kernel, uclinux-h8-devel, linux-hexagon, linux-ia64,
linux-m68k, linux-metag, linux-mips, linux-am33-list, nios2-dev,
openrisc, linux-parisc, linuxppc-dev, linux-s390, linux-sh,
sparclinux, user-mode-linux-devel, user-mode-linux-user,
linux-xtensa
In-Reply-To: <20180328152714.6103-1-shea@shealevy.com>
On 03/28/2018 10:26 AM, Shea Levy wrote:
> Now only those architectures that have custom initrd free requirements
> need to define free_initrd_mem.
...
> --- a/arch/arc/mm/init.c
> +++ b/arch/arc/mm/init.c
> @@ -229,10 +229,3 @@ void __ref free_initmem(void)
> {
> free_initmem_default(-1);
> }
> -
> -#ifdef CONFIG_BLK_DEV_INITRD
> -void __init free_initrd_mem(unsigned long start, unsigned long end)
> -{
> - free_reserved_area((void *)start, (void *)end, -1, "initrd");
> -}
> -#endif
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 3f972e83909b..19d1c5594e2d 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -47,6 +47,7 @@ config ARM
> select HARDIRQS_SW_RESEND
> select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
> select HAVE_ARCH_BITREVERSE if (CPU_32v7M || CPU_32v7) && !CPU_32v6
> + select HAVE_ARCH_FREE_INITRD_MEM
> select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
> select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
> select HAVE_ARCH_MMAP_RND_BITS if MMU
Isn't this why weak symbols were invented?
Confused,
Rob
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: David Miller @ 2018-03-28 15:55 UTC (permalink / raw)
To: benh
Cc: okaya, torvalds, alexander.duyck, will.deacon, arnd, jgg,
David.Laight, oohall, linuxppc-dev, linux-rdma, alexander.h.duyck,
paulmck, netdev, linus971
In-Reply-To: <1522249996.21446.25.camel@kernel.crashing.org>
From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Thu, 29 Mar 2018 02:13:16 +1100
> Let's fix all archs, it's way easier than fixing all drivers. Half of
> the archs are unused or dead anyway.
Agreed.
^ permalink raw reply
* Re: [PATCH 2/2] smp: introduce kick_active_cpus_sync()
From: Paul E. McKenney @ 2018-03-28 13:56 UTC (permalink / raw)
To: Yury Norov
Cc: Chris Metcalf, Christopher Lameter, Russell King - ARM Linux,
Mark Rutland, Steven Rostedt, Mathieu Desnoyers, Catalin Marinas,
Will Deacon, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, linux-arm-kernel, linuxppc-dev, kvm-ppc,
linux-mm, linux-kernel, luto
In-Reply-To: <20180328133605.u7pftfxpn3jbqire@yury-thinkpad>
On Wed, Mar 28, 2018 at 04:36:05PM +0300, Yury Norov wrote:
> On Mon, Mar 26, 2018 at 05:45:55AM -0700, Paul E. McKenney wrote:
> > On Sun, Mar 25, 2018 at 11:11:54PM +0300, Yury Norov wrote:
> > > On Sun, Mar 25, 2018 at 12:23:28PM -0700, Paul E. McKenney wrote:
> > > > On Sun, Mar 25, 2018 at 08:50:04PM +0300, Yury Norov wrote:
> > > > > kick_all_cpus_sync() forces all CPUs to sync caches by sending broadcast IPI.
> > > > > If CPU is in extended quiescent state (idle task or nohz_full userspace), this
> > > > > work may be done at the exit of this state. Delaying synchronization helps to
> > > > > save power if CPU is in idle state and decrease latency for real-time tasks.
> > > > >
> > > > > This patch introduces kick_active_cpus_sync() and uses it in mm/slab and arm64
> > > > > code to delay syncronization.
> > > > >
> > > > > For task isolation (https://lkml.org/lkml/2017/11/3/589), IPI to the CPU running
> > > > > isolated task would be fatal, as it breaks isolation. The approach with delaying
> > > > > of synchronization work helps to maintain isolated state.
> > > > >
> > > > > I've tested it with test from task isolation series on ThunderX2 for more than
> > > > > 10 hours (10k giga-ticks) without breaking isolation.
> > > > >
> > > > > Signed-off-by: Yury Norov <ynorov@caviumnetworks.com>
> > > > > ---
> > > > > arch/arm64/kernel/insn.c | 2 +-
> > > > > include/linux/smp.h | 2 ++
> > > > > kernel/smp.c | 24 ++++++++++++++++++++++++
> > > > > mm/slab.c | 2 +-
> > > > > 4 files changed, 28 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c
> > > > > index 2718a77da165..9d7c492e920e 100644
> > > > > --- a/arch/arm64/kernel/insn.c
> > > > > +++ b/arch/arm64/kernel/insn.c
> > > > > @@ -291,7 +291,7 @@ int __kprobes aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt)
> > > > > * synchronization.
> > > > > */
> > > > > ret = aarch64_insn_patch_text_nosync(addrs[0], insns[0]);
> > > > > - kick_all_cpus_sync();
> > > > > + kick_active_cpus_sync();
> > > > > return ret;
> > > > > }
> > > > > }
> > > > > diff --git a/include/linux/smp.h b/include/linux/smp.h
> > > > > index 9fb239e12b82..27215e22240d 100644
> > > > > --- a/include/linux/smp.h
> > > > > +++ b/include/linux/smp.h
> > > > > @@ -105,6 +105,7 @@ int smp_call_function_any(const struct cpumask *mask,
> > > > > smp_call_func_t func, void *info, int wait);
> > > > >
> > > > > void kick_all_cpus_sync(void);
> > > > > +void kick_active_cpus_sync(void);
> > > > > void wake_up_all_idle_cpus(void);
> > > > >
> > > > > /*
> > > > > @@ -161,6 +162,7 @@ smp_call_function_any(const struct cpumask *mask, smp_call_func_t func,
> > > > > }
> > > > >
> > > > > static inline void kick_all_cpus_sync(void) { }
> > > > > +static inline void kick_active_cpus_sync(void) { }
> > > > > static inline void wake_up_all_idle_cpus(void) { }
> > > > >
> > > > > #ifdef CONFIG_UP_LATE_INIT
> > > > > diff --git a/kernel/smp.c b/kernel/smp.c
> > > > > index 084c8b3a2681..0358d6673850 100644
> > > > > --- a/kernel/smp.c
> > > > > +++ b/kernel/smp.c
> > > > > @@ -724,6 +724,30 @@ void kick_all_cpus_sync(void)
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
> > > > >
> > > > > +/**
> > > > > + * kick_active_cpus_sync - Force CPUs that are not in extended
> > > > > + * quiescent state (idle or nohz_full userspace) sync by sending
> > > > > + * IPI. Extended quiescent state CPUs will sync at the exit of
> > > > > + * that state.
> > > > > + */
> > > > > +void kick_active_cpus_sync(void)
> > > > > +{
> > > > > + int cpu;
> > > > > + struct cpumask kernel_cpus;
> > > > > +
> > > > > + smp_mb();
> > > > > +
> > > > > + cpumask_clear(&kernel_cpus);
> > > > > + preempt_disable();
> > > > > + for_each_online_cpu(cpu) {
> > > > > + if (!rcu_eqs_special_set(cpu))
> > > >
> > > > If we get here, the CPU is not in a quiescent state, so we therefore
> > > > must IPI it, correct?
> > > >
> > > > But don't you also need to define rcu_eqs_special_exit() so that RCU
> > > > can invoke it when it next leaves its quiescent state? Or are you able
> > > > to ignore the CPU in that case? (If you are able to ignore the CPU in
> > > > that case, I could give you a lower-cost function to get your job done.)
> > > >
> > > > Thanx, Paul
> > >
> > > What's actually needed for synchronization is issuing memory barrier on target
> > > CPUs before we start executing kernel code.
> > >
> > > smp_mb() is implicitly called in smp_call_function*() path for it. In
> > > rcu_eqs_special_set() -> rcu_dynticks_eqs_exit() path, smp_mb__after_atomic()
> > > is called just before rcu_eqs_special_exit().
> > >
> > > So I think, rcu_eqs_special_exit() may be left untouched. Empty
> > > rcu_eqs_special_exit() in new RCU path corresponds empty do_nothing() in old
> > > IPI path.
> > >
> > > Or my understanding of smp_mb__after_atomic() is wrong? By default,
> > > smp_mb__after_atomic() is just alias to smp_mb(). But some
> > > architectures define it differently. x86, for example, aliases it to
> > > just barrier() with a comment: "Atomic operations are already
> > > serializing on x86".
> > >
> > > I was initially thinking that it's also fine to leave
> > > rcu_eqs_special_exit() empty in this case, but now I'm not sure...
> > >
> > > Anyway, answering to your question, we shouldn't ignore quiescent
> > > CPUs, and rcu_eqs_special_set() path is really needed as it issues
> > > memory barrier on them.
> >
> > An alternative approach would be for me to make something like this
> > and export it:
> >
> > bool rcu_cpu_in_eqs(int cpu)
> > {
> > struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
> > int snap;
> >
> > smp_mb(); /* Obtain consistent snapshot, pairs with update. */
> > snap = READ_ONCE(&rdtp->dynticks);
> > smp_mb(); /* See above. */
> > return !(snap & RCU_DYNTICK_CTRL_CTR);
> > }
> >
> > Then you could replace your use of rcu_cpu_in_eqs() above with
>
> Did you mean replace rcu_eqs_special_set()?
Yes, apologies for my confusion, and good show figuring it out. ;-)
> > the new rcu_cpu_in_eqs(). This would avoid the RMW atomic, and, more
> > important, the unnecessary write to ->dynticks.
> >
> > Or am I missing something?
> >
> > Thanx, Paul
>
> This will not work because EQS CPUs will not be charged to call
> smp_mb() on exit of EQS.
Actually, CPUs are guaranteed to do a value-returning atomic increment
of ->dynticks on EQS exit, which implies smp_mb() both before and after
that atomic increment.
> Lets sync our understanding of IPI and RCU mechanisms.
>
> Traditional IPI scheme looks like this:
>
> CPU1: CPU2:
> touch shared resource(); /* running any code */
> smp_mb();
> smp_call_function(); ---> handle_IPI()
EQS exit here, so implied
smp_mb() on both sides of the
->dynticks increment.
> {
> /* Make resource visible */
> smp_mb();
> do_nothing();
> }
>
> And new RCU scheme for eqs CPUs looks like this:
>
> CPU1: CPU2:
> touch shared resource(); /* Running EQS */
> smp_mb();
>
> if (RCU_DYNTICK_CTRL_CTR)
> set(RCU_DYNTICK_CTRL_MASK); /* Still in EQS */
>
> /* And later */
> rcu_dynticks_eqs_exit()
> {
> if (RCU_DYNTICK_CTRL_MASK) {
> /* Make resource visible */
> smp_mb();
> rcu_eqs_special_exit();
> }
> }
>
> Is it correct?
You are missing the atomic_add_return() that is already in
rcu_dynticks_eqs_exit(), and this value-returning atomic operation again
implies smp_mb() both before and after. So you should be covered without
needing to worry about RCU_DYNTICK_CTRL_MASK.
Or am I missing something subtle here?
Thanx, Paul
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Benjamin Herrenschmidt @ 2018-03-28 15:13 UTC (permalink / raw)
To: okaya, Linus Torvalds
Cc: Alexander Duyck, Will Deacon, Arnd Bergmann, Jason Gunthorpe,
David Laight, Oliver,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT), linux-rdma,
Alexander Duyck, Paul E. McKenney, netdev, linus971
In-Reply-To: <b02cc752d90c5e5ae2cc8cc4f67429a7@codeaurora.org>
On Wed, 2018-03-28 at 07:41 -0400, okaya@codeaurora.org wrote:
> Yes, we want to get there indeed. It is because of some arch not
> implementing writel properly. Maintainers want to play safe.
>
> That is why I asked if IA64 and other well known archs follow the
> strongly ordered rule at this moment like PPC and ARM.
>
> Or should we go and inform every arch about this before yanking wmb()?
>
> Maintainers are afraid of introducing a regression.
Let's fix all archs, it's way easier than fixing all drivers. Half of
the archs are unused or dead anyway.
> >
> > The above code makes no sense, and just looks stupid to me. It also
> > generates pointlessly bad code on x86, so it's bad there too.
> >
> > Linus
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Benjamin Herrenschmidt @ 2018-03-28 15:12 UTC (permalink / raw)
To: David Laight, Will Deacon
Cc: Linus Torvalds, Alexander Duyck, Sinan Kaya, Arnd Bergmann,
Jason Gunthorpe, Oliver,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT),
linux-rdma@vger.kernel.org, Paul E. McKenney,
netdev@vger.kernel.org
In-Reply-To: <a642dc060b484d98bc728eb1f4b5c600@AcuMS.aculab.com>
On Wed, 2018-03-28 at 11:30 +0000, David Laight wrote:
> From: Benjamin Herrenschmidt
> > Sent: 28 March 2018 10:56
>
> ...
> > For example, let's say I have a device with a reset bit and the spec
> > says the reset bit needs to be set for at least 10us.
> >
> > This is wrong:
> >
> > writel(1, RESET_REG);
> > usleep(10);
> > writel(0, RESET_REG);
> >
> > Because of write posting, the first write might arrive to the device
> > right before the second one.
> >
> > The typical "fix" is to turn that into:
> >
> > writel(1, RESET_REG);
> > readl(RESET_REG); /* Flush posted writes */
>
> Would a writel(1, RESET_REG) here provide enough synchronsiation?
Probably yes. It's one of those things where you try to deal with the
fact that 90% of driver writers barely understand the basic stuff and
so you need the "default" accessors to be hardened as much as possible.
We still need to get a reasonably definition of the semantics of the
relaxed ones vs. WC memory but let's get through that exercise first
and hopefully for the last time.
> > usleep(10);
> > writel(0, RESET_REG);
> >
> > *However* the issue here, at least on power, is that the CPU can issue
> > that readl but doesn't necessarily wait for it to complete (ie, the
> > data to return), before proceeding to the usleep. Now a usleep contains
> > a bunch of loads and stores and is probably fine, but a udelay which
> > just loops on the timebase may not be.
> >
> > Thus we may still violate the timing requirement.
>
> I've seem that sort of code (with udelay() and no read back) quite often.
> How many were in linux I don't know.
>
> For small delays I usually fix it by repeated writes (of the same value)
> to the device register. That can guarantee very short intervals.
As long as you know the bus frequency...
> The only time I've actually seen buffered writes break timing was
> between a 286 and an 8859 interrupt controller.
:-)
The problem for me is not so much what I've seen, I realize that most
of the issues we are talking about are the kind that will hit once in a
thousand times or less.
But we *can* reason about them in a way that can effectively prevent
the problem completely and when your cluster has 10000 machine, 1/1000
starts becoming significant.
These days the vast majority of IO devices either are 99% DMA driven so
that a bit of overhead on MMIOs is irrelevant, or have one fast path
(low latency IB etc...) that needs some finer control, and the rest is
all setup which can be paranoid at will.
So I think we should aim for the "default" accessors most people use to
be as hadened as we can think of. I favor correctness over performance
in all cases. But then we also define a reasonable semantic for the
relaxed ones (well, we sort-of do have one, we might have to make it a
bit more precise in some areas) that allows the few MMIO fast path that
care to be optimized.
> If you wrote to the mask then enabled interrupts the first IACK cycle
> could be too close to write and break the cycle recovery time.
> That clobbered most of the interrupt controller registers.
> That probably affected every 286 board ever built!
> Not sure how much software added the required extra bus cycle.
>
> > What we did inside readl, with the twi;isync sequence (which basically
> > means, trap on return value with "trap never" as a condition, followed
> > by isync that ensures all excpetion conditions are resolved), is force
> > the CPU to "consume" the data from the read before moving on.
> >
> > This effectively makes readl fully synchronous (we would probably avoid
> > that if we were to implement a readl_relaxed).
>
> I've always wondered exactly what the twi;isync were for - always seemed
> very heavy handed for most mmio reads.
> Particularly if you are doing mmio reads from a data fifo.
If you do that you should use the "s" version of the accessors. Those
will only do the above trick at the end of the access series. Also a
FIFO needs special care about endianness anyway, so you should use
those accessors regardless. (Hint: you never endian swap a FIFO even on
BE on a LE device, unless something's been wired very badly in HW).
> Perhaps there should be a writel_wait() that is allowed to do a read back
> for such code paths?
I think what we have is fine, we just define that the standard
writel/readl as fairly simple and hardened, and we look at providing a
somewhat reasonable set of relaxed variants for optimizing fast path.
We pretty much already are there, we just need to be better at defining
the semantics.
And for the super high perf case, which thankfully is either seldom
(server high perf network stuff) or very arch specific (ARM SoC stuff),
then arch specific driver hacks will always remain the norm.
Cheers,
Ben.
> David
>
^ permalink raw reply
* [RFC PATCH 3/3] powerpc/64s: always flush non-local CPUs from single threaded mms
From: Nicholas Piggin @ 2018-03-28 15:29 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nicholas Piggin, Anton Blanchard, Aneesh Kumar K . V,
Benjamin Herrenschmidt
In-Reply-To: <20180328152951.6422-1-npiggin@gmail.com>
Go one step further, if we're going to put a tlbie on the bus
at all, make it count. Always flush all others and restore our
mm to a local one.
---
arch/powerpc/mm/tlb-radix.c | 45 +++++++++++++++++++++++++++------------------
1 file changed, 27 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
index f00acacf48f1..ba48539e799e 100644
--- a/arch/powerpc/mm/tlb-radix.c
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -424,10 +424,16 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
return;
preempt_disable();
- if (!mm_is_thread_local(mm))
- _tlbie_va(vmaddr, pid, psize, RIC_FLUSH_TLB);
- else
+ if (mm_is_thread_local(mm)) {
_tlbiel_va(vmaddr, pid, psize, RIC_FLUSH_TLB);
+ } else {
+ if (mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else {
+ _tlbie_va(vmaddr, pid, psize, RIC_FLUSH_TLB);
+ }
+ }
preempt_enable();
}
@@ -496,14 +502,14 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
nr_pages > tlb_single_page_flush_ceiling);
}
- if (full) {
+ if (!local && mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else if (full) {
if (local) {
_tlbiel_pid(pid, RIC_FLUSH_TLB);
} else {
- if (mm_is_singlethreaded(mm)) {
- _tlbie_pid(pid, RIC_FLUSH_ALL);
- mm_reset_thread_local(mm);
- } else if (mm_needs_flush_escalation(mm)) {
+ if (mm_needs_flush_escalation(mm)) {
_tlbie_pid(pid, RIC_FLUSH_ALL);
} else {
_tlbie_pid(pid, RIC_FLUSH_TLB);
@@ -618,19 +624,17 @@ static inline void __radix__flush_tlb_range_psize(struct mm_struct *mm,
nr_pages > tlb_single_page_flush_ceiling);
}
- if (full) {
+ if (!local && mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else if (full) {
if (local) {
_tlbiel_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
} else {
- if (mm_is_singlethreaded(mm)) {
- _tlbie_pid(pid, RIC_FLUSH_ALL);
- mm_reset_thread_local(mm);
- } else {
- if (mm_needs_flush_escalation(mm))
- also_pwc = true;
+ if (mm_needs_flush_escalation(mm))
+ also_pwc = true;
- _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
- }
+ _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
}
} else {
if (local)
@@ -676,7 +680,12 @@ void radix__flush_tlb_collapsed_pmd(struct mm_struct *mm, unsigned long addr)
if (mm_is_thread_local(mm)) {
_tlbiel_va_range(addr, end, pid, PAGE_SIZE, mmu_virtual_psize, true);
} else {
- _tlbie_va_range(addr, end, pid, PAGE_SIZE, mmu_virtual_psize, true);
+ if (mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else {
+ _tlbie_va_range(addr, end, pid, PAGE_SIZE, mmu_virtual_psize, true);
+ }
}
preempt_enable();
--
2.16.1
^ permalink raw reply related
* [RFC PATCH 2/3] powerpc/64s/radix: reset mm_cpumask for single thread process when possible
From: Nicholas Piggin @ 2018-03-28 15:29 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nicholas Piggin, Anton Blanchard, Aneesh Kumar K . V,
Benjamin Herrenschmidt
In-Reply-To: <20180328152951.6422-1-npiggin@gmail.com>
When a single-threaded process has a non-local mm_cpumask and requires
a full PID tlbie invalidation, use that as an opportunity to reset the
cpumask back to the current CPU we're running on.
No other thread can concurrently switch to this mm, because it must
have had a reference on mm_users before it could use_mm. mm_users can
be asynchronously incremented e.g., by mmget_not_zero, but those
users should not be doing use_mm.
---
arch/powerpc/include/asm/mmu_context.h | 33 +++++++++++++++-----
arch/powerpc/include/asm/tlb.h | 7 +++++
arch/powerpc/mm/tlb-radix.c | 57 +++++++++++++++++++++++++---------
3 files changed, 75 insertions(+), 22 deletions(-)
diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h
index 3a15b6db9501..9d373c8fe9fa 100644
--- a/arch/powerpc/include/asm/mmu_context.h
+++ b/arch/powerpc/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/sched.h>
+#include <linux/sched/mm.h>
#include <linux/spinlock.h>
#include <asm/mmu.h>
#include <asm/cputable.h>
@@ -102,13 +103,6 @@ static inline void mm_context_add_copro(struct mm_struct *mm)
static inline void mm_context_remove_copro(struct mm_struct *mm)
{
- int c;
-
- c = atomic_dec_if_positive(&mm->context.copros);
-
- /* Detect imbalance between add and remove */
- WARN_ON(c < 0);
-
/*
* Need to broadcast a global flush of the full mm before
* decrementing active_cpus count, as the next TLBI may be
@@ -119,10 +113,15 @@ static inline void mm_context_remove_copro(struct mm_struct *mm)
* for the time being. Invalidations will remain global if
* used on hash.
*/
- if (c == 0 && radix_enabled()) {
+ if (radix_enabled()) {
flush_all_mm(mm);
dec_mm_active_cpus(mm);
}
+
+ /* Detect imbalance between add and remove */
+ WARN_ON(!atomic_read(&mm->context.copros));
+ atomic_dec(&mm->context.copros);
+
}
#else
static inline void inc_mm_active_cpus(struct mm_struct *mm) { }
@@ -162,6 +161,24 @@ static inline void activate_mm(struct mm_struct *prev, struct mm_struct *next)
static inline void enter_lazy_tlb(struct mm_struct *mm,
struct task_struct *tsk)
{
+#ifdef CONFIG_PPC_BOOK3S_64
+ /*
+ * Under radix, we do not want to keep lazy PIDs around because
+ * even if the CPU does not access userspace, it can still bring
+ * in translations through speculation and prefetching.
+ *
+ * Switching away here allows us to trim back the mm_cpumask in
+ * cases where we know the process is not running on some CPUs
+ * (see mm/tlb-radix.c).
+ */
+ if (radix_enabled() && mm != &init_mm) {
+ mmgrab(&init_mm);
+ tsk->active_mm = &init_mm;
+ switch_mm_irqs_off(mm, tsk->active_mm, tsk);
+ mmdrop(mm);
+ }
+#endif
+
/* 64-bit Book3E keeps track of current PGD in the PACA */
#ifdef CONFIG_PPC_BOOK3E_64
get_paca()->pgd = NULL;
diff --git a/arch/powerpc/include/asm/tlb.h b/arch/powerpc/include/asm/tlb.h
index a7eabff27a0f..c4f43bcc6494 100644
--- a/arch/powerpc/include/asm/tlb.h
+++ b/arch/powerpc/include/asm/tlb.h
@@ -76,6 +76,13 @@ static inline int mm_is_thread_local(struct mm_struct *mm)
return false;
return cpumask_test_cpu(smp_processor_id(), mm_cpumask(mm));
}
+static inline void mm_reset_thread_local(struct mm_struct *mm)
+{
+ WARN_ON(!(atomic_read(&mm->mm_users) == 1 && current->mm == mm));
+ atomic_set(&mm->context.active_cpus, 1);
+ cpumask_clear(mm_cpumask(mm));
+ cpumask_set_cpu(smp_processor_id(), mm_cpumask(mm));
+}
#else /* CONFIG_PPC_BOOK3S_64 */
static inline int mm_is_thread_local(struct mm_struct *mm)
{
diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
index a07f5372a4bf..f00acacf48f1 100644
--- a/arch/powerpc/mm/tlb-radix.c
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -341,6 +341,15 @@ void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmadd
}
EXPORT_SYMBOL(radix__local_flush_tlb_page);
+static bool mm_is_singlethreaded(struct mm_struct *mm)
+{
+ if (atomic_read(&mm->context.copros) > 0)
+ return false;
+ if (atomic_read(&mm->mm_users) == 1 && current->mm == mm)
+ return true;
+ return false;
+}
+
static bool mm_needs_flush_escalation(struct mm_struct *mm)
{
/*
@@ -348,7 +357,9 @@ static bool mm_needs_flush_escalation(struct mm_struct *mm)
* caching PTEs and not flushing them properly when
* RIC = 0 for a PID/LPID invalidate
*/
- return atomic_read(&mm->context.copros) != 0;
+ if (atomic_read(&mm->context.copros) > 0)
+ return true;
+ return false;
}
#ifdef CONFIG_SMP
@@ -362,12 +373,17 @@ void radix__flush_tlb_mm(struct mm_struct *mm)
preempt_disable();
if (!mm_is_thread_local(mm)) {
- if (mm_needs_flush_escalation(mm))
+ if (mm_is_singlethreaded(mm)) {
_tlbie_pid(pid, RIC_FLUSH_ALL);
- else
+ mm_reset_thread_local(mm);
+ } else if (mm_needs_flush_escalation(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ } else {
_tlbie_pid(pid, RIC_FLUSH_TLB);
- } else
+ }
+ } else {
_tlbiel_pid(pid, RIC_FLUSH_TLB);
+ }
preempt_enable();
}
EXPORT_SYMBOL(radix__flush_tlb_mm);
@@ -381,10 +397,13 @@ void radix__flush_all_mm(struct mm_struct *mm)
return;
preempt_disable();
- if (!mm_is_thread_local(mm))
+ if (!mm_is_thread_local(mm)) {
_tlbie_pid(pid, RIC_FLUSH_ALL);
- else
+ if (mm_is_singlethreaded(mm))
+ mm_reset_thread_local(mm);
+ } else {
_tlbiel_pid(pid, RIC_FLUSH_ALL);
+ }
preempt_enable();
}
EXPORT_SYMBOL(radix__flush_all_mm);
@@ -481,10 +500,14 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
if (local) {
_tlbiel_pid(pid, RIC_FLUSH_TLB);
} else {
- if (mm_needs_flush_escalation(mm))
+ if (mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else if (mm_needs_flush_escalation(mm)) {
_tlbie_pid(pid, RIC_FLUSH_ALL);
- else
+ } else {
_tlbie_pid(pid, RIC_FLUSH_TLB);
+ }
}
} else {
bool hflush = false;
@@ -596,13 +619,19 @@ static inline void __radix__flush_tlb_range_psize(struct mm_struct *mm,
}
if (full) {
- if (!local && mm_needs_flush_escalation(mm))
- also_pwc = true;
-
- if (local)
+ if (local) {
_tlbiel_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
- else
- _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL: RIC_FLUSH_TLB);
+ } else {
+ if (mm_is_singlethreaded(mm)) {
+ _tlbie_pid(pid, RIC_FLUSH_ALL);
+ mm_reset_thread_local(mm);
+ } else {
+ if (mm_needs_flush_escalation(mm))
+ also_pwc = true;
+
+ _tlbie_pid(pid, also_pwc ? RIC_FLUSH_ALL : RIC_FLUSH_TLB);
+ }
+ }
} else {
if (local)
_tlbiel_va_range(start, end, pid, page_size, psize, also_pwc);
--
2.16.1
^ permalink raw reply related
* [RFC PATCH 1/3] powerpc/64s: do not flush TLB when relaxing access
From: Nicholas Piggin @ 2018-03-28 15:29 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nicholas Piggin, Anton Blanchard, Aneesh Kumar K . V,
Benjamin Herrenschmidt
In-Reply-To: <20180328152951.6422-1-npiggin@gmail.com>
Book3S does not require TLB flushing when protection is being relaxed.
>From Power ISA v3.0B, p.1090:
Setting a Reference or Change Bit or Upgrading Access Authority
(PTE Subject to Atomic Hardware Updates)
If the only change being made to a valid PTE that is subject to
atomic hardware updates is to set the Refer- ence or Change bit to
1 or to add access authorities, a simpler sequence suffices
because the translation hardware will refetch the PTE if an access
is attempted for which the only problems were reference and/or
change bits needing to be set or insufficient access authority.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/mm/pgtable-book3s64.c | 1 -
arch/powerpc/mm/pgtable.c | 3 ++-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/mm/pgtable-book3s64.c b/arch/powerpc/mm/pgtable-book3s64.c
index 422e80253a33..de49cedcbc84 100644
--- a/arch/powerpc/mm/pgtable-book3s64.c
+++ b/arch/powerpc/mm/pgtable-book3s64.c
@@ -40,7 +40,6 @@ int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
if (changed) {
__ptep_set_access_flags(vma->vm_mm, pmdp_ptep(pmdp),
pmd_pte(entry), address);
- flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
}
return changed;
}
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index 9f361ae571e9..5b07a626df5b 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -224,7 +224,8 @@ int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address,
if (!is_vm_hugetlb_page(vma))
assert_pte_locked(vma->vm_mm, address);
__ptep_set_access_flags(vma->vm_mm, ptep, entry, address);
- flush_tlb_page(vma, address);
+ if (!IS_ENABLED(CONFIG_PPC_BOOK3S_64))
+ flush_tlb_page(vma, address);
}
return changed;
}
--
2.16.1
^ permalink raw reply related
* [RFC PATCH 0/3] powerpc tlbie reductions again
From: Nicholas Piggin @ 2018-03-28 15:29 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nicholas Piggin, Anton Blanchard, Aneesh Kumar K . V,
Benjamin Herrenschmidt
Last time this came up, there was concern about whether we can trim
the mm cpumask, and what concurrency there is vs use_mm(). I've had
more of a look and still think this is okay. I haven't thought of a
good way to add debug checks to ensure it though.
When doing a parallel kernel build on a 2 socket P9 system, this
series causes tlbie (broadcast) to go from 1.37 million (22k/sec) to
181 thousand (3k/sec).
tlbiel (local) flushes increase from 20.2 to 23.7 million. Due to
requiring 128 tlbiel (vs 1 tlbie) to flush a PID, and also we set the
cutoff higher before we switch from va range to full PID flush, when
doing tlbiel.
End result performance was very little changed, very tiny improvement
maybe but well under 1%. Kernel compile mostly stays off the
interconnect, and this is a small system, and without nMMU
involvement. Any of these factors could make broadcast tlbie reduction
more important.
Remaining work - ensuring correctness of this stuff, implementations
for hash, understanding and testing nMMU cases better, using IPIs for
some/all types of invalidations, then possibly looking at doing
something more fancy with the PID allocator.
Nicholas Piggin (3):
powerpc/64s: do not flush TLB when relaxing access
powerpc/64s/radix: reset mm_cpumask for single thread process when
possible
powerpc/64s: always flush non-local CPUs from single threaded mms
arch/powerpc/include/asm/mmu_context.h | 33 ++++++++++----
arch/powerpc/include/asm/tlb.h | 7 +++
arch/powerpc/mm/pgtable-book3s64.c | 1 -
arch/powerpc/mm/pgtable.c | 3 +-
arch/powerpc/mm/tlb-radix.c | 78 +++++++++++++++++++++++++---------
5 files changed, 92 insertions(+), 30 deletions(-)
--
2.16.1
^ permalink raw reply
* [PATCH] Extract initrd free logic from arch-specific code.
From: Shea Levy @ 2018-03-28 15:26 UTC (permalink / raw)
To: linux-riscv, linux-kernel
Cc: Christoph Hellwig, Shea Levy, Richard Henderson, Ivan Kokshaysky,
Matt Turner, Vineet Gupta, Russell King, Catalin Marinas,
Will Deacon, Mark Salter, Aurelien Jacquiot, Mikael Starvik,
Jesper Nilsson, Yoshinori Sato, Richard Kuo, Tony Luck,
Fenghua Yu, Geert Uytterhoeven, James Hogan, Michal Simek,
Ralf Baechle, David Howells, Ley Foon Tan, Jonas Bonn,
Stefan Kristiansson, Stafford Horne, James E.J. Bottomley,
Helge Deller, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Palmer Dabbelt, Albert Ou, Martin Schwidefsky,
Heiko Carstens, Chen Liqin, Lennox Wu, Rich Felker,
David S. Miller, Jeff Dike, Richard Weinberger, Guan Xuetao,
Thomas Gleixner, Ingo Molnar, H. Peter Anvin, x86, Chris Zankel,
Max Filippov, Kate Stewart, Greg Kroah-Hartman,
Philippe Ombredanne, Eugeniy Paltsev, Al Viro, Vladimir Murzin,
Linus Walleij, Michal Hocko, Andrew Morton, Sudip Mukherjee,
Marc Zyngier, Rob Herring, Kees Cook, Vlastimil Babka,
Balbir Singh, Christophe Leroy, Joe Perches,
Oliver O'Halloran, Dan Williams, Wei Yang,
Christian König, Arnd Bergmann, Deepa Dinamani,
Daniel Thompson, Rob Landley, Florian Fainelli, linux-alpha,
linux-snps-arc, linux-arm-kernel, adi-buildroot-devel,
linux-c6x-dev, linux-cris-kernel, uclinux-h8-devel, linux-hexagon,
linux-ia64, linux-m68k, linux-metag, linux-mips, linux-am33-list,
nios2-dev, openrisc, linux-parisc, linuxppc-dev, linux-s390,
linux-sh, sparclinux, user-mode-linux-devel, user-mode-linux-user,
linux-xtensa
In-Reply-To: <20180325221853.10839-1-shea@shealevy.com>
Now only those architectures that have custom initrd free requirements
need to define free_initrd_mem.
Signed-off-by: Shea Levy <shea@shealevy.com>
---
arch/alpha/mm/init.c | 8 --------
arch/arc/mm/init.c | 7 -------
arch/arm/Kconfig | 1 +
arch/arm64/Kconfig | 1 +
arch/blackfin/Kconfig | 1 +
arch/c6x/mm/init.c | 7 -------
arch/cris/Kconfig | 1 +
arch/frv/mm/init.c | 11 -----------
arch/h8300/mm/init.c | 7 -------
arch/hexagon/Kconfig | 1 +
arch/ia64/Kconfig | 1 +
arch/m32r/Kconfig | 1 +
arch/m32r/mm/init.c | 11 -----------
arch/m68k/mm/init.c | 7 -------
arch/metag/Kconfig | 1 +
arch/microblaze/mm/init.c | 7 -------
arch/mips/Kconfig | 1 +
arch/mn10300/Kconfig | 1 +
arch/nios2/mm/init.c | 7 -------
arch/openrisc/mm/init.c | 7 -------
arch/parisc/mm/init.c | 7 -------
arch/powerpc/mm/mem.c | 7 -------
arch/riscv/mm/init.c | 6 ------
arch/s390/Kconfig | 1 +
arch/score/Kconfig | 1 +
arch/sh/mm/init.c | 7 -------
arch/sparc/Kconfig | 1 +
arch/tile/Kconfig | 1 +
arch/um/kernel/mem.c | 7 -------
arch/unicore32/Kconfig | 1 +
arch/x86/Kconfig | 1 +
arch/xtensa/Kconfig | 1 +
init/initramfs.c | 7 +++++++
usr/Kconfig | 4 ++++
34 files changed, 28 insertions(+), 113 deletions(-)
diff --git a/arch/alpha/mm/init.c b/arch/alpha/mm/init.c
index 9d74520298ab..55f7c8efa962 100644
--- a/arch/alpha/mm/init.c
+++ b/arch/alpha/mm/init.c
@@ -291,11 +291,3 @@ free_initmem(void)
{
free_initmem_default(-1);
}
-
-#ifdef CONFIG_BLK_DEV_INITRD
-void
-free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
diff --git a/arch/arc/mm/init.c b/arch/arc/mm/init.c
index ba145065c579..7bcf23ab1756 100644
--- a/arch/arc/mm/init.c
+++ b/arch/arc/mm/init.c
@@ -229,10 +229,3 @@ void __ref free_initmem(void)
{
free_initmem_default(-1);
}
-
-#ifdef CONFIG_BLK_DEV_INITRD
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 3f972e83909b..19d1c5594e2d 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,7 @@ config ARM
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
select HAVE_ARCH_BITREVERSE if (CPU_32v7M || CPU_32v7) && !CPU_32v6
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
select HAVE_ARCH_MMAP_RND_BITS if MMU
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index cb03e93f03cf..de93620870af 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -85,6 +85,7 @@ config ARM64
select HAVE_ALIGNED_STRUCT_PAGE if SLUB
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_BITREVERSE
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_HUGE_VMAP
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
diff --git a/arch/blackfin/Kconfig b/arch/blackfin/Kconfig
index d9c2866ba618..6c6dae9fe894 100644
--- a/arch/blackfin/Kconfig
+++ b/arch/blackfin/Kconfig
@@ -15,6 +15,7 @@ config BLACKFIN
def_bool y
select HAVE_ARCH_KGDB
select HAVE_ARCH_TRACEHOOK
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_TRACER
diff --git a/arch/c6x/mm/init.c b/arch/c6x/mm/init.c
index 4cc72b0d1c1d..a11cb657182a 100644
--- a/arch/c6x/mm/init.c
+++ b/arch/c6x/mm/init.c
@@ -66,13 +66,6 @@ void __init mem_init(void)
mem_init_print_info(NULL);
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
void __init free_initmem(void)
{
free_initmem_default(-1);
diff --git a/arch/cris/Kconfig b/arch/cris/Kconfig
index cd5a0865c97f..5425f77e5664 100644
--- a/arch/cris/Kconfig
+++ b/arch/cris/Kconfig
@@ -76,6 +76,7 @@ config CRIS
select HAVE_DEBUG_BUGVERBOSE if ETRAX_ARCH_V32
select HAVE_NMI
select DMA_DIRECT_OPS if PCI
+ select HAVE_ARCH_FREE_INITRD_MEM
config HZ
int
diff --git a/arch/frv/mm/init.c b/arch/frv/mm/init.c
index cf464100e838..345edc4dc462 100644
--- a/arch/frv/mm/init.c
+++ b/arch/frv/mm/init.c
@@ -131,14 +131,3 @@ void free_initmem(void)
free_initmem_default(-1);
#endif
} /* end free_initmem() */
-
-/*****************************************************************************/
-/*
- * free the initial ramdisk memory
- */
-#ifdef CONFIG_BLK_DEV_INITRD
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-} /* end free_initrd_mem() */
-#endif
diff --git a/arch/h8300/mm/init.c b/arch/h8300/mm/init.c
index 015287ac8ce8..37574332b202 100644
--- a/arch/h8300/mm/init.c
+++ b/arch/h8300/mm/init.c
@@ -102,13 +102,6 @@ void __init mem_init(void)
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
void
free_initmem(void)
{
diff --git a/arch/hexagon/Kconfig b/arch/hexagon/Kconfig
index 76d2f20d525e..69a16cd2e253 100644
--- a/arch/hexagon/Kconfig
+++ b/arch/hexagon/Kconfig
@@ -17,6 +17,7 @@ config HEXAGON
# GENERIC_ALLOCATOR is used by dma_alloc_coherent()
select GENERIC_ALLOCATOR
select GENERIC_IRQ_SHOW
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_KGDB
select HAVE_ARCH_TRACEHOOK
select NO_IOPORT_MAP
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index bbe12a038d21..366ef1dd3cd4 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -28,6 +28,7 @@ config IA64
select HAVE_DYNAMIC_FTRACE if (!ITANIUM)
select HAVE_FUNCTION_TRACER
select TTY
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_TRACEHOOK
select HAVE_DMA_API_DEBUG
select HAVE_MEMBLOCK
diff --git a/arch/m32r/Kconfig b/arch/m32r/Kconfig
index dd84ee194579..9e41bdff45c3 100644
--- a/arch/m32r/Kconfig
+++ b/arch/m32r/Kconfig
@@ -21,6 +21,7 @@ config M32R
select CPU_NO_EFFICIENT_FFS
select DMA_DIRECT_OPS
select ARCH_NO_COHERENT_DMA_MMAP if !MMU
+ select HAVE_ARCH_FREE_INITRD_MEM
config SBUS
bool
diff --git a/arch/m32r/mm/init.c b/arch/m32r/mm/init.c
index 93abc8c3a46e..e2b5f09209ee 100644
--- a/arch/m32r/mm/init.c
+++ b/arch/m32r/mm/init.c
@@ -139,14 +139,3 @@ void free_initmem(void)
{
free_initmem_default(-1);
}
-
-#ifdef CONFIG_BLK_DEV_INITRD
-/*======================================================================*
- * free_initrd_mem() :
- * orig : arch/sh/mm/init.c
- *======================================================================*/
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
diff --git a/arch/m68k/mm/init.c b/arch/m68k/mm/init.c
index e85acd131fa8..e20bef09258c 100644
--- a/arch/m68k/mm/init.c
+++ b/arch/m68k/mm/init.c
@@ -172,10 +172,3 @@ void __init mem_init(void)
mem_init_print_info(NULL);
print_memmap();
}
-
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
diff --git a/arch/metag/Kconfig b/arch/metag/Kconfig
index c7b62a339539..5be7f1693b1b 100644
--- a/arch/metag/Kconfig
+++ b/arch/metag/Kconfig
@@ -7,6 +7,7 @@ config METAG
select GENERIC_IRQ_SHOW
select GENERIC_SMP_IDLE_THREAD
select HAVE_64BIT_ALIGNED_ACCESS
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_TRACEHOOK
select HAVE_C_RECORDMCOUNT
select HAVE_DEBUG_KMEMLEAK
diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c
index df6de7ccdc2e..ea058dfda222 100644
--- a/arch/microblaze/mm/init.c
+++ b/arch/microblaze/mm/init.c
@@ -187,13 +187,6 @@ void __init setup_memory(void)
paging_init();
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
void free_initmem(void)
{
free_initmem_default(-1);
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8128c3b68d6b..c033cd1e0c52 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -27,6 +27,7 @@ config MIPS
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
select HANDLE_DOMAIN_IRQ
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS if MMU
diff --git a/arch/mn10300/Kconfig b/arch/mn10300/Kconfig
index e9d8d60bd28b..5aa4f1aa309f 100644
--- a/arch/mn10300/Kconfig
+++ b/arch/mn10300/Kconfig
@@ -6,6 +6,7 @@ config MN10300
select HAVE_UID16
select GENERIC_IRQ_SHOW
select ARCH_WANT_IPC_PARSE_VERSION
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_KGDB
select GENERIC_ATOMIC64
diff --git a/arch/nios2/mm/init.c b/arch/nios2/mm/init.c
index c92fe4234009..3df75ff8c768 100644
--- a/arch/nios2/mm/init.c
+++ b/arch/nios2/mm/init.c
@@ -82,13 +82,6 @@ void __init mmu_init(void)
flush_tlb_all();
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
void __ref free_initmem(void)
{
free_initmem_default(-1);
diff --git a/arch/openrisc/mm/init.c b/arch/openrisc/mm/init.c
index 6972d5d6f23f..c1a3dcf9ad40 100644
--- a/arch/openrisc/mm/init.c
+++ b/arch/openrisc/mm/init.c
@@ -222,13 +222,6 @@ void __init mem_init(void)
return;
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
void free_initmem(void)
{
free_initmem_default(-1);
diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c
index cab32ee824d2..3643399230f3 100644
--- a/arch/parisc/mm/init.c
+++ b/arch/parisc/mm/init.c
@@ -932,10 +932,3 @@ void flush_tlb_all(void)
spin_unlock(&sid_lock);
}
#endif
-
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index fe8c61149fb8..e85b2a3cd264 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -404,13 +404,6 @@ void free_initmem(void)
free_initmem_default(POISON_FREE_INITMEM);
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
/*
* This is called when a page has been modified by the kernel.
* It just marks the page as not i-cache clean. We do the i-cache
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index c77df8142be2..36f83fe8a726 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -62,9 +62,3 @@ void free_initmem(void)
{
free_initmem_default(0);
}
-
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
-}
-#endif /* CONFIG_BLK_DEV_INITRD */
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index eaee7087886f..94a9879f1ea8 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -125,6 +125,7 @@ config S390
select GENERIC_TIME_VSYSCALL
select HAVE_ALIGNED_STRUCT_PAGE if SLUB
select HAVE_ARCH_AUDITSYSCALL
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_JUMP_LABEL
select CPU_NO_EFFICIENT_FFS if !HAVE_MARCH_Z9_109_FEATURES
select HAVE_ARCH_SECCOMP_FILTER
diff --git a/arch/score/Kconfig b/arch/score/Kconfig
index d881f99c9ddd..2beff03b2429 100644
--- a/arch/score/Kconfig
+++ b/arch/score/Kconfig
@@ -16,6 +16,7 @@ config SCORE
select MODULES_USE_ELF_REL
select CLONE_BACKWARDS
select CPU_NO_EFFICIENT_FFS
+ select HAVE_ARCH_FREE_INITRD_MEM
choice
prompt "System type"
diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c
index ce0bbaa7e404..7451459d0725 100644
--- a/arch/sh/mm/init.c
+++ b/arch/sh/mm/init.c
@@ -477,13 +477,6 @@ void free_initmem(void)
free_initmem_default(-1);
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
#ifdef CONFIG_MEMORY_HOTPLUG
int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap,
bool want_memblock)
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 8767e45f1b2b..06d543e35caf 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -18,6 +18,7 @@ config SPARC
select OF_PROMTREE
select HAVE_IDE
select HAVE_OPROFILE
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_KGDB if !SMP || SPARC64
select HAVE_ARCH_TRACEHOOK
select HAVE_EXIT_THREAD
diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig
index ef9d403cbbe4..7fb36c15762c 100644
--- a/arch/tile/Kconfig
+++ b/arch/tile/Kconfig
@@ -16,6 +16,7 @@ config TILE
select GENERIC_PENDING_IRQ if SMP
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_CONTEXT_TRACKING
diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c
index 3c0e470ea646..2d26eec92126 100644
--- a/arch/um/kernel/mem.c
+++ b/arch/um/kernel/mem.c
@@ -170,13 +170,6 @@ void free_initmem(void)
{
}
-#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
-#endif
-
/* Allocate and free page tables. */
pgd_t *pgd_alloc(struct mm_struct *mm)
diff --git a/arch/unicore32/Kconfig b/arch/unicore32/Kconfig
index 462e59a7ae78..4da24cf467de 100644
--- a/arch/unicore32/Kconfig
+++ b/arch/unicore32/Kconfig
@@ -4,6 +4,7 @@ config UNICORE32
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_MIGHT_HAVE_PC_PARPORT
select ARCH_MIGHT_HAVE_PC_SERIO
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_MEMBLOCK
select HAVE_GENERIC_DMA_COHERENT
select HAVE_KERNEL_GZIP
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 0fa71a78ec99..51ce64e6d848 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -111,6 +111,7 @@ config X86
select HAVE_ALIGNED_STRUCT_PAGE if SLUB
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_HUGE_VMAP if X86_64 || X86_PAE
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_KASAN if X86_64
select HAVE_ARCH_KGDB
diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig
index c921e8bccdc8..01a4a0e42118 100644
--- a/arch/xtensa/Kconfig
+++ b/arch/xtensa/Kconfig
@@ -16,6 +16,7 @@ config XTENSA
select GENERIC_PCI_IOMAP
select GENERIC_SCHED_CLOCK
select GENERIC_STRNCPY_FROM_USER if KASAN
+ select HAVE_ARCH_FREE_INITRD_MEM
select HAVE_ARCH_KASAN if MMU
select HAVE_CC_STACKPROTECTOR
select HAVE_DEBUG_KMEMLEAK
diff --git a/init/initramfs.c b/init/initramfs.c
index 7e99a0038942..d319058eb464 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -526,6 +526,13 @@ extern unsigned long __initramfs_size;
#include <linux/initrd.h>
#include <linux/kexec.h>
+#ifndef CONFIG_HAVE_ARCH_FREE_INITRD_MEM
+void __init free_initrd_mem(unsigned long start, unsigned long end)
+{
+ free_reserved_area((void *)start, (void *)end, -1, "initrd");
+}
+#endif
+
static void __init free_initrd(void)
{
#ifdef CONFIG_KEXEC_CORE
diff --git a/usr/Kconfig b/usr/Kconfig
index 43658b8a975e..7a94f6df39bf 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -233,3 +233,7 @@ config INITRAMFS_COMPRESSION
default ".lzma" if RD_LZMA
default ".bz2" if RD_BZIP2
default ""
+
+config HAVE_ARCH_FREE_INITRD_MEM
+ bool
+ default n
--
2.16.2
^ permalink raw reply related
* Re: [PATCH v3 41/41] cxlflash: Handle spurious interrupts
From: Matthew R. Ochs @ 2018-03-28 15:03 UTC (permalink / raw)
To: Uma Krishnan
Cc: linux-scsi, James Bottomley, Martin K. Petersen, Manoj N. Kumar,
linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522082142-58975-1-git-send-email-ukrishn@linux.vnet.ibm.com>
On Mon, Mar 26, 2018 at 11:35:42AM -0500, Uma Krishnan wrote:
> The following Oops can occur when there is heavy I/O traffic and the host
> is reset by a tool such as sg_reset.
>
> [c000200fff3fbc90] c00800001690117c process_cmd_doneq+0x104/0x500
> [cxlflash] (unreliable)
> [c000200fff3fbd80] c008000016901648 cxlflash_rrq_irq+0xd0/0x150 [cxlflash]
> [c000200fff3fbde0] c000000000193130 __handle_irq_event_percpu+0xa0/0x310
> [c000200fff3fbea0] c0000000001933d8 handle_irq_event_percpu+0x38/0x90
> [c000200fff3fbee0] c000000000193494 handle_irq_event+0x64/0xb0
> [c000200fff3fbf10] c000000000198ea0 handle_fasteoi_irq+0xc0/0x230
> [c000200fff3fbf40] c00000000019182c generic_handle_irq+0x4c/0x70
> [c000200fff3fbf60] c00000000001794c __do_irq+0x7c/0x1c0
> [c000200fff3fbf90] c00000000002a390 call_do_irq+0x14/0x24
> [c000200e5828fab0] c000000000017b2c do_IRQ+0x9c/0x130
> [c000200e5828fb00] c000000000009b04 h_virt_irq_common+0x114/0x120
>
> When a context is reset, the pending commands are flushed and the AFU
> is notified. Before the AFU handles this request there could be command
> completion interrupts queued to PHB which are yet to be delivered to the
> context. In this scenario, a context could receive an interrupt for a
> command that has been flushed, leading to a possible crash when the memory
> for the flushed command is accessed.
>
> To resolve this problem, a boolean will indicate if the hardware queue is
> ready to process interrupts or not. This can be evaluated in the interrupt
> handler before proessing an interrupt.
>
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
^ permalink raw reply
* Re: [PATCH v3 40/41] cxlflash: Remove commmands from pending list on timeout
From: Matthew R. Ochs @ 2018-03-28 14:50 UTC (permalink / raw)
To: Uma Krishnan
Cc: linux-scsi, James Bottomley, Martin K. Petersen, Manoj N. Kumar,
linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522082134-58938-1-git-send-email-ukrishn@linux.vnet.ibm.com>
On Mon, Mar 26, 2018 at 11:35:34AM -0500, Uma Krishnan wrote:
> The following Oops can occur if an internal command sent to the AFU does
> not complete within the timeout:
>
> [c000000ff101b810] c008000016020d94 term_mc+0xfc/0x1b0 [cxlflash]
> [c000000ff101b8a0] c008000016020fb0 term_afu+0x168/0x280 [cxlflash]
> [c000000ff101b930] c0080000160232ec cxlflash_pci_error_detected+0x184/0x230
> [cxlflash]
> [c000000ff101b9e0] c00800000d95d468 cxl_vphb_error_detected+0x90/0x150[cxl]
> [c000000ff101ba20] c00800000d95f27c cxl_pci_error_detected+0xa4/0x240 [cxl]
> [c000000ff101bac0] c00000000003eaf8 eeh_report_error+0xd8/0x1b0
> [c000000ff101bb20] c00000000003d0b8 eeh_pe_dev_traverse+0x98/0x170
> [c000000ff101bbb0] c00000000003f438 eeh_handle_normal_event+0x198/0x580
> [c000000ff101bc60] c00000000003fba4 eeh_handle_event+0x2a4/0x338
> [c000000ff101bd10] c0000000000400b8 eeh_event_handler+0x1f8/0x200
> [c000000ff101bdc0] c00000000013da48 kthread+0x1a8/0x1b0
> [c000000ff101be30] c00000000000b528 ret_from_kernel_thread+0x5c/0xb4
>
> When an internal command times out, the command buffer is freed while it
> is still in the pending commands list of the context. This corrupts the
> list and when the context is cleaned up, a crash is encountered.
>
> To resolve this issue, when an AFU command or TMF command times out, the
> command should be deleted from the hardware queue pending command list
> before freeing the buffer.
>
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
^ permalink raw reply
* Re: [PATCH 2/2] smp: introduce kick_active_cpus_sync()
From: Paul E. McKenney @ 2018-03-28 14:45 UTC (permalink / raw)
To: Yury Norov
Cc: Chris Metcalf, Christopher Lameter, Russell King - ARM Linux,
Mark Rutland, Steven Rostedt, Mathieu Desnoyers, Catalin Marinas,
Will Deacon, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, linux-arm-kernel, linuxppc-dev, kvm-ppc,
linux-mm, linux-kernel, luto
In-Reply-To: <20180328144140.g4qwtciikffaescu@yury-thinkpad>
On Wed, Mar 28, 2018 at 05:41:40PM +0300, Yury Norov wrote:
> On Wed, Mar 28, 2018 at 06:56:17AM -0700, Paul E. McKenney wrote:
> > On Wed, Mar 28, 2018 at 04:36:05PM +0300, Yury Norov wrote:
> > > On Mon, Mar 26, 2018 at 05:45:55AM -0700, Paul E. McKenney wrote:
> > > > On Sun, Mar 25, 2018 at 11:11:54PM +0300, Yury Norov wrote:
> > > > > On Sun, Mar 25, 2018 at 12:23:28PM -0700, Paul E. McKenney wrote:
> > > > > > On Sun, Mar 25, 2018 at 08:50:04PM +0300, Yury Norov wrote:
> > > > > > > kick_all_cpus_sync() forces all CPUs to sync caches by sending broadcast IPI.
> > > > > > > If CPU is in extended quiescent state (idle task or nohz_full userspace), this
> > > > > > > work may be done at the exit of this state. Delaying synchronization helps to
> > > > > > > save power if CPU is in idle state and decrease latency for real-time tasks.
> > > > > > >
> > > > > > > This patch introduces kick_active_cpus_sync() and uses it in mm/slab and arm64
> > > > > > > code to delay syncronization.
> > > > > > >
> > > > > > > For task isolation (https://lkml.org/lkml/2017/11/3/589), IPI to the CPU running
> > > > > > > isolated task would be fatal, as it breaks isolation. The approach with delaying
> > > > > > > of synchronization work helps to maintain isolated state.
> > > > > > >
> > > > > > > I've tested it with test from task isolation series on ThunderX2 for more than
> > > > > > > 10 hours (10k giga-ticks) without breaking isolation.
> > > > > > >
> > > > > > > Signed-off-by: Yury Norov <ynorov@caviumnetworks.com>
> > > > > > > ---
> > > > > > > arch/arm64/kernel/insn.c | 2 +-
> > > > > > > include/linux/smp.h | 2 ++
> > > > > > > kernel/smp.c | 24 ++++++++++++++++++++++++
> > > > > > > mm/slab.c | 2 +-
> > > > > > > 4 files changed, 28 insertions(+), 2 deletions(-)
> > > > > > >
> > > > > > > diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c
> > > > > > > index 2718a77da165..9d7c492e920e 100644
> > > > > > > --- a/arch/arm64/kernel/insn.c
> > > > > > > +++ b/arch/arm64/kernel/insn.c
> > > > > > > @@ -291,7 +291,7 @@ int __kprobes aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt)
> > > > > > > * synchronization.
> > > > > > > */
> > > > > > > ret = aarch64_insn_patch_text_nosync(addrs[0], insns[0]);
> > > > > > > - kick_all_cpus_sync();
> > > > > > > + kick_active_cpus_sync();
> > > > > > > return ret;
> > > > > > > }
> > > > > > > }
> > > > > > > diff --git a/include/linux/smp.h b/include/linux/smp.h
> > > > > > > index 9fb239e12b82..27215e22240d 100644
> > > > > > > --- a/include/linux/smp.h
> > > > > > > +++ b/include/linux/smp.h
> > > > > > > @@ -105,6 +105,7 @@ int smp_call_function_any(const struct cpumask *mask,
> > > > > > > smp_call_func_t func, void *info, int wait);
> > > > > > >
> > > > > > > void kick_all_cpus_sync(void);
> > > > > > > +void kick_active_cpus_sync(void);
> > > > > > > void wake_up_all_idle_cpus(void);
> > > > > > >
> > > > > > > /*
> > > > > > > @@ -161,6 +162,7 @@ smp_call_function_any(const struct cpumask *mask, smp_call_func_t func,
> > > > > > > }
> > > > > > >
> > > > > > > static inline void kick_all_cpus_sync(void) { }
> > > > > > > +static inline void kick_active_cpus_sync(void) { }
> > > > > > > static inline void wake_up_all_idle_cpus(void) { }
> > > > > > >
> > > > > > > #ifdef CONFIG_UP_LATE_INIT
> > > > > > > diff --git a/kernel/smp.c b/kernel/smp.c
> > > > > > > index 084c8b3a2681..0358d6673850 100644
> > > > > > > --- a/kernel/smp.c
> > > > > > > +++ b/kernel/smp.c
> > > > > > > @@ -724,6 +724,30 @@ void kick_all_cpus_sync(void)
> > > > > > > }
> > > > > > > EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
> > > > > > >
> > > > > > > +/**
> > > > > > > + * kick_active_cpus_sync - Force CPUs that are not in extended
> > > > > > > + * quiescent state (idle or nohz_full userspace) sync by sending
> > > > > > > + * IPI. Extended quiescent state CPUs will sync at the exit of
> > > > > > > + * that state.
> > > > > > > + */
> > > > > > > +void kick_active_cpus_sync(void)
> > > > > > > +{
> > > > > > > + int cpu;
> > > > > > > + struct cpumask kernel_cpus;
> > > > > > > +
> > > > > > > + smp_mb();
> > > > > > > +
> > > > > > > + cpumask_clear(&kernel_cpus);
> > > > > > > + preempt_disable();
> > > > > > > + for_each_online_cpu(cpu) {
> > > > > > > + if (!rcu_eqs_special_set(cpu))
> > > > > >
> > > > > > If we get here, the CPU is not in a quiescent state, so we therefore
> > > > > > must IPI it, correct?
> > > > > >
> > > > > > But don't you also need to define rcu_eqs_special_exit() so that RCU
> > > > > > can invoke it when it next leaves its quiescent state? Or are you able
> > > > > > to ignore the CPU in that case? (If you are able to ignore the CPU in
> > > > > > that case, I could give you a lower-cost function to get your job done.)
> > > > > >
> > > > > > Thanx, Paul
> > > > >
> > > > > What's actually needed for synchronization is issuing memory barrier on target
> > > > > CPUs before we start executing kernel code.
> > > > >
> > > > > smp_mb() is implicitly called in smp_call_function*() path for it. In
> > > > > rcu_eqs_special_set() -> rcu_dynticks_eqs_exit() path, smp_mb__after_atomic()
> > > > > is called just before rcu_eqs_special_exit().
> > > > >
> > > > > So I think, rcu_eqs_special_exit() may be left untouched. Empty
> > > > > rcu_eqs_special_exit() in new RCU path corresponds empty do_nothing() in old
> > > > > IPI path.
> > > > >
> > > > > Or my understanding of smp_mb__after_atomic() is wrong? By default,
> > > > > smp_mb__after_atomic() is just alias to smp_mb(). But some
> > > > > architectures define it differently. x86, for example, aliases it to
> > > > > just barrier() with a comment: "Atomic operations are already
> > > > > serializing on x86".
> > > > >
> > > > > I was initially thinking that it's also fine to leave
> > > > > rcu_eqs_special_exit() empty in this case, but now I'm not sure...
> > > > >
> > > > > Anyway, answering to your question, we shouldn't ignore quiescent
> > > > > CPUs, and rcu_eqs_special_set() path is really needed as it issues
> > > > > memory barrier on them.
> > > >
> > > > An alternative approach would be for me to make something like this
> > > > and export it:
> > > >
> > > > bool rcu_cpu_in_eqs(int cpu)
> > > > {
> > > > struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
> > > > int snap;
> > > >
> > > > smp_mb(); /* Obtain consistent snapshot, pairs with update. */
> > > > snap = READ_ONCE(&rdtp->dynticks);
> > > > smp_mb(); /* See above. */
> > > > return !(snap & RCU_DYNTICK_CTRL_CTR);
> > > > }
> > > >
> > > > Then you could replace your use of rcu_cpu_in_eqs() above with
> > >
> > > Did you mean replace rcu_eqs_special_set()?
> >
> > Yes, apologies for my confusion, and good show figuring it out. ;-)
> >
> > > > the new rcu_cpu_in_eqs(). This would avoid the RMW atomic, and, more
> > > > important, the unnecessary write to ->dynticks.
> > > >
> > > > Or am I missing something?
> > > >
> > > > Thanx, Paul
> > >
> > > This will not work because EQS CPUs will not be charged to call
> > > smp_mb() on exit of EQS.
> >
> > Actually, CPUs are guaranteed to do a value-returning atomic increment
> > of ->dynticks on EQS exit, which implies smp_mb() both before and after
> > that atomic increment.
> >
> > > Lets sync our understanding of IPI and RCU mechanisms.
> > >
> > > Traditional IPI scheme looks like this:
> > >
> > > CPU1: CPU2:
> > > touch shared resource(); /* running any code */
> > > smp_mb();
> > > smp_call_function(); ---> handle_IPI()
> >
> > EQS exit here, so implied
> > smp_mb() on both sides of the
> > ->dynticks increment.
> >
> > > {
> > > /* Make resource visible */
> > > smp_mb();
> > > do_nothing();
> > > }
> > >
> > > And new RCU scheme for eqs CPUs looks like this:
> > >
> > > CPU1: CPU2:
> > > touch shared resource(); /* Running EQS */
> > > smp_mb();
> > >
> > > if (RCU_DYNTICK_CTRL_CTR)
> > > set(RCU_DYNTICK_CTRL_MASK); /* Still in EQS */
> > >
> > > /* And later */
> > > rcu_dynticks_eqs_exit()
> > > {
> > > if (RCU_DYNTICK_CTRL_MASK) {
> > > /* Make resource visible */
> > > smp_mb();
> > > rcu_eqs_special_exit();
> > > }
> > > }
> > >
> > > Is it correct?
> >
> > You are missing the atomic_add_return() that is already in
> > rcu_dynticks_eqs_exit(), and this value-returning atomic operation again
> > implies smp_mb() both before and after. So you should be covered without
> > needing to worry about RCU_DYNTICK_CTRL_MASK.
> >
> > Or am I missing something subtle here?
>
> Ah, now I understand, thank you. I'll collect other comments for more, and
> submit v2 with this change.
Very good, looking forward to seeing v2.
Thanx, Paul
^ permalink raw reply
* Re: [PATCH v3 39/41] cxlflash: Synchronize reset and remove ops
From: Matthew R. Ochs @ 2018-03-28 14:43 UTC (permalink / raw)
To: Uma Krishnan
Cc: linux-scsi, James Bottomley, Martin K. Petersen, Manoj N. Kumar,
linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522082127-58900-1-git-send-email-ukrishn@linux.vnet.ibm.com>
On Mon, Mar 26, 2018 at 11:35:27AM -0500, Uma Krishnan wrote:
> The following Oops can be encountered if a device removal or system
> shutdown is initiated while an EEH recovery is in process:
>
> [c000000ff2f479c0] c008000015256f18 cxlflash_pci_slot_reset+0xa0/0x100
> [cxlflash]
> [c000000ff2f47a30] c00800000dae22e0 cxl_pci_slot_reset+0x168/0x290 [cxl]
> [c000000ff2f47ae0] c00000000003ef1c eeh_report_reset+0xec/0x170
> [c000000ff2f47b20] c00000000003d0b8 eeh_pe_dev_traverse+0x98/0x170
> [c000000ff2f47bb0] c00000000003f80c eeh_handle_normal_event+0x56c/0x580
> [c000000ff2f47c60] c00000000003fba4 eeh_handle_event+0x2a4/0x338
> [c000000ff2f47d10] c0000000000400b8 eeh_event_handler+0x1f8/0x200
> [c000000ff2f47dc0] c00000000013da48 kthread+0x1a8/0x1b0
> [c000000ff2f47e30] c00000000000b528 ret_from_kernel_thread+0x5c/0xb4
>
> The remove handler frees AFU memory while the EEH recovery is in progress,
> leading to a race condition. This can result in a crash if the recovery
> thread tries to access this memory.
>
> To resolve this issue, the cxlflash remove handler will evaluate the
> device state and yield to any active reset or probing threads.
>
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Looks good!
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
^ permalink raw reply
* Re: [PATCH 2/2] smp: introduce kick_active_cpus_sync()
From: Yury Norov @ 2018-03-28 14:41 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Chris Metcalf, Christopher Lameter, Russell King - ARM Linux,
Mark Rutland, Steven Rostedt, Mathieu Desnoyers, Catalin Marinas,
Will Deacon, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, linux-arm-kernel, linuxppc-dev, kvm-ppc,
linux-mm, linux-kernel, luto
In-Reply-To: <20180328135617.GQ3675@linux.vnet.ibm.com>
On Wed, Mar 28, 2018 at 06:56:17AM -0700, Paul E. McKenney wrote:
> On Wed, Mar 28, 2018 at 04:36:05PM +0300, Yury Norov wrote:
> > On Mon, Mar 26, 2018 at 05:45:55AM -0700, Paul E. McKenney wrote:
> > > On Sun, Mar 25, 2018 at 11:11:54PM +0300, Yury Norov wrote:
> > > > On Sun, Mar 25, 2018 at 12:23:28PM -0700, Paul E. McKenney wrote:
> > > > > On Sun, Mar 25, 2018 at 08:50:04PM +0300, Yury Norov wrote:
> > > > > > kick_all_cpus_sync() forces all CPUs to sync caches by sending broadcast IPI.
> > > > > > If CPU is in extended quiescent state (idle task or nohz_full userspace), this
> > > > > > work may be done at the exit of this state. Delaying synchronization helps to
> > > > > > save power if CPU is in idle state and decrease latency for real-time tasks.
> > > > > >
> > > > > > This patch introduces kick_active_cpus_sync() and uses it in mm/slab and arm64
> > > > > > code to delay syncronization.
> > > > > >
> > > > > > For task isolation (https://lkml.org/lkml/2017/11/3/589), IPI to the CPU running
> > > > > > isolated task would be fatal, as it breaks isolation. The approach with delaying
> > > > > > of synchronization work helps to maintain isolated state.
> > > > > >
> > > > > > I've tested it with test from task isolation series on ThunderX2 for more than
> > > > > > 10 hours (10k giga-ticks) without breaking isolation.
> > > > > >
> > > > > > Signed-off-by: Yury Norov <ynorov@caviumnetworks.com>
> > > > > > ---
> > > > > > arch/arm64/kernel/insn.c | 2 +-
> > > > > > include/linux/smp.h | 2 ++
> > > > > > kernel/smp.c | 24 ++++++++++++++++++++++++
> > > > > > mm/slab.c | 2 +-
> > > > > > 4 files changed, 28 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c
> > > > > > index 2718a77da165..9d7c492e920e 100644
> > > > > > --- a/arch/arm64/kernel/insn.c
> > > > > > +++ b/arch/arm64/kernel/insn.c
> > > > > > @@ -291,7 +291,7 @@ int __kprobes aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt)
> > > > > > * synchronization.
> > > > > > */
> > > > > > ret = aarch64_insn_patch_text_nosync(addrs[0], insns[0]);
> > > > > > - kick_all_cpus_sync();
> > > > > > + kick_active_cpus_sync();
> > > > > > return ret;
> > > > > > }
> > > > > > }
> > > > > > diff --git a/include/linux/smp.h b/include/linux/smp.h
> > > > > > index 9fb239e12b82..27215e22240d 100644
> > > > > > --- a/include/linux/smp.h
> > > > > > +++ b/include/linux/smp.h
> > > > > > @@ -105,6 +105,7 @@ int smp_call_function_any(const struct cpumask *mask,
> > > > > > smp_call_func_t func, void *info, int wait);
> > > > > >
> > > > > > void kick_all_cpus_sync(void);
> > > > > > +void kick_active_cpus_sync(void);
> > > > > > void wake_up_all_idle_cpus(void);
> > > > > >
> > > > > > /*
> > > > > > @@ -161,6 +162,7 @@ smp_call_function_any(const struct cpumask *mask, smp_call_func_t func,
> > > > > > }
> > > > > >
> > > > > > static inline void kick_all_cpus_sync(void) { }
> > > > > > +static inline void kick_active_cpus_sync(void) { }
> > > > > > static inline void wake_up_all_idle_cpus(void) { }
> > > > > >
> > > > > > #ifdef CONFIG_UP_LATE_INIT
> > > > > > diff --git a/kernel/smp.c b/kernel/smp.c
> > > > > > index 084c8b3a2681..0358d6673850 100644
> > > > > > --- a/kernel/smp.c
> > > > > > +++ b/kernel/smp.c
> > > > > > @@ -724,6 +724,30 @@ void kick_all_cpus_sync(void)
> > > > > > }
> > > > > > EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
> > > > > >
> > > > > > +/**
> > > > > > + * kick_active_cpus_sync - Force CPUs that are not in extended
> > > > > > + * quiescent state (idle or nohz_full userspace) sync by sending
> > > > > > + * IPI. Extended quiescent state CPUs will sync at the exit of
> > > > > > + * that state.
> > > > > > + */
> > > > > > +void kick_active_cpus_sync(void)
> > > > > > +{
> > > > > > + int cpu;
> > > > > > + struct cpumask kernel_cpus;
> > > > > > +
> > > > > > + smp_mb();
> > > > > > +
> > > > > > + cpumask_clear(&kernel_cpus);
> > > > > > + preempt_disable();
> > > > > > + for_each_online_cpu(cpu) {
> > > > > > + if (!rcu_eqs_special_set(cpu))
> > > > >
> > > > > If we get here, the CPU is not in a quiescent state, so we therefore
> > > > > must IPI it, correct?
> > > > >
> > > > > But don't you also need to define rcu_eqs_special_exit() so that RCU
> > > > > can invoke it when it next leaves its quiescent state? Or are you able
> > > > > to ignore the CPU in that case? (If you are able to ignore the CPU in
> > > > > that case, I could give you a lower-cost function to get your job done.)
> > > > >
> > > > > Thanx, Paul
> > > >
> > > > What's actually needed for synchronization is issuing memory barrier on target
> > > > CPUs before we start executing kernel code.
> > > >
> > > > smp_mb() is implicitly called in smp_call_function*() path for it. In
> > > > rcu_eqs_special_set() -> rcu_dynticks_eqs_exit() path, smp_mb__after_atomic()
> > > > is called just before rcu_eqs_special_exit().
> > > >
> > > > So I think, rcu_eqs_special_exit() may be left untouched. Empty
> > > > rcu_eqs_special_exit() in new RCU path corresponds empty do_nothing() in old
> > > > IPI path.
> > > >
> > > > Or my understanding of smp_mb__after_atomic() is wrong? By default,
> > > > smp_mb__after_atomic() is just alias to smp_mb(). But some
> > > > architectures define it differently. x86, for example, aliases it to
> > > > just barrier() with a comment: "Atomic operations are already
> > > > serializing on x86".
> > > >
> > > > I was initially thinking that it's also fine to leave
> > > > rcu_eqs_special_exit() empty in this case, but now I'm not sure...
> > > >
> > > > Anyway, answering to your question, we shouldn't ignore quiescent
> > > > CPUs, and rcu_eqs_special_set() path is really needed as it issues
> > > > memory barrier on them.
> > >
> > > An alternative approach would be for me to make something like this
> > > and export it:
> > >
> > > bool rcu_cpu_in_eqs(int cpu)
> > > {
> > > struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
> > > int snap;
> > >
> > > smp_mb(); /* Obtain consistent snapshot, pairs with update. */
> > > snap = READ_ONCE(&rdtp->dynticks);
> > > smp_mb(); /* See above. */
> > > return !(snap & RCU_DYNTICK_CTRL_CTR);
> > > }
> > >
> > > Then you could replace your use of rcu_cpu_in_eqs() above with
> >
> > Did you mean replace rcu_eqs_special_set()?
>
> Yes, apologies for my confusion, and good show figuring it out. ;-)
>
> > > the new rcu_cpu_in_eqs(). This would avoid the RMW atomic, and, more
> > > important, the unnecessary write to ->dynticks.
> > >
> > > Or am I missing something?
> > >
> > > Thanx, Paul
> >
> > This will not work because EQS CPUs will not be charged to call
> > smp_mb() on exit of EQS.
>
> Actually, CPUs are guaranteed to do a value-returning atomic increment
> of ->dynticks on EQS exit, which implies smp_mb() both before and after
> that atomic increment.
>
> > Lets sync our understanding of IPI and RCU mechanisms.
> >
> > Traditional IPI scheme looks like this:
> >
> > CPU1: CPU2:
> > touch shared resource(); /* running any code */
> > smp_mb();
> > smp_call_function(); ---> handle_IPI()
>
> EQS exit here, so implied
> smp_mb() on both sides of the
> ->dynticks increment.
>
> > {
> > /* Make resource visible */
> > smp_mb();
> > do_nothing();
> > }
> >
> > And new RCU scheme for eqs CPUs looks like this:
> >
> > CPU1: CPU2:
> > touch shared resource(); /* Running EQS */
> > smp_mb();
> >
> > if (RCU_DYNTICK_CTRL_CTR)
> > set(RCU_DYNTICK_CTRL_MASK); /* Still in EQS */
> >
> > /* And later */
> > rcu_dynticks_eqs_exit()
> > {
> > if (RCU_DYNTICK_CTRL_MASK) {
> > /* Make resource visible */
> > smp_mb();
> > rcu_eqs_special_exit();
> > }
> > }
> >
> > Is it correct?
>
> You are missing the atomic_add_return() that is already in
> rcu_dynticks_eqs_exit(), and this value-returning atomic operation again
> implies smp_mb() both before and after. So you should be covered without
> needing to worry about RCU_DYNTICK_CTRL_MASK.
>
> Or am I missing something subtle here?
Ah, now I understand, thank you. I'll collect other comments for more, and
submit v2 with this change.
Yury
^ permalink raw reply
* Re: [v2, 01/10] powerpc: Add security feature flags for Spectre/Meltdown
From: Michael Ellerman @ 2018-03-28 14:13 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev; +Cc: mauricfo
In-Reply-To: <20180327120153.31612-1-mpe@ellerman.id.au>
On Tue, 2018-03-27 at 12:01:44 UTC, Michael Ellerman wrote:
> This commit adds security feature flags to reflect the settings we
> receive from firmware regarding Spectre/Meltdown mitigations.
>
> The feature names reflect the names we are given by firmware on bare
> metal machines. See the hostboot source for details.
>
> Arguably these could be firmware features, but that then requires them
> to be read early in boot so they're available prior to asm feature
> patching, but we don't actually want to use them for patching. We may
> also want to dynamically update them in future, which would be
> incompatible with the way firmware features work (at the moment at
> least). So for now just make them separate flags.
>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Series applied to powerpc next.
https://git.kernel.org/powerpc/c/9a868f634349e62922c226834aa23e
cheers
^ permalink raw reply
* Re: [v3,1/8] powerpc: Add ppc_breakpoint_available()
From: Michael Ellerman @ 2018-03-28 14:13 UTC (permalink / raw)
To: Michael Neuling; +Cc: kvm-ppc, npiggin, david, linuxppc-dev, pedromfc
In-Reply-To: <20180327043724.13862-2-mikey@neuling.org>
On Tue, 2018-03-27 at 04:37:17 UTC, Michael Neuling wrote:
> Add ppc_breakpoint_available() to determine if a breakpoint is
> available currently via the DAWR or DABR.
>
> Signed-off-by: Michael Neuling <mikey@neuling.org>
Series applied to powerpc next, thanks.
https://git.kernel.org/powerpc/c/404b27d66ed657ebccb08a9c8f8f65
cheers
^ permalink raw reply
* Re: [v2] powerpc/perf: Fix the kernel address leak to userspace via SDAR
From: Michael Ellerman @ 2018-03-28 14:13 UTC (permalink / raw)
To: Madhavan Srinivasan; +Cc: Madhavan Srinivasan, linuxppc-dev
In-Reply-To: <1521632426-30770-3-git-send-email-maddy@linux.vnet.ibm.com>
On Wed, 2018-03-21 at 11:40:26 UTC, Madhavan Srinivasan wrote:
> Sampled Data Address Register (SDAR) is a 64-bit
> register that contains the effective address of
> the storage operand of an instruction that was
> being executed, possibly out-of-order, at or around
> the time that the Performance Monitor alert occurred.
>
> In certain scenario SDAR happen to contain the kernel
> address even for userspace only sampling. Add checks
> to prevent it.
>
> Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Applied to powerpc next, thanks.
https://git.kernel.org/powerpc/c/cd1231d7035fea894118d5155ff984
cheers
^ permalink raw reply
* Re: [v2] powerpc/perf: Fix kernel address leak to userspace via BHRB buffer
From: Michael Ellerman @ 2018-03-28 14:13 UTC (permalink / raw)
To: Madhavan Srinivasan; +Cc: Madhavan Srinivasan, linuxppc-dev
In-Reply-To: <1521632426-30770-2-git-send-email-maddy@linux.vnet.ibm.com>
On Wed, 2018-03-21 at 11:40:25 UTC, Madhavan Srinivasan wrote:
> The current Branch History Rolling Buffer (BHRB) code does
> not check for any privilege levels before updating the data
> from BHRB. This leaks kernel addresses to userspace even when
> profiling only with userspace privileges. Add proper checks
> to prevent it.
>
> Acked-by: Balbir Singh <bsingharora@gmail.com>
> Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Applied to powerpc next, thanks.
https://git.kernel.org/powerpc/c/bb19af816025d495376bd76bf6fbcf
cheers
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox