* Re: [PATCH 5/8] powerpc/64s: implement queued spinlocks and rwlocks
From: Will Deacon @ 2020-07-02 8:02 UTC (permalink / raw)
To: Nicholas Piggin
Cc: linux-arch, Peter Zijlstra, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <20200702074839.1057733-6-npiggin@gmail.com>
On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote:
> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h
> new file mode 100644
> index 000000000000..f84da77b6bb7
> --- /dev/null
> +++ b/arch/powerpc/include/asm/qspinlock.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_POWERPC_QSPINLOCK_H
> +#define _ASM_POWERPC_QSPINLOCK_H
> +
> +#include <asm-generic/qspinlock_types.h>
> +
> +#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
> +
> +#define smp_mb__after_spinlock() smp_mb()
> +
> +static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
> +{
> + smp_mb();
> + return atomic_read(&lock->val);
> +}
Why do you need the smp_mb() here?
Will
^ permalink raw reply
* Re: [PATCH 2/8] powerpc/pseries: use smp_rmb() in H_CONFER spin yield
From: Peter Zijlstra @ 2020-07-02 8:28 UTC (permalink / raw)
To: Nicholas Piggin
Cc: linux-arch, linuxppc-dev, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, Will Deacon
In-Reply-To: <20200702074839.1057733-3-npiggin@gmail.com>
On Thu, Jul 02, 2020 at 05:48:33PM +1000, Nicholas Piggin wrote:
> There is no need for rmb(), this allows faster lwsync here.
Since you determined this; I'm thinking you actually understand the
ordering here. How about recording this understanding in a comment?
Also, should the lock->slock load not use READ_ONCE() ?
^ permalink raw reply
* Re: [PATCH v5 3/3] mm/page_alloc: Keep memoryless cpuless node 0 offline
From: Michal Hocko @ 2020-07-02 8:41 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Gautham R Shenoy, David Hildenbrand, Linus Torvalds, linux-kernel,
linux-mm, Satheesh Rajendran, Mel Gorman, Kirill A. Shutemov,
Andrew Morton, linuxppc-dev, Christopher Lameter, Vlastimil Babka
In-Reply-To: <20200702064408.GD17918@linux.vnet.ibm.com>
On Thu 02-07-20 12:14:08, Srikar Dronamraju wrote:
> * Michal Hocko <mhocko@kernel.org> [2020-07-01 14:21:10]:
>
> > > >>>>>>
> > > >>>>>> 2. Also existence of dummy node also leads to inconsistent information. The
> > > >>>>>> number of online nodes is inconsistent with the information in the
> > > >>>>>> device-tree and resource-dump
> > > >>>>>>
> > > >>>>>> 3. When the dummy node is present, single node non-Numa systems end up showing
> > > >>>>>> up as NUMA systems and numa_balancing gets enabled. This will mean we take
> > > >>>>>> the hit from the unnecessary numa hinting faults.
> > > >>>>>
> > > >>>>> I have to say that I dislike the node online/offline state and directly
> > > >>>>> exporting that to the userspace. Users should only care whether the node
> > > >>>>> has memory/cpus. Numa nodes can be online without any memory. Just
> > > >>>>> offline all the present memory blocks but do not physically hot remove
> > > >>>>> them and you are in the same situation. If users are confused by an
> > > >>>>> output of tools like numactl -H then those could be updated and hide
> > > >>>>> nodes without any memory&cpus.
> > > >>>>>
> > > >>>>> The autonuma problem sounds interesting but again this patch doesn't
> > > >>>>> really solve the underlying problem because I strongly suspect that the
> > > >>>>> problem is still there when a numa node gets all its memory offline as
> > > >>>>> mentioned above.
> >
> > I would really appreciate a feedback to these two as well.
>
> 1. Its not just numactl that's to be fixed but all tools/utilities that
> depend on /sys/devices/system/node/online. Are we saying to not rely/believe
> in the output given by the kernel but do further verification?
No, what we are saying is that even an online node might have zero
number of online pages/cpus. So the online status is not really
something that matters. If people are confused by that output then user
space tools can make their confusion go away. I really do not understand
why the kernel should do any logic there.
> Also how would the user space differentiate between the case where the
> Kernel missed marking a node as offline to the case where the memory was
> offlined on a cpuless node but node wasn't offline?.
What I am arguing is that those two shouldn't be any different. Really!
> 2. Regarding the autonuma, the case of offline memory is user/admin driven,
> so if there is a performance hit, its something that's driven by his
> user/admin actions. Also how often do we see users offline complete memory
> of cpuless node on a 2 node system?
How often do we see crippled HW configurations like that? Really if
autonuma should be made more clever for one case it should recognize the
other as well.
> > > [ 0.009726] SRAT: PXM 1 -> APIC 0x00 -> Node 0
> > > [ 0.009727] SRAT: PXM 1 -> APIC 0x01 -> Node 0
> > > [ 0.009727] SRAT: PXM 1 -> APIC 0x02 -> Node 0
> > > [ 0.009728] SRAT: PXM 1 -> APIC 0x03 -> Node 0
> > > [ 0.009731] ACPI: SRAT: Node 0 PXM 1 [mem 0x00000000-0x0009ffff]
> > > [ 0.009732] ACPI: SRAT: Node 0 PXM 1 [mem 0x00100000-0xbfffffff]
> > > [ 0.009733] ACPI: SRAT: Node 0 PXM 1 [mem 0x100000000-0x13fffffff]
> >
> > This begs a question whether ppc can do the same thing?
>
> Certainly ppc can be made to adapt to this situation but that would be a
> workaround. Do we have a reason why we think node 0 is unique and special?
It is not. As replied in other email in this thread. I would hope for
having less hacks in the numa initialization. Cleaning up the mess is
would be a lot of work and testing on all NUMA capable architectures.
This is a heritage from the past I am afraid. All that I am arguing here
is that your touch to the generic code with a very simple looking patch
might have side effects which are pretty much impossible to review.
Moreover it seems that nothing but ppc really needs this treatment.
So fixing it in ppc specific code sounds much more safe.
Normally I would really push for a generic solution but after getting
burned several times in this area I do not dare anymore. The problem is
not in the code complexity but in how spread it is in places where you
do not expect side effects.
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* [Bug 208181] BUG: KASAN: stack-out-of-bounds in strcmp+0x58/0xd8
From: bugzilla-daemon @ 2020-07-02 8:47 UTC (permalink / raw)
To: linuxppc-dev
In-Reply-To: <bug-208181-206035@https.bugzilla.kernel.org/>
https://bugzilla.kernel.org/show_bug.cgi?id=208181
--- Comment #10 from Christophe Leroy (christophe.leroy@csgroup.eu) ---
I reproduced the problem and bisected it to commit
https://github.com/torvalds/linux/commit/d2a91cef9bbdeb87b7449fdab1a6be6000930210
I'll investigate the issue
Can you confirm this commit is also the curprit from your side ?
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply
* Re: [PATCH v2 06/10] powerpc/perf: power10 Performance Monitoring support
From: kernel test robot @ 2020-07-02 9:06 UTC (permalink / raw)
To: Athira Rajeev, mpe; +Cc: linuxppc-dev, mikey, maddy, kbuild-all
In-Reply-To: <1593595262-1433-7-git-send-email-atrajeev@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]
Hi Athira,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on powerpc/next]
[also build test WARNING on tip/perf/core v5.8-rc3 next-20200702]
[cannot apply to kvm-ppc/kvm-ppc-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Athira-Rajeev/powerpc-perf-Add-support-for-power10-PMU-Hardware/20200701-181147
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc-allyesconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> arch/powerpc/perf/power10-pmu.c:393:5: warning: no previous prototype for 'init_power10_pmu' [-Wmissing-prototypes]
393 | int init_power10_pmu(void)
| ^~~~~~~~~~~~~~~~
vim +/init_power10_pmu +393 arch/powerpc/perf/power10-pmu.c
392
> 393 int init_power10_pmu(void)
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 69717 bytes --]
^ permalink raw reply
* Re: [PATCH v2 10/10] powerpc/perf: Add extended regs support for power10 platform
From: kernel test robot @ 2020-07-02 9:40 UTC (permalink / raw)
To: Athira Rajeev, mpe; +Cc: linuxppc-dev, mikey, maddy, kbuild-all
In-Reply-To: <1593595262-1433-11-git-send-email-atrajeev@linux.vnet.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]
Hi Athira,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on powerpc/next]
[also build test ERROR on tip/perf/core v5.8-rc3 next-20200702]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Athira-Rajeev/powerpc-perf-Add-support-for-power10-PMU-Hardware/20200701-181147
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc-pmac32_defconfig (attached as .config)
compiler: powerpc-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
{standard input}: Assembler messages:
>> {standard input}:84: Error: unsupported relocation against SPRN_SIER2
>> {standard input}:91: Error: unsupported relocation against SPRN_SIER3
>> {standard input}:119: Error: unsupported relocation against SPRN_MMCR3
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 24961 bytes --]
^ permalink raw reply
* Re: [PATCH 5/8] powerpc/64s: implement queued spinlocks and rwlocks
From: Nicholas Piggin @ 2020-07-02 10:25 UTC (permalink / raw)
To: Will Deacon
Cc: linux-arch, Peter Zijlstra, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <20200702080219.GB16113@willie-the-truck>
Excerpts from Will Deacon's message of July 2, 2020 6:02 pm:
> On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote:
>> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h
>> new file mode 100644
>> index 000000000000..f84da77b6bb7
>> --- /dev/null
>> +++ b/arch/powerpc/include/asm/qspinlock.h
>> @@ -0,0 +1,20 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +#ifndef _ASM_POWERPC_QSPINLOCK_H
>> +#define _ASM_POWERPC_QSPINLOCK_H
>> +
>> +#include <asm-generic/qspinlock_types.h>
>> +
>> +#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
>> +
>> +#define smp_mb__after_spinlock() smp_mb()
>> +
>> +static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
>> +{
>> + smp_mb();
>> + return atomic_read(&lock->val);
>> +}
>
> Why do you need the smp_mb() here?
A long and sad tale that ends here 51d7d5205d338
Should probably at least refer to that commit from here, since this one
is not going to git blame back there. I'll add something.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH 5/8] powerpc/64s: implement queued spinlocks and rwlocks
From: Will Deacon @ 2020-07-02 10:35 UTC (permalink / raw)
To: Nicholas Piggin
Cc: linux-arch, Peter Zijlstra, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <1593685459.r2tfxtfdp6.astroid@bobo.none>
On Thu, Jul 02, 2020 at 08:25:43PM +1000, Nicholas Piggin wrote:
> Excerpts from Will Deacon's message of July 2, 2020 6:02 pm:
> > On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote:
> >> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h
> >> new file mode 100644
> >> index 000000000000..f84da77b6bb7
> >> --- /dev/null
> >> +++ b/arch/powerpc/include/asm/qspinlock.h
> >> @@ -0,0 +1,20 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> +#ifndef _ASM_POWERPC_QSPINLOCK_H
> >> +#define _ASM_POWERPC_QSPINLOCK_H
> >> +
> >> +#include <asm-generic/qspinlock_types.h>
> >> +
> >> +#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
> >> +
> >> +#define smp_mb__after_spinlock() smp_mb()
> >> +
> >> +static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
> >> +{
> >> + smp_mb();
> >> + return atomic_read(&lock->val);
> >> +}
> >
> > Why do you need the smp_mb() here?
>
> A long and sad tale that ends here 51d7d5205d338
>
> Should probably at least refer to that commit from here, since this one
> is not going to git blame back there. I'll add something.
Is this still an issue, though?
See 38b850a73034 (where we added a similar barrier on arm64) and then
c6f5d02b6a0f (where we removed it).
Will
^ permalink raw reply
* Re: [PATCH 2/8] powerpc/pseries: use smp_rmb() in H_CONFER spin yield
From: Nicholas Piggin @ 2020-07-02 10:36 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-arch, Will Deacon, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <20200702082840.GC4781@hirez.programming.kicks-ass.net>
Excerpts from Peter Zijlstra's message of July 2, 2020 6:28 pm:
> On Thu, Jul 02, 2020 at 05:48:33PM +1000, Nicholas Piggin wrote:
>> There is no need for rmb(), this allows faster lwsync here.
>
> Since you determined this; I'm thinking you actually understand the
> ordering here. How about recording this understanding in a comment?
>
> Also, should the lock->slock load not use READ_ONCE() ?
Yeah, good point. Maybe I'll drop it from this series, doesn't really
belong I just saw the cleanup and didn't want to forget it.
We we just ordering the two loads in this function, and !SMP isn't a
concern (i.e., no issues of !SMP guest on SMP HV), but yeah fixing
the lack of comment is warranted, thanks.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH 5/8] powerpc/64s: implement queued spinlocks and rwlocks
From: Nicholas Piggin @ 2020-07-02 10:47 UTC (permalink / raw)
To: Will Deacon
Cc: linux-arch, Peter Zijlstra, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <20200702103506.GA16418@willie-the-truck>
Excerpts from Will Deacon's message of July 2, 2020 8:35 pm:
> On Thu, Jul 02, 2020 at 08:25:43PM +1000, Nicholas Piggin wrote:
>> Excerpts from Will Deacon's message of July 2, 2020 6:02 pm:
>> > On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote:
>> >> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h
>> >> new file mode 100644
>> >> index 000000000000..f84da77b6bb7
>> >> --- /dev/null
>> >> +++ b/arch/powerpc/include/asm/qspinlock.h
>> >> @@ -0,0 +1,20 @@
>> >> +/* SPDX-License-Identifier: GPL-2.0 */
>> >> +#ifndef _ASM_POWERPC_QSPINLOCK_H
>> >> +#define _ASM_POWERPC_QSPINLOCK_H
>> >> +
>> >> +#include <asm-generic/qspinlock_types.h>
>> >> +
>> >> +#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
>> >> +
>> >> +#define smp_mb__after_spinlock() smp_mb()
>> >> +
>> >> +static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
>> >> +{
>> >> + smp_mb();
>> >> + return atomic_read(&lock->val);
>> >> +}
>> >
>> > Why do you need the smp_mb() here?
>>
>> A long and sad tale that ends here 51d7d5205d338
>>
>> Should probably at least refer to that commit from here, since this one
>> is not going to git blame back there. I'll add something.
>
> Is this still an issue, though?
>
> See 38b850a73034 (where we added a similar barrier on arm64) and then
> c6f5d02b6a0f (where we removed it).
>
Oh nice, I didn't know that went away. Thanks for the heads up.
I'm going to say I'm too scared to remove it while changing the
spinlock algorithm, but I'll open an issue and we should look at
removing it.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH 5/8] powerpc/64s: implement queued spinlocks and rwlocks
From: Will Deacon @ 2020-07-02 10:48 UTC (permalink / raw)
To: Nicholas Piggin
Cc: linux-arch, Peter Zijlstra, Boqun Feng, linux-kernel, kvm-ppc,
virtualization, Ingo Molnar, Waiman Long, linuxppc-dev
In-Reply-To: <1593686722.w9psaqk7yp.astroid@bobo.none>
On Thu, Jul 02, 2020 at 08:47:05PM +1000, Nicholas Piggin wrote:
> Excerpts from Will Deacon's message of July 2, 2020 8:35 pm:
> > On Thu, Jul 02, 2020 at 08:25:43PM +1000, Nicholas Piggin wrote:
> >> Excerpts from Will Deacon's message of July 2, 2020 6:02 pm:
> >> > On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote:
> >> >> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h
> >> >> new file mode 100644
> >> >> index 000000000000..f84da77b6bb7
> >> >> --- /dev/null
> >> >> +++ b/arch/powerpc/include/asm/qspinlock.h
> >> >> @@ -0,0 +1,20 @@
> >> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> >> +#ifndef _ASM_POWERPC_QSPINLOCK_H
> >> >> +#define _ASM_POWERPC_QSPINLOCK_H
> >> >> +
> >> >> +#include <asm-generic/qspinlock_types.h>
> >> >> +
> >> >> +#define _Q_PENDING_LOOPS (1 << 9) /* not tuned */
> >> >> +
> >> >> +#define smp_mb__after_spinlock() smp_mb()
> >> >> +
> >> >> +static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
> >> >> +{
> >> >> + smp_mb();
> >> >> + return atomic_read(&lock->val);
> >> >> +}
> >> >
> >> > Why do you need the smp_mb() here?
> >>
> >> A long and sad tale that ends here 51d7d5205d338
> >>
> >> Should probably at least refer to that commit from here, since this one
> >> is not going to git blame back there. I'll add something.
> >
> > Is this still an issue, though?
> >
> > See 38b850a73034 (where we added a similar barrier on arm64) and then
> > c6f5d02b6a0f (where we removed it).
> >
>
> Oh nice, I didn't know that went away. Thanks for the heads up.
>
> I'm going to say I'm too scared to remove it while changing the
> spinlock algorithm, but I'll open an issue and we should look at
> removing it.
Makes sense to me -- it certainly needs a deeper look! In the meantime,
please put some of this in a comment next to the barrier.
Cheers,
Will
^ permalink raw reply
* Re: [PATCH v6 1/2] crash_core, vmcoreinfo: Append 'MAX_PHYSMEM_BITS' to vmcoreinfo
From: Catalin Marinas @ 2020-07-02 11:00 UTC (permalink / raw)
To: Bhupesh Sharma
Cc: Mark Rutland, Kazuhito Hagio, Will Deacon, x86, kexec,
linux-kernel, Paul Mackerras, James Morse, Boris Petkov,
Thomas Gleixner, bhupesh.linux, linuxppc-dev, Ingo Molnar,
linux-arm-kernel, Dave Anderson
In-Reply-To: <1589395957-24628-2-git-send-email-bhsharma@redhat.com>
On Thu, May 14, 2020 at 12:22:36AM +0530, Bhupesh Sharma wrote:
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index 9f1557b98468..18175687133a 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -413,6 +413,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> VMCOREINFO_STRUCT_SIZE(mem_section);
> VMCOREINFO_OFFSET(mem_section, section_mem_map);
> + VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
> #endif
> VMCOREINFO_STRUCT_SIZE(page);
> VMCOREINFO_STRUCT_SIZE(pglist_data);
I can queue this patch via the arm64 tree (together with the second one)
but I'd like an ack from the kernel/crash_core.c maintainers. They don't
seem to have been cc'ed either (only the kexec list).
Thanks.
--
Catalin
^ permalink raw reply
* [powerpc][next-20200701] Hung task timeouts during regression test runs
From: Sachin Sant @ 2020-07-02 11:23 UTC (permalink / raw)
To: linuxppc-dev, linux-block; +Cc: axboe, Linux Next Mailing List, ming.lei
Starting with linux-next 20200701 release I am observing automated regressions
tests taking longer time to complete. A test which took 10 minutes with next-20200630
took more than 60 minutes against next-20200701.
Following hung task timeout messages were seen during these runs
[ 1718.848351] Not tainted 5.8.0-rc3-next-20200701-autotest #1
[ 1718.848356] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1718.848362] NetworkManager D 0 2626 1 0x00040080
[ 1718.848367] Call Trace:
[ 1718.848374] [c0000008b0f6b8f0] [c000000000c6d558] schedule+0x78/0x130 (unreliable)
[ 1718.848382] [c0000008b0f6bad0] [c00000000001b070] __switch_to+0x2e0/0x480
[ 1718.848388] [c0000008b0f6bb30] [c000000000c6ce9c] __schedule+0x2cc/0x910
[ 1718.848394] [c0000008b0f6bc10] [c000000000c6d558] schedule+0x78/0x130
[ 1718.848401] [c0000008b0f6bc40] [c0000000005d5a64] jbd2_log_wait_commit+0xd4/0x1a0
[ 1718.848408] [c0000008b0f6bcc0] [c00000000055fb6c] ext4_sync_file+0x1cc/0x480
[ 1718.848415] [c0000008b0f6bd20] [c000000000493530] vfs_fsync_range+0x70/0xf0
[ 1718.848421] [c0000008b0f6bd60] [c000000000493638] do_fsync+0x58/0xd0
[ 1718.848427] [c0000008b0f6bda0] [c0000000004936d8] sys_fsync+0x28/0x40
[ 1718.848433] [c0000008b0f6bdc0] [c000000000035e28] system_call_exception+0xf8/0x1c0
[ 1718.848440] [c0000008b0f6be20] [c00000000000ca70] system_call_common+0xf0/0x278
Comparing next-20200630 with next-20200701 one possible candidate seems to
be following commit:
commit 37f4a24c2469a10a4c16c641671bd766e276cf9f
blk-mq: centralise related handling into blk_mq_get_driver_tag
Reverting this commit allows the test to complete in 10 minutes.
Thanks
-Sachin
^ permalink raw reply
* Re: [PATCH 01/11] kexec_file: allow archs to handle special regions while locating memory hole
From: Dave Young @ 2020-07-02 11:47 UTC (permalink / raw)
To: Hari Bathini
Cc: Pingfan Liu, Petr Tesarik, Kexec-ml, Mahesh J Salgaonkar,
Mimi Zohar, lkml, linuxppc-dev, Sourabh Jain, Vivek Goyal,
Andrew Morton, Thiago Jung Bauermann, Eric Biederman
In-Reply-To: <0e145e84-a6cf-4da3-1a1a-331a7e1ac1fa@linux.ibm.com>
On 07/02/20 at 12:01am, Hari Bathini wrote:
>
>
> On 01/07/20 1:16 pm, Dave Young wrote:
> > On 06/29/20 at 05:26pm, Hari Bathini wrote:
> >> Hi Petr,
> >>
> >> On 29/06/20 5:09 pm, Petr Tesarik wrote:
> >>> Hi Hari,
> >>>
> >>> is there any good reason to add two more functions with a very similar
> >>> name to an existing function? AFAICS all you need is a way to call a
> >>> PPC64-specific function from within kexec_add_buffer (PATCH 4/11), so
> >>> you could add something like this:
> >>>
> >>> int __weak arch_kexec_locate_mem_hole(struct kexec_buf *kbuf)
> >>> {
> >>> return 0;
> >>> }
> >>>
> >>> Call this function from kexec_add_buffer where appropriate and then
> >>> override it for PPC64 (it roughly corresponds to your
> >>> kexec_locate_mem_hole_ppc64() from PATCH 4/11).
> >>>
> >>> FWIW it would make it easier for me to follow the resulting code.
> >>
> >> Right, Petr.
> >>
> >> I was trying out a few things before I ended up with what I sent here.
> >> Bu yeah.. I did realize arch_kexec_locate_mem_hole() would have been better
> >> after sending out v1. Will take care of that in v2.
> >
> > Another way is use arch private function to locate mem hole, then set
> > kbuf->mem, and then call kexec_add_buf, it will skip the common locate
> > hole function.
>
> Dave, I did think about it. But there are a couple of places this can get
> tricky. One is ima_add_kexec_buffer() and the other is kexec_elf_load().
> These call sites could be updated to set kbuf->mem before kexec_add_buffer().
> But the current approach seemed like the better option for it creates a
> single point of control in setting up segment buffers and also, makes adding
> any new segments simpler, arch-specific segments or otherwise.
>
Ok, thanks for the explanation.
^ permalink raw reply
* [PATCH 2/2] powerpc/kasan: Fix shadow pages allocation failure
From: Christophe Leroy @ 2020-07-02 11:52 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
erhard_f
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <3667deb0911affbf999b99f87c31c77d5e870cd2.1593690707.git.christophe.leroy@csgroup.eu>
Doing kasan pages allocation in MMU_init is too early, kernel doesn't
have access yet to the entire memory space and memblock_alloc() fails
when the kernel is a bit big.
Do it from kasan_init() instead.
Reported-by: Erhard F. <erhard_f@mailbox.org>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=208181
Fixes: 2edb16efc899 ("powerpc/32: Add KASAN support")
Fixes: d2a91cef9bbd ("powerpc/kasan: Fix shadow pages allocation failure")
Cc: stable@vger.kernel.org
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/mm/kasan/kasan_init_32.c | 25 +++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/arch/powerpc/mm/kasan/kasan_init_32.c b/arch/powerpc/mm/kasan/kasan_init_32.c
index 4813c6d50889..019b0c0bbbf3 100644
--- a/arch/powerpc/mm/kasan/kasan_init_32.c
+++ b/arch/powerpc/mm/kasan/kasan_init_32.c
@@ -120,11 +120,24 @@ static void __init kasan_unmap_early_shadow_vmalloc(void)
void __init kasan_mmu_init(void)
{
int ret;
+
+ if (early_mmu_has_feature(MMU_FTR_HPTE_TABLE) ||
+ IS_ENABLED(CONFIG_KASAN_VMALLOC)) {
+ ret = kasan_init_shadow_page_tables(KASAN_SHADOW_START, KASAN_SHADOW_END);
+
+ if (ret)
+ panic("kasan: kasan_init_shadow_page_tables() failed");
+ }
+}
+
+void __init kasan_init(void)
+{
struct memblock_region *reg;
for_each_memblock(memory, reg) {
phys_addr_t base = reg->base;
phys_addr_t top = min(base + reg->size, total_lowmem);
+ int ret;
if (base >= top)
continue;
@@ -134,18 +147,6 @@ void __init kasan_mmu_init(void)
panic("kasan: kasan_init_region() failed");
}
- if (early_mmu_has_feature(MMU_FTR_HPTE_TABLE) ||
- IS_ENABLED(CONFIG_KASAN_VMALLOC)) {
- ret = kasan_init_shadow_page_tables(KASAN_SHADOW_START, KASAN_SHADOW_END);
-
- if (ret)
- panic("kasan: kasan_init_shadow_page_tables() failed");
- }
-
-}
-
-void __init kasan_init(void)
-{
kasan_remap_early_shadow_ro();
clear_page(kasan_early_shadow_page);
--
2.25.0
^ permalink raw reply related
* [PATCH 1/2] Revert "powerpc/kasan: Fix shadow pages allocation failure"
From: Christophe Leroy @ 2020-07-02 11:52 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
erhard_f
Cc: linuxppc-dev, linux-kernel
This reverts commit d2a91cef9bbdeb87b7449fdab1a6be6000930210.
This commit moved too much work in kasan_init(). The allocation
of shadow pages has to be moved for the reason explained in that
patch, but the allocation of page tables still need to be done
before switching to the final hash table.
First revert the incorrect commit, following patch redoes it
properly.
Reported-by: Erhard F. <erhard_f@mailbox.org>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=208181
Fixes: d2a91cef9bbd ("powerpc/kasan: Fix shadow pages allocation failure")
Cc: stable@vger.kernel.org
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/include/asm/kasan.h | 2 ++
arch/powerpc/mm/init_32.c | 2 ++
arch/powerpc/mm/kasan/kasan_init_32.c | 4 +---
3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/kasan.h b/arch/powerpc/include/asm/kasan.h
index be85c7005fb1..d635b96c7ea6 100644
--- a/arch/powerpc/include/asm/kasan.h
+++ b/arch/powerpc/include/asm/kasan.h
@@ -27,10 +27,12 @@
#ifdef CONFIG_KASAN
void kasan_early_init(void);
+void kasan_mmu_init(void);
void kasan_init(void);
void kasan_late_init(void);
#else
static inline void kasan_init(void) { }
+static inline void kasan_mmu_init(void) { }
static inline void kasan_late_init(void) { }
#endif
diff --git a/arch/powerpc/mm/init_32.c b/arch/powerpc/mm/init_32.c
index 5a5469eb3174..bf1717f8d5f4 100644
--- a/arch/powerpc/mm/init_32.c
+++ b/arch/powerpc/mm/init_32.c
@@ -171,6 +171,8 @@ void __init MMU_init(void)
btext_unmap();
#endif
+ kasan_mmu_init();
+
setup_kup();
/* Shortly after that, the entire linear mapping will be available */
diff --git a/arch/powerpc/mm/kasan/kasan_init_32.c b/arch/powerpc/mm/kasan/kasan_init_32.c
index 0760e1e754e4..4813c6d50889 100644
--- a/arch/powerpc/mm/kasan/kasan_init_32.c
+++ b/arch/powerpc/mm/kasan/kasan_init_32.c
@@ -117,7 +117,7 @@ static void __init kasan_unmap_early_shadow_vmalloc(void)
kasan_update_early_region(k_start, k_end, __pte(0));
}
-static void __init kasan_mmu_init(void)
+void __init kasan_mmu_init(void)
{
int ret;
struct memblock_region *reg;
@@ -146,8 +146,6 @@ static void __init kasan_mmu_init(void)
void __init kasan_init(void)
{
- kasan_mmu_init();
-
kasan_remap_early_shadow_ro();
clear_page(kasan_early_shadow_page);
--
2.25.0
^ permalink raw reply related
* Re: [powerpc][next-20200701] Hung task timeouts during regression test runs
From: Ming Lei @ 2020-07-02 11:52 UTC (permalink / raw)
To: Sachin Sant; +Cc: linux-block, axboe, Linux Next Mailing List, linuxppc-dev
In-Reply-To: <CDAB3931-FAAD-443A-A9CD-362E527043A1@linux.vnet.ibm.com>
On Thu, Jul 02, 2020 at 04:53:04PM +0530, Sachin Sant wrote:
> Starting with linux-next 20200701 release I am observing automated regressions
> tests taking longer time to complete. A test which took 10 minutes with next-20200630
> took more than 60 minutes against next-20200701.
>
> Following hung task timeout messages were seen during these runs
>
> [ 1718.848351] Not tainted 5.8.0-rc3-next-20200701-autotest #1
> [ 1718.848356] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 1718.848362] NetworkManager D 0 2626 1 0x00040080
> [ 1718.848367] Call Trace:
> [ 1718.848374] [c0000008b0f6b8f0] [c000000000c6d558] schedule+0x78/0x130 (unreliable)
> [ 1718.848382] [c0000008b0f6bad0] [c00000000001b070] __switch_to+0x2e0/0x480
> [ 1718.848388] [c0000008b0f6bb30] [c000000000c6ce9c] __schedule+0x2cc/0x910
> [ 1718.848394] [c0000008b0f6bc10] [c000000000c6d558] schedule+0x78/0x130
> [ 1718.848401] [c0000008b0f6bc40] [c0000000005d5a64] jbd2_log_wait_commit+0xd4/0x1a0
> [ 1718.848408] [c0000008b0f6bcc0] [c00000000055fb6c] ext4_sync_file+0x1cc/0x480
> [ 1718.848415] [c0000008b0f6bd20] [c000000000493530] vfs_fsync_range+0x70/0xf0
> [ 1718.848421] [c0000008b0f6bd60] [c000000000493638] do_fsync+0x58/0xd0
> [ 1718.848427] [c0000008b0f6bda0] [c0000000004936d8] sys_fsync+0x28/0x40
> [ 1718.848433] [c0000008b0f6bdc0] [c000000000035e28] system_call_exception+0xf8/0x1c0
> [ 1718.848440] [c0000008b0f6be20] [c00000000000ca70] system_call_common+0xf0/0x278
>
> Comparing next-20200630 with next-20200701 one possible candidate seems to
> be following commit:
>
> commit 37f4a24c2469a10a4c16c641671bd766e276cf9f
> blk-mq: centralise related handling into blk_mq_get_driver_tag
>
> Reverting this commit allows the test to complete in 10 minutes.
Hello,
Thanks for the report.
Please try the following fix:
https://lore.kernel.org/linux-block/20200702062041.GC2452799@T590/raw
Thanks,
Ming
^ permalink raw reply
* Re: [PATCH 04/11] ppc64/kexec_file: avoid stomping memory used by special regions
From: Dave Young @ 2020-07-02 11:54 UTC (permalink / raw)
To: Hari Bathini
Cc: Thiago Jung Bauermann, Pingfan Liu, Kexec-ml, Petr Tesarik, lkml,
Sourabh Jain, Mahesh J Salgaonkar, linuxppc-dev, Mimi Zohar,
Andrew Morton, Vivek Goyal, Eric Biederman
In-Reply-To: <6e96ae5a-91fd-726e-1eda-314f2317d8b4@linux.ibm.com>
On 07/01/20 at 11:48pm, Hari Bathini wrote:
>
>
> On 01/07/20 1:10 pm, Dave Young wrote:
> > Hi Hari,
> > On 06/27/20 at 12:35am, Hari Bathini wrote:
> >> crashkernel region could have an overlap with special memory regions
> >> like opal, rtas, tce-table & such. These regions are referred to as
> >> exclude memory ranges. Setup this ranges during image probe in order
> >> to avoid them while finding the buffer for different kdump segments.
> >> Implement kexec_locate_mem_hole_ppc64() that locates a memory hole
> >> accounting for these ranges. Also, override arch_kexec_add_buffer()
> >> to locate a memory hole & later call __kexec_add_buffer() function
> >> with kbuf->mem set to skip the generic locate memory hole lookup.
> >>
> >> Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
> >> ---
> >> arch/powerpc/include/asm/crashdump-ppc64.h | 10 +
> >> arch/powerpc/include/asm/kexec.h | 7 -
> >> arch/powerpc/kexec/elf_64.c | 7 +
> >> arch/powerpc/kexec/file_load_64.c | 292 ++++++++++++++++++++++++++++
> >> 4 files changed, 312 insertions(+), 4 deletions(-)
> >> create mode 100644 arch/powerpc/include/asm/crashdump-ppc64.h
> >>
> > [snip]
> >> /**
> >> + * get_exclude_memory_ranges - Get exclude memory ranges. This list includes
> >> + * regions like opal/rtas, tce-table, initrd,
> >> + * kernel, htab which should be avoided while
> >> + * setting up kexec load segments.
> >> + * @mem_ranges: Range list to add the memory ranges to.
> >> + *
> >> + * Returns 0 on success, negative errno on error.
> >> + */
> >> +static int get_exclude_memory_ranges(struct crash_mem **mem_ranges)
> >> +{
> >> + int ret;
> >> +
> >> + ret = add_tce_mem_ranges(mem_ranges);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_initrd_mem_range(mem_ranges);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_htab_mem_range(mem_ranges);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_kernel_mem_range(mem_ranges);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_rtas_mem_range(mem_ranges, false);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_opal_mem_range(mem_ranges, false);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + ret = add_reserved_ranges(mem_ranges);
> >> + if (ret)
> >> + goto out;
> >> +
> >> + /* exclude memory ranges should be sorted for easy lookup */
> >> + sort_memory_ranges(*mem_ranges);
> >> +out:
> >> + if (ret)
> >> + pr_err("Failed to setup exclude memory ranges\n");
> >> + return ret;
> >> +}
> >
> > I'm confused about the "overlap with crashkernel memory", does that mean
> > those normal kernel used memory could be put in crashkernel reserved
>
> There are regions that could overlap with crashkernel region but they are
> not normal kernel used memory though. These are regions that kernel and/or
> f/w chose to place at a particular address for real mode accessibility
> and/or memory layout between kernel & f/w kind of thing.
>
> > memory range? If so why can't just skip those areas while crashkernel
> > doing the reservation?
>
> crashkernel region has a dependency to be in the first memory block for it
> to be accessible in real mode. Accommodating this requirement while addressing
> other requirements would mean something like what we have now. A list of
> possible special memory regions in crashkernel region to take care of.
>
> I have plans to split crashkernel region into low & high to have exclusive
> regions for crashkernel, even if that means to have two of them. But that
> is for another day with its own set of complexities to deal with...
Ok, I was not aware the powerpc crashkernel reservation is not
dynamically reserved. But seems powerpc need those tricks at least for
the time being like you said.
Thanks
Dave
^ permalink raw reply
* Re: [PATCH 04/11] ppc64/kexec_file: avoid stomping memory used by special regions
From: Dave Young @ 2020-07-02 11:59 UTC (permalink / raw)
To: piliu
Cc: Thiago Jung Bauermann, Kexec-ml, Mahesh J Salgaonkar,
Petr Tesarik, lkml, linuxppc-dev, Sourabh Jain, Vivek Goyal,
Andrew Morton, Mimi Zohar, Hari Bathini, Eric Biederman
In-Reply-To: <bc8fe308-5ce9-1ca6-c832-7c3a75a732d2@redhat.com>
> > I'm confused about the "overlap with crashkernel memory", does that mean
> > those normal kernel used memory could be put in crashkernel reserved
> > memory range? If so why can't just skip those areas while crashkernel
> > doing the reservation?
> I raised the same question in another mail. As Hari's answer, "kexec -p"
> skips these ranges in user space. And the same logic should be done in
> "kexec -s -p"
See it, thanks! The confusion also applied to the userspace
implementation though. Seems they have to be special cases because of
the powerpc crashkernel reservation implemtation in kernel limitation
Thanks
Dave
^ permalink raw reply
* [Bug 208181] BUG: KASAN: stack-out-of-bounds in strcmp+0x58/0xd8
From: bugzilla-daemon @ 2020-07-02 12:00 UTC (permalink / raw)
To: linuxppc-dev
In-Reply-To: <bug-208181-206035@https.bugzilla.kernel.org/>
https://bugzilla.kernel.org/show_bug.cgi?id=208181
--- Comment #11 from Christophe Leroy (christophe.leroy@csgroup.eu) ---
The issue is that that commit moved more code than described into kasan_init():
Kasan Pages allocation have to be moved into kasan_init() but page tables
allocation must remain before the switch to the final hash table.
Problem only occurs on book3s/32 having hash MMU.
See proposed fix at
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=187165 (2
patches).
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply
* Re: [PATCH v6 1/2] crash_core, vmcoreinfo: Append 'MAX_PHYSMEM_BITS' to vmcoreinfo
From: Dave Young @ 2020-07-02 12:08 UTC (permalink / raw)
To: Catalin Marinas
Cc: Mark Rutland, Kazuhito Hagio, Bhupesh Sharma, x86, kexec,
linux-kernel, linuxppc-dev, Paul Mackerras, Boris Petkov,
James Morse, Thomas Gleixner, bhupesh.linux, Will Deacon,
Ingo Molnar, linux-arm-kernel, Dave Anderson
In-Reply-To: <20200702110003.GC22241@gaia>
Hi Catalin,
On 07/02/20 at 12:00pm, Catalin Marinas wrote:
> On Thu, May 14, 2020 at 12:22:36AM +0530, Bhupesh Sharma wrote:
> > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > index 9f1557b98468..18175687133a 100644
> > --- a/kernel/crash_core.c
> > +++ b/kernel/crash_core.c
> > @@ -413,6 +413,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> > VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> > VMCOREINFO_STRUCT_SIZE(mem_section);
> > VMCOREINFO_OFFSET(mem_section, section_mem_map);
> > + VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
> > #endif
> > VMCOREINFO_STRUCT_SIZE(page);
> > VMCOREINFO_STRUCT_SIZE(pglist_data);
>
> I can queue this patch via the arm64 tree (together with the second one)
> but I'd like an ack from the kernel/crash_core.c maintainers. They don't
> seem to have been cc'ed either (only the kexec list).
For the VMCOREINFO part, I'm fine with the changes, but since I do not
understand the arm64 pieces so I would like to leave to arm64 people to
review. If arm64 bits are good enough, feel free to add:
Acked-by: Dave Young <dyoung@redhat.com>
Thanks
Dave
^ permalink raw reply
* Re: [powerpc][next-20200701] Hung task timeouts during regression test runs
From: Sachin Sant @ 2020-07-02 13:16 UTC (permalink / raw)
To: Ming Lei; +Cc: linux-block, axboe, Linux Next Mailing List, linuxppc-dev
In-Reply-To: <20200702115216.GF2452799@T590>
> On 02-Jul-2020, at 5:22 PM, Ming Lei <ming.lei@redhat.com> wrote:
>
> On Thu, Jul 02, 2020 at 04:53:04PM +0530, Sachin Sant wrote:
>> Starting with linux-next 20200701 release I am observing automated regressions
>> tests taking longer time to complete. A test which took 10 minutes with next-20200630
>> took more than 60 minutes against next-20200701.
>>
>> Following hung task timeout messages were seen during these runs
>>
>> [ 1718.848351] Not tainted 5.8.0-rc3-next-20200701-autotest #1
>> [ 1718.848356] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> [ 1718.848362] NetworkManager D 0 2626 1 0x00040080
>> [ 1718.848367] Call Trace:
>> [ 1718.848374] [c0000008b0f6b8f0] [c000000000c6d558] schedule+0x78/0x130 (unreliable)
>> [ 1718.848382] [c0000008b0f6bad0] [c00000000001b070] __switch_to+0x2e0/0x480
>> [ 1718.848388] [c0000008b0f6bb30] [c000000000c6ce9c] __schedule+0x2cc/0x910
>> [ 1718.848394] [c0000008b0f6bc10] [c000000000c6d558] schedule+0x78/0x130
>> [ 1718.848401] [c0000008b0f6bc40] [c0000000005d5a64] jbd2_log_wait_commit+0xd4/0x1a0
>> [ 1718.848408] [c0000008b0f6bcc0] [c00000000055fb6c] ext4_sync_file+0x1cc/0x480
>> [ 1718.848415] [c0000008b0f6bd20] [c000000000493530] vfs_fsync_range+0x70/0xf0
>> [ 1718.848421] [c0000008b0f6bd60] [c000000000493638] do_fsync+0x58/0xd0
>> [ 1718.848427] [c0000008b0f6bda0] [c0000000004936d8] sys_fsync+0x28/0x40
>> [ 1718.848433] [c0000008b0f6bdc0] [c000000000035e28] system_call_exception+0xf8/0x1c0
>> [ 1718.848440] [c0000008b0f6be20] [c00000000000ca70] system_call_common+0xf0/0x278
>>
>> Comparing next-20200630 with next-20200701 one possible candidate seems to
>> be following commit:
>>
>> commit 37f4a24c2469a10a4c16c641671bd766e276cf9f
>> blk-mq: centralise related handling into blk_mq_get_driver_tag
>>
>> Reverting this commit allows the test to complete in 10 minutes.
>
> Hello,
>
> Thanks for the report.
>
> Please try the following fix:
>
> https://lore.kernel.org/linux-block/20200702062041.GC2452799@T590/raw
The fix works for me.
Tested-by : Sachin Sant <sachinp@linux.vnet.ibm.com>
Thanks
-Sachin
>
>
> Thanks,
> Ming
^ permalink raw reply
* Re: [PATCH V3 (RESEND) 2/3] mm/sparsemem: Enable vmem_altmap support in vmemmap_alloc_block_buf()
From: Catalin Marinas @ 2020-07-02 14:07 UTC (permalink / raw)
To: Anshuman Khandual
Cc: x86, H. Peter Anvin, Peter Zijlstra, Dave Hansen, linuxppc-dev,
linux-kernel, linux-mm, Ingo Molnar, Paul Mackerras,
Andy Lutomirski, Borislav Petkov, Thomas Gleixner, Will Deacon,
Andrew Morton, linux-arm-kernel
In-Reply-To: <1592442930-9380-3-git-send-email-anshuman.khandual@arm.com>
On Thu, Jun 18, 2020 at 06:45:29AM +0530, Anshuman Khandual wrote:
> There are many instances where vmemap allocation is often switched between
> regular memory and device memory just based on whether altmap is available
> or not. vmemmap_alloc_block_buf() is used in various platforms to allocate
> vmemmap mappings. Lets also enable it to handle altmap based device memory
> allocation along with existing regular memory allocations. This will help
> in avoiding the altmap based allocation switch in many places.
>
> While here also implement a regular memory allocation fallback mechanism
> when the first preferred device memory allocation fails. This will ensure
> preserving the existing semantics on powerpc platform. To summarize there
> are three different methods to call vmemmap_alloc_block_buf().
>
> (., NULL, false) /* Allocate from system RAM */
> (., altmap, false) /* Allocate from altmap without any fallback */
> (., altmap, true) /* Allocate from altmap with fallback (system RAM) */
[...]
> diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
> index bc73abf0bc25..01e25b56eccb 100644
> --- a/arch/powerpc/mm/init_64.c
> +++ b/arch/powerpc/mm/init_64.c
> @@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
> * fall back to system memory if the altmap allocation fail.
> */
> if (altmap && !altmap_cross_boundary(altmap, start, page_size)) {
> - p = altmap_alloc_block_buf(page_size, altmap);
> - if (!p)
> - pr_debug("altmap block allocation failed, falling back to system memory");
> + p = vmemmap_alloc_block_buf(page_size, node,
> + altmap, true);
> + } else {
> + p = vmemmap_alloc_block_buf(page_size, node,
> + NULL, false);
> }
> - if (!p)
> - p = vmemmap_alloc_block_buf(page_size, node);
> if (!p)
> return -ENOMEM;
Is the fallback argument actually necessary. It may be cleaner to just
leave the code as is with the choice between altmap and NULL. If an arch
needs a fallback (only powerpc), they have the fallback in place
already. I don't see the powerpc code any better after this change.
I'm fine with the altmap argument though.
--
Catalin
^ permalink raw reply
* [PATCH] docs: powerpc: Clarify book3s/32 MMU families
From: Christophe Leroy @ 2020-07-02 14:09 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, corbet
Cc: linuxppc-dev, linux-kernel, linux-doc
Documentation wrongly tells that book3s/32 CPU have hash MMU.
603 and e300 core only have software loaded TLB.
755, 7450 family and e600 core have both hash MMU and software loaded
TLB. This can be selected by setting a bit in HID2 (755) or
HID0 (others). At the time being this is not supported by the kernel.
Make this explicit in the documentation.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
Documentation/powerpc/cpu_families.rst | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/Documentation/powerpc/cpu_families.rst b/Documentation/powerpc/cpu_families.rst
index 1e063c5440c3..9b84e045e713 100644
--- a/Documentation/powerpc/cpu_families.rst
+++ b/Documentation/powerpc/cpu_families.rst
@@ -9,7 +9,9 @@ and are supported by arch/powerpc.
Book3S (aka sPAPR)
------------------
-- Hash MMU
+- Hash MMU (except 603 and e300)
+- Software loaded TLB (603 and e300)
+- Selectable Software loaded TLB in addition to hash MMU (755, 7450, e600)
- Mix of 32 & 64 bit::
+--------------+ +----------------+
@@ -24,9 +26,9 @@ Book3S (aka sPAPR)
| |
| |
v v
- +--------------+ +----------------+ +-------+
- | 604 | | 750 (G3) | ---> | 750CX |
- +--------------+ +----------------+ +-------+
+ +--------------+ +-----+ +----------------+ +-------+
+ | 604 | | 755 | <--- | 750 (G3) | ---> | 750CX |
+ +--------------+ +-----+ +----------------+ +-------+
| | |
| | |
v v v
--
2.25.0
^ permalink raw reply related
* Re: [PATCH 18/20] block: refator submit_bio_noacct
From: Qian Cai @ 2020-07-02 14:10 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, linux-xtensa, linux-nvdimm, linux-s390, linux-m68k,
linux-nvme, linux-kernel, linux-raid, dm-devel, linux-bcache,
linuxppc-dev, drbd-dev
In-Reply-To: <20200629193947.2705954-19-hch@lst.de>
On Mon, Jun 29, 2020 at 09:39:45PM +0200, Christoph Hellwig wrote:
> Split out a __submit_bio_noacct helper for the actual de-recursion
> algorithm, and simplify the loop by using a continue when we can't
> enter the queue for a bio.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
Reverting this commit and its dependencies,
5a6c35f9af41 block: remove direct_make_request
ff93ea0ce763 block: shortcut __submit_bio_noacct for blk-mq drivers
fixed the stack-out-of-bounds during boot,
https://lore.kernel.org/linux-block/000000000000bcdeaa05a97280e4@google.com/
[ 55.573431][ T1373] BUG: KASAN: stack-out-of-bounds in bio_alloc_bioset+0x493/0x4a0
bio_alloc_bioset+0x493/0x4a0:
bio_list_empty at include/linux/bio.h:561
(inlined by) bio_alloc_bioset at block/bio.c:482
[ 55.581140][ T1373] Read of size 8 at addr ffffc9000a7df1e0 by task mount/1373
[ 55.588409][ T1373]
[ 55.590615][ T1373] CPU: 2 PID: 1373 Comm: mount Not tainted 5.8.0-rc3-next-20200702 #2
[ 55.598672][ T1373] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 07/10/2019
[ 55.607972][ T1373] Call Trace:
[ 55.607980][ T1373] dump_stack+0x9d/0xe0
[ 55.607984][ T1373] ? bio_alloc_bioset+0x493/0x4a0
[ 55.607992][ T1373] ? bio_alloc_bioset+0x493/0x4a0
[ 55.625007][ T1373] print_address_description.constprop.8.cold.10+0x56/0x44e
[ 55.632191][ T1373] ? bio_alloc_bioset+0x493/0x4a0
[ 55.637100][ T1373] ? bio_alloc_bioset+0x493/0x4a0
[ 55.642011][ T1373] kasan_report.cold.11+0x37/0x7c
[ 55.646923][ T1373] ? bio_alloc_bioset+0x493/0x4a0
[ 55.651968][ T1373] bio_alloc_bioset+0x493/0x4a0
[ 55.651971][ T1373] ? bvec_alloc+0x290/0x290
[ 55.651975][ T1373] ? mark_lock+0x147/0x1800
[ 55.651978][ T1373] ? mark_lock+0x147/0x1800
[ 55.651981][ T1373] bio_clone_fast+0xe/0x30
[ 55.651983][ T1373] bio_split+0x8a/0x4c0
[ 55.651986][ T1373] ? print_irqtrace_events+0x270/0x270
[ 55.651990][ T1373] __blk_queue_split+0xc42/0x13e0
[ 55.651998][ T1373] ? __lock_acquire+0xc57/0x4da0
Startin[ 55.693322][ T1373] ? __blk_rq_map_sg+0x14c0/0x14c0
[ 55.699711][ T1373] ? lockdep_hardirqs_on_prepare+0x550/0x550
[ 55.705602][ T1373] ? mark_held_locks+0xb0/0x110
[ 55.705605][ T1373] ? lockdep_hardirqs_on_prepare+0x550/0x550
[ 55.705608][ T1373] ? lockdep_hardirqs_on_prepare+0x550/0x550
[ 55.705611][ T1373] ? find_held_lock+0x33/0x1c0
[ 55.705614][ T1373] ? find_held_lock+0x33/0x1c0
[ 55.705618][ T1373] blk_mq_submit_bio+0x19e/0x1e20
[ 55.705621][ T1373] ? lock_downgrade+0x720/0x720
[ 55.705624][ T1373] ? blk_mq_try_issue_directly+0x140/0x140
[ 55.705628][ T1373] ? rcu_read_lock_sched_held+0xaa/0xd0
[ 55.705631][ T1373] ? rcu_read_lock_bh_held+0xc0/0xc0
[ 55.705635][ T1373] ? blk_queue_enter+0x83c/0x9a0
[ 55.705647][ T1373] ? submit_bio_checks+0x1cc0/0x1cc0
[ 55.767384][ T1373] submit_bio_noacct+0x9c0/0xeb0
[ 55.772212][ T1373] ? blk_queue_enter+0x9a0/0x9a0
[ 55.777038][ T1373] ? lockdep_hardirqs_on_prepare+0x550/0x550
[ 55.782913][ T1373] ? trace_hardirqs_on+0x20/0x1b5
[ 55.787825][ T1373] ? submit_bio+0xe7/0x480
[ 55.792125][ T1373] submit_bio+0xe7/0x480
[ 55.796252][ T1373] ? bio_associate_blkg_from_css+0x4a3/0xd30
[ 55.802124][ T1373] ? submit_bio_noacct+0xeb0/0xeb0
[ 55.807124][ T1373] ? lock_downgrade+0x720/0x720
[ 55.811862][ T1373] ? rcu_read_unlock+0x50/0x50
[ 55.816512][ T1373] ? lockdep_init_map_waits+0x267/0x7b0
[ 55.821948][ T1373] ? lockdep_init_map_waits+0x267/0x7b0
g LVM event acti[ 55.827386][ T1373] ? __raw_spin_lock_init+0x34/0x100
[ 55.833957][ T1373] submit_bio_wait+0xf9/0x200
vation on device[ 55.838521][ T1373] ? submit_bio_wait_endio+0x30/0x30
[ 55.845091][ T1373] xfs_rw_bdev+0x3ca/0x4d0
[ 55.849396][ T1373] xlog_do_io+0x149/0x320
[ 55.853611][ T1373] xlog_bread+0x1e/0xb0
[ 55.857651][ T1373] xlog_find_verify_log_record+0xba/0x4c0
[ 55.863264][ T1373] ? xlog_header_check_mount+0xb0/0xb0
[ 55.868615][ T1373] xlog_find_zeroed+0x2bc/0x4c0
8:3...
[ 55.873356][ T1373] ? print_irqtrace_events+0x270/0x270
[ 55.880093][ T1373] ? xlog_find_verify_log_record+0x4c0/0x4c0
[ 55.885966][ T1373] ? __lock_acquire+0x1920/0x4da0
[ 55.890881][ T1373] xlog_find_head+0xd4/0x790
[ 55.895355][ T1373] ? xlog_find_zeroed+0x4c0/0x4c0
[ 55.900269][ T1373] ? rcu_read_lock_sched_held+0xaa/0xd0
[ 55.905708][ T1373] ? rcu_read_lock_bh_held+0xc0/0xc0
[ 55.910885][ T1373] ? sugov_update_single+0x18d/0x4f0
[ 55.916058][ T1373] xlog_find_tail+0xc2/0x810
[ 55.920534][ T1373] ? mark_lock+0x147/0x1800
[ 55.924921][ T1373] ? xlog_verify_head+0x4c0/0x4c0
[ 55.929834][ T1373] ? debug_show_held_locks+0x30/0x50
[ 55.935007][ T1373] ? print_irqtrace_events+0x270/0x270
[ 55.940358][ T1373] ? try_to_wake_up+0x6d1/0xf40
[ 55.945094][ T1373] ? mark_held_locks+0xb0/0x110
[ 55.949835][ T1373] ? lockdep_hardirqs_on_prepare+0x38c/0x550
[ 55.955708][ T1373] ? _raw_spin_unlock_irqrestore+0x39/0x40
[ 55.961410][ T1373] ? trace_hardirqs_on+0x20/0x1b5
[ 55.966324][ T1373] xlog_recover+0x7c/0x480
[ 55.970627][ T1373] ? xlog_buf_readahead+0x110/0x110
[ 55.975715][ T1373] ? migrate_swap_stop+0xbf0/0xbf0
[ 55.980718][ T1373] ? lockdep_init_map_waits+0x267/0x7b0
[ 55.986156][ T1373] ? __raw_spin_lock_init+0x34/0x100
[ 55.991333][ T1373] xfs_log_mount+0x541/0x660
[ 55.995809][ T1373] xfs_mountfs+0xccd/0x1a00
[ 56.000202][ T1373] ? queue_work_node+0x190/0x190
[ 56.005028][ T1373] ? rcu_read_lock_sched_held+0xaa/0xd0
[ 56.010466][ T1373] ? xfs_default_resblks+0x50/0x50
[ 56.015464][ T1373] ? xfs_filestream_get_parent+0xa0/0xa0
[ 56.020989][ T1373] ? init_timer_key+0x285/0x320
[ 56.025727][ T1373] ? lockdep_init_map_waits+0x267/0x7b0
[ 56.031165][ T1373] ? xfs_filestream_get_parent+0xa0/0xa0
[ 56.036689][ T1373] ? xfs_mru_cache_create+0x358/0x560
[ 56.041951][ T1373] xfs_fc_fill_super+0x6d3/0xd50
[ 56.046777][ T1373] get_tree_bdev+0x40a/0x690
[ 56.051257][ T1373] ? xfs_fs_inode_init_once+0xc0/0xc0
[ 56.056523][ T1373] vfs_get_tree+0x84/0x2c0
[ 56.060827][ T1373] do_mount+0xf93/0x1630
[ 56.064953][ T1373] ? rcu_read_lock_bh_held+0xc0/0xc0
[ 56.070129][ T1373] ? copy_mount_string+0x20/0x20
[ 56.074956][ T1373] ? _copy_from_user+0xbe/0x100
[ 56.079696][ T1373] ? memdup_user+0x4f/0x80
[ 56.083999][ T1373] __x64_sys_mount+0x15d/0x1b0
2m OK ] St[ 56.088654][ T1373] do_syscall_64+0x5f/0x310
[ 56.094437][ T1373] ? trace_hardirqs_off+0x12/0x1a0
[ 56.099439][ T1373] ? asm_exc_page_fault+0x8/0x30
[ 56.104267][ T1373] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 56.110055][ T1373] RIP: 0033:0x7f3bc2c8a9ee
[ 56.114357][ T1373] Code: Bad RIP value.
[ 56.118309][ T1373] RSP: 002b:00007fffd4675718 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
arted File Syste[ 56.126629][ T1373] RAX: ffffffffffffffda RBX: 000055a59d34c9c0 RCX: 00007f3bc2c8a9ee
[ 56.135900][ T1373] RDX: 000055a59d34cba0 RSI: 000055a59d34cc00 RDI: 000055a59d34e900
[ 56.143779][ T1373] RBP: 00007f3bc3a36184 R08: 0000000000000000 R09: 0000000000000003
[ 56.151661][ T1373] R10: 00000000c0ed0000 R11: 0000000000000246 R12: 0000000000000000
[ 56.159541][ T1373] R13: 00000000c0ed0000 R14: 000055a59d34e900 R15: 000055a59d34cba0
[ 56.167422][ T1373]
[ 56.169626][ T1373]
[ 56.171831][ T1373] addr ffffc9000a7df1e0 is located in stack of task mount/1373 at offset 48 in frame:
[ 56.181287][ T1373] submit_bio_noacct+0x0/0xeb0
submit_bio_noacct at block/blk-core.c:1198
[ 56.185939][ T1373]
[ 56.188144][ T1373] this frame has 2 objects:
m Check on /dev/[ 56.192532][ T1373] [32, 48) 'bio_list'
[ 56.192534][ T1373] [96, 128) 'bio_list_on_stack'
[ 56.197872][ T1373]
[ 56.204894][ T1373] Memory state around the buggy address:
[ 56.210420][ T1373] ffffc9000a7df080: f2 f2 f2 f2 f2 00 f2 f2 f2 f2 f2 f2 f2 00 00 00
[ 56.218389][ T1373] ffffc9000a7df100: 00 00 f2 f2 f2 00 00 00 00 00 00 00 00 00 00 00
disk/by-uuid/D10[ 56.226359][ T1373] >ffffc9000a7df180: 00 00 00 00 00 00 f1 f1 f1 f1 00 00 f2 f2 f2 f2
[ 56.235718][ T1373] ^
[ 56.242817][ T1373] ffffc9000a7df200: f2 f2 00 00 00 00 f3 f3 f3 f3 00 00 00 00 00 00
[ 56.250790][ T1373] ffffc9000a7df280: 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f2
[ 56.258757][ T1373] ==================================================================
> ---
> block/blk-core.c | 131 +++++++++++++++++++++++++----------------------
> 1 file changed, 71 insertions(+), 60 deletions(-)
>
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 1caeb01e127768..b82f48c86e6f7a 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -1085,6 +1085,74 @@ static blk_qc_t do_make_request(struct bio *bio)
> return ret;
> }
>
> +/*
> + * The loop in this function may be a bit non-obvious, and so deserves some
> + * explanation:
> + *
> + * - Before entering the loop, bio->bi_next is NULL (as all callers ensure
> + * that), so we have a list with a single bio.
> + * - We pretend that we have just taken it off a longer list, so we assign
> + * bio_list to a pointer to the bio_list_on_stack, thus initialising the
> + * bio_list of new bios to be added. ->submit_bio() may indeed add some more
> + * bios through a recursive call to submit_bio_noacct. If it did, we find a
> + * non-NULL value in bio_list and re-enter the loop from the top.
> + * - In this case we really did just take the bio of the top of the list (no
> + * pretending) and so remove it from bio_list, and call into ->submit_bio()
> + * again.
> + *
> + * bio_list_on_stack[0] contains bios submitted by the current ->submit_bio.
> + * bio_list_on_stack[1] contains bios that were submitted before the current
> + * ->submit_bio_bio, but that haven't been processed yet.
> + */
> +static blk_qc_t __submit_bio_noacct(struct bio *bio)
> +{
> + struct bio_list bio_list_on_stack[2];
> + blk_qc_t ret = BLK_QC_T_NONE;
> +
> + BUG_ON(bio->bi_next);
> +
> + bio_list_init(&bio_list_on_stack[0]);
> + current->bio_list = bio_list_on_stack;
> +
> + do {
> + struct request_queue *q = bio->bi_disk->queue;
> + struct bio_list lower, same;
> +
> + if (unlikely(bio_queue_enter(bio) != 0))
> + continue;
> +
> + /*
> + * Create a fresh bio_list for all subordinate requests.
> + */
> + bio_list_on_stack[1] = bio_list_on_stack[0];
> + bio_list_init(&bio_list_on_stack[0]);
> +
> + ret = do_make_request(bio);
> +
> + /*
> + * Sort new bios into those for a lower level and those for the
> + * same level.
> + */
> + bio_list_init(&lower);
> + bio_list_init(&same);
> + while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL)
> + if (q == bio->bi_disk->queue)
> + bio_list_add(&same, bio);
> + else
> + bio_list_add(&lower, bio);
> +
> + /*
> + * Now assemble so we handle the lowest level first.
> + */
> + bio_list_merge(&bio_list_on_stack[0], &lower);
> + bio_list_merge(&bio_list_on_stack[0], &same);
> + bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]);
> + } while ((bio = bio_list_pop(&bio_list_on_stack[0])));
> +
> + current->bio_list = NULL;
> + return ret;
> +}
> +
> /**
> * submit_bio_noacct - re-submit a bio to the block device layer for I/O
> * @bio: The bio describing the location in memory and on the device.
> @@ -1096,17 +1164,8 @@ static blk_qc_t do_make_request(struct bio *bio)
> */
> blk_qc_t submit_bio_noacct(struct bio *bio)
> {
> - /*
> - * bio_list_on_stack[0] contains bios submitted by the current
> - * ->submit_bio.
> - * bio_list_on_stack[1] contains bios that were submitted before the
> - * current ->submit_bio_bio, but that haven't been processed yet.
> - */
> - struct bio_list bio_list_on_stack[2];
> - blk_qc_t ret = BLK_QC_T_NONE;
> -
> if (!submit_bio_checks(bio))
> - goto out;
> + return BLK_QC_T_NONE;
>
> /*
> * We only want one ->submit_bio to be active at a time, else
> @@ -1120,58 +1179,10 @@ blk_qc_t submit_bio_noacct(struct bio *bio)
> */
> if (current->bio_list) {
> bio_list_add(¤t->bio_list[0], bio);
> - goto out;
> + return BLK_QC_T_NONE;
> }
>
> - /* following loop may be a bit non-obvious, and so deserves some
> - * explanation.
> - * Before entering the loop, bio->bi_next is NULL (as all callers
> - * ensure that) so we have a list with a single bio.
> - * We pretend that we have just taken it off a longer list, so
> - * we assign bio_list to a pointer to the bio_list_on_stack,
> - * thus initialising the bio_list of new bios to be
> - * added. ->submit_bio() may indeed add some more bios
> - * through a recursive call to submit_bio_noacct. If it
> - * did, we find a non-NULL value in bio_list and re-enter the loop
> - * from the top. In this case we really did just take the bio
> - * of the top of the list (no pretending) and so remove it from
> - * bio_list, and call into ->submit_bio() again.
> - */
> - BUG_ON(bio->bi_next);
> - bio_list_init(&bio_list_on_stack[0]);
> - current->bio_list = bio_list_on_stack;
> - do {
> - struct request_queue *q = bio->bi_disk->queue;
> -
> - if (likely(bio_queue_enter(bio) == 0)) {
> - struct bio_list lower, same;
> -
> - /* Create a fresh bio_list for all subordinate requests */
> - bio_list_on_stack[1] = bio_list_on_stack[0];
> - bio_list_init(&bio_list_on_stack[0]);
> - ret = do_make_request(bio);
> -
> - /* sort new bios into those for a lower level
> - * and those for the same level
> - */
> - bio_list_init(&lower);
> - bio_list_init(&same);
> - while ((bio = bio_list_pop(&bio_list_on_stack[0])) != NULL)
> - if (q == bio->bi_disk->queue)
> - bio_list_add(&same, bio);
> - else
> - bio_list_add(&lower, bio);
> - /* now assemble so we handle the lowest level first */
> - bio_list_merge(&bio_list_on_stack[0], &lower);
> - bio_list_merge(&bio_list_on_stack[0], &same);
> - bio_list_merge(&bio_list_on_stack[0], &bio_list_on_stack[1]);
> - }
> - bio = bio_list_pop(&bio_list_on_stack[0]);
> - } while (bio);
> - current->bio_list = NULL; /* deactivate */
> -
> -out:
> - return ret;
> + return __submit_bio_noacct(bio);
> }
> EXPORT_SYMBOL(submit_bio_noacct);
>
> --
> 2.26.2
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox