All of lore.kernel.org
 help / color / mirror / Atom feed
From: Shrikanth Hegde <sshegde@linux.ibm.com>
To: "Michal Suchánek" <msuchanek@suse.de>
Cc: "Christophe Leroy (CS GROUP)" <chleroy@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	harshpb@linux.ibm.com, Gautam Menghani <gautam@linux.ibm.com>,
	amachhiw@linux.ibm.com,
	Christian Zigotzky <chzigotzky@xenosoft.de>,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	"kvm-ppc@vger.kernel.org" <kvm-ppc@vger.kernel.org>,
	Madhavan Srinivasan <maddy@linux.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	"R.T.Dickinson" <rtd2@xtra.co.nz>,
	hypexed@yahoo.com.au, mad skateman <madskateman@gmail.com>,
	Christian Zigotzky <info@xenosoft.de>,
	Samir M <samir@linux.ibm.com>,
	Venkat Rao Bagalkote <venkat88@linux.ibm.com>,
	Narayana Murty N <nnmlinux@linux.ibm.com>,
	Hari Bathini <hbathini@linux.ibm.com>
Subject: Re: [Linux PPC] Disable PREEMPT
Date: Thu, 17 Sep 2026 15:58:42 +0530	[thread overview]
Message-ID: <b2a5d53f-05e8-48ae-adb7-12fc74523b4e@linux.ibm.com> (raw)
In-Reply-To: <aqu9hA1Bq58GFrVw@kunlun.suse.cz>

Hi Michal,

On 9/17/26 3:44 PM, Michal Suchánek wrote:
> Hello,
> 
> On Wed, Sep 16, 2026 at 11:41:38PM +0530, Shrikanth Hegde wrote:
>> Hi Michal,
>>
>>>
>>> It seems easy to re-create.
>>> The key is running bcc/bpf program in parallel to kernel build.
>>>
>>> For example, these two in parallal, leads to stack corruption panic.
>>> Stacktrace does differ sometimes.
>>>
>>> ./funccount  rcu* -d 100
>>> make -j 32
>>>
>>> Boom. Will try to debug this further.
>>
>> A bit more debugging.
>>
>> 1. A few functions seems to crash, specially ones that call schedule.
>>     So picked schedule itself for tracing
>> 2. Used plain ftrace. No crash.
>> 3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
>> 4. Did some preempt count validation with annotations. Seems to be holding fine.
>> 5. Since unlock can call schedule while tracing schedule, there is possible looping.
>>     Need to see further why it happens to be crashing there. Maybe regs or paca
>>     handling needs a closer look.
>>
>> For now, below patch helps to keep the system stable under kprobes, bcc tools.
>> I will look further into see why exactly it is crashing and below one is going to
>> be the proper fix or not.
>>
>> Feel free to give it a try.
> 
> thanks for the patch.
> 
> While it is mangled by the e-mail client in different way compared to
> the previous patch it does seem to mitigate the issue.

Thanks for trying.

> 
> As it is unlikely the board where KVM lockup was originally observed is
> running kpbobes it will probably not address the original problem.
> 

Give it a try.

I am suspecting both lockup, stack corruptions are secondary symptoms here.
While recreate, I have different type of traces.

> Thanks
> 
> Michal
> 
>>
>> ---
>>   arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
>> index f8208c027148..bcb1e8a8995d 100644
>> --- a/arch/powerpc/kernel/kprobes-ftrace.c
>> +++ b/arch/powerpc/kernel/kprobes-ftrace.c
>> @@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
>>   		__this_cpu_write(current_kprobe, NULL);
>>   	}
>>   out:
>> +	preempt_disable_notrace();
>>   	ftrace_test_recursion_unlock(bit);
>> +	preempt_enable_no_resched_notrace();
>>   }
>>   NOKPROBE_SYMBOL(kprobe_ftrace_handler);
>> -- 
>> 2.52.0


  reply	other threads:[~2026-09-17 10:29 UTC|newest]

Thread overview: 78+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-14 14:02 [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT Christian Zigotzky
2026-02-17  5:02 ` Shrikanth Hegde
2026-02-17  8:52   ` Christian Zigotzky
2026-02-17  9:03     ` Shrikanth Hegde
2026-02-17  9:44       ` Christian Zigotzky
2026-02-17 10:17         ` Madhavan Srinivasan
2026-02-17 10:26           ` Christian Zigotzky
2026-02-17 11:34             ` Shrikanth Hegde
2026-02-21  5:30               ` Christian Zigotzky
2026-02-23  3:45                 ` Shrikanth Hegde
2026-02-23 16:48                   ` [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' Christian Zigotzky
2026-02-23 18:07                     ` Christophe Leroy (CS GROUP)
2026-02-23 19:38                       ` Christophe Leroy (CS GROUP)
2026-02-23 22:40                         ` Kees Cook
2026-02-24  6:30                           ` Christian Zigotzky
2026-02-24  7:18                           ` Christophe Leroy (CS GROUP)
2026-02-25  5:54               ` [Linux PPC] Disable PREEMPT Christian Zigotzky
2026-02-25  6:23                 ` Shrikanth Hegde
2026-02-25  7:16                 ` Christophe Leroy (CS GROUP)
2026-02-25 11:36                   ` Peter Zijlstra
2026-02-26  5:37                   ` Christian Zigotzky
2026-02-26 13:24                     ` Christophe Leroy (CS GROUP)
2026-09-03  9:53                       ` Michal Suchánek
2026-09-03 10:56                         ` Shrikanth Hegde
2026-09-03 11:25                           ` Michal Suchánek
2026-09-03 11:31                             ` Shrikanth Hegde
2026-09-03 11:40                               ` Michal Suchánek
2026-09-03 12:04                                 ` Shrikanth Hegde
2026-09-03 12:25                                   ` Michal Suchánek
2026-09-03 12:36                                     ` Harsh Prateek Bora
2026-09-03 12:55                                       ` Michal Suchánek
2026-09-03 15:17                                         ` Amit Machhiwal
2026-09-03 13:28                                       ` Michal Suchánek
2026-09-04  7:39                                         ` Michal Suchánek
2026-09-21  8:06                                         ` Michal Suchánek
2026-09-21 16:46                                           ` Michal Suchánek
2026-09-21 17:20                                             ` Shrikanth Hegde
2026-09-22  8:22                                               ` Michal Suchánek
2026-09-03 17:22                             ` Shrikanth Hegde
2026-09-04  7:35                               ` Michal Suchánek
2026-09-04  9:04                                 ` Shrikanth Hegde
2026-09-10  8:59                                   ` Michal Suchánek
2026-09-10 10:41                                     ` Shrikanth Hegde
2026-09-10 11:29                                       ` Michal Suchánek
2026-09-10 12:16                                         ` Michal Suchánek
2026-09-11 17:09                                           ` Shrikanth Hegde
2026-09-14 10:58                                             ` Michal Suchánek
2026-09-15  5:47                                               ` Narayana Murty N
2026-09-15  7:35                                                 ` Michal Suchánek
2026-09-15  9:08                                                   ` Shrikanth Hegde
2026-09-16 18:11                                                     ` Shrikanth Hegde
2026-09-17 10:14                                                       ` Michal Suchánek
2026-09-17 10:28                                                         ` Shrikanth Hegde [this message]
2026-09-17 13:12                                                           ` Michal Suchánek
2026-09-17 14:15                                                             ` Shrikanth Hegde
2026-09-18  2:39                                                               ` Christian Zigotzky
2026-09-18  7:10                                                                 ` Michal Suchánek
2026-09-20 11:49                                                                   ` Christian Zigotzky
2026-09-20 13:04                                                                     ` Christian Zigotzky
2026-09-21  5:12                                                                       ` Shrikanth Hegde
2026-09-23  5:09                                                                         ` Christian Zigotzky
2026-09-23  5:13                                                                           ` Shrikanth Hegde
2026-09-23 16:01                                                                             ` Christian Zigotzky
2026-09-23 18:42                                                                               ` Shrikanth Hegde
2026-09-24 17:47                                                                                 ` Christian Zigotzky
2026-09-25  6:40                                                                                   ` Shrikanth Hegde
2026-09-18  9:26                                                               ` Shrikanth Hegde
2026-09-18 11:52                                                                 ` Michal Suchánek
2026-09-04  8:55                               ` Christian Zigotzky
2026-09-04  9:20                                 ` Christian Zigotzky
2026-09-04  9:24                                   ` Shrikanth Hegde
2026-09-04  9:34                                     ` Christian Zigotzky
2026-09-05 13:07                                       ` Christian Zigotzky
2026-09-07  8:50                                         ` Michal Suchánek
2026-09-07 11:32                                           ` Christian Zigotzky
2026-09-10  8:54                                             ` Michal Suchánek
2026-09-10 10:31                                               ` Michal Suchánek
2026-09-10 11:31                                                 ` Michal Suchánek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b2a5d53f-05e8-48ae-adb7-12fc74523b4e@linux.ibm.com \
    --to=sshegde@linux.ibm.com \
    --cc=amachhiw@linux.ibm.com \
    --cc=chleroy@kernel.org \
    --cc=chzigotzky@xenosoft.de \
    --cc=gautam@linux.ibm.com \
    --cc=harshpb@linux.ibm.com \
    --cc=hbathini@linux.ibm.com \
    --cc=hypexed@yahoo.com.au \
    --cc=info@xenosoft.de \
    --cc=kvm-ppc@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=maddy@linux.ibm.com \
    --cc=madskateman@gmail.com \
    --cc=mpe@ellerman.id.au \
    --cc=msuchanek@suse.de \
    --cc=nnmlinux@linux.ibm.com \
    --cc=peterz@infradead.org \
    --cc=rtd2@xtra.co.nz \
    --cc=samir@linux.ibm.com \
    --cc=venkat88@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.