Kernel KVM-PPC virtualization development
 help / color / mirror / Atom feed
From: Shrikanth Hegde <sshegde@linux.ibm.com>
To: "Michal Suchánek" <msuchanek@suse.de>
Cc: "Christophe Leroy (CS GROUP)" <chleroy@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	harshpb@linux.ibm.com, Gautam Menghani <gautam@linux.ibm.com>,
	amachhiw@linux.ibm.com,
	Christian Zigotzky <chzigotzky@xenosoft.de>,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	"kvm-ppc@vger.kernel.org" <kvm-ppc@vger.kernel.org>,
	Madhavan Srinivasan <maddy@linux.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	"R.T.Dickinson" <rtd2@xtra.co.nz>,
	hypexed@yahoo.com.au, mad skateman <madskateman@gmail.com>,
	Christian Zigotzky <info@xenosoft.de>,
	Samir M <samir@linux.ibm.com>,
	Venkat Rao Bagalkote <venkat88@linux.ibm.com>,
	Narayana Murty N <nnmlinux@linux.ibm.com>,
	Hari Bathini <hbathini@linux.ibm.com>
Subject: Re: [Linux PPC] Disable PREEMPT
Date: Thu, 17 Sep 2026 15:58:42 +0530	[thread overview]
Message-ID: <b2a5d53f-05e8-48ae-adb7-12fc74523b4e@linux.ibm.com> (raw)
In-Reply-To: <aqu9hA1Bq58GFrVw@kunlun.suse.cz>

Hi Michal,

On 9/17/26 3:44 PM, Michal Suchánek wrote:
> Hello,
> 
> On Wed, Sep 16, 2026 at 11:41:38PM +0530, Shrikanth Hegde wrote:
>> Hi Michal,
>>
>>>
>>> It seems easy to re-create.
>>> The key is running bcc/bpf program in parallel to kernel build.
>>>
>>> For example, these two in parallal, leads to stack corruption panic.
>>> Stacktrace does differ sometimes.
>>>
>>> ./funccount  rcu* -d 100
>>> make -j 32
>>>
>>> Boom. Will try to debug this further.
>>
>> A bit more debugging.
>>
>> 1. A few functions seems to crash, specially ones that call schedule.
>>     So picked schedule itself for tracing
>> 2. Used plain ftrace. No crash.
>> 3. Used kprobes. Boom. Crash appears. But this helped to avoid BPF JIT, or BCC part of it.
>> 4. Did some preempt count validation with annotations. Seems to be holding fine.
>> 5. Since unlock can call schedule while tracing schedule, there is possible looping.
>>     Need to see further why it happens to be crashing there. Maybe regs or paca
>>     handling needs a closer look.
>>
>> For now, below patch helps to keep the system stable under kprobes, bcc tools.
>> I will look further into see why exactly it is crashing and below one is going to
>> be the proper fix or not.
>>
>> Feel free to give it a try.
> 
> thanks for the patch.
> 
> While it is mangled by the e-mail client in different way compared to
> the previous patch it does seem to mitigate the issue.

Thanks for trying.

> 
> As it is unlikely the board where KVM lockup was originally observed is
> running kpbobes it will probably not address the original problem.
> 

Give it a try.

I am suspecting both lockup, stack corruptions are secondary symptoms here.
While recreate, I have different type of traces.

> Thanks
> 
> Michal
> 
>>
>> ---
>>   arch/powerpc/kernel/kprobes-ftrace.c | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/powerpc/kernel/kprobes-ftrace.c b/arch/powerpc/kernel/kprobes-ftrace.c
>> index f8208c027148..bcb1e8a8995d 100644
>> --- a/arch/powerpc/kernel/kprobes-ftrace.c
>> +++ b/arch/powerpc/kernel/kprobes-ftrace.c
>> @@ -63,7 +63,9 @@ void kprobe_ftrace_handler(unsigned long nip, unsigned long parent_nip,
>>   		__this_cpu_write(current_kprobe, NULL);
>>   	}
>>   out:
>> +	preempt_disable_notrace();
>>   	ftrace_test_recursion_unlock(bit);
>> +	preempt_enable_no_resched_notrace();
>>   }
>>   NOKPROBE_SYMBOL(kprobe_ftrace_handler);
>> -- 
>> 2.52.0


  reply	other threads:[~2026-09-17 10:29 UTC|newest]

Thread overview: 78+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-14 14:02 [Linux PPC] KVM PR and KVM HV do not work if the kernel was compiled with PREEMPT Christian Zigotzky
2026-02-17  5:02 ` Shrikanth Hegde
2026-02-17  8:52   ` Christian Zigotzky
2026-02-17  9:03     ` Shrikanth Hegde
2026-02-17  9:44       ` Christian Zigotzky
2026-02-17 10:17         ` Madhavan Srinivasan
2026-02-17 10:26           ` Christian Zigotzky
2026-02-17 11:34             ` Shrikanth Hegde
2026-02-21  5:30               ` Christian Zigotzky
2026-02-23  3:45                 ` Shrikanth Hegde
2026-02-23 16:48                   ` [Linux PPC] 7.0-rc1: e500_mmu.c:923:26: error: assignment to 'struct tlbe_priv *' from incompatible pointer type 'struct tlbe_ref *' Christian Zigotzky
2026-02-23 18:07                     ` Christophe Leroy (CS GROUP)
2026-02-23 19:38                       ` Christophe Leroy (CS GROUP)
2026-02-23 22:40                         ` Kees Cook
2026-02-24  6:30                           ` Christian Zigotzky
2026-02-24  7:18                           ` Christophe Leroy (CS GROUP)
2026-02-25  5:54               ` [Linux PPC] Disable PREEMPT Christian Zigotzky
2026-02-25  6:23                 ` Shrikanth Hegde
2026-02-25  7:16                 ` Christophe Leroy (CS GROUP)
2026-02-25 11:36                   ` Peter Zijlstra
2026-02-26  5:37                   ` Christian Zigotzky
2026-02-26 13:24                     ` Christophe Leroy (CS GROUP)
2026-09-03  9:53                       ` Michal Suchánek
2026-09-03 10:56                         ` Shrikanth Hegde
2026-09-03 11:25                           ` Michal Suchánek
2026-09-03 11:31                             ` Shrikanth Hegde
2026-09-03 11:40                               ` Michal Suchánek
2026-09-03 12:04                                 ` Shrikanth Hegde
2026-09-03 12:25                                   ` Michal Suchánek
2026-09-03 12:36                                     ` Harsh Prateek Bora
2026-09-03 12:55                                       ` Michal Suchánek
2026-09-03 15:17                                         ` Amit Machhiwal
2026-09-03 13:28                                       ` Michal Suchánek
2026-09-04  7:39                                         ` Michal Suchánek
2026-09-21  8:06                                         ` Michal Suchánek
2026-09-21 16:46                                           ` Michal Suchánek
2026-09-21 17:20                                             ` Shrikanth Hegde
2026-09-22  8:22                                               ` Michal Suchánek
2026-09-03 17:22                             ` Shrikanth Hegde
2026-09-04  7:35                               ` Michal Suchánek
2026-09-04  9:04                                 ` Shrikanth Hegde
2026-09-10  8:59                                   ` Michal Suchánek
2026-09-10 10:41                                     ` Shrikanth Hegde
2026-09-10 11:29                                       ` Michal Suchánek
2026-09-10 12:16                                         ` Michal Suchánek
2026-09-11 17:09                                           ` Shrikanth Hegde
2026-09-14 10:58                                             ` Michal Suchánek
2026-09-15  5:47                                               ` Narayana Murty N
2026-09-15  7:35                                                 ` Michal Suchánek
2026-09-15  9:08                                                   ` Shrikanth Hegde
2026-09-16 18:11                                                     ` Shrikanth Hegde
2026-09-17 10:14                                                       ` Michal Suchánek
2026-09-17 10:28                                                         ` Shrikanth Hegde [this message]
2026-09-17 13:12                                                           ` Michal Suchánek
2026-09-17 14:15                                                             ` Shrikanth Hegde
2026-09-18  2:39                                                               ` Christian Zigotzky
2026-09-18  7:10                                                                 ` Michal Suchánek
2026-09-20 11:49                                                                   ` Christian Zigotzky
2026-09-20 13:04                                                                     ` Christian Zigotzky
2026-09-21  5:12                                                                       ` Shrikanth Hegde
2026-09-23  5:09                                                                         ` Christian Zigotzky
2026-09-23  5:13                                                                           ` Shrikanth Hegde
2026-09-23 16:01                                                                             ` Christian Zigotzky
2026-09-23 18:42                                                                               ` Shrikanth Hegde
2026-09-24 17:47                                                                                 ` Christian Zigotzky
2026-09-25  6:40                                                                                   ` Shrikanth Hegde
2026-09-18  9:26                                                               ` Shrikanth Hegde
2026-09-18 11:52                                                                 ` Michal Suchánek
2026-09-04  8:55                               ` Christian Zigotzky
2026-09-04  9:20                                 ` Christian Zigotzky
2026-09-04  9:24                                   ` Shrikanth Hegde
2026-09-04  9:34                                     ` Christian Zigotzky
2026-09-05 13:07                                       ` Christian Zigotzky
2026-09-07  8:50                                         ` Michal Suchánek
2026-09-07 11:32                                           ` Christian Zigotzky
2026-09-10  8:54                                             ` Michal Suchánek
2026-09-10 10:31                                               ` Michal Suchánek
2026-09-10 11:31                                                 ` Michal Suchánek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b2a5d53f-05e8-48ae-adb7-12fc74523b4e@linux.ibm.com \
    --to=sshegde@linux.ibm.com \
    --cc=amachhiw@linux.ibm.com \
    --cc=chleroy@kernel.org \
    --cc=chzigotzky@xenosoft.de \
    --cc=gautam@linux.ibm.com \
    --cc=harshpb@linux.ibm.com \
    --cc=hbathini@linux.ibm.com \
    --cc=hypexed@yahoo.com.au \
    --cc=info@xenosoft.de \
    --cc=kvm-ppc@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=maddy@linux.ibm.com \
    --cc=madskateman@gmail.com \
    --cc=mpe@ellerman.id.au \
    --cc=msuchanek@suse.de \
    --cc=nnmlinux@linux.ibm.com \
    --cc=peterz@infradead.org \
    --cc=rtd2@xtra.co.nz \
    --cc=samir@linux.ibm.com \
    --cc=venkat88@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox