public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
To: Guenter Roeck <linux@roeck-us.net>
Cc: Michael Ellerman <mpe@ellerman.id.au>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	ajd@linux.ibm.com, aneesh.kumar@linux.ibm.com,
	atrajeev@linux.vnet.ibm.com, christophe.leroy@csgroup.eu,
	cuigaosheng1@huawei.com, david@redhat.com, farosas@linux.ibm.com,
	geoff@infradead.org, gustavoars@kernel.org, haren@linux.ibm.com,
	hbathini@linux.ibm.com, joel@jms.id.au, lihuafei1@huawei.com,
	linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
	lukas.bulwahn@gmail.com, mikey@neuling.org, nathan@kernel.org,
	nathanl@linux.ibm.com, nicholas@linux.ibm.com, npiggin@gmail.com,
	pali@kernel.org, paul@paul-moore.com, rmclure@linux.ibm.com,
	ruscur@russell.cc, windhl@126.com,
	wsa+renesas@sang-engineering.com, ye.xingchen@zte.com.cn,
	yuanjilin@cdjrlc.com, zhengyongjun3@huawei.com
Subject: Re: [GIT PULL] Please pull powerpc/linux.git powerpc-6.1-1 tag
Date: Wed, 12 Oct 2022 12:37:50 -0600	[thread overview]
Message-ID: <Y0cJfoPLGeuzm/gQ@zx2c4.com> (raw)
In-Reply-To: <20221012174826.GB2995920@roeck-us.net>

On Wed, Oct 12, 2022 at 10:48:26AM -0700, Guenter Roeck wrote:
> > I've also managed to not hit this bug a few times. When it triggers,
> > after "kprobes: kprobe jump-optimization is enabled. All kprobes are
> > optimized if possible.", there's a long hang - tens seconds before it
> > continues. When it doesn't trigger, there's no hang at that point in the
> > boot process.
> > 
> 
> That probably explains why my attempts to bisect the problem were
> unsuccessful.

So I just did this:

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 2fe28eeb2f38..2d70bc09db7e 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1212,6 +1212,7 @@ static void __cold try_to_generate_entropy(void)
        struct entropy_timer_state stack;
        unsigned int i, num_different = 0;
        unsigned long last = random_get_entropy();
+       return;

        for (i = 0; i < NUM_TRIAL_SAMPLES - 1; ++i) {
                stack.entropy = random_get_entropy();

And then ran it, and now we get the lockup from the idle process:

udhcpc: started, v1.33.0
udhcpc: sending discover
watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:0]
Modules linked in:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.0.0-28380-gde492c83cae0-dirty #10
Hardware name: PowerMac3,1 PPC970FX 0x3c0301 PowerMac
NIP:  c0000000000300f8 LR: c0000000000304e8 CTR: c00000000001a410
REGS: c0000000028c79a8 TRAP: 0900   Not tainted  (6.0.0-28380-gde492c83cae0-dirty)
MSR:  800000000000b032 <SF,EE,FP,ME,IR,DR,RI>  CR: 24088442  XER: 00000000
IRQMASK: 0
GPR00: c0000000000304e8 c0000000028c7b30 c000000001435500 c0000000028c79a8
GPR04: c0000000013366c0 0000000000000000 000000000010029c 0000000000000000
GPR08: c000000002d3bbb0 0000000000000000 c000000002883d00 c000000002915500
GPR12: 0000000044088442 c000000002e00000 0000000000000007 0000000002295698
GPR16: 00000000039400e8 0000000002295258 0000000002295660 00000000022953d0
GPR20: 0000000002295b10 00000000022b34d0 0000000002295b38 0000000003945500
GPR24: 0000000003945500 0000000000080000 c000000002883d80 c000000002883d00
GPR28: c00000000290d0c0 0000000000000001 c00000000290d018 c00000000290cc78
NIP [c0000000000300f8] .replay_soft_interrupts+0x28/0x2d0
LR [c0000000000304e8] .arch_local_irq_restore+0x148/0x1a0
Call Trace:
[c0000000028c7b30] [c0000000000304e8] .arch_local_irq_restore+0x148/0x1a0 (unreliable)
[c0000000028c7bb0] [c00000000001a388] .arch_cpu_idle+0xb8/0x140
[c0000000028c7c30] [c000000000fd4940] .default_idle_call+0x80/0xc8
[c0000000028c7ca0] [c000000000148480] .do_idle+0x150/0x1a0
[c0000000028c7d50] [c000000000148748] .cpu_startup_entry+0x38/0x40
[c0000000028c7dd0] [c0000000000113a8] .rest_init+0x168/0x170
[c0000000028c7e60] [c000000002004224] .arch_post_acpi_subsys_init+0x0/0x24
[c0000000028c7ed0] [c000000002004ba8] .start_kernel+0x8d0/0x924
[c0000000028c7f90] [c00000000000d4ac] start_here_common+0x1c/0x20
Instruction dump:
60000000 60000000 7c0802a6 f8010010 f821fe01 60000000 60000000 38610078
e92d0af8 f92101f8 39200000 4803a491 <60000000> 39200000 e9410180 f92101b0
Kernel panic - not syncing: softlockup: hung tasks
CPU: 0 PID: 0 Comm: swapper/0 Tainted: G             L     6.0.0-28380-gde492c83cae0-dirty #10
Hardware name: PowerMac3,1 PPC970FX 0x3c0301 PowerMac
Call Trace:
[c0000000028c74a0] [c000000000f93b90] .dump_stack_lvl+0x7c/0xc4 (unreliable)
[c0000000028c7530] [c0000000000d2a58] .panic+0x180/0x438
[c0000000028c75e0] [c000000000232424] .watchdog_timer_fn+0x3a4/0x410
[c0000000028c76a0] [c0000000001cb964] .__hrtimer_run_queues+0x1f4/0x590
[c0000000028c77a0] [c0000000001cc354] .hrtimer_interrupt+0x134/0x300
[c0000000028c7860] [c000000000021cd4] .timer_interrupt+0x1c4/0x5d0
[c0000000028c7930] [c0000000000302f8] .replay_soft_interrupts+0x228/0x2d0
[c0000000028c7b30] [c0000000000304e8] .arch_local_irq_restore+0x148/0x1a0
[c0000000028c7bb0] [c00000000001a388] .arch_cpu_idle+0xb8/0x140
[c0000000028c7c30] [c000000000fd4940] .default_idle_call+0x80/0xc8
[c0000000028c7ca0] [c000000000148480] .do_idle+0x150/0x1a0
[c0000000028c7d50] [c000000000148748] .cpu_startup_entry+0x38/0x40
[c0000000028c7dd0] [c0000000000113a8] .rest_init+0x168/0x170
[c0000000028c7e60] [c000000002004224] .arch_post_acpi_subsys_init+0x0/0x24
[c0000000028c7ed0] [c000000002004ba8] .start_kernel+0x8d0/0x924
[c0000000028c7f90] [c00000

  reply	other threads:[~2022-10-12 18:38 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-09 11:01 [GIT PULL] Please pull powerpc/linux.git powerpc-6.1-1 tag Michael Ellerman
2022-10-09 21:17 ` pr-tracker-bot
2022-10-10 19:25 ` Jason A. Donenfeld
2022-10-10 20:03   ` Jason A. Donenfeld
2022-10-10 22:26     ` Jason A. Donenfeld
2022-10-11  0:00       ` Andrew Donnellan
2022-10-11  0:13         ` Jason A. Donenfeld
2022-10-11  1:44           ` Michael Ellerman
2022-10-11  2:57             ` Jason A. Donenfeld
2022-10-11  9:34               ` Michael Ellerman
2022-10-11  1:53     ` Michael Ellerman
2022-10-11  2:57       ` Jason A. Donenfeld
2022-10-11  9:35         ` Michael Ellerman
2022-10-11 11:10           ` Nicholas Piggin
2022-10-12 14:18 ` Guenter Roeck
2022-10-12 15:49   ` Jason A. Donenfeld
2022-10-12 16:44     ` Guenter Roeck
2022-10-12 17:20       ` Jason A. Donenfeld
2022-10-12 17:48         ` Guenter Roeck
2022-10-12 18:37           ` Jason A. Donenfeld [this message]
2022-10-13  5:17             ` Nicholas Piggin
2022-10-12 22:16         ` Guenter Roeck
2022-10-13  0:03           ` Michael Ellerman
2022-10-13  0:21             ` Guenter Roeck
2022-10-13  5:03               ` Nicholas Piggin
2022-10-13  5:19                 ` Jason A. Donenfeld
2022-10-13  5:20                 ` Guenter Roeck
2022-10-13  5:22               ` Nicholas Piggin
2022-10-13  4:43         ` Guenter Roeck
2022-10-13  5:14           ` Nicholas Piggin
2022-10-13 18:55             ` Guenter Roeck
2022-10-12 16:45     ` Jason A. Donenfeld
2022-10-12 16:49       ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Y0cJfoPLGeuzm/gQ@zx2c4.com \
    --to=jason@zx2c4.com \
    --cc=ajd@linux.ibm.com \
    --cc=aneesh.kumar@linux.ibm.com \
    --cc=atrajeev@linux.vnet.ibm.com \
    --cc=christophe.leroy@csgroup.eu \
    --cc=cuigaosheng1@huawei.com \
    --cc=david@redhat.com \
    --cc=farosas@linux.ibm.com \
    --cc=geoff@infradead.org \
    --cc=gustavoars@kernel.org \
    --cc=haren@linux.ibm.com \
    --cc=hbathini@linux.ibm.com \
    --cc=joel@jms.id.au \
    --cc=lihuafei1@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=lukas.bulwahn@gmail.com \
    --cc=mikey@neuling.org \
    --cc=mpe@ellerman.id.au \
    --cc=nathan@kernel.org \
    --cc=nathanl@linux.ibm.com \
    --cc=nicholas@linux.ibm.com \
    --cc=npiggin@gmail.com \
    --cc=pali@kernel.org \
    --cc=paul@paul-moore.com \
    --cc=rmclure@linux.ibm.com \
    --cc=ruscur@russell.cc \
    --cc=torvalds@linux-foundation.org \
    --cc=windhl@126.com \
    --cc=wsa+renesas@sang-engineering.com \
    --cc=ye.xingchen@zte.com.cn \
    --cc=yuanjilin@cdjrlc.com \
    --cc=zhengyongjun3@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox