From: Bjorn Andersson <bjorn.andersson@linaro.org>
To: Mark Brown <broonie@kernel.org>
Cc: Andy Gross <agross@kernel.org>,
kernel-build-reports@lists.linaro.org,
linux-next@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>
Subject: Re: next/master boot: 257 boots: 8 failed, 237 passed with 8 offline, 2 untried/unknown, 2 conflicts (next-20191028)
Date: Mon, 28 Oct 2019 13:02:19 -0700 [thread overview]
Message-ID: <20191028200219.GS571@minitux> (raw)
In-Reply-To: <20191028191121.GH5015@sirena.co.uk>
On Mon 28 Oct 12:11 PDT 2019, Mark Brown wrote:
> On Mon, Oct 28, 2019 at 11:40:19AM -0700, Bjorn Andersson wrote:
> > On Mon 28 Oct 10:48 PDT 2019, Mark Brown wrote:
> > > On Mon, Oct 28, 2019 at 08:03:08AM -0700, kernelci.org bot wrote:
>
> > > Today's -next (anf Friday's) fails to boot on db820c:
>
> > > > defconfig:
> > > > gcc-8:
> > > > apq8096-db820c: 1 failed lab
>
> > > It looks like it deadlocks somewhere, the last things in the log are a
> > > failure to start ufshcd-qcom and then an RCU stall some time later:
>
> > db820c has been failing intermittently for a while now, it seems that
> > booting with kpti enabled causes something to go wrong. There are
> > nothing strange in the kernel logs and ftrace seems to indicate that all
> > the CPUs are idling nicely.
>
> Oh dear. Adding Catalin and Will. Is it definitely KPTI that's
> triggering stuff? It did turn up some bugs on other systems, though
> it's a bit strange it's only manifesting in KernelCI...
I did a test recently where I booted my db820c 100 times with kpti=yes
and 100 times with kpti=no on the kernel command line, and the result
was 90% failure to reach console vs 0%. Going back and looking at the
logs for the 10% indicated that the boot CPU was fine, but I had stalls
reported on other CPUs.
In an effort to rule out driver bugs I reduced the DT to CPUs, the core
clocks, gic, timers and serial driver, and I still saw the problem.
I have not looked at this with jtag and hence do not know what secure
world is doing.
Regards,
Bjorn
WARNING: multiple messages have this Message-ID (diff)
From: Bjorn Andersson <bjorn.andersson@linaro.org>
To: Mark Brown <broonie@kernel.org>
Cc: kernel-build-reports@lists.linaro.org,
linux-arm-msm@vger.kernel.org, Andy Gross <agross@kernel.org>,
linux-next@vger.kernel.org,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
linux-arm-kernel@lists.infradead.org
Subject: Re: next/master boot: 257 boots: 8 failed, 237 passed with 8 offline, 2 untried/unknown, 2 conflicts (next-20191028)
Date: Mon, 28 Oct 2019 13:02:19 -0700 [thread overview]
Message-ID: <20191028200219.GS571@minitux> (raw)
In-Reply-To: <20191028191121.GH5015@sirena.co.uk>
On Mon 28 Oct 12:11 PDT 2019, Mark Brown wrote:
> On Mon, Oct 28, 2019 at 11:40:19AM -0700, Bjorn Andersson wrote:
> > On Mon 28 Oct 10:48 PDT 2019, Mark Brown wrote:
> > > On Mon, Oct 28, 2019 at 08:03:08AM -0700, kernelci.org bot wrote:
>
> > > Today's -next (anf Friday's) fails to boot on db820c:
>
> > > > defconfig:
> > > > gcc-8:
> > > > apq8096-db820c: 1 failed lab
>
> > > It looks like it deadlocks somewhere, the last things in the log are a
> > > failure to start ufshcd-qcom and then an RCU stall some time later:
>
> > db820c has been failing intermittently for a while now, it seems that
> > booting with kpti enabled causes something to go wrong. There are
> > nothing strange in the kernel logs and ftrace seems to indicate that all
> > the CPUs are idling nicely.
>
> Oh dear. Adding Catalin and Will. Is it definitely KPTI that's
> triggering stuff? It did turn up some bugs on other systems, though
> it's a bit strange it's only manifesting in KernelCI...
I did a test recently where I booted my db820c 100 times with kpti=yes
and 100 times with kpti=no on the kernel command line, and the result
was 90% failure to reach console vs 0%. Going back and looking at the
logs for the 10% indicated that the boot CPU was fine, but I had stalls
reported on other CPUs.
In an effort to rule out driver bugs I reduced the DT to CPUs, the core
clocks, gic, timers and serial driver, and I still saw the problem.
I have not looked at this with jtag and hence do not know what secure
world is doing.
Regards,
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-10-28 20:02 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <5db7032c.1c69fb81.888b0.b521@mx.google.com>
2019-10-28 17:48 ` next/master boot: 257 boots: 8 failed, 237 passed with 8 offline, 2 untried/unknown, 2 conflicts (next-20191028) Mark Brown
2019-10-28 17:48 ` Mark Brown
2019-10-28 18:40 ` Bjorn Andersson
2019-10-28 18:40 ` Bjorn Andersson
2019-10-28 19:11 ` Mark Brown
2019-10-28 19:11 ` Mark Brown
2019-10-28 20:02 ` Bjorn Andersson [this message]
2019-10-28 20:02 ` Bjorn Andersson
2019-10-28 20:14 ` Will Deacon
2019-10-28 20:14 ` Will Deacon
2019-10-28 20:23 ` Mark Brown
2019-10-28 20:23 ` Mark Brown
2019-10-28 15:03 kernelci.org bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191028200219.GS571@minitux \
--to=bjorn.andersson@linaro.org \
--cc=agross@kernel.org \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=kernel-build-reports@lists.linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.