spacemit.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: "Troy Mitchell" <troy.mitchell@linux.spacemit.com>
To: "Alex Elder" <elder@riscstar.com>,
	"Manivannan Sadhasivam" <mani@kernel.org>,
	<spacemit@lists.linux.dev>, "Bruno Banelli" <bbanelli@gmail.com>,
	"Troy Mitchell" <troy.mitchell@linux.spacemit.com>
Cc: <linux-pci@vger.kernel.org>, <linux-riscv@lists.infradead.org>,
	"Yixun Lan" <dlan@kernel.org>,
	"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
	"Krzysztof Wilczynski" <kwilczynski@kernel.org>,
	"Rob Herring" <robh@kernel.org>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Uwe Kleine-Koenig" <u.kleine-koenig@baylibre.com>,
	"Javier Martinez Canillas" <javierm@redhat.com>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter
Date: Thu, 03 Sep 2026 09:06:25 +0800	[thread overview]
Message-ID: <DL5A18T0GPB5.32JI7ECB3UCRX@linux.spacemit.com> (raw)
In-Reply-To: <7a29717b-dc62-4421-b783-7b78e4375268@riscstar.com>

[-- Attachment #1: Type: text/plain, Size: 3897 bytes --]

On Wed Sep 2, 2026 at 9:56 PM +08, Alex Elder wrote:
> On 9/2/26 8:42 AM, Manivannan Sadhasivam wrote:
>> On Tue, Aug 25, 2026 at 07:22:49AM +0200, Bruno Banelli wrote:
>>> Hi,
>>>
>>> On a Milk-V Jupiter, probing the PCIe controller at ca800000 (port C, the
>>> card slot) permanently wedges the CPU that runs the probe when one
>>> particular add-in card is installed.  The CPU stops responding to NMI, and
>>> because the probe is asynchronous, kernel_init() then blocks forever in
>>> async_synchronize_full() and the machine never finishes booting.
>>>
>>> The same card, in the same slot, on the same board, does *not* hang the
>>> vendor 6.6 kernel -- it reports "Phy link never came up" and boots normally.
>>> Six other cards do not hang mainline either.  So whatever the electrical
>>> cause, this looks like a robustness problem in pcie-spacemit-k1: an endpoint
>>> should not be able to hang a host-side DBI register access.
>>>
>>>
>>> HARDWARE
>>> --------
>>>    Milk-V Jupiter v1.1, SpacemiT M1 (socinfo: CPU[M1-8571] REV[C] DRO[130]),
>>>    16 GiB LPDDR4X.
>>>    Firmware: stock vendor U-Boot 2022.10 (k1-bl-v2.2.9), unmodified.
>>>    Port B (ca400000, M.2) has a Samsung PM9B1 NVMe and works throughout.
>>>    Port C (ca800000) is the card slot -- an x8-length connector, silkscreened
>>>    PCIE_X2, wired x2.
>>>
>>>
>>> REPRODUCED ON
>>> -------------
>>>    v7.1 and v7.2, riscv defconfig (plus PHY_SPACEMIT_K1_USB2, USB_DWC3,
>>>    SPACEMIT_K1_TSENSOR, IGB, IGC, NVMe/ext4 built in).
>>>    gcc 13.3.0 (cross) and gcc 16.2.0 (native, Debian sid).
>>>    Identical failure in all combinations.  Not a regression -- port C has
>>>    never worked with this card on mainline.
>>>
>>> Command line:
>>>    console=ttyS0,115200 earlycon root=/dev/nvme0n1p2 rootwait rw
>>>    swiotlb=65536 clk_ignore_unused pd_ignore_unused
>>>
>>>
>>> SYMPTOM
>>> -------
>>> Port C prints its address ranges and then never speaks again.  (Log below is
>>> from a run with port B disabled in DT, so nothing is interleaved.)
>>>
>>>    [ 1.290074] spacemit-k1-pcie ca800000.pcie: host bridge /soc/pcie-bus/pcie@ca800000 ranges:
>>>    [ 1.297283] spacemit-k1-pcie ca800000.pcie:       IO 0x00b7002000..0x00b7101fff -> 0x0000000000
>>>    [ 1.312783] spacemit-k1-pcie ca800000.pcie:      MEM 0x00a0000000..0x00afffffff -> 0x00a0000000
>>>    [ 1.326753] spacemit-k1-pcie ca800000.pcie:      MEM 0x00b0000000..0x00b6ffffff -> 0x00b0000000
>>>    [22.348490] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
>>>    [22.351764] rcu:     4-...0: (12 GPs behind) idle=051c/1/0x4000000000000000 softirq=43/43 fqs=1908
>>>    [22.367040] Sending NMI from CPU 2 to CPUs 4:
>>>    [32.367049] After 10 seconds, these CPUS still haven't responded to the NMI: 4
>>>
>>> The CPU ignoring an NMI for ten seconds is why I read this as an MMIO access
>>> that never receives a completion rather than a spin or a deadlock.
>>>
>>>
>>> LOCALISATION
>>> ------------
>>> I added a dev_info() before each step of k1_pcie_init() (patch at the end of
>>> this mail).  The last marker port C prints is the one immediately before the
>>> first DBI access:
>>>
>>>    [1.347050] spacemit-k1-pcie ca800000.pcie: K1DBG 1 toggle_soft_reset
>>>    [1.362635] spacemit-k1-pcie ca800000.pcie: K1DBG 2 enable_resources
>>>    [1.370918] spacemit-k1-pcie ca800000.pcie: K1DBG 3 first DBI write (vendor/device ID)
>>>    <nothing further>
>> 
>> Sounds weird that an endpoint is causing DBI write hang.
>> 
>> Can Alex or someone from Spacemit look into this issue?
>
> I think Yixun Lan or maybe Troy Mitchell should investigate.
>
> I have not done anything on the Milk-V Jupiter and have no
> access to any board of that type.
This seems to be device-specific. Some of our team members are already looking into it.

-- 
Troy Mitchell


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

      reply	other threads:[~2026-09-03  1:06 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-25  5:22 [BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter Bruno Banelli
2026-09-02 13:42 ` Manivannan Sadhasivam
2026-09-02 13:56   ` Alex Elder
2026-09-03  1:06     ` Troy Mitchell [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DL5A18T0GPB5.32JI7ECB3UCRX@linux.spacemit.com \
    --to=troy.mitchell@linux.spacemit.com \
    --cc=bbanelli@gmail.com \
    --cc=bhelgaas@google.com \
    --cc=dakr@kernel.org \
    --cc=dlan@kernel.org \
    --cc=elder@riscstar.com \
    --cc=javierm@redhat.com \
    --cc=kwilczynski@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=lpieralisi@kernel.org \
    --cc=mani@kernel.org \
    --cc=robh@kernel.org \
    --cc=spacemit@lists.linux.dev \
    --cc=u.kleine-koenig@baylibre.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).