From: Marc Zyngier <maz@kernel.org>
To: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: <tglx@kernel.org>, <radu@rendec.net>, <jason@lakedaemon.net>,
<linux-arm-kernel@lists.infradead.org>,
<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] irqchip/gic-v3-its: Size the collection table for all possible CPUs
Date: Mon, 16 Nov 2026 09:42:36 +0000 [thread overview]
Message-ID: <86jymd159v.wl-maz@kernel.org> (raw)
In-Reply-To: <20261116022330.273840-1-ruanjinjie@huawei.com>
On Mon, 16 Nov 2026 02:23:30 +0000,
Jinjie Ruan <ruanjinjie@huawei.com> wrote:
>
> The ITS collection table is sized from the hardware default described by
> GITS_BASER: a single ITS page holding entry-size objects. With the usual
> 16-byte collection entry and a 4K ITS page that is 256 collections.
>
> its_cpu_init_collections() installs one collection per CPU using MAPC with
> the CPU number as the collection ID, so on a system with more than 256
> possible CPUs the MAPC commands for the remaining CPUs fail with
> E_ITS_MAPC_COLLECTION_OOR (0x010903) and their redistributors end up
> without a valid collection. LPIs targeted at those CPUs, which is where
> managed MSI/MSI-X affinity places a large part of the device interrupts,
> are then dropped by the hardware: the DMA completes but the completion
> interrupt never arrives, so drivers fall back to polling and time out.
>
> On a 384-CPU HiSilicon HIP12 system running v7.4-rc2 every ITS reported
>
> ITS@0x488000000: allocated 256 Interrupt Collections (flat, esz 16, psz 4K, shr 1)
>
> and all four NVMe controllers logged repeated
>
> nvme nvme0: I/O tag 256 (1100) QID 182 timeout, completion polled
>
> while the hinic driver failed to probe with -EIO. With the table sized
> for num_possible_cpus() it holds 512 entries and both symptoms are gone.
Please drop all these examples, as they serve absolutely no purpose.
Everybody understands that losing interrupts is deadly. The whole
thing is also filled with implementation details that are not really
relevant, and can be described in a more general way.
>
> Handle GITS_BASER_TYPE_COLLECTION in its_alloc_tables() the same way as
> the device and VCPU tables, requesting order_base_2(num_possible_cpus())
> IDs.
Just say something like:
"The collection table is currently sized at a single ITS page size,
and the driver's allocation policy is to allocate a single collection
per CPU. On systems with more CPUs that the ITS can fit in a single
page, no LPI can be routed to the CPUs that don't have an allocated
collection, and the system grinds to a halt.
Change the allocation policy to allow two-level tables, similarly to
the other tables."
>
> Fixes: 1ac19ca6bf97 ("irqchip: GICv3: ITS: tables allocators")
> Cc: stable@vger.kernel.org
This is not a fix. This is new feature support for implementation that
are both very wide and have a stupidly small ITS page size. These
systems have *never* worked, and therefore there is nothing to fix.
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> ---
> drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index c952c069f937..319b6edefd37 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -2716,6 +2716,21 @@ static int its_alloc_tables(struct its_node *its)
> indirect = its_parse_indirect_baser(its, baser, &order,
> ITS_MAX_VPEID_BITS);
> break;
> +
> + case GITS_BASER_TYPE_COLLECTION:
> + /*
> + * Size the collection table for every possible CPU.
> + * One collection per CPU is installed via MAPC with
> + * col_id == cpu, so a table sized only for the
> + * hardware default page (e.g. 256 entries with
> + * esz 16 / psz 4K) is too small on systems with more
> + * CPUs than that: MAPC for the remaining CPUs fails
> + * with E_ITS_MAPC_COLLECTION_OOR and LPIs targeted at
> + * those CPUs are never delivered.
> + */
This comment serves no purpose, and the code says everything that
needs to be known.
> + indirect = its_parse_indirect_baser(its, baser, &order,
> + order_base_2(num_possible_cpus()));
> + break;
> }
>
> err = its_setup_baser(its, baser, cache, shr, order, indirect);
Finally, it really looks like we could do better, since *all* table
types are now able to use two-level structures. I'd like to see
something like the below (untested).
M.
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 282e4bafb084f..a058cb8373ef1 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2682,7 +2682,7 @@ static int its_alloc_tables(struct its_node *its)
u64 val = its_read_baser(its, baser);
u64 type = GITS_BASER_TYPE(val);
bool indirect = false;
- u32 order;
+ u32 order, nr_ids;
if (type == GITS_BASER_TYPE_NONE)
continue;
@@ -2696,8 +2696,11 @@ static int its_alloc_tables(struct its_node *its)
switch (type) {
case GITS_BASER_TYPE_DEVICE:
- indirect = its_parse_indirect_baser(its, baser, &order,
- device_ids(its));
+ nr_ids = device_ids(its);
+ break;
+
+ case GITS_BASER_TYPE_COLLECTION:
+ nr_ids = order_base_2(num_possible_cpus());
break;
case GITS_BASER_TYPE_VCPU:
@@ -2713,11 +2716,11 @@ static int its_alloc_tables(struct its_node *its)
}
}
- indirect = its_parse_indirect_baser(its, baser, &order,
- ITS_MAX_VPEID_BITS);
+ nr_ids = ITS_MAX_VPEID_BITS;
break;
}
+ indirect = its_parse_indirect_baser(its, baser, &order, nr_ids);
err = its_setup_baser(its, baser, cache, shr, order, indirect);
if (err < 0) {
its_free_tables(its);
--
Without deviation from the norm, progress is not possible.
prev parent reply other threads:[~2026-11-16 9:42 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 2:23 [PATCH] irqchip/gic-v3-its: Size the collection table for all possible CPUs Jinjie Ruan
2026-11-16 9:42 ` Marc Zyngier [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=86jymd159v.wl-maz@kernel.org \
--to=maz@kernel.org \
--cc=jason@lakedaemon.net \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=radu@rendec.net \
--cc=ruanjinjie@huawei.com \
--cc=tglx@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox