* [PATCH 0/8] Organize the SMMUv3 invalidation flow so iommupt can use it
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
[ This is part of the patch pile to move SMMUv3 over to the generic page
table:
1) Introduction of new gather items and RISCV usage
https://patch.msgid.link/r/0-v2-b5156f657dc1+25f-iommu_riscv_inv_jgg@nvidia.com
2) Remove SMMUv3 struct arm_smmu_cmdq_ent
https://patch.msgid.link/r/0-v2-47b2bf710ad5+716ac-smmu_no_cmdq_ent_jgg@nvidia.com
3) Organize the SMMUv3 invalidation flow so iommupt can use it
4) Use the generic iommu page table for SMMUv3
It depends on #2 only
The whole branch is here:
https://github.com/jgunthorpe/linux/commits/iommu_pt_arm64/
]
iommupt has a design that focuses on building a single iommu_iotlb_gather
for arbitary batches of map/unmap operations. The gather uses the free
list and it captures invalidations of tables, leaves and supports mixed
levels.
The introduction of PT_FEAT_DETAILED_GATHER provides some additional
information that is useful for ARM: the damage bitmaps for the table and
level changes.
Prior to switching SMMUv3 over to use iommupt prepare for this by
reworking the internal invalidation to work on the same data format that
iommupt will produce. Bridge the invalidations generated by io-pgtable
into the new format. The conversion is simple enough, io-pgtable generates
invalidation operations that have only a single set bit in
table_levels_bitmap/leaf_levels_bitmap, so we can convert the io-pgtable
provided size into the proper level leaf or table bit.
When iommupt uses this mechanism it will fill in full bitmaps reflecting
the union of all invalidations contained in the gather, and this series
provides an implementation that can work this way.
Like the other drivers the general algorithm focuses on trying to issue a
single command per gather or at most 512 single invalidations. If that
isn't possible then it falls back to full invalidation. Since table and
leaf invalidation are combined together there is no waste of invaliding
tables prior to performing a full invalidation.
On its own this provides value as the invalidation has a number of
rough spots:
- Non-leaf invalidation actually expands into a TLBI for every
translation granule because the inner logic doesn't special case the
walk vs leaf condition. Now that a table_levels_bitmap is used to
describe the walk invalidation it properly generates a RIL with optimal
TTL or only one single invalidation.
- RIL doesn't calculate perfect hints for SVA because the SVA rules are
different from the io-pgtable-arm rules that the RIL algorithm works
with. SVA can now express the combined leaf and table invalidation that
the MM callback represents and get the right TTL, with an optimization
for the common 4k only scenario.
- RIL didn't generate a single invalidation like VT-d and AMD do,
instead it tries to generate an exact coverage with many
smaller invalidations. Switch it to match the other drivers single
range approach for performance and consistency. Since ARM has a much
more flexible range definition the over invalidation is far smaller
than other systems.
The approach is to introduce a new struct arm_smmu_tlbi which
describes the invalidation, pre-compute into the tlbi the single and
range commands from the start/last and bitmaps, and then apply the
correct pre-computed command to each of items in the invalidation
list.
The RIL and single calculations are revised to use the new bitmaps
and accurately generate TTL/stride/etc.
Some of this design is to support another series to remove the batch on
the stack. Now that we have the invalidation list and the tlbi it is
simple to just expand the invs list directly into commands instead of
using the temporary on-stack batch array. Eventually removing batch will
save ~1k of stack usage here.
Jason Gunthorpe (8):
iommu/arm-smmu-v3: Pass the parameters for the invalidation in a
struct
iommu/arm-smmu-v3: Move pgsize out of arm_smmu_inv
iommu/arm-smmu-v3: Optimize range invalidation for latency
iommu/arm-smmu-v3: Keep track in the arm_smmu_invs if RIL is used
iommu/arm-smmu-v3: Precompute the invalidation commands
iommu/arm-smmu-v3: Populate the tlbi at the top of the call chain
iommu/arm-smmu-v3: Change how the tlbi describes the invalidation
iommu/arm-smmu-v3: Support the DS expansion of RIL's SCALE
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 32 +-
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c | 30 +-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 439 ++++++++++++------
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 54 ++-
4 files changed, 382 insertions(+), 173 deletions(-)
base-commit: 82440c340635733f86ab9b1ade899ea21ef9da0b
--
2.43.0
^ permalink raw reply
* [PATCH 6/8] iommu/arm-smmu-v3: Populate the tlbi at the top of the call chain
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
In-Reply-To: <0-v1-5b1ac97a5403+6588f-smmu_tlbi_jgg@nvidia.com>
Each of these has their own unique situation, populate the tlbi right
at the top and pass it into arm_smmu_domain_inv_range(). They will
diverge further when the iommupt invalidation scheme is introduced.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 21 ++++----
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 51 +++++++++----------
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 13 +++--
3 files changed, 45 insertions(+), 40 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 4649717df151ba..c708fefb053771 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -129,16 +129,19 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn,
{
struct arm_smmu_domain *smmu_domain =
container_of(mn, struct arm_smmu_domain, mmu_notifier);
- size_t size;
+ struct arm_smmu_tlbi tlbi = {
+ .smmu_domain = smmu_domain,
+ .iova = start,
+ /*
+ * The mm_types defines vm_end as the first byte after the end
+ * address, different from IOMMU subsystem using the last
+ * address of an address range.
+ */
+ .size = end - start,
+ .iopte_granule = PAGE_SIZE,
+ };
- /*
- * The mm_types defines vm_end as the first byte after the end address,
- * different from IOMMU subsystem using the last address of an address
- * range. So do a simple translation here by calculating size correctly.
- */
- size = end - start;
-
- arm_smmu_domain_inv_range(smmu_domain, start, size, PAGE_SIZE, false);
+ arm_smmu_domain_tlbi(&tlbi);
}
static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9aa08f782e8986..94f742de90330c 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2642,25 +2642,12 @@ static void arm_smmu_domain_tlbi_inv(struct arm_smmu_tlbi *tlbi,
}
}
-void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
- unsigned long iova, size_t size,
- unsigned int granule, bool leaf)
+void arm_smmu_domain_tlbi(struct arm_smmu_tlbi *tlbi)
{
- struct arm_smmu_tlbi tlbi = {
- .smmu_domain = smmu_domain,
- .iova = iova,
- .size = size,
- .iopte_granule = granule,
- .leaf_only = leaf,
- };
struct arm_smmu_invs *invs;
- if (!size || size == SIZE_MAX) {
- tlbi.single.use_full_inv = true;
- tlbi.range.use_full_inv = true;
- } else {
- arm_smmu_tlbi_calc_single(&tlbi);
- }
+ if (!tlbi->single.use_full_inv)
+ arm_smmu_tlbi_calc_single(tlbi);
/*
* An invalidation request must follow some IOPTE change and then load
@@ -2690,14 +2677,14 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
smp_mb();
rcu_read_lock();
- invs = rcu_dereference(smmu_domain->invs);
+ invs = rcu_dereference(tlbi->smmu_domain->invs);
/* Only precaculate RIL if it will be used. */
if (invs->has_range_inv) {
- if (!tlbi.range.use_full_inv)
- arm_smmu_tlbi_calc_range(&tlbi);
+ if (!tlbi->range.use_full_inv)
+ arm_smmu_tlbi_calc_range(tlbi);
} else {
- tlbi.range.use_full_inv = true;
+ tlbi->range.use_full_inv = true;
}
/*
@@ -2708,10 +2695,10 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned long flags;
read_lock_irqsave(&invs->rwlock, flags);
- arm_smmu_domain_tlbi_inv(&tlbi, invs);
+ arm_smmu_domain_tlbi_inv(tlbi, invs);
read_unlock_irqrestore(&invs->rwlock, flags);
} else {
- arm_smmu_domain_tlbi_inv(&tlbi, invs);
+ arm_smmu_domain_tlbi_inv(tlbi, invs);
}
rcu_read_unlock();
@@ -2731,8 +2718,14 @@ static void arm_smmu_tlb_inv_walk(unsigned long iova, size_t size,
size_t granule, void *cookie)
{
struct arm_smmu_domain *smmu_domain = cookie;
+ struct arm_smmu_tlbi tlbi = {
+ .smmu_domain = smmu_domain,
+ .iova = iova,
+ .size = size,
+ .iopte_granule = granule,
+ };
- arm_smmu_domain_inv_range(smmu_domain, iova, size, granule, false);
+ arm_smmu_domain_tlbi(&tlbi);
}
static const struct iommu_flush_ops arm_smmu_flush_ops = {
@@ -3994,14 +3987,18 @@ static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain)
static void arm_smmu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *gather)
{
- struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_tlbi tlbi = {
+ .smmu_domain = to_smmu_domain(domain),
+ .iova = gather->start,
+ .size = gather->end - gather->start + 1,
+ .iopte_granule = gather->pgsize,
+ .leaf_only = true,
+ };
if (!gather->pgsize)
return;
- arm_smmu_domain_inv_range(smmu_domain, gather->start,
- gather->end - gather->start + 1,
- gather->pgsize, true);
+ arm_smmu_domain_tlbi(&tlbi);
}
static phys_addr_t
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index b5e214b428d644..95da62d64df171 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1168,13 +1168,18 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master,
struct arm_smmu_domain *smmu_domain, ioasid_t pasid,
struct arm_smmu_cd *cd, struct iommu_domain *old);
-void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
- unsigned long iova, size_t size,
- unsigned int granule, bool leaf);
+void arm_smmu_domain_tlbi(struct arm_smmu_tlbi *tlbi);
static inline void arm_smmu_domain_inv(struct arm_smmu_domain *smmu_domain)
{
- arm_smmu_domain_inv_range(smmu_domain, 0, 0, 0, false);
+ /* Prefilled for invalidate all */
+ struct arm_smmu_tlbi tlbi = {
+ .smmu_domain = smmu_domain,
+ .single.use_full_inv = true,
+ .range.use_full_inv = true,
+ };
+
+ arm_smmu_domain_tlbi(&tlbi);
}
void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu,
--
2.43.0
^ permalink raw reply related
* Re: [PATCH v2 3/3] arm64: dts: allwinner: A133: add support for Baijie Helper A133 board
From: Alexander Sverdlin @ 2026-05-18 20:05 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: linux-sunxi, Andre Przywara, devicetree, linux-arm-kernel,
linux-kernel
In-Reply-To: <agr9m_tidBr6Cu2h@collins>
Hi Paul,
thanks for the review!
On Mon, 2026-05-18 at 13:52 +0200, Paul Kocialkowski wrote:
>
> > diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a133-baije-core.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-a133-baije-core.dtsi
> > new file mode 100644
> > index 000000000000..65b094f30bf5
> > --- /dev/null
> > +++ b/arch/arm64/boot/dts/allwinner/sun50i-a133-baije-core.dtsi
[]
> You should add:
>
> chosen {
> stdout-path = "serial0:115200n8";
> };
I actually have it in .dts, but it's theoretically possible to deploy
the core board in a way that serial0 is *not* a console, so the above
probably will not be valid in all cases in .dtsi.
>
>
>
> > +®_dcdc2 {
> > + regulator-always-on;
> > + regulator-min-microvolt = <500000>;
> > + regulator-max-microvolt = <1300000>;
>
> Should be:
> regulator-min-microvolt = <900000>;
> regulator-max-microvolt = <1300000>;
0.81..1.2v according to A133 Datasheet Revision 1.1 Jul.14, 2020?
>
> > +®_dcdc4 {
> > + regulator-always-on;
> > + regulator-min-microvolt = <500000>;
> > + regulator-max-microvolt = <1300000>;
> > + regulator-name = "vdd-sys";
>
> Should be:
> regulator-min-microvolt = <810000>;
> regulator-max-microvolt = <990000>;
> regulator-name = "vcc-usb-sys";
I'm a bit puzzled here: datasheet says 0.9..1.0v
and it has no "Typ" value, similar to VDD_CPU, but
VDD_SYS is not part of OPP tables, so who is going
to adjust this? Or shall it be just
regulator-min-microvolt = <950000>;
regulator-max-microvolt = <950000>;
?
>
> > +};
> > +
> > +®_dcdc5 {
> > + regulator-always-on;
> > + regulator-min-microvolt = <800000>;
> > + regulator-max-microvolt = <1840000>;
> > + regulator-name = "vcc-dram";
>
> Should be:
> regulator-min-microvolt = <1100000>;
> regulator-max-microvolt = <1100000>;
> regulator-name = "vcc-dram-2";
>
> ALDO2 is the main DRAM supply, this is the second one.
Core schematics mentions 1.1V/1.2/1.35/1.5 on this rail...
Currently U-Boot has CONFIG_AXP_DCDC5_VOLT=1100, but potentially
this is adjustable, right? At some point LPDDR4 chips they
are soldering today will be unavailable. And in the current
market it will happen rather sooner than later...
>
> > +};
> > +
> > +/* DCDC6 unused */
> > +
> > +®_dldo1 {
> > + regulator-min-microvolt = <700000>;
> > + regulator-max-microvolt = <3300000>;
> > + regulator-enable-ramp-delay = <1000>;
>
> Should be:
> regulator-min-microvolt = <1800000>;
> regulator-max-microvolt = <1800000>;
> regulator-name = "vcc-pg";
Do suggest to drop vendor's
regulator-enable-ramp-delay = <1000>;
in all cases?
> >
> > diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a133-baijie-helper.dts b/arch/arm64/boot/dts/allwinner/sun50i-a133-baijie-helper.dts
> > new file mode 100644
> > index 000000000000..ccbca5d0a40c
> > --- /dev/null
> > +++ b/arch/arm64/boot/dts/allwinner/sun50i-a133-baijie-helper.dts
[]
> > + aliases {
> > + serial0 = &uart0;
>
> The is best added to the core dtsi.
>
> > + };
> > +
> > + chosen {
> > + stdout-path = "serial0:115200n8";
>
> Ditto.
But it only physically materializes in Helperboard, the carrier.
Potentially this one can be left floating or used for something else.
--
Alexander Sverdlin.
^ permalink raw reply
* Re: [PATCH v2 0/5] mm: reduce mmap_lock contention and improve page fault performance
From: Suren Baghdasaryan @ 2026-05-18 19:56 UTC (permalink / raw)
To: Barry Song
Cc: Lorenzo Stoakes, Matthew Wilcox, akpm, linux-mm, david, liam,
vbabka, rppt, mhocko, jack, pfalcato, wanglian, chentao,
lianux.mm, kunwu.chan, liyangouwen1, chrisl, kasong, shikemeng,
nphamcs, bhe, youngjun.park, linux-arm-kernel, linux-kernel,
loongarch, linuxppc-dev, linux-riscv, linux-s390, Nanzhe Zhao
In-Reply-To: <CAGsJ_4zqLfdWoTH9s7FFaqWWj0mESfikYgr7=GcV64qcuXrPxA@mail.gmail.com>
On Mon, May 18, 2026 at 4:26 AM Barry Song <baohua@kernel.org> wrote:
>
> On Mon, May 18, 2026 at 5:47 PM Lorenzo Stoakes <ljs@kernel.org> wrote:
> >
> > On Sun, May 17, 2026 at 04:45:15PM +0800, Barry Song wrote:
> > > On Sat, May 2, 2026 at 1:58 AM Matthew Wilcox <willy@infradead.org> wrote:
> > > >
> > > > On Sat, May 02, 2026 at 01:44:34AM +0800, Barry Song wrote:
> > > > > On Fri, May 1, 2026 at 10:57 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > > > >
> > > > > > On Fri, May 01, 2026 at 06:49:58AM +0800, Barry Song wrote:
> > > > > > > 1. There is no deterministic latency for I/O completion. It depends on
> > > > > > > both the hardware and the software stack (bio/request queues and the
> > > > > > > block scheduler). Sometimes the latency is short; at other times it can
> > > > > > > be quite long. In such cases, a high-priority thread performing operations
> > > > > > > such as mprotect, unmap, prctl_set_vma, or madvise may be forced to wait
> > > > > > > for an unpredictable amount of time.
> > > > > >
> > > > > > But does that actually happen? I find it hard to believe that thread A
> > > > > > unmaps a VMA while thread B is in the middle of taking a page fault in
> > > > > > that same VMA. mprotect() and madvise() are more likely to happen, but
> > > > > > it still seems really unlikely to me.
> > > > >
> > > > > It doesn’t have to involve unmapping or applying mprotect to
> > > > > the entire VMA—just a portion of it is sufficient.
> > > >
> > > > Yes, but that still fails to answer "does this actually happen". How much
> > > > performance is all this complexity in the page fault handler buying us?
> > > > If you don't answer this question, I'm just going to go in and rip it
> > > > all out.
> > > >
> > >
> > > Hi Matthew (and Lorenzo, Jan, and anyone else who may be
> > > waiting for answers),
> > >
> > > As promised during LSF/MM/BPF, we conducted thorough
> > > testing on Android phones to determine whether performing
> > > I/O in `filemap_fault()` can block `vma_start_write()`.
> > > I wanted to give a quick update on this question.
> > >
> > > Nanzhe at Xiaomi created tracing scripts and ran various
> > > applications on Android devices with I/O performed under
> > > the VMA lock in `filemap_fault()`. We found that:
> > >
> > > 1. There are very few cases where unmap() is blocked by
> > > page faults. I assume this is due to buggy user code
> > > or poor synchronization between reads and unmap().
> > > So I assume it is not a problem.
> > >
> > > 2. We observed many cases where `vma_start_write()`
> > > is blocked by page-fault I/O in some applications.
> > > The blocking occurs in the `dup_mmap()` path during
> > > fork().
> > >
> > > With Suren's commit fb49c455323ff ("fork: lock VMAs of
> > > the parent process when forking"), we now always hold
> > > `vma_write_lock()` for each VMA. Note that the
> > > `mmap_lock` write lock is also held, which could lead to
> > > chained waiting if page-fault I/O is performed without
> > > releasing the VMA lock.
> >
> > Hm but did you observe this 'chained waiting'? And what were the latencies?
>
> We have clearly observed that the `fork()` operations of many
> popular Android apps, such as iQiyi, Baidu Tieba, and 10086,
> end up waiting on page-fault (PF) I/O when the VMA lock is
> held during I/O operations. This has already become a
> practical issue. I also believe this can lead to chained
> waiting, since the global `mmap_lock` blocks all threads that
> need to acquire it.
>
>
> >
> > >
> > > My gut feeling is that Suren's commit may be overshooting,
> > > so my rough idea is that we might want to do something like
> > > the following (we haven't tested it yet and it might be
> > > wrong):
> >
> > Yeah I'm really not sure about that.
> >
> > Prior to the VMA locks, the mmap write lock would have guaranteed no concurrent
> > page faults, which is really what fb49c455323ff is about.
> >
> > So Suren's patch was essentially restoring the _existing_ forking behaviour, and
> > now you're saying 'let's change the forking behaviour that's been like that for
> > forever'.
>
>
> I am afraid not. Before we introduced the per-VMA lock, we
> were not performing I/O while holding `mmap_lock`. A page fault
> that needed I/O would drop the `mmap_lock` read lock and allow
> `fork()` to proceed.
>
> Now, you are suggesting performing I/O while holding the VMA
> lock, which changes the requirements and introduces this
> problem.
>
> >
> > I think you would _really_ have to be sure that's safe. And forking is a very
> > dangerous time in terms of complexity and sensitivity and 'weird stuff'
> > happening so I'd tread _very_ carefully here.
>
> Yep. I think my original proposal did not require any changes
> to `fork()`, since it simply preserved the current behavior of
> dropping the VMA lock before performing I/O. In that model,
> `fork()` would not end up waiting on I/O at all.
>
> What you are suggesting now appears to be performing I/O while
> holding the VMA lock, which in turn introduces the need to
> change `fork()`.
>
> >
> > >
> > > diff --git a/mm/mmap.c b/mm/mmap.c
> > > index 2311ae7c2ff4..5ddaf297f31a 100644
> > > --- a/mm/mmap.c
> > > +++ b/mm/mmap.c
> > > @@ -1762,7 +1762,13 @@ __latent_entropy int dup_mmap(struct mm_struct
> > > *mm, struct mm_struct *oldmm)
> > > for_each_vma(vmi, mpnt) {
> > > struct file *file;
> > >
> > > - retval = vma_start_write_killable(mpnt);
> > > + /*
> > > + * For anonymous or writable private VMAs, prevent
> > > + * concurrent CoW faults.
> > > + */
> >
> > To nit pick I think the comment's confusing but also tells you you don't need to
> > specific anon check - writable private is sufficient. And it's not really just
> > CoW that's the issue, it's anon_vma population _at all_ as well as CoW.
> >
> > > + if (!mpnt->vm_file || (!(mpnt->vm_flags & VM_SHARED) &&
> > > + (mpnt->vm_flags & VM_WRITE)))
> > > + retval = vma_start_write_killable(mpnt);
> >
> > I think this has to be VM_MAYWRITE, because somebody could otherwise mprotect()
> > it R/W.
> >
> > I also don't understand why !mpnt->vm_file for a read-only anon mapping (more
> > likely PROT_NONE) is here, just do the second check?
> >
> > (Also please use the new interface, so !vma_test(mpnt, VMA_SHARED_BIT) &&
> > vma_test(mpnt, VMA_MAYWRITE_BIT))
>
> Yep, I can definitely refine the check further. But before
> doing that, I'd first like to confirm that we are aligned on
> the direction.
>
> If you still intend to hold the VMA lock while performing I/O,
> then I think we should fix `fork()` to avoid taking
> `vma_start_write()`.
>
> >
> > > if (retval < 0)
> > > goto loop_out;
> > > if (mpnt->vm_flags & VM_DONTCOPY) {
> > >
> > > Based on the above, we may want to re-check whether fork()
> > > can be blocked by page faults. At the same time, if Suren,
> > > you, or anyone else has any comments, please feel free to
> > > share them.
> > >
> > > Best Regards
> > > Barry
> >
> > Technical commentary above is sort of 'just cos' :) because I really question
> > doing this honestly.
>
> I think we either need to fix `fork()`, or keep the current
> behavior of dropping the VMA lock before performing I/O.
I see. So, this problem arises from the fact that we are changing the
pagefaults requiring I/O operation to hold VMA lock...
And you want to lock VMA on fork only if vma_is_anonymous(vma) ||
is_cow_mapping(vma->vm_flags). So, we will be blocking page faults for
anonymous and COW VMAs only while holding mmap_write_lock, preventing
any VMA modification. On the surface, that looks ok to me but I might
be missing some corner cases. If nobody sees any obvious issues, I
think it's worth a try.
>
> >
> > I'd also like to get Suren's input, however.
>
> Yes. of course.
>
> >
> > Thanks, Lorenzo
>
> Best Regards
> Barry
^ permalink raw reply
* Re: [PATCH 1/8] mm: Add ptep_try_install() for lockless empty-slot installs
From: Tejun Heo @ 2026-05-18 19:53 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: David Vernet, Andrea Righi, Changwoo Min, Alexei Starovoitov,
Andrii Nakryiko, Daniel Borkmann, Martin KaFai Lau,
Kumar Kartikeya Dwivedi, Catalin Marinas, Will Deacon,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
Andrew Morton, Mike Rapoport, Emil Tsalapatis, sched-ext, bpf,
x86, linux-arm-kernel, linux-mm, linux-kernel
In-Reply-To: <9ba50fd2-077e-4291-9276-9adb18186873@kernel.org>
Hello,
On Mon, May 18, 2026 at 10:06:29AM +0200, David Hildenbrand (Arm) wrote:
> On 5/17/26 23:12, Tejun Heo wrote:
> > +static inline bool ptep_try_install(pte_t *ptep, pte_t new_pte)
> > +{
> > + return false;
> > +}
> > +#endif
>
> Ehm, what?
>
> This is a very, very, very bad generic idea/interface.
>
> On which ptes is this supposed to be used? User ptes or kernel ptes?
>
> Surely we don't want this on user ptes.
Yeah, this is only for the BPF arena PTEs which are already managed in their
own way. I'd be happy to place / gate it however appropriate.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH 7/8] sched_ext: Sub-allocator over kernel-claimed BPF arena pages
From: Tejun Heo @ 2026-05-18 19:51 UTC (permalink / raw)
To: Peter Zijlstra
Cc: David Vernet, Andrea Righi, Changwoo Min, Alexei Starovoitov,
Andrii Nakryiko, Daniel Borkmann, Martin KaFai Lau,
Kumar Kartikeya Dwivedi, Catalin Marinas, Will Deacon,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
Andrew Morton, David Hildenbrand, Mike Rapoport, Emil Tsalapatis,
sched-ext, bpf, x86, linux-arm-kernel, linux-mm, linux-kernel
In-Reply-To: <20260518072042.GP3102624@noisy.programming.kicks-ass.net>
Hello,
On Mon, May 18, 2026 at 09:20:42AM +0200, Peter Zijlstra wrote:
...
> Should this really be part of scx rather than be part of the bpf-arena
> thing proper?
It's just a layer on top of arena. If bpf folks are okay with it, I don't
see why it can't be a common utility thing on the bpf side.
Thanks.
--
tejun
^ permalink raw reply
* [PATCH 2/8] iommu/arm-smmu-v3: Move pgsize out of arm_smmu_inv
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
In-Reply-To: <0-v1-5b1ac97a5403+6588f-smmu_tlbi_jgg@nvidia.com>
pgsize is a constant property of the domain, it is the base translation
granule of the page table (4k, 16k, 64k) in log2.
Store it to the struct arm_smmu_domain based on how the page table was created.
Add smmu_domain to the tlbi and just get tg from the domain.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 1 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 27 ++++++++-----------
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 ++-
3 files changed, 14 insertions(+), 17 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index f1f8e01a7e9142..4649717df151ba 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -324,6 +324,7 @@ struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev,
* ARM_SMMU_FEAT_RANGE_INV is present
*/
smmu_domain->domain.pgsize_bitmap = PAGE_SIZE;
+ smmu_domain->tgsz_lg2 = PAGE_SHIFT;
smmu_domain->stage = ARM_SMMU_DOMAIN_SVA;
smmu_domain->smmu = smmu;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index f7881d21eb06cd..cd0ab518712cd6 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2379,14 +2379,13 @@ static void arm_smmu_tlb_inv_context(void *cookie)
static void arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
struct arm_smmu_cmdq_batch *cmds,
struct arm_smmu_cmd *cmd,
- struct arm_smmu_tlbi *tlbi,
- size_t pgsize)
+ struct arm_smmu_tlbi *tlbi)
{
size_t inv_range = tlbi->iopte_granule;
unsigned long iova = tlbi->iova;
unsigned long end = iova + tlbi->size;
unsigned long num_pages = 0;
- unsigned int tg = pgsize;
+ unsigned int tg = tlbi->smmu_domain->tgsz_lg2;
u64 orig_data0 = cmd->data[0];
u8 ttl = 0, tg_enc = 0;
@@ -2494,7 +2493,7 @@ static void arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
return;
}
- arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, tlbi, inv->pgsize);
+ arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, tlbi);
}
static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
@@ -2596,6 +2595,7 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned int granule, bool leaf)
{
struct arm_smmu_tlbi tlbi = {
+ .smmu_domain = smmu_domain,
.iova = iova,
.size = size,
.iopte_granule = granule,
@@ -2851,6 +2851,7 @@ static int arm_smmu_domain_finalise(struct arm_smmu_domain *smmu_domain,
return -ENOMEM;
smmu_domain->domain.pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
+ smmu_domain->tgsz_lg2 = __ffs(pgtbl_cfg.pgsize_bitmap);
smmu_domain->domain.geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1;
smmu_domain->domain.geometry.force_aperture = true;
if (enable_dirty && smmu_domain->stage == ARM_SMMU_DOMAIN_S1)
@@ -3085,15 +3086,13 @@ static void arm_smmu_disable_iopf(struct arm_smmu_master *master,
static struct arm_smmu_inv *
arm_smmu_master_build_inv(struct arm_smmu_master *master,
- enum arm_smmu_inv_type type, u32 id, ioasid_t ssid,
- size_t pgsize)
+ enum arm_smmu_inv_type type, u32 id, ioasid_t ssid)
{
struct arm_smmu_invs *build_invs = master->build_invs;
struct arm_smmu_inv *cur, inv = {
.smmu = master->smmu,
.type = type,
.id = id,
- .pgsize = pgsize,
};
if (WARN_ON(build_invs->num_invs >= build_invs->max_invs))
@@ -3145,28 +3144,24 @@ arm_smmu_master_build_invs(struct arm_smmu_master *master, bool ats_enabled,
ioasid_t ssid, struct arm_smmu_domain *smmu_domain)
{
const bool nesting = smmu_domain->nest_parent;
- size_t pgsize = 0, i;
+ size_t i;
iommu_group_mutex_assert(master->dev);
master->build_invs->num_invs = 0;
- /* Range-based invalidation requires the leaf pgsize for calculation */
- if (master->smmu->features & ARM_SMMU_FEAT_RANGE_INV)
- pgsize = __ffs(smmu_domain->domain.pgsize_bitmap);
-
switch (smmu_domain->stage) {
case ARM_SMMU_DOMAIN_SVA:
case ARM_SMMU_DOMAIN_S1:
if (!arm_smmu_master_build_inv(master, INV_TYPE_S1_ASID,
smmu_domain->cd.asid,
- IOMMU_NO_PASID, pgsize))
+ IOMMU_NO_PASID))
return NULL;
break;
case ARM_SMMU_DOMAIN_S2:
if (!arm_smmu_master_build_inv(master, INV_TYPE_S2_VMID,
smmu_domain->s2_cfg.vmid,
- IOMMU_NO_PASID, pgsize))
+ IOMMU_NO_PASID))
return NULL;
break;
default:
@@ -3178,7 +3173,7 @@ arm_smmu_master_build_invs(struct arm_smmu_master *master, bool ats_enabled,
if (nesting) {
if (!arm_smmu_master_build_inv(
master, INV_TYPE_S2_VMID_S1_CLEAR,
- smmu_domain->s2_cfg.vmid, IOMMU_NO_PASID, 0))
+ smmu_domain->s2_cfg.vmid, IOMMU_NO_PASID))
return NULL;
}
@@ -3189,7 +3184,7 @@ arm_smmu_master_build_invs(struct arm_smmu_master *master, bool ats_enabled,
*/
if (!arm_smmu_master_build_inv(
master, nesting ? INV_TYPE_ATS_FULL : INV_TYPE_ATS,
- master->streams[i].id, ssid, 0))
+ master->streams[i].id, ssid))
return NULL;
}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 4e97aa5d011a98..0c63069400d22d 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -734,7 +734,6 @@ struct arm_smmu_inv {
u8 nsize_opcode;
u32 id; /* ASID or VMID or SID */
union {
- size_t pgsize; /* ARM_SMMU_FEAT_RANGE_INV */
u32 ssid; /* INV_TYPE_ATS */
};
@@ -803,6 +802,7 @@ static inline struct arm_smmu_invs *arm_smmu_invs_alloc(size_t num_invs)
}
struct arm_smmu_tlbi {
+ struct arm_smmu_domain *smmu_domain;
unsigned long iova;
size_t size;
unsigned int iopte_granule;
@@ -1049,6 +1049,7 @@ struct arm_smmu_domain {
spinlock_t devices_lock;
bool enforce_cache_coherency : 1;
bool nest_parent : 1;
+ u8 tgsz_lg2;
struct mmu_notifier mmu_notifier;
};
--
2.43.0
^ permalink raw reply related
* [PATCH 13/13] drm/meson: dw-hdmi: Use suspend_late/resume_early/resume_noirq pm ops
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
meson_drv_pm_suspend()/drm_mode_config_helper_suspend() and
meson_drv_pm_resume()/drm_mode_config_helper_resume() is called
after/before the suspend/resume pm ops of dw-hdmi.
This result in atomic_disable() being called after
meson_dw_hdmi_pm_suspend() and atomic_enable() before
meson_dw_hdmi_pm_resume()/dw_hdmi_resume() is called.
Suspend (without changes):
- meson_dw_hdmi_pm_suspend()
- meson_drv_pm_suspend()
- drm_mode_config_helper_suspend()
- atomic_disable()
Resume (without changes):
- resume_irq() <<-- system freeze at hdmi-tx irq on G12B
- meson_drv_pm_resume()
- drm_mode_config_helper_resume()
- atomic_enable()
- meson_dw_hdmi_pm_resume()
- meson_dw_hdmi_init()
- dw_hdmi_resume()
- dw_hdmi_init_hw()
Change to use suspend_late/resume_early pm ops for system suspend to
ensure pm ops for dw-hdmi is run after/before meson_drv pm ops.
The G12B can trigger an IRQ (stat=0x40) very early during resume_irq
that causes a system freeze. Move the meson_dw_hdmi_init() call to
resume_noirq pm ops to resolve the system freeze.
Also move meson_dw_hdmi_init() to run before interrupt allocation to
ensure HW has been initialized when first IRQ is triggered.
Suspend (with changes):
- meson_drv_pm_suspend()
- drm_mode_config_helper_suspend()
- atomic_disable()
- meson_dw_hdmi_pm_suspend_late()
Resume (with changes):
- meson_dw_hdmi_pm_resume_noirq()
- meson_dw_hdmi_init()
- resume_irq()
- meson_dw_hdmi_pm_resume_early()
- dw_hdmi_resume()
- dw_hdmi_init_hw()
- meson_drv_pm_resume()
- drm_mode_config_helper_resume()
- atomic_enable()
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
This can be tested using CONFIG_PM_DEBUG with a simulated suspend:
echo N > /sys/module/printk/parameters/console_suspend
echo 1 > /sys/power/pm_debug_messages
echo platform > /sys/power/pm_test
echo mem > /sys/power/state
or using something like following for real suspend/resume:
echo N > /sys/module/printk/parameters/console_suspend
rtcwake -m mem -s 5 -d /dev/rtc1
On S905X a simulated platform pm_test could only be tested. On G12A both
simulated and real suspend works. And on G12B the board seems to freeze
at a later stage when taking out of real suspend, however platform
pm_test works.
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 27 +++++++++++++++++++--------
1 file changed, 19 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 7b465e216759..16f15466a6f4 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -696,6 +696,8 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
if (IS_ERR(dw_plat_data->regm))
return PTR_ERR(dw_plat_data->regm);
+ meson_dw_hdmi_init(meson_dw_hdmi);
+
irq = platform_get_irq(pdev, 0);
if (irq < 0)
return irq;
@@ -706,8 +708,6 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
return dev_err_probe(dev, ret,
"Failed to request hdmi top irq\n");
- meson_dw_hdmi_init(meson_dw_hdmi);
-
/* Bridge / Connector */
dw_plat_data->priv_data = meson_dw_hdmi;
@@ -763,7 +763,7 @@ static const struct component_ops meson_dw_hdmi_ops = {
.unbind = meson_dw_hdmi_unbind,
};
-static int __maybe_unused meson_dw_hdmi_pm_suspend(struct device *dev)
+static int __maybe_unused meson_dw_hdmi_pm_suspend_late(struct device *dev)
{
struct meson_dw_hdmi *meson_dw_hdmi = dev_get_drvdata(dev);
@@ -777,7 +777,19 @@ static int __maybe_unused meson_dw_hdmi_pm_suspend(struct device *dev)
return 0;
}
-static int __maybe_unused meson_dw_hdmi_pm_resume(struct device *dev)
+static int __maybe_unused meson_dw_hdmi_pm_resume_early(struct device *dev)
+{
+ struct meson_dw_hdmi *meson_dw_hdmi = dev_get_drvdata(dev);
+
+ if (!meson_dw_hdmi)
+ return 0;
+
+ dw_hdmi_resume(meson_dw_hdmi->hdmi);
+
+ return 0;
+}
+
+static int __maybe_unused meson_dw_hdmi_pm_resume_noirq(struct device *dev)
{
struct meson_dw_hdmi *meson_dw_hdmi = dev_get_drvdata(dev);
@@ -786,8 +798,6 @@ static int __maybe_unused meson_dw_hdmi_pm_resume(struct device *dev)
meson_dw_hdmi_init(meson_dw_hdmi);
- dw_hdmi_resume(meson_dw_hdmi->hdmi);
-
return 0;
}
@@ -802,8 +812,9 @@ static void meson_dw_hdmi_remove(struct platform_device *pdev)
}
static const struct dev_pm_ops meson_dw_hdmi_pm_ops = {
- SET_SYSTEM_SLEEP_PM_OPS(meson_dw_hdmi_pm_suspend,
- meson_dw_hdmi_pm_resume)
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(meson_dw_hdmi_pm_suspend_late,
+ meson_dw_hdmi_pm_resume_early)
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(NULL, meson_dw_hdmi_pm_resume_noirq)
};
static const struct of_device_id meson_dw_hdmi_of_table[] = {
--
2.54.0
^ permalink raw reply related
* [PATCH v2 09/11] drm/rockchip: dw_hdmi: Configure HDMI PHY in atomic_mode_set()
From: Jonas Karlman @ 2026-05-18 19:37 UTC (permalink / raw)
To: Heiko Stübner, Sandy Huang, Andy Yan, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter
Cc: dri-devel, linux-rockchip, linux-arm-kernel, linux-kernel,
Jonas Karlman
In-Reply-To: <20260518193748.2482823-1-jonas@kwiboo.se>
The HDMI helpers negotiated TMDS character rate and output bpc are
available from the connector state. Change the encoder helper from
mode_set() to atomic_mode_set() so these values can be used to configure
the HDMI PHY using phy_configure().
This has no impact until the dw-hdmi bridge is fully converted into a
HDMI bridge and HDMI helpers are used to assign hdmi.tmds_char_rate.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
v2: No change
---
drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c | 29 +++++++++++++++++----
1 file changed, 24 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c b/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
index 6c4923b9b659..4e7fd4b80d76 100644
--- a/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
+++ b/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
@@ -226,11 +226,22 @@ dw_hdmi_rockchip_mode_valid(struct dw_hdmi *dw_hdmi, void *data,
return MODE_OK;
}
-static void dw_hdmi_rockchip_encoder_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adj_mode)
+static void
+dw_hdmi_rockchip_encoder_atomic_mode_set(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
{
struct rockchip_hdmi *hdmi = to_rockchip_hdmi(encoder);
+ struct drm_display_mode *adj_mode = &crtc_state->adjusted_mode;
+
+ if (hdmi->phy && conn_state->hdmi.tmds_char_rate) {
+ union phy_configure_opts opts = {};
+
+ opts.hdmi.bpc = conn_state->hdmi.output_bpc;
+ opts.hdmi.tmds_char_rate = conn_state->hdmi.tmds_char_rate;
+
+ phy_configure(hdmi->phy, &opts);
+ }
clk_set_rate(hdmi->ref_clk, adj_mode->clock * 1000);
}
@@ -270,15 +281,23 @@ dw_hdmi_rockchip_encoder_atomic_check(struct drm_encoder *encoder,
struct drm_connector_state *conn_state)
{
struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc_state);
+ struct rockchip_hdmi *hdmi = to_rockchip_hdmi(encoder);
+ union phy_configure_opts opts = {};
s->output_mode = ROCKCHIP_OUT_MODE_AAAA;
s->output_type = DRM_MODE_CONNECTOR_HDMIA;
- return 0;
+ if (!hdmi->phy || !conn_state->hdmi.tmds_char_rate)
+ return 0;
+
+ opts.hdmi.bpc = conn_state->hdmi.output_bpc;
+ opts.hdmi.tmds_char_rate = conn_state->hdmi.tmds_char_rate;
+
+ return phy_validate(hdmi->phy, PHY_MODE_HDMI, PHY_HDMI_MODE_TMDS, &opts);
}
static const struct drm_encoder_helper_funcs dw_hdmi_rockchip_encoder_helper_funcs = {
- .mode_set = dw_hdmi_rockchip_encoder_mode_set,
+ .atomic_mode_set = dw_hdmi_rockchip_encoder_atomic_mode_set,
.enable = dw_hdmi_rockchip_encoder_enable,
.atomic_check = dw_hdmi_rockchip_encoder_atomic_check,
};
--
2.54.0
^ permalink raw reply related
* [PATCH 11/13] drm/bridge: dw-hdmi: Export dw_hdmi_schedule_hpd_work() helper
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Andrzej Hajda,
Robert Foss, Laurent Pinchart, Jonas Karlman, Jernej Skrabec,
Luca Ceresoli, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter
Cc: Jerome Brunet, Martin Blumenstingl, dri-devel, linux-amlogic,
linux-arm-kernel, linux-kernel, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Export a dw_hdmi_schedule_hpd_work() helper that schedule the HPD
delayed work. Primarily to be used by the meson dw-hdmi driver.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/bridge/synopsys/dw-hdmi.c | 11 +++++++++--
include/drm/bridge/dw_hdmi.h | 2 ++
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
index 99dd62b6becf..54d75317ce9c 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
@@ -3014,8 +3014,7 @@ static irqreturn_t dw_hdmi_hardirq(int irq, void *dev_id)
status == connector_status_connected ?
"plugin" : "plugout");
- mod_delayed_work(system_percpu_wq, &hdmi->hpd_work,
- msecs_to_jiffies(HOTPLUG_DEBOUNCE_MS));
+ dw_hdmi_schedule_hpd_work(hdmi);
hdmi_writeb(hdmi, intr_stat, HDMI_IH_PHY_STAT0);
hdmi_writeb(hdmi, ~HDMI_IH_PHY_STAT0_HPD, HDMI_IH_MUTE_PHY_STAT0);
@@ -3025,6 +3024,14 @@ static irqreturn_t dw_hdmi_hardirq(int irq, void *dev_id)
return ret;
}
+void dw_hdmi_schedule_hpd_work(struct dw_hdmi *hdmi)
+{
+ if (!IS_ERR_OR_NULL(hdmi))
+ mod_delayed_work(system_percpu_wq, &hdmi->hpd_work,
+ msecs_to_jiffies(HOTPLUG_DEBOUNCE_MS));
+}
+EXPORT_SYMBOL_GPL(dw_hdmi_schedule_hpd_work);
+
static void dw_hdmi_hpd_work(struct work_struct *work)
{
struct dw_hdmi *hdmi = container_of(work, struct dw_hdmi, hpd_work.work);
diff --git a/include/drm/bridge/dw_hdmi.h b/include/drm/bridge/dw_hdmi.h
index 10013b8d3adb..c56d1775f04a 100644
--- a/include/drm/bridge/dw_hdmi.h
+++ b/include/drm/bridge/dw_hdmi.h
@@ -184,6 +184,8 @@ struct dw_hdmi *dw_hdmi_bind(struct platform_device *pdev,
void dw_hdmi_resume(struct dw_hdmi *hdmi);
+void dw_hdmi_schedule_hpd_work(struct dw_hdmi *hdmi);
+
int dw_hdmi_set_plugged_cb(struct dw_hdmi *hdmi, hdmi_codec_plugged_cb fn,
struct device *codec_dev);
void dw_hdmi_set_sample_non_pcm(struct dw_hdmi *hdmi, unsigned int non_pcm);
--
2.54.0
^ permalink raw reply related
* [PATCH 10/13] drm/meson: dw-hdmi: Use dev_err_probe() to report errors
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Change to use dev_err_probe() to report bind() errors consistently.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 65 +++++++++++++--------------
1 file changed, 30 insertions(+), 35 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index d0cf2042d41c..1dd59196ff7f 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -650,10 +650,9 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
DRM_DEBUG_DRIVER("\n");
match = of_device_get_match_data(dev);
- if (!match) {
- dev_err(dev, "failed to get match data\n");
- return -ENODEV;
- }
+ if (!match)
+ return dev_err_probe(dev, -ENODEV,
+ "Failed to get match data\n");
meson_dw_hdmi = devm_kzalloc(dev, sizeof(*meson_dw_hdmi),
GFP_KERNEL);
@@ -667,50 +666,45 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
ret = devm_regulator_get_enable_optional(dev, "hdmi");
if (ret < 0 && ret != -ENODEV)
- return ret;
+ return dev_err_probe(dev, ret,
+ "Failed to get/enable hdmi regulator\n");
meson_dw_hdmi->hdmitx_apb = devm_reset_control_get_exclusive(dev,
"hdmitx_apb");
- if (IS_ERR(meson_dw_hdmi->hdmitx_apb)) {
- dev_err(dev, "Failed to get hdmitx_apb reset\n");
- return PTR_ERR(meson_dw_hdmi->hdmitx_apb);
- }
+ if (IS_ERR(meson_dw_hdmi->hdmitx_apb))
+ return dev_err_probe(dev, PTR_ERR(meson_dw_hdmi->hdmitx_apb),
+ "Failed to get hdmitx_apb reset\n");
meson_dw_hdmi->hdmitx_ctrl = devm_reset_control_get_exclusive(dev,
"hdmitx");
- if (IS_ERR(meson_dw_hdmi->hdmitx_ctrl)) {
- dev_err(dev, "Failed to get hdmitx reset\n");
- return PTR_ERR(meson_dw_hdmi->hdmitx_ctrl);
- }
+ if (IS_ERR(meson_dw_hdmi->hdmitx_ctrl))
+ return dev_err_probe(dev, PTR_ERR(meson_dw_hdmi->hdmitx_ctrl),
+ "Failed to get hdmitx reset\n");
meson_dw_hdmi->hdmitx_phy = devm_reset_control_get_exclusive(dev,
"hdmitx_phy");
- if (IS_ERR(meson_dw_hdmi->hdmitx_phy)) {
- dev_err(dev, "Failed to get hdmitx_phy reset\n");
- return PTR_ERR(meson_dw_hdmi->hdmitx_phy);
- }
+ if (IS_ERR(meson_dw_hdmi->hdmitx_phy))
+ return dev_err_probe(dev, PTR_ERR(meson_dw_hdmi->hdmitx_phy),
+ "Failed to get hdmitx_phy reset\n");
meson_dw_hdmi->hdmitx = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(meson_dw_hdmi->hdmitx))
return PTR_ERR(meson_dw_hdmi->hdmitx);
clk = devm_clk_get_enabled(dev, "isfr");
- if (IS_ERR(clk)) {
- dev_err(dev, "Unable to get isfr pclk\n");
- return PTR_ERR(clk);
- }
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "Failed to get isfr pclk\n");
clk = devm_clk_get_enabled(dev, "iahb");
- if (IS_ERR(clk)) {
- dev_err(dev, "Unable to get iahb pclk\n");
- return PTR_ERR(clk);
- }
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "Failed to get iahb pclk\n");
clk = devm_clk_get_enabled(dev, "venci");
- if (IS_ERR(clk)) {
- dev_err(dev, "Unable to get venci pclk\n");
- return PTR_ERR(clk);
- }
+ if (IS_ERR(clk))
+ return dev_err_probe(dev, PTR_ERR(clk),
+ "Failed to get venci pclk\n");
dw_plat_data->regm = devm_regmap_init(dev, NULL, meson_dw_hdmi,
&meson_dw_hdmi_regmap_config);
@@ -724,10 +718,9 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
ret = devm_request_threaded_irq(dev, irq, dw_hdmi_top_irq,
dw_hdmi_top_thread_irq, IRQF_SHARED,
"dw_hdmi_top_irq", meson_dw_hdmi);
- if (ret) {
- dev_err(dev, "Failed to request hdmi top irq\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to request hdmi top irq\n");
meson_dw_hdmi_init(meson_dw_hdmi);
@@ -752,14 +745,16 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
meson_dw_hdmi->hdmi = dw_hdmi_probe(pdev, &meson_dw_hdmi->dw_plat_data);
if (IS_ERR(meson_dw_hdmi->hdmi)) {
devm_free_irq(dev, irq, meson_dw_hdmi);
- return PTR_ERR(meson_dw_hdmi->hdmi);
+ return dev_err_probe(dev, PTR_ERR(meson_dw_hdmi->hdmi),
+ "Failed to probe dw-hdmi bridge\n");
}
meson_dw_hdmi->bridge = of_drm_find_and_get_bridge(dev->of_node);
if (!meson_dw_hdmi->bridge) {
devm_free_irq(dev, irq, meson_dw_hdmi);
dw_hdmi_remove(meson_dw_hdmi->hdmi);
- return -ENODEV;
+ return dev_err_probe(dev, -ENODEV,
+ "Failed to find dw-hdmi bridge\n");
}
DRM_DEBUG_DRIVER("HDMI controller initialized\n");
--
2.54.0
^ permalink raw reply related
* [PATCH 12/13] drm/meson: dw-hdmi: Use dw_hdmi_schedule_hpd_work() helper
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Change to use the dw-hdmi HPD delayed work to handle HPD events.
Also merge top and bottom half IRQ handlers to simplify IRQ handling now
that HPD event is handled using a delayed work.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 26 +++++---------------------
1 file changed, 5 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 1dd59196ff7f..7b465e216759 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -145,7 +145,6 @@ struct meson_dw_hdmi {
struct reset_control *hdmitx_apb;
struct reset_control *hdmitx_ctrl;
struct reset_control *hdmitx_phy;
- u32 irq_stat;
struct dw_hdmi *hdmi;
struct drm_bridge *bridge;
};
@@ -498,10 +497,10 @@ static irqreturn_t dw_hdmi_top_irq(int irq, void *dev_id)
stat = dw_hdmi->data->top_read(dw_hdmi, HDMITX_TOP_INTR_STAT);
dw_hdmi->data->top_write(dw_hdmi, HDMITX_TOP_INTR_STAT_CLR, stat);
- /* HPD Events, handle in the threaded interrupt handler */
+ /* HPD Events, handle in delayed work */
if (stat & (HDMITX_TOP_INTR_HPD_RISE | HDMITX_TOP_INTR_HPD_FALL)) {
- dw_hdmi->irq_stat = stat;
- return IRQ_WAKE_THREAD;
+ dw_hdmi_schedule_hpd_work(dw_hdmi->hdmi);
+ return IRQ_HANDLED;
}
/* HDMI Controller Interrupt */
@@ -513,20 +512,6 @@ static irqreturn_t dw_hdmi_top_irq(int irq, void *dev_id)
return IRQ_HANDLED;
}
-/* Threaded interrupt handler to manage HPD events */
-static irqreturn_t dw_hdmi_top_thread_irq(int irq, void *dev_id)
-{
- struct meson_dw_hdmi *dw_hdmi = dev_id;
- u32 stat = dw_hdmi->irq_stat;
-
- /* HPD Events */
- if (stat & (HDMITX_TOP_INTR_HPD_RISE | HDMITX_TOP_INTR_HPD_FALL) &&
- dw_hdmi->bridge)
- drm_helper_hpd_irq_event(dw_hdmi->bridge->dev);
-
- return IRQ_HANDLED;
-}
-
/* DW HDMI Regmap */
static int meson_dw_hdmi_reg_read(void *context, unsigned int reg,
@@ -715,9 +700,8 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
if (irq < 0)
return irq;
- ret = devm_request_threaded_irq(dev, irq, dw_hdmi_top_irq,
- dw_hdmi_top_thread_irq, IRQF_SHARED,
- "dw_hdmi_top_irq", meson_dw_hdmi);
+ ret = devm_request_irq(dev, irq, dw_hdmi_top_irq, IRQF_SHARED,
+ "dw_hdmi_top_irq", meson_dw_hdmi);
if (ret)
return dev_err_probe(dev, ret,
"Failed to request hdmi top irq\n");
--
2.54.0
^ permalink raw reply related
* [PATCH 09/13] drm/meson: dw-hdmi: Use devm_clk_get_enabled() helper
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Change to use the devm_clk_get_enabled() helper instead of using an
open-coded variant.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 48 +++++++++------------------
1 file changed, 16 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index fcd2426af9fc..d0cf2042d41c 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -634,29 +634,6 @@ static void meson_dw_hdmi_init(struct meson_dw_hdmi *meson_dw_hdmi)
}
-static void meson_disable_clk(void *data)
-{
- clk_disable_unprepare(data);
-}
-
-static int meson_enable_clk(struct device *dev, char *name)
-{
- struct clk *clk;
- int ret;
-
- clk = devm_clk_get(dev, name);
- if (IS_ERR(clk)) {
- dev_err(dev, "Unable to get %s pclk\n", name);
- return PTR_ERR(clk);
- }
-
- ret = clk_prepare_enable(clk);
- if (!ret)
- ret = devm_add_action_or_reset(dev, meson_disable_clk, clk);
-
- return ret;
-}
-
static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
void *data)
{
@@ -666,6 +643,7 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
struct drm_device *drm = data;
struct meson_drm *priv = drm->dev_private;
struct dw_hdmi_plat_data *dw_plat_data;
+ struct clk *clk;
int irq;
int ret;
@@ -716,17 +694,23 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
if (IS_ERR(meson_dw_hdmi->hdmitx))
return PTR_ERR(meson_dw_hdmi->hdmitx);
- ret = meson_enable_clk(dev, "isfr");
- if (ret)
- return ret;
+ clk = devm_clk_get_enabled(dev, "isfr");
+ if (IS_ERR(clk)) {
+ dev_err(dev, "Unable to get isfr pclk\n");
+ return PTR_ERR(clk);
+ }
- ret = meson_enable_clk(dev, "iahb");
- if (ret)
- return ret;
+ clk = devm_clk_get_enabled(dev, "iahb");
+ if (IS_ERR(clk)) {
+ dev_err(dev, "Unable to get iahb pclk\n");
+ return PTR_ERR(clk);
+ }
- ret = meson_enable_clk(dev, "venci");
- if (ret)
- return ret;
+ clk = devm_clk_get_enabled(dev, "venci");
+ if (IS_ERR(clk)) {
+ dev_err(dev, "Unable to get venci pclk\n");
+ return PTR_ERR(clk);
+ }
dw_plat_data->regm = devm_regmap_init(dev, NULL, meson_dw_hdmi,
&meson_dw_hdmi_regmap_config);
--
2.54.0
^ permalink raw reply related
* [PATCH 06/13] drm/meson: encoder_hdmi: Use bridge connector CEC notifier
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
The dw-hdmi bridge detect() func now updates EDID and CEC phys addr for
the connector and any registered generic CEC notifier.
Replace the open-coded CEC notifier handling with use of the bridge CEC
notifier op.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/Kconfig | 1 +
drivers/gpu/drm/meson/meson_encoder_hdmi.c | 85 +++++-----------------
2 files changed, 19 insertions(+), 67 deletions(-)
diff --git a/drivers/gpu/drm/meson/Kconfig b/drivers/gpu/drm/meson/Kconfig
index 417f79829cf8..0dc5ca21e4fc 100644
--- a/drivers/gpu/drm/meson/Kconfig
+++ b/drivers/gpu/drm/meson/Kconfig
@@ -13,6 +13,7 @@ config DRM_MESON
select REGMAP_MMIO
select MESON_CANVAS
select CEC_CORE if CEC_NOTIFIER
+ select DRM_DISPLAY_HDMI_CEC_NOTIFIER_HELPER if CEC_NOTIFIER
config DRM_MESON_DW_HDMI
tristate "HDMI Synopsys Controller support for Amlogic Meson Display"
diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.c b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
index 1b9a1d9ed3d3..45104ef35344 100644
--- a/drivers/gpu/drm/meson/meson_encoder_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
@@ -16,8 +16,6 @@
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
-#include <media/cec-notifier.h>
-
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_bridge_connector.h>
@@ -41,7 +39,6 @@ struct meson_encoder_hdmi {
struct drm_connector *connector;
struct meson_drm *priv;
unsigned long output_bus_fmt;
- struct cec_notifier *cec_notifier;
};
#define bridge_to_meson_encoder_hdmi(x) \
@@ -57,14 +54,6 @@ static int meson_encoder_hdmi_attach(struct drm_bridge *bridge,
&encoder_hdmi->bridge, flags);
}
-static void meson_encoder_hdmi_detach(struct drm_bridge *bridge)
-{
- struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
-
- cec_notifier_conn_unregister(encoder_hdmi->cec_notifier);
- encoder_hdmi->cec_notifier = NULL;
-}
-
static void meson_encoder_hdmi_set_vclk(struct meson_encoder_hdmi *encoder_hdmi,
const struct drm_display_mode *mode)
{
@@ -321,27 +310,9 @@ static int meson_encoder_hdmi_atomic_check(struct drm_bridge *bridge,
return 0;
}
-static void meson_encoder_hdmi_hpd_notify(struct drm_bridge *bridge,
- struct drm_connector *connector,
- enum drm_connector_status status)
-{
- struct meson_encoder_hdmi *encoder_hdmi = bridge_to_meson_encoder_hdmi(bridge);
-
- if (!encoder_hdmi->cec_notifier)
- return;
-
- if (status == connector_status_connected)
- cec_notifier_set_phys_addr(encoder_hdmi->cec_notifier,
- connector->display_info.source_physical_address);
- else
- cec_notifier_phys_addr_invalidate(encoder_hdmi->cec_notifier);
-}
-
static const struct drm_bridge_funcs meson_encoder_hdmi_bridge_funcs = {
.attach = meson_encoder_hdmi_attach,
- .detach = meson_encoder_hdmi_detach,
.mode_valid = meson_encoder_hdmi_mode_valid,
- .hpd_notify = meson_encoder_hdmi_hpd_notify,
.atomic_enable = meson_encoder_hdmi_atomic_enable,
.atomic_disable = meson_encoder_hdmi_atomic_disable,
.atomic_get_input_bus_fmts = meson_encoder_hdmi_get_inp_bus_fmts,
@@ -374,9 +345,9 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
meson_encoder_hdmi->bridge.next_bridge = of_drm_find_and_get_bridge(remote);
if (!meson_encoder_hdmi->bridge.next_bridge) {
- ret = dev_err_probe(priv->dev, -EPROBE_DEFER,
- "Failed to find HDMI transceiver bridge\n");
- goto err_put_node;
+ of_node_put(remote);
+ return dev_err_probe(priv->dev, -EPROBE_DEFER,
+ "Failed to find HDMI transceiver bridge\n");
}
/* HDMI Encoder Bridge */
@@ -384,6 +355,13 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
meson_encoder_hdmi->bridge.type = DRM_MODE_CONNECTOR_HDMIA;
meson_encoder_hdmi->bridge.interlace_allowed = true;
+ pdev = of_find_device_by_node(remote);
+ of_node_put(remote);
+ if (pdev) {
+ meson_encoder_hdmi->bridge.ops |= DRM_BRIDGE_OP_HDMI_CEC_NOTIFIER;
+ meson_encoder_hdmi->bridge.hdmi_cec_dev = &pdev->dev;
+ }
+
drm_bridge_add(&meson_encoder_hdmi->bridge);
meson_encoder_hdmi->priv = priv;
@@ -391,30 +369,24 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
/* Encoder */
ret = drm_simple_encoder_init(priv->drm, &meson_encoder_hdmi->encoder,
DRM_MODE_ENCODER_TMDS);
- if (ret) {
- dev_err_probe(priv->dev, ret, "Failed to init HDMI encoder\n");
- goto err_put_node;
- }
+ if (ret)
+ return dev_err_probe(priv->dev, ret, "Failed to init HDMI encoder\n");
meson_encoder_hdmi->encoder.possible_crtcs = BIT(0);
/* Attach HDMI Encoder Bridge to Encoder */
ret = drm_bridge_attach(&meson_encoder_hdmi->encoder, &meson_encoder_hdmi->bridge, NULL,
DRM_BRIDGE_ATTACH_NO_CONNECTOR);
- if (ret) {
- dev_err_probe(priv->dev, ret, "Failed to attach bridge\n");
- goto err_put_node;
- }
+ if (ret)
+ return dev_err_probe(priv->dev, ret, "Failed to attach bridge\n");
/* Initialize & attach Bridge Connector */
meson_encoder_hdmi->connector = drm_bridge_connector_init(priv->drm,
&meson_encoder_hdmi->encoder);
- if (IS_ERR(meson_encoder_hdmi->connector)) {
- ret = dev_err_probe(priv->dev,
- PTR_ERR(meson_encoder_hdmi->connector),
- "Unable to create HDMI bridge connector\n");
- goto err_put_node;
- }
+ if (IS_ERR(meson_encoder_hdmi->connector))
+ return dev_err_probe(priv->dev,
+ PTR_ERR(meson_encoder_hdmi->connector),
+ "Unable to create HDMI bridge connector\n");
/*
* We should have now in place:
@@ -437,32 +409,11 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
/* Handle this here until handled by drm_bridge_connector_init() */
meson_encoder_hdmi->connector->ycbcr_420_allowed = true;
- pdev = of_find_device_by_node(remote);
- of_node_put(remote);
- if (pdev) {
- struct cec_connector_info conn_info;
- struct cec_notifier *notifier;
-
- cec_fill_conn_info_from_drm(&conn_info, meson_encoder_hdmi->connector);
-
- notifier = cec_notifier_conn_register(&pdev->dev, NULL, &conn_info);
- if (!notifier) {
- put_device(&pdev->dev);
- return -ENOMEM;
- }
-
- meson_encoder_hdmi->cec_notifier = notifier;
- }
-
priv->encoders[MESON_ENC_HDMI] = meson_encoder_hdmi;
dev_dbg(priv->dev, "HDMI encoder initialized\n");
return 0;
-
-err_put_node:
- of_node_put(remote);
- return ret;
}
void meson_encoder_hdmi_remove(struct meson_drm *priv)
--
2.54.0
^ permalink raw reply related
* [PATCH 08/13] drm/meson: dw-hdmi: Use local dev variable consistently in bind()
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Replace indirect struct device accesses via pdev->dev with the local dev
parameter already available in meson_dw_hdmi_bind(), for consistency and
readability.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 30099bf71aad..fcd2426af9fc 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -671,9 +671,9 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
DRM_DEBUG_DRIVER("\n");
- match = of_device_get_match_data(&pdev->dev);
+ match = of_device_get_match_data(dev);
if (!match) {
- dev_err(&pdev->dev, "failed to get match data\n");
+ dev_err(dev, "failed to get match data\n");
return -ENODEV;
}
@@ -771,7 +771,7 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
return PTR_ERR(meson_dw_hdmi->hdmi);
}
- meson_dw_hdmi->bridge = of_drm_find_and_get_bridge(pdev->dev.of_node);
+ meson_dw_hdmi->bridge = of_drm_find_and_get_bridge(dev->of_node);
if (!meson_dw_hdmi->bridge) {
devm_free_irq(dev, irq, meson_dw_hdmi);
dw_hdmi_remove(meson_dw_hdmi->hdmi);
--
2.54.0
^ permalink raw reply related
* [PATCH 02/13] drm/meson: dw-hdmi: Protect from possible NULL pointer dereference
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl, Sam Ravnborg
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
The IRQ handler can be called at any time after the call to
devm_request_threaded_irq() completes, even before dw_hdmi->bridge has
been assigned later in meson_dw_hdmi_bind().
Protect from a possible NULL pointer dereference in IRQ handler by only
calling drm_helper_hpd_irq_event() when bridge has been assigned.
Fixes: e67f6037ae1b ("drm/meson: split out encoder from meson_dw_hdmi")
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
I only observed this NULL pointer dereference one time, without being
able to reliably re-create a similar timing scenario. I still think this
is an issue that possible could happen and likely should be fixed.
Note that patches later in this series will fully replace this change.
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 993f6d5d4b29..eafe7daf6ff1 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -520,7 +520,8 @@ static irqreturn_t dw_hdmi_top_thread_irq(int irq, void *dev_id)
u32 stat = dw_hdmi->irq_stat;
/* HPD Events */
- if (stat & (HDMITX_TOP_INTR_HPD_RISE | HDMITX_TOP_INTR_HPD_FALL)) {
+ if (stat & (HDMITX_TOP_INTR_HPD_RISE | HDMITX_TOP_INTR_HPD_FALL) &&
+ dw_hdmi->bridge) {
bool hpd_connected = false;
if (stat & HDMITX_TOP_INTR_HPD_RISE)
--
2.54.0
^ permalink raw reply related
* [PATCH 03/13] drm/meson: dw-hdmi: Call dw_hdmi_remove() consistently
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
dw-hdmi export two similar pair of functions to probe()/remove() and
bind()/unbind(), update to use dw_hdmi_remove() for consistency.
dw_hdmi_probe() will drm_bridge_add() the dw-hdmi bridge, so it is
expected that of_drm_find_and_get_bridge() always return the dw-hdmi
bridge. Regardless, validate that the dw-hdmi can be found for
consistent error handling.
Also always ensure the IRQ handler and bridge is released before
dw_hdmi_remove() is called.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index eafe7daf6ff1..9aafdc768f2b 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -775,10 +775,17 @@ static int meson_dw_hdmi_bind(struct device *dev, struct device *master,
platform_set_drvdata(pdev, meson_dw_hdmi);
meson_dw_hdmi->hdmi = dw_hdmi_probe(pdev, &meson_dw_hdmi->dw_plat_data);
- if (IS_ERR(meson_dw_hdmi->hdmi))
+ if (IS_ERR(meson_dw_hdmi->hdmi)) {
+ devm_free_irq(dev, irq, meson_dw_hdmi);
return PTR_ERR(meson_dw_hdmi->hdmi);
+ }
meson_dw_hdmi->bridge = of_drm_find_and_get_bridge(pdev->dev.of_node);
+ if (!meson_dw_hdmi->bridge) {
+ devm_free_irq(dev, irq, meson_dw_hdmi);
+ dw_hdmi_remove(meson_dw_hdmi->hdmi);
+ return -ENODEV;
+ }
DRM_DEBUG_DRIVER("HDMI controller initialized\n");
@@ -793,8 +800,8 @@ static void meson_dw_hdmi_unbind(struct device *dev, struct device *master,
int irq = platform_get_irq(pdev, 0);
devm_free_irq(dev, irq, meson_dw_hdmi);
- dw_hdmi_unbind(meson_dw_hdmi->hdmi);
drm_bridge_put(meson_dw_hdmi->bridge);
+ dw_hdmi_remove(meson_dw_hdmi->hdmi);
}
static const struct component_ops meson_dw_hdmi_ops = {
--
2.54.0
^ permalink raw reply related
* [PATCH 07/13] drm/meson: encoder_hdmi: Report ycbcr_420_allowed from encoder
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
The bridge connector report ycbcr_420_allowed support when all bridges
in the chain support ycbcr_420_allowed.
Report ycbcr_420_allowed on the encoder bridge so that the bridge
connector automatically can report correct ycbcr_420_allowed support.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_encoder_hdmi.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.c b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
index 45104ef35344..484675cb8284 100644
--- a/drivers/gpu/drm/meson/meson_encoder_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
@@ -354,6 +354,7 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
meson_encoder_hdmi->bridge.of_node = priv->dev->of_node;
meson_encoder_hdmi->bridge.type = DRM_MODE_CONNECTOR_HDMIA;
meson_encoder_hdmi->bridge.interlace_allowed = true;
+ meson_encoder_hdmi->bridge.ycbcr_420_allowed = true;
pdev = of_find_device_by_node(remote);
of_node_put(remote);
@@ -406,9 +407,6 @@ int meson_encoder_hdmi_probe(struct meson_drm *priv)
drm_connector_attach_max_bpc_property(meson_encoder_hdmi->connector, 8, 8);
- /* Handle this here until handled by drm_bridge_connector_init() */
- meson_encoder_hdmi->connector->ycbcr_420_allowed = true;
-
priv->encoders[MESON_ENC_HDMI] = meson_encoder_hdmi;
dev_dbg(priv->dev, "HDMI encoder initialized\n");
--
2.54.0
^ permalink raw reply related
* [PATCH 05/13] drm/meson: encoder_hdmi: Use CEC phys addr from display_info
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
The dw-hdmi bridge detect() func now updates EDID for the connector.
Something that ensures that display_info.source_physical_address has an
updated CEC phys addr when the hpd_notify() func is called.
Change to use display_info source_physical_address directly instead of
re-reading EDID to set the CEC phys addr at HPD interrupt.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_encoder_hdmi.c | 26 ++++------------------
1 file changed, 4 insertions(+), 22 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_encoder_hdmi.c b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
index 55c0601df3c6..1b9a1d9ed3d3 100644
--- a/drivers/gpu/drm/meson/meson_encoder_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_encoder_hdmi.c
@@ -330,28 +330,10 @@ static void meson_encoder_hdmi_hpd_notify(struct drm_bridge *bridge,
if (!encoder_hdmi->cec_notifier)
return;
- if (status == connector_status_connected) {
- const struct drm_edid *drm_edid;
- const struct edid *edid;
-
- drm_edid = drm_bridge_edid_read(encoder_hdmi->bridge.next_bridge,
- encoder_hdmi->connector);
- if (!drm_edid)
- return;
-
- /*
- * FIXME: The CEC physical address should be set using
- * cec_notifier_set_phys_addr(encoder_hdmi->cec_notifier,
- * connector->display_info.source_physical_address) from a path
- * that has read the EDID and called
- * drm_edid_connector_update().
- */
- edid = drm_edid_raw(drm_edid);
-
- cec_notifier_set_phys_addr_from_edid(encoder_hdmi->cec_notifier, edid);
-
- drm_edid_free(drm_edid);
- } else
+ if (status == connector_status_connected)
+ cec_notifier_set_phys_addr(encoder_hdmi->cec_notifier,
+ connector->display_info.source_physical_address);
+ else
cec_notifier_phys_addr_invalidate(encoder_hdmi->cec_notifier);
}
--
2.54.0
^ permalink raw reply related
* [PATCH 04/13] drm/meson: dw-hdmi: Drop call to drm_bridge_hpd_notify()
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
Calls to both drm_helper_hpd_irq_event() and drm_bridge_hpd_notify() in
the IRQ handler causes multiple hotplug uevents and modesets during an
HPD interrupt.
Change to only call drm_helper_hpd_irq_event() in IRQ handler to ensure
only one hotplug uevent is triggered when the connection status or EDID
has changed.
The bridge connectors detect() func help ensure that any hpd_notify()
func is called for all bridges in the chain.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 9aafdc768f2b..30099bf71aad 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -521,17 +521,8 @@ static irqreturn_t dw_hdmi_top_thread_irq(int irq, void *dev_id)
/* HPD Events */
if (stat & (HDMITX_TOP_INTR_HPD_RISE | HDMITX_TOP_INTR_HPD_FALL) &&
- dw_hdmi->bridge) {
- bool hpd_connected = false;
-
- if (stat & HDMITX_TOP_INTR_HPD_RISE)
- hpd_connected = true;
-
+ dw_hdmi->bridge)
drm_helper_hpd_irq_event(dw_hdmi->bridge->dev);
- drm_bridge_hpd_notify(dw_hdmi->bridge,
- hpd_connected ? connector_status_connected
- : connector_status_disconnected);
- }
return IRQ_HANDLED;
}
--
2.54.0
^ permalink raw reply related
* [PATCH 01/13] drm/meson: dw-hdmi: Report connector status based on HPD bit
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jerome Brunet, Martin Blumenstingl
Cc: dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
In-Reply-To: <20260518194744.2483580-1-jonas@kwiboo.se>
G12A added support for RX-SENSE, status for both HPD and RX-SENSE is
reported in the TOP_STAT0 register.
Limit read_hpd() phy ops to only report connected status based on the
HPD status bit in TOP_STAT0, to help ensure that EDID can be read from
the sink in the connector detect() func.
Fixes: 3b7c1237a72a ("drm/meson: Add G12A support for the DW-HDMI Glue")
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
drivers/gpu/drm/meson/meson_dw_hdmi.c | 6 ++++--
drivers/gpu/drm/meson/meson_dw_hdmi.h | 3 +++
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.c b/drivers/gpu/drm/meson/meson_dw_hdmi.c
index 2a8756da569b..993f6d5d4b29 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.c
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.c
@@ -457,9 +457,11 @@ static enum drm_connector_status dw_hdmi_read_hpd(struct dw_hdmi *hdmi,
void *data)
{
struct meson_dw_hdmi *dw_hdmi = (struct meson_dw_hdmi *)data;
+ unsigned int stat;
- return !!dw_hdmi->data->top_read(dw_hdmi, HDMITX_TOP_STAT0) ?
- connector_status_connected : connector_status_disconnected;
+ stat = dw_hdmi->data->top_read(dw_hdmi, HDMITX_TOP_STAT0);
+ return stat & HDMITX_TOP_STAT0_HPD ? connector_status_connected :
+ connector_status_disconnected;
}
static void dw_hdmi_setup_hpd(struct dw_hdmi *hdmi,
diff --git a/drivers/gpu/drm/meson/meson_dw_hdmi.h b/drivers/gpu/drm/meson/meson_dw_hdmi.h
index 08e1c14e4ea0..cb4616daf667 100644
--- a/drivers/gpu/drm/meson/meson_dw_hdmi.h
+++ b/drivers/gpu/drm/meson/meson_dw_hdmi.h
@@ -157,4 +157,7 @@
*/
#define HDMITX_TOP_STAT0 (0x00E)
+#define HDMITX_TOP_STAT0_HPD BIT(0)
+#define HDMITX_TOP_STAT0_RXSENSE BIT(1)
+
#endif /* __MESON_DW_HDMI_H */
--
2.54.0
^ permalink raw reply related
* [PATCH 00/13] drm/meson: dw-hdmi: Misc cleanup and use CEC notifier helpers
From: Jonas Karlman @ 2026-05-18 19:47 UTC (permalink / raw)
To: Neil Armstrong, Kevin Hilman, Heiko Stuebner
Cc: Jerome Brunet, Martin Blumenstingl, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
dri-devel, linux-amlogic, linux-arm-kernel, linux-kernel,
Jonas Karlman, linux-rockchip
This series include misc cleanup of the meson-dw-hdmi driver, changes to
use the bridge CEC notifier op and to use the dw-hdmi delayed work for
HPD event handling.
Patch 1 ensure connector status is based on HPD bit
Patch 2 protect from a possible NULL pointer dereference during bind()
Patch 4 reduce number of hotplug uevents and hpd_notify() calls
Patch 6 changes to use bridge connector CEC notifier
Patch 3,5,7-10 cleanup code for consistency
Patch 11-12 changes to use dw-hdmi HPD delayed work at HPD event
Patch 13 changes to use suspend_late/resume_early/resume_noirq pm ops
This series depends on improvements made in the series "drm: bridge:
dw_hdmi: Misc enable/disable, CEC and EDID cleanup" [1].
[1] https://patchwork.freedesktop.org/series/134727/
This series is part of a multi series effort to:
- drm: bridge: dw_hdmi: Misc enable/disable, CEC and EDID cleanup [v7]
- drm/meson: hdmi: Misc cleanup and use CEC notifier helpers [v1]
- drm/bridge: dw-hdmi: Improve input/output bus format handling
- drm/bridge: dw-hdmi: Convert to a HDMI bridge and use of bridge connector
- drm/bridge: dw-hdmi: Add and use tmds_char_rate_valid() plat data ops
- phy: rockchip: inno-hdmi: Change TMDS rate handling to configure() ops [v4]
- drm/rockchip: dw_hdmi: Misc cleanup and propagate bus format [v2]
- drm/rockchip: dw_hdmi: Enable YCbCr and Deep Color modes
Link to snapshot: https://github.com/Kwiboo/linux-rockchip/commits/next-20260518-rk-hdmi-v5/
Jonas Karlman (13):
drm/meson: dw-hdmi: Report connector status based on HPD bit
drm/meson: dw-hdmi: Protect from possible NULL pointer dereference
drm/meson: dw-hdmi: Call dw_hdmi_remove() consistently
drm/meson: dw-hdmi: Drop call to drm_bridge_hpd_notify()
drm/meson: encoder_hdmi: Use CEC phys addr from display_info
drm/meson: encoder_hdmi: Use bridge connector CEC notifier
drm/meson: encoder_hdmi: Report ycbcr_420_allowed from encoder
drm/meson: dw-hdmi: Use local dev variable consistently in bind()
drm/meson: dw-hdmi: Use devm_clk_get_enabled() helper
drm/meson: dw-hdmi: Use dev_err_probe() to report errors
drm/bridge: dw-hdmi: Export dw_hdmi_schedule_hpd_work() helper
drm/meson: dw-hdmi: Use dw_hdmi_schedule_hpd_work() helper
drm/meson: dw-hdmi: Use suspend_late/resume_early/resume_noirq pm ops
drivers/gpu/drm/bridge/synopsys/dw-hdmi.c | 11 +-
drivers/gpu/drm/meson/Kconfig | 1 +
drivers/gpu/drm/meson/meson_dw_hdmi.c | 169 +++++++++------------
drivers/gpu/drm/meson/meson_dw_hdmi.h | 3 +
drivers/gpu/drm/meson/meson_encoder_hdmi.c | 107 +++----------
include/drm/bridge/dw_hdmi.h | 2 +
6 files changed, 106 insertions(+), 187 deletions(-)
--
2.54.0
^ permalink raw reply
* [PATCH 5/8] iommu/arm-smmu-v3: Precompute the invalidation commands
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
In-Reply-To: <0-v1-5b1ac97a5403+6588f-smmu_tlbi_jgg@nvidia.com>
Store the required cmd data in the tlbi and just copy it out when
processing each item in the invs list. The cmd form only depends on
if the instance supports RIL or not, otherwise it is always the same.
This avoids redundant calculations for each invs entry.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 132 +++++++++++---------
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 11 ++
2 files changed, 81 insertions(+), 62 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 3b0b273fcde829..9aa08f782e8986 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2398,32 +2398,15 @@ static bool arm_smmu_ttl_addr_aligned(u64 address, unsigned int tg,
return !(address & GENMASK_U64(pgsz_lg2 - 1, 0));
}
-static void arm_smmu_cmdq_batch_add_ril(struct arm_smmu_device *smmu,
- struct arm_smmu_cmdq_batch *cmds,
- struct arm_smmu_cmd *cmd, bool leaf,
- u64 address, unsigned int num,
- unsigned int scale, u8 ttl, u8 tg_enc)
-{
- cmd->data[0] |= FIELD_PREP(CMDQ_TLBI_0_NUM, num) |
- FIELD_PREP(CMDQ_TLBI_0_SCALE, scale);
- cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_LEAF, leaf) |
- FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
- FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) | address;
- arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd);
-}
-
/*
- * Issue a single range TLBI command covering [iova, iova+size). Returns true if
- * successful, false if the range is too large for a single command.
+ * Generate a single range TLBI command covering [iova, iova+size). Sets
+ * use_full_inv if the range is too large for a single command.
*
* The algorithm finds the smallest SCALE where the range (in tg-sized pages)
* fits in the 5-bit NUM field (max 32 units of 2^SCALE pages). This may widen
* the invalidation range.
*/
-static bool arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
- struct arm_smmu_cmdq_batch *cmds,
- struct arm_smmu_cmd *cmd,
- struct arm_smmu_tlbi *tlbi)
+static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
{
unsigned int tg_lg2 = tlbi->smmu_domain->tgsz_lg2;
u64 cur_tg = tlbi->iova >> tg_lg2;
@@ -2433,9 +2416,6 @@ static bool arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
unsigned int scale;
u8 ttl = 0;
- if (!tlbi->size)
- return false;
-
/*
* Determine what level the granule is at. For non-leaf, both
* io-pgtable and SVA pass a nominal last-level granule because they
@@ -2452,10 +2432,13 @@ static bool arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
if (num_tg == 1) {
if (!ttl)
ttl = 3;
- arm_smmu_cmdq_batch_add_ril(smmu, cmds, cmd, tlbi->leaf_only,
- cur_tg << tg_lg2, 0, 0, ttl,
- tg_enc);
- return true;
+ tlbi->range.data0 = 0;
+ tlbi->range.data1 =
+ FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
+ FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
+ FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
+ (cur_tg << tg_lg2);
+ return;
}
/*
@@ -2474,7 +2457,8 @@ static bool arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
/*
* Range too large for a single command, use full invalidation.
*/
- return false;
+ tlbi->range.use_full_inv = true;
+ return;
}
/* 16K granule TTL=1 is reserved (Section 4.4.1) */
@@ -2485,38 +2469,31 @@ static bool arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
if (ttl && !arm_smmu_ttl_addr_aligned(cur_tg << tg_lg2, tg_lg2, ttl))
ttl = 0;
- arm_smmu_cmdq_batch_add_ril(smmu, cmds, cmd, tlbi->leaf_only,
- cur_tg << tg_lg2,
- DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1,
- scale, ttl, tg_enc);
- return true;
+ tlbi->range.data0 =
+ FIELD_PREP(CMDQ_TLBI_0_NUM,
+ DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1) |
+ FIELD_PREP(CMDQ_TLBI_0_SCALE, scale);
+ tlbi->range.data1 = FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
+ FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
+ FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
+ (cur_tg << tg_lg2);
}
/*
* One TLBI command per IOTLB entry, assuming the entries are all at least
- * iopte_granule sized. Returns false if too many commands would be needed which
- * indicates too high a latency. The threshold is similar to MAX_DVM_OPS in
- * arch/arm64/include/asm/tlbflush.h for the 4k PAGE_SIZE.
+ * iopte_granule sized. Sets use_full_inv if too many commands would be needed
+ * which indicates too high a latency. The threshold is similar to MAX_DVM_OPS
+ * in arch/arm64/include/asm/tlbflush.h for the 4k PAGE_SIZE.
*/
-static bool arm_smmu_cmdq_batch_add_single(struct arm_smmu_device *smmu,
- struct arm_smmu_cmdq_batch *cmds,
- struct arm_smmu_cmd *cmd,
- struct arm_smmu_tlbi *tlbi)
+static void arm_smmu_tlbi_calc_single(struct arm_smmu_tlbi *tlbi)
{
unsigned long num_ops = tlbi->size / tlbi->iopte_granule;
- unsigned long iova = tlbi->iova;
- unsigned long i;
- if (!num_ops || num_ops > 512)
- return false;
-
- for (i = 0; i < num_ops; i++) {
- cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
- (iova & ~GENMASK_U64(11, 0));
- arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd);
- iova += tlbi->iopte_granule;
+ if (!num_ops || num_ops > 512) {
+ tlbi->single.use_full_inv = true;
+ return;
}
- return true;
+ tlbi->single.num = num_ops;
}
static void arm_smmu_inv_all_cmd(struct arm_smmu_inv *inv,
@@ -2536,16 +2513,32 @@ static bool arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
struct arm_smmu_cmd *cmd,
struct arm_smmu_tlbi *tlbi)
{
+ u64 iova = tlbi->iova;
+ unsigned int i;
+
if (inv->smmu->features & ARM_SMMU_FEAT_RANGE_INV) {
- if (arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, tlbi))
- return false;
- } else {
- if (arm_smmu_cmdq_batch_add_single(inv->smmu, cmds, cmd, tlbi))
- return false;
+ if (tlbi->range.use_full_inv) {
+ arm_smmu_inv_all_cmd(inv, cmds, cmd);
+ return true;
+ }
+ cmd->data[0] |= tlbi->range.data0;
+ cmd->data[1] = tlbi->range.data1;
+ arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, cmd);
+ return false;
}
- arm_smmu_inv_all_cmd(inv, cmds, cmd);
- return true;
+ if (tlbi->single.use_full_inv) {
+ arm_smmu_inv_all_cmd(inv, cmds, cmd);
+ return true;
+ }
+
+ for (i = 0; i < tlbi->single.num; i++) {
+ cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
+ (iova & ~GENMASK_U64(11, 0));
+ iova += tlbi->iopte_granule;
+ arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, cmd);
+ }
+ return false;
}
static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
@@ -2564,8 +2557,8 @@ static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
return false;
}
-static void __arm_smmu_domain_inv_range(struct arm_smmu_tlbi *tlbi,
- struct arm_smmu_invs *invs)
+static void arm_smmu_domain_tlbi_inv(struct arm_smmu_tlbi *tlbi,
+ struct arm_smmu_invs *invs)
{
struct arm_smmu_cmdq_batch cmds = {};
bool used_s12_vmall = false;
@@ -2662,6 +2655,13 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
};
struct arm_smmu_invs *invs;
+ if (!size || size == SIZE_MAX) {
+ tlbi.single.use_full_inv = true;
+ tlbi.range.use_full_inv = true;
+ } else {
+ arm_smmu_tlbi_calc_single(&tlbi);
+ }
+
/*
* An invalidation request must follow some IOPTE change and then load
* an invalidation array. In the meantime, a domain attachment mutates
@@ -2692,6 +2692,14 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
rcu_read_lock();
invs = rcu_dereference(smmu_domain->invs);
+ /* Only precaculate RIL if it will be used. */
+ if (invs->has_range_inv) {
+ if (!tlbi.range.use_full_inv)
+ arm_smmu_tlbi_calc_range(&tlbi);
+ } else {
+ tlbi.range.use_full_inv = true;
+ }
+
/*
* Avoid locking unless ATS is being used. No ATC invalidation can be
* going on after a domain is detached.
@@ -2700,10 +2708,10 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned long flags;
read_lock_irqsave(&invs->rwlock, flags);
- __arm_smmu_domain_inv_range(&tlbi, invs);
+ arm_smmu_domain_tlbi_inv(&tlbi, invs);
read_unlock_irqrestore(&invs->rwlock, flags);
} else {
- __arm_smmu_domain_inv_range(&tlbi, invs);
+ arm_smmu_domain_tlbi_inv(&tlbi, invs);
}
rcu_read_unlock();
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index d58fe91a96325f..b5e214b428d644 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -809,6 +809,17 @@ struct arm_smmu_tlbi {
size_t size;
unsigned int iopte_granule;
bool leaf_only;
+
+ struct {
+ bool use_full_inv;
+ u16 num;
+ } single;
+
+ struct {
+ bool use_full_inv;
+ u32 data0;
+ u64 data1;
+ } range;
};
struct arm_smmu_evtq {
--
2.43.0
^ permalink raw reply related
* [PATCH 8/8] iommu/arm-smmu-v3: Support the DS expansion of RIL's SCALE
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
In-Reply-To: <0-v1-5b1ac97a5403+6588f-smmu_tlbi_jgg@nvidia.com>
If DS is supported then SCALE can go up to 39. Detect the IDR and compute
a scale max that is compatible for the entire invs list.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 16 +++++++++++++---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +++++-
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0841ab053f903e..8b727aef9ac277 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -991,6 +991,8 @@ static inline int arm_smmu_invs_iter_next_cmp(struct arm_smmu_invs *invs_l,
static void arm_smmu_invs_update_caps(struct arm_smmu_invs *invs,
const struct arm_smmu_inv *inv)
{
+ unsigned int scale_max;
+
if (arm_smmu_inv_is_ats(inv))
invs->has_ats = true;
@@ -998,6 +1000,9 @@ static void arm_smmu_invs_update_caps(struct arm_smmu_invs *invs,
return;
invs->has_range_inv = true;
+ scale_max = (inv->smmu->features & ARM_SMMU_FEAT_DS) ? 39 : 31;
+ if (!invs->range_inv_scale_max || scale_max < invs->range_inv_scale_max)
+ invs->range_inv_scale_max = scale_max;
}
/**
@@ -2448,7 +2453,8 @@ static unsigned int arm_smmu_compute_ttl(u8 leaf_bitmap, u8 table_bitmap,
* fits in the 5-bit NUM field (max 32 units of 2^SCALE pages). This may widen
* the invalidation range.
*/
-static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
+static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi,
+ unsigned int scale_max)
{
unsigned int tg_lg2 = tlbi->smmu_domain->tgsz_lg2;
unsigned int ttl = arm_smmu_compute_ttl(
@@ -2492,7 +2498,7 @@ static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
* on the address beyond it must be aligned to tg (so long as TTL=0)
*/
scale = fls64((num_tg - 1) / 32);
- if (scale > 31) {
+ if (scale > scale_max) {
/*
* Range too large for a single command, use full invalidation.
*/
@@ -2743,7 +2749,8 @@ void arm_smmu_domain_tlbi(struct arm_smmu_tlbi *tlbi)
/* Only precaculate RIL if it will be used. */
if (invs->has_range_inv) {
if (!tlbi->range.use_full_inv)
- arm_smmu_tlbi_calc_range(tlbi);
+ arm_smmu_tlbi_calc_range(tlbi,
+ invs->range_inv_scale_max);
} else {
tlbi->range.use_full_inv = true;
}
@@ -5172,6 +5179,9 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
/* Maximum number of outstanding stalls */
smmu->evtq.max_stalls = FIELD_GET(IDR5_STALL_MAX, reg);
+ if (reg & IDR5_DS)
+ smmu->features |= ARM_SMMU_FEAT_DS;
+
/* Page sizes */
if (reg & IDR5_GRAN64K)
smmu->pgsize_bitmap |= SZ_64K | SZ_512M;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index d6c548ade41f01..e56524e7bfb198 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -64,6 +64,7 @@ struct arm_vsmmu;
#define ARM_SMMU_IDR5 0x14
#define IDR5_STALL_MAX GENMASK(31, 16)
+#define IDR5_DS (1 << 7)
#define IDR5_GRAN64K (1 << 6)
#define IDR5_GRAN16K (1 << 5)
#define IDR5_GRAN4K (1 << 4)
@@ -415,7 +416,7 @@ struct arm_smmu_cmd {
#define CMDQ_TLBI_0_NUM GENMASK_ULL(16, 12)
#define CMDQ_TLBI_RANGE_NUM_MAX 31
-#define CMDQ_TLBI_0_SCALE GENMASK_ULL(24, 20)
+#define CMDQ_TLBI_0_SCALE GENMASK_ULL(25, 20)
#define CMDQ_TLBI_0_VMID GENMASK_ULL(47, 32)
#define CMDQ_TLBI_0_ASID GENMASK_ULL(63, 48)
#define CMDQ_TLBI_1_LEAF (1UL << 0)
@@ -756,6 +757,7 @@ static inline bool arm_smmu_inv_is_ats(const struct arm_smmu_inv *inv)
* @rwlock: optional rwlock to fence ATS operations
* @has_ats: flag if the array contains an INV_TYPE_ATS or INV_TYPE_ATS_FULL
* @has_range_inv: flag if any entry's SMMU supports range invalidation
+ * @range_inv_scale_max: max SCALE usable by all range-capable SMMUs
* @rcu: rcu head for kfree_rcu()
* @inv: flexible invalidation array
*
@@ -786,6 +788,7 @@ struct arm_smmu_invs {
rwlock_t rwlock;
bool has_ats;
bool has_range_inv;
+ u8 range_inv_scale_max;
struct rcu_head rcu;
struct arm_smmu_inv inv[] __counted_by(max_invs);
};
@@ -953,6 +956,7 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_HD (1 << 22)
#define ARM_SMMU_FEAT_S2FWB (1 << 23)
#define ARM_SMMU_FEAT_BBML2 (1 << 24)
+#define ARM_SMMU_FEAT_DS (1 << 25)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
--
2.43.0
^ permalink raw reply related
* [PATCH 7/8] iommu/arm-smmu-v3: Change how the tlbi describes the invalidation
From: Jason Gunthorpe @ 2026-05-18 19:43 UTC (permalink / raw)
To: iommu, Joerg Roedel, Jean-Philippe Brucker, linux-arm-kernel,
Robin Murphy, Will Deacon
Cc: David Matlack, Pasha Tatashin, patches, Pranjal Shrivastava,
Samiullah Khawaja, Mostafa Saleh
In-Reply-To: <0-v1-5b1ac97a5403+6588f-smmu_tlbi_jgg@nvidia.com>
The RIL logic has long had a FIXME that there is not enough
information to properly compute the RIL. There is also subtly not
enough information to properly compute the single stride either.
Change tlbi to use the information format that iommupt is going to
use for ARM. This prepares the invalidation code to support iommupt
and fixes two small limitations with the current code.
iommupt is designed to accumulate all invalidation into a single
gather, then the iommu driver should issue a small number of commands
to execute the gather to control invalidation latency. This is in
contrast to io-pgtable-arm.c which generates many gather flushes and
direct walk cache flushes as it progresses.
To accommodate this the gather will accumulate "damage" in bitmaps,
one for leaf changes and one for table changes. This is enough
information for SMMUv3 to compute the proper stride for single
invalidation and to generate ideal hints for range invalidation.
Change the inner workings of the tlbi process to directly use this
new-style gather description with the idea that the iommupt
conversion will just direct assign the gather fields to the tlbi.
Rework the three places creating the tlbi to express their needs in
terms of the new bitmaps.
1) Simple iotlb invalidation always gets a single range of leaf
levels, so it can set a single leaf bit
2) Walk invalidation always gets a single table level so it can set a
single table bit.
This corrects a weakness in the existing design where single
invalidation would walk the entire table level issuing 4k
invalidations, now it will just push a single invalidation.
3) SVA invalidation has no idea what the MM did, so it will set all
the bits in the bitmaps.
This corrects another weakness where the RIL invalidation logic
was generating hints assuming the #2 rules which isn't correct
for SVA.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 28 ++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 163 +++++++++++++-----
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 20 ++-
3 files changed, 157 insertions(+), 54 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index c708fefb053771..d7f88866469846 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -129,17 +129,33 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn,
{
struct arm_smmu_domain *smmu_domain =
container_of(mn, struct arm_smmu_domain, mmu_notifier);
+ unsigned int tg_lg2 = smmu_domain->tgsz_lg2;
struct arm_smmu_tlbi tlbi = {
.smmu_domain = smmu_domain,
- .iova = start,
+ .start = start,
+ .last = end - 1,
/*
- * The mm_types defines vm_end as the first byte after the end
- * address, different from IOMMU subsystem using the last
- * address of an address range.
+ * No information comes from the mm, assume the worst case that
+ * it changed every table level. The way this is hooked into the
+ * mm is tricky, the range won't be expanded to include an
+ * entire table level if one was removed like the iommu gather
+ * does. Thus even if this is a 4k invalidation it may be
+ * including any table level too.
*/
- .size = end - start,
- .iopte_granule = PAGE_SIZE,
+ .table_levels_bitmap = 0xfe,
};
+ unsigned int pmd_lg2sz = (tg_lg2 - 3) * 1 + tg_lg2;
+
+ /*
+ * If the size is small then we can infer the invalidation is PTE only
+ * and set the PTE level only. Otherwise it could be some other
+ * combination so just set them all. This allows RIL to use TTL=3 in
+ * cases of PTE only changes.
+ */
+ if (end - start < BIT_U64(pmd_lg2sz))
+ tlbi.leaf_levels_bitmap = 1;
+ else
+ tlbi.leaf_levels_bitmap = 0xff;
arm_smmu_domain_tlbi(&tlbi);
}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 94f742de90330c..0841ab053f903e 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2312,8 +2312,8 @@ static struct arm_smmu_cmd arm_smmu_atc_inv_to_cmd(u32 sid, int ssid,
* This has the unpleasant side-effect of invalidating all PASID-tagged
* ATC entries within the address range.
*/
- page_start = tlbi->iova >> inval_grain_shift;
- page_end = (tlbi->iova + tlbi->size - 1) >> inval_grain_shift;
+ page_start = tlbi->start >> inval_grain_shift;
+ page_end = tlbi->last >> inval_grain_shift;
/*
* In an ATS Invalidate Request, the address must be aligned on the
@@ -2399,7 +2399,49 @@ static bool arm_smmu_ttl_addr_aligned(u64 address, unsigned int tg,
}
/*
- * Generate a single range TLBI command covering [iova, iova+size). Sets
+ * Compute the TTL hint from leaf/table level bitmaps. 0 ttlt means no hint
+ * invalidate all levels.
+ */
+static unsigned int arm_smmu_compute_ttl(u8 leaf_bitmap, u8 table_bitmap,
+ unsigned int tg)
+{
+ int ttl;
+
+ if (leaf_bitmap) {
+ if (is_power_of_2(leaf_bitmap))
+ ttl = 3 - (int)__ffs(leaf_bitmap);
+ else
+ ttl = 0;
+
+ if (table_bitmap) {
+ int table_ttl = 3 - (int)__ffs(table_bitmap) + 1;
+
+ /*
+ * A RIL invalidation with !leaf_only clears out all
+ * table levels above the leaf level ttl only.
+ */
+ if (table_ttl > ttl)
+ ttl = 0;
+ }
+ } else if (table_bitmap) {
+ ttl = 3 - (int)__ffs(table_bitmap) + 1;
+ } else {
+ /* Both bitmaps zero is not allowed */
+ return 0;
+ }
+
+ /* 16K granule, ARM TTL=1 is reserved (SMMUv3 F.b Section 4.4.1) */
+ if (tg == 14 && ttl == 1)
+ return 0;
+
+ /* ARM levels -1 and 0 cannot be hinted */
+ if (ttl <= 0 || ttl > 3)
+ return 0;
+ return ttl;
+}
+
+/*
+ * Generate a single range TLBI command covering [start, last]. Sets
* use_full_inv if the range is too large for a single command.
*
* The algorithm finds the smallest SCALE where the range (in tg-sized pages)
@@ -2409,20 +2451,13 @@ static bool arm_smmu_ttl_addr_aligned(u64 address, unsigned int tg,
static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
{
unsigned int tg_lg2 = tlbi->smmu_domain->tgsz_lg2;
- u64 cur_tg = tlbi->iova >> tg_lg2;
- u64 last_tg = (tlbi->iova + tlbi->size - 1) >> tg_lg2;
+ unsigned int ttl = arm_smmu_compute_ttl(
+ tlbi->leaf_levels_bitmap, tlbi->table_levels_bitmap, tg_lg2);
+ u64 cur_tg = tlbi->start >> tg_lg2;
+ u64 last_tg = tlbi->last >> tg_lg2;
u64 num_tg = last_tg - cur_tg + 1;
u8 tg_enc = (tg_lg2 - 10) / 2;
unsigned int scale;
- u8 ttl = 0;
-
- /*
- * Determine what level the granule is at. For non-leaf, both
- * io-pgtable and SVA pass a nominal last-level granule because they
- * don't know what level(s) actually apply, so leave TTL=0.
- */
- if (tlbi->leaf_only)
- ttl = 4 - ((ilog2(tlbi->iopte_granule) - 3) / (tg_lg2 - 3));
/*
* SMMUv3 F.b Section 4.4.1: TG!=0, NUM==0, SCALE==0, TTL==0 is
@@ -2430,14 +2465,18 @@ static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
* a TTL hint to target only the exact leaf entry.
*/
if (num_tg == 1) {
- if (!ttl)
+ /*
+ * The two io-pgtable ops filling the tlbi won't generate ttl=0.
+ * sva sets constants for single page that give ttl=3
+ */
+ if (WARN_ON(!ttl))
ttl = 3;
tlbi->range.data0 = 0;
- tlbi->range.data1 =
- FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
- FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
- FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
- (cur_tg << tg_lg2);
+ tlbi->range.data1 = FIELD_PREP(CMDQ_TLBI_1_LEAF,
+ !tlbi->table_levels_bitmap) |
+ FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
+ FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
+ (cur_tg << tg_lg2);
return;
}
@@ -2461,10 +2500,6 @@ static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
return;
}
- /* 16K granule TTL=1 is reserved (Section 4.4.1) */
- if (tg_lg2 == 14 && ttl == 1)
- ttl = 0;
-
/* Verify address alignment for the TTL hint */
if (ttl && !arm_smmu_ttl_addr_aligned(cur_tg << tg_lg2, tg_lg2, ttl))
ttl = 0;
@@ -2473,27 +2508,52 @@ static void arm_smmu_tlbi_calc_range(struct arm_smmu_tlbi *tlbi)
FIELD_PREP(CMDQ_TLBI_0_NUM,
DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1) |
FIELD_PREP(CMDQ_TLBI_0_SCALE, scale);
- tlbi->range.data1 = FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
- FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
- FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
- (cur_tg << tg_lg2);
+ tlbi->range.data1 =
+ FIELD_PREP(CMDQ_TLBI_1_LEAF, !tlbi->table_levels_bitmap) |
+ FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) |
+ FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) |
+ (cur_tg << tg_lg2);
}
/*
- * One TLBI command per IOTLB entry, assuming the entries are all at least
- * iopte_granule sized. Sets use_full_inv if too many commands would be needed
- * which indicates too high a latency. The threshold is similar to MAX_DVM_OPS
- * in arch/arm64/include/asm/tlbflush.h for the 4k PAGE_SIZE.
+ * Compute the stride for non-RIL single-page invalidation. Returns the log2
+ * stride of the lowest affected level. Single invalidation removes all IOPTEs
+ * that contain the IOVA invalidated, and we can reliably assume that the
+ * architected page size and table sizes (not contiguous!) are reflected in the
+ * IOTLB. Thus if there is a 2M leaf entry we only need to issue a single IOTLB
+ * invalidation within that 2M IOVA.
+ */
+static u8 arm_smmu_tlbi_calc_stride(struct arm_smmu_tlbi *tlbi)
+{
+ unsigned int tg_lg2 = tlbi->smmu_domain->tgsz_lg2;
+ u8 combined = tlbi->table_levels_bitmap | tlbi->leaf_levels_bitmap;
+
+ if (!combined)
+ return U8_MAX;
+ return (tg_lg2 - 3) * __ffs(combined) + tg_lg2;
+}
+
+/*
+ * One TLBI command per stride-sized entry. Sets use_full_inv if too many
+ * commands would be needed. The threshold is similar to MAX_DVM_OPS in
+ * arch/arm64/include/asm/tlbflush.h.
*/
static void arm_smmu_tlbi_calc_single(struct arm_smmu_tlbi *tlbi)
{
- unsigned long num_ops = tlbi->size / tlbi->iopte_granule;
+ u8 stride_lg2 = arm_smmu_tlbi_calc_stride(tlbi);
+ unsigned long num_ops;
+ if (stride_lg2 == U8_MAX) {
+ tlbi->single.use_full_inv = true;
+ return;
+ }
+ num_ops = (tlbi->last - tlbi->start + 1) >> stride_lg2;
if (!num_ops || num_ops > 512) {
tlbi->single.use_full_inv = true;
return;
}
tlbi->single.num = num_ops;
+ tlbi->single.stride_lg2 = stride_lg2;
}
static void arm_smmu_inv_all_cmd(struct arm_smmu_inv *inv,
@@ -2513,7 +2573,7 @@ static bool arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
struct arm_smmu_cmd *cmd,
struct arm_smmu_tlbi *tlbi)
{
- u64 iova = tlbi->iova;
+ u64 iova = tlbi->start;
unsigned int i;
if (inv->smmu->features & ARM_SMMU_FEAT_RANGE_INV) {
@@ -2533,9 +2593,10 @@ static bool arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
}
for (i = 0; i < tlbi->single.num; i++) {
- cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_LEAF, tlbi->leaf_only) |
+ cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_LEAF,
+ !tlbi->table_levels_bitmap) |
(iova & ~GENMASK_U64(11, 0));
- iova += tlbi->iopte_granule;
+ iova += BIT_U64(tlbi->single.stride_lg2);
arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, cmd);
}
return false;
@@ -2714,15 +2775,21 @@ static void arm_smmu_tlb_inv_page_nosync(struct iommu_iotlb_gather *gather,
iommu_iotlb_gather_add_page(domain, gather, iova, granule);
}
+/*
+ * Called by io-pgtable-arm.c for each single table level it wants to remove.
+ * size is the size of the table level and granule is the tg in bytes.
+ */
static void arm_smmu_tlb_inv_walk(unsigned long iova, size_t size,
size_t granule, void *cookie)
{
struct arm_smmu_domain *smmu_domain = cookie;
+ unsigned int tg_lg2 = smmu_domain->tgsz_lg2;
struct arm_smmu_tlbi tlbi = {
.smmu_domain = smmu_domain,
- .iova = iova,
- .size = size,
- .iopte_granule = granule,
+ .start = iova,
+ .last = iova + size - 1,
+ .table_levels_bitmap =
+ BIT((ilog2(size) - tg_lg2) / (tg_lg2 - 3)),
};
arm_smmu_domain_tlbi(&tlbi);
@@ -3984,15 +4051,23 @@ static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain)
arm_smmu_tlb_inv_context(smmu_domain);
}
+/*
+ * Called by io-pgtable-arm.c for each run of same pgsize leaf only
+ * invalidation. If it has to change to a different leaf level then it flushes
+ * the gather and starts a fresh one. Thus this always targets only a single
+ * leaf level.
+ */
static void arm_smmu_iotlb_sync(struct iommu_domain *domain,
struct iommu_iotlb_gather *gather)
{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ unsigned int tg = smmu_domain->tgsz_lg2;
struct arm_smmu_tlbi tlbi = {
- .smmu_domain = to_smmu_domain(domain),
- .iova = gather->start,
- .size = gather->end - gather->start + 1,
- .iopte_granule = gather->pgsize,
- .leaf_only = true,
+ .smmu_domain = smmu_domain,
+ .start = gather->start,
+ .last = gather->end,
+ .leaf_levels_bitmap =
+ BIT((ilog2(gather->pgsize) - tg) / (tg - 3)),
};
if (!gather->pgsize)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 95da62d64df171..d6c548ade41f01 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -805,14 +805,26 @@ static inline struct arm_smmu_invs *arm_smmu_invs_alloc(size_t num_invs)
struct arm_smmu_tlbi {
struct arm_smmu_domain *smmu_domain;
- unsigned long iova;
- size_t size;
- unsigned int iopte_granule;
- bool leaf_only;
+ unsigned long start;
+ unsigned long last;
+ /*
+ * Level bitmaps use iommupt numbering: bit 0 is the leaf-only level
+ * (ARM level 3), bit 1 is the next level up (ARM level 2), etc. These
+ * match the iommu_iotlb_gather.pt fields. Each set bit indicates a
+ * change at that level. The contiguous hint has no effect on
+ * invalidation processing because HW can ignore the hint.
+ *
+ * If leaf_levels_bitmap is 0 then this is a walk cache only
+ * invalidation. If table_levels_bitmap is 0 then this is a leaf only
+ * invalidation.
+ */
+ u8 leaf_levels_bitmap;
+ u8 table_levels_bitmap;
struct {
bool use_full_inv;
u16 num;
+ u8 stride_lg2;
} single;
struct {
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox