[PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Aaron Kling via B4 Relay]

The reason for this is to properly support the spi nor chip on the
Jetson Xavier NX module. Prior to this, it would time out on all
transfers and sometimes even trigger a cbb fault, locking up the entire
unit. With this, reading and writing to the flash memory works as
expected.

---
Aaron Kling (3):
      spi: dt-bindings: tegra: Support IOMMU property for Tegra194 QSPI
      spi: dt-bindings: tegra: Support dma-coherent property for QSPI
      arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

 Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 6 +++++-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi                        | 8 ++++++++
 2 files changed, 13 insertions(+), 1 deletion(-)
---
base-commit: e98d21c170b01ddef366f023bbfcf6b31509fa83
change-id: 20260515-tegra194-qspi-iommu-e4e4644d5fdf

Best regards,
-- 
Aaron Kling <webgeek1234@gmail.com>

[PATCH 1/3] spi: dt-bindings: tegra: Support IOMMU property for Tegra194 QSPI

[Aaron Kling via B4 Relay]

From: Aaron Kling <webgeek1234@gmail.com>

This is supported via an external dma controller

Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
---
 Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml b/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
index 909c204b8adf81..62233eb3101aeb 100644
--- a/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
+++ b/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
@@ -78,7 +78,9 @@ allOf:
         compatible:
           not:
             contains:
-              const: nvidia,tegra234-qspi
+              enum:
+                - nvidia,tegra194-qspi
+                - nvidia,tegra234-qspi
     then:
       properties:
         iommus: false

-- 
2.53.0

[PATCH 2/3] spi: dt-bindings: tegra: Support dma-coherent property for QSPI

[Aaron Kling via B4 Relay]

From: Aaron Kling <webgeek1234@gmail.com>

DMA coherency is supported by at least the Tegra194 and Tegra234
controllers.

Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
---
 Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml b/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
index 62233eb3101aeb..433bca02f86b13 100644
--- a/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
+++ b/Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml
@@ -47,6 +47,8 @@ properties:
   iommus:
     maxItems: 1
 
+  dma-coherent: true
+
 patternProperties:
   "@[0-9a-f]+$":
     type: object

-- 
2.53.0

[PATCH 3/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Aaron Kling via B4 Relay]

From: Aaron Kling <webgeek1234@gmail.com>

Without iommu and dma enabled, flash storage such as the spi-nor on the
p3668 module times out and cannot complete any transfers.

Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
---
 arch/arm64/boot/dts/nvidia/tegra194.dtsi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
index 1d659454a6f9fe..0e0a20befe9e68 100644
--- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
@@ -923,6 +923,10 @@ spi@3270000 {
 				 <&bpmp TEGRA194_CLK_QSPI0_PM>;
 			clock-names = "qspi", "qspi_out";
 			resets = <&bpmp TEGRA194_RESET_QSPI0>;
+			iommus = <&smmu TEGRA194_SID_GPCDMA_0>;
+			dma-coherent;
+			dmas = <&gpcdma 5>, <&gpcdma 5>;
+			dma-names = "rx", "tx";
 			status = "disabled";
 		};
 
@@ -1013,6 +1017,10 @@ spi@3300000 {
 				 <&bpmp TEGRA194_CLK_QSPI1_PM>;
 			clock-names = "qspi", "qspi_out";
 			resets = <&bpmp TEGRA194_RESET_QSPI1>;
+			iommus = <&smmu TEGRA194_SID_GPCDMA_0>;
+			dma-coherent;
+			dmas = <&gpcdma 6>, <&gpcdma 6>;
+			dma-names = "rx", "tx";
 			status = "disabled";
 		};
 

-- 
2.53.0

Re: [PATCH 1/3] spi: dt-bindings: tegra: Support IOMMU property for Tegra194 QSPI

[Krzysztof Kozlowski]

On Fri, May 15, 2026 at 03:35:27PM -0500, Aaron Kling wrote:
> This is supported via an external dma controller
> 
> Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> ---
>  Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>

Best regards,
Krzysztof

Re: [PATCH 2/3] spi: dt-bindings: tegra: Support dma-coherent property for QSPI

[Krzysztof Kozlowski]

On Fri, May 15, 2026 at 03:35:28PM -0500, Aaron Kling wrote:
> DMA coherency is supported by at least the Tegra194 and Tegra234
> controllers.
> 
> Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> ---
>  Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 2 ++
>  1 file changed, 2 insertions(+)

Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>

Best regards,
Krzysztof

Re: [PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Jon Hunter]

On 15/05/2026 21:35, Aaron Kling via B4 Relay wrote:
> The reason for this is to properly support the spi nor chip on the
> Jetson Xavier NX module. Prior to this, it would time out on all
> transfers and sometimes even trigger a cbb fault, locking up the entire
> unit. With this, reading and writing to the flash memory works as
> expected.

What kernel's do you see this on? With the latest mainline/-next I do 
see ...

  tegra-qspi 3270000.spi: cannot use DMA: -19
  tegra-qspi 3270000.spi: falling back to PIO

But I don't see the crash. However, on linux-6.1.y I do see the crash ...

  tegra-qspi 3270000.spi: cannot use DMA: -19
  tegra-qspi 3270000.spi: falling back to PIO
  tegra-qspi 3270000.spi: transfer timeout
  tegra-qspi 3270000.spi: error in transfer, fifo status 0x20400006
  CPU:0, Error: cbb-noc@2300000, irq=15

So I believe recent upstream changes in the Tegra210 QSPI driver have 
fixed this.

This series does fix the issue on linux-6.1.y but I believe that is 
because this is really enabling DMA support and so PIO is still broken. 
Ideally, PIO should work if DMA support is missing in device-tree.

IMO this series simply enables DMA support. May be we should clarify 
this in the commit message, but otherwise, I am fine with these changes.

Jon

-- 
nvpublic

Re: [PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Jon Hunter]

On 19/05/2026 15:43, Jon Hunter wrote:
> 
> On 15/05/2026 21:35, Aaron Kling via B4 Relay wrote:
>> The reason for this is to properly support the spi nor chip on the
>> Jetson Xavier NX module. Prior to this, it would time out on all
>> transfers and sometimes even trigger a cbb fault, locking up the entire
>> unit. With this, reading and writing to the flash memory works as
>> expected.
> 
> What kernel's do you see this on? With the latest mainline/-next I do 
> see ...
> 
>   tegra-qspi 3270000.spi: cannot use DMA: -19
>   tegra-qspi 3270000.spi: falling back to PIO
> 
> But I don't see the crash. However, on linux-6.1.y I do see the crash ...
> 
>   tegra-qspi 3270000.spi: cannot use DMA: -19
>   tegra-qspi 3270000.spi: falling back to PIO
>   tegra-qspi 3270000.spi: transfer timeout
>   tegra-qspi 3270000.spi: error in transfer, fifo status 0x20400006
>   CPU:0, Error: cbb-noc@2300000, irq=15
> 
> So I believe recent upstream changes in the Tegra210 QSPI driver have 
> fixed this.
> 
> This series does fix the issue on linux-6.1.y but I believe that is 
> because this is really enabling DMA support and so PIO is still broken. 
> Ideally, PIO should work if DMA support is missing in device-tree.
> 
> IMO this series simply enables DMA support. May be we should clarify 
> this in the commit message, but otherwise, I am fine with these changes.

BTW, that said. This is a much simpler way to avoid the hang on earlier 
kernels. So I would be happy to get this merged and backport to stable 
as a fix. Again may be we just need to clarify this a bit more in the 
commit message.

Jon

-- 
nvpublic

Re: [PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Aaron Kling]

On Tue, May 19, 2026 at 10:50 AM Jon Hunter <jonathanh@nvidia.com> wrote:
>
>
> On 19/05/2026 15:43, Jon Hunter wrote:
> >
> > On 15/05/2026 21:35, Aaron Kling via B4 Relay wrote:
> >> The reason for this is to properly support the spi nor chip on the
> >> Jetson Xavier NX module. Prior to this, it would time out on all
> >> transfers and sometimes even trigger a cbb fault, locking up the entire
> >> unit. With this, reading and writing to the flash memory works as
> >> expected.
> >
> > What kernel's do you see this on? With the latest mainline/-next I do
> > see ...
> >
> >   tegra-qspi 3270000.spi: cannot use DMA: -19
> >   tegra-qspi 3270000.spi: falling back to PIO
> >
> > But I don't see the crash. However, on linux-6.1.y I do see the crash ...
> >
> >   tegra-qspi 3270000.spi: cannot use DMA: -19
> >   tegra-qspi 3270000.spi: falling back to PIO
> >   tegra-qspi 3270000.spi: transfer timeout
> >   tegra-qspi 3270000.spi: error in transfer, fifo status 0x20400006
> >   CPU:0, Error: cbb-noc@2300000, irq=15
> >
> > So I believe recent upstream changes in the Tegra210 QSPI driver have
> > fixed this.
> >
> > This series does fix the issue on linux-6.1.y but I believe that is
> > because this is really enabling DMA support and so PIO is still broken.
> > Ideally, PIO should work if DMA support is missing in device-tree.
> >
> > IMO this series simply enables DMA support. May be we should clarify
> > this in the commit message, but otherwise, I am fine with these changes.
>
> BTW, that said. This is a much simpler way to avoid the hang on earlier
> kernels. So I would be happy to get this merged and backport to stable
> as a fix. Again may be we just need to clarify this a bit more in the
> commit message.

I could update the commit message to be more like 'Enable DMA support
for Tegra194 QSPI', sure.

The primary kernel version I'm working on currently is 6.18,
specifically googles android common kernel fork of it. I also
replicated the issue on 6.12 and android-mainline which is currently
7.0-ish.

I was not normally seeing issues on boot, though I think I did hit
that a couple times when jumping around versions. Where I had trouble
was active use of the flash chip, like read/writing the slot metadata
for a/b support or writing the flash for inline bootloader updates.
The issue wasn't always triggering a cbb panic, but it was (almost?)
always timing out and failing to read or write the flash chip.

Aaron

Re: [PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Jon Hunter]

On 19/05/2026 17:48, Aaron Kling wrote:

...

> I could update the commit message to be more like 'Enable DMA support
> for Tegra194 QSPI', sure.

Thanks.

> The primary kernel version I'm working on currently is 6.18,
> specifically googles android common kernel fork of it. I also
> replicated the issue on 6.12 and android-mainline which is currently
> 7.0-ish.

So I also observed issues with Linux v6.18. However, Linux v7.0 is 
working fine for me.

> I was not normally seeing issues on boot, though I think I did hit
> that a couple times when jumping around versions. Where I had trouble
> was active use of the flash chip, like read/writing the slot metadata
> for a/b support or writing the flash for inline bootloader updates.
> The issue wasn't always triggering a cbb panic, but it was (almost?)
> always timing out and failing to read or write the flash chip.

OK, then you are doing more significant testing that I am. I was just 
seeing issues on boot, but with Linux v7.0 I no longer see issues on boot.

Jon

-- 
nvpublic

Re: [PATCH 0/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Jon Hunter]

On 19/05/2026 17:48, Aaron Kling wrote:

...

>> BTW, that said. This is a much simpler way to avoid the hang on earlier
>> kernels. So I would be happy to get this merged and backport to stable
>> as a fix. Again may be we just need to clarify this a bit more in the
>> commit message.
> 
> I could update the commit message to be more like 'Enable DMA support
> for Tegra194 QSPI', sure.

We should still describe in the commit message that this is also prevent 
a hang/crash from occurring that is being observed with PIO mode to give 
the full picture.

Jon

-- 
nvpublic

Re: [PATCH 1/3] spi: dt-bindings: tegra: Support IOMMU property for Tegra194 QSPI

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

On Fri, May 15, 2026 at 03:35:27PM -0500, Aaron Kling via B4 Relay wrote:
> From: Aaron Kling <webgeek1234@gmail.com>
> 
> This is supported via an external dma controller

You say this yourself: it's supported via an external controller,
therefore the QSPI controller on Tegra194 (and earlier) does not need
the iommus property because it does not itself perform the DMA.

The GPC DMA has the iommus property, QSPI doesn't need it. This is
different on Tegra234 and later where the QSPI has a built-in DMA
controller and does not rely on GPC DMA any longer.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 2/3] spi: dt-bindings: tegra: Support dma-coherent property for QSPI

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

On Fri, May 15, 2026 at 03:35:28PM -0500, Aaron Kling via B4 Relay wrote:
> From: Aaron Kling <webgeek1234@gmail.com>
> 
> DMA coherency is supported by at least the Tegra194 and Tegra234
> controllers.
> 
> Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> ---
>  Documentation/devicetree/bindings/spi/nvidia,tegra210-quad.yaml | 2 ++
>  1 file changed, 2 insertions(+)

Similarly, DMA coherency does not apply on Tegra194 because the device
itself does not perform DMA. dma-coherent should go on the DMA
controller's DT node, which it already is.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 3/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On Fri, May 15, 2026 at 03:35:29PM -0500, Aaron Kling via B4 Relay wrote:
> From: Aaron Kling <webgeek1234@gmail.com>
> 
> Without iommu and dma enabled, flash storage such as the spi-nor on the
> p3668 module times out and cannot complete any transfers.
> 
> Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> ---
>  arch/arm64/boot/dts/nvidia/tegra194.dtsi | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> index 1d659454a6f9fe..0e0a20befe9e68 100644
> --- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> +++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> @@ -923,6 +923,10 @@ spi@3270000 {
>  				 <&bpmp TEGRA194_CLK_QSPI0_PM>;
>  			clock-names = "qspi", "qspi_out";
>  			resets = <&bpmp TEGRA194_RESET_QSPI0>;
> +			iommus = <&smmu TEGRA194_SID_GPCDMA_0>;
> +			dma-coherent;
> +			dmas = <&gpcdma 5>, <&gpcdma 5>;
> +			dma-names = "rx", "tx";

dmas and dma-names are the only properties that you should need here.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 3/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Aaron Kling]

On Wed, May 20, 2026 at 6:49 AM Thierry Reding <thierry.reding@gmail.com> wrote:
>
> On Fri, May 15, 2026 at 03:35:29PM -0500, Aaron Kling via B4 Relay wrote:
> > From: Aaron Kling <webgeek1234@gmail.com>
> >
> > Without iommu and dma enabled, flash storage such as the spi-nor on the
> > p3668 module times out and cannot complete any transfers.
> >
> > Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> > ---
> >  arch/arm64/boot/dts/nvidia/tegra194.dtsi | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> > diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > index 1d659454a6f9fe..0e0a20befe9e68 100644
> > --- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > +++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > @@ -923,6 +923,10 @@ spi@3270000 {
> >                                <&bpmp TEGRA194_CLK_QSPI0_PM>;
> >                       clock-names = "qspi", "qspi_out";
> >                       resets = <&bpmp TEGRA194_RESET_QSPI0>;
> > +                     iommus = <&smmu TEGRA194_SID_GPCDMA_0>;
> > +                     dma-coherent;
> > +                     dmas = <&gpcdma 5>, <&gpcdma 5>;
> > +                     dma-names = "rx", "tx";
>
> dmas and dma-names are the only properties that you should need here.

This is where I started and it didn't work. Without the iommu
properties, I get endless mmu faults upon accessing the qspi flash
device:

[   62.313833] tegra-mc 2c00000.memory-controller: axisw: secure write
@0x00000003ffffff00: VPR violation ((null))
[   62.324542] arm-smmu 12000000.iommu: Unhandled context fault:
fsr=0x402, iova=0xcd21d000, fsynr=0x80012, cbfrsynra=0x820, cb=1

I know downstream kernels aren't always (normally...) the arbiter of
sanity, but I did pull these from the Nvidia downstream 5.10 kernel,
which is where Nvidia was starting to try to align with mainline
paradigms. All of these properties are set there.

If the qspi node isn't supposed to set iommu directly, then there's a
larger problem here that I don't even know where to begin to look
into. All I can say is that dma fails with faults if they are not set
and works when they are.

Aaron

Re: [PATCH 3/3] arm64: tegra: Add iommu and dma properties for Tegra194 QSPI

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 2849 bytes --]

On Wed, May 20, 2026 at 12:29:49PM -0500, Aaron Kling wrote:
> On Wed, May 20, 2026 at 6:49 AM Thierry Reding <thierry.reding@gmail.com> wrote:
> >
> > On Fri, May 15, 2026 at 03:35:29PM -0500, Aaron Kling via B4 Relay wrote:
> > > From: Aaron Kling <webgeek1234@gmail.com>
> > >
> > > Without iommu and dma enabled, flash storage such as the spi-nor on the
> > > p3668 module times out and cannot complete any transfers.
> > >
> > > Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
> > > ---
> > >  arch/arm64/boot/dts/nvidia/tegra194.dtsi | 8 ++++++++
> > >  1 file changed, 8 insertions(+)
> > >
> > > diff --git a/arch/arm64/boot/dts/nvidia/tegra194.dtsi b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > > index 1d659454a6f9fe..0e0a20befe9e68 100644
> > > --- a/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > > +++ b/arch/arm64/boot/dts/nvidia/tegra194.dtsi
> > > @@ -923,6 +923,10 @@ spi@3270000 {
> > >                                <&bpmp TEGRA194_CLK_QSPI0_PM>;
> > >                       clock-names = "qspi", "qspi_out";
> > >                       resets = <&bpmp TEGRA194_RESET_QSPI0>;
> > > +                     iommus = <&smmu TEGRA194_SID_GPCDMA_0>;
> > > +                     dma-coherent;
> > > +                     dmas = <&gpcdma 5>, <&gpcdma 5>;
> > > +                     dma-names = "rx", "tx";
> >
> > dmas and dma-names are the only properties that you should need here.
> 
> This is where I started and it didn't work. Without the iommu
> properties, I get endless mmu faults upon accessing the qspi flash
> device:
> 
> [   62.313833] tegra-mc 2c00000.memory-controller: axisw: secure write
> @0x00000003ffffff00: VPR violation ((null))
> [   62.324542] arm-smmu 12000000.iommu: Unhandled context fault:
> fsr=0x402, iova=0xcd21d000, fsynr=0x80012, cbfrsynra=0x820, cb=1
> 
> I know downstream kernels aren't always (normally...) the arbiter of
> sanity, but I did pull these from the Nvidia downstream 5.10 kernel,
> which is where Nvidia was starting to try to align with mainline
> paradigms. All of these properties are set there.
> 
> If the qspi node isn't supposed to set iommu directly, then there's a
> larger problem here that I don't even know where to begin to look
> into. All I can say is that dma fails with faults if they are not set
> and works when they are.

I think this is a problem similar to what we experienced with I2C a
little while ago, which we fixed like so:

    cdbf26251d3b ("i2c: tegra: Allocate DMA memory for DMA engine")

There's another patch currently under review that tries to simplify this
further using new support from the dmaengine subsystem:

    https://patchwork.ozlabs.org/project/linux-tegra/patch/20260518114013.62065-2-akhilrajeev@nvidia.com/

Doing something analogous for QSPI should fix this.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]