[PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
---
 arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
index cff34d1c74b60..cd34ce5dfd63a 100644
--- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
+++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
@@ -792,8 +792,8 @@ soc: soc@0 {
 
 		#address-cells = <2>;
 		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 
 		gcc: clock-controller@100000 {
 			compatible = "qcom,x1e80100-gcc";
-- 
2.51.0

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Stephan Gerhold]

On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
> Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
> The upper address space is used to support more than 32GB of memory.
> 
> This fixes issues when DMA buffers are allocated outside the 36-bit range.
> 
> Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
> Signed-off-by: Jonathan Marek <jonathan@marek.ca>
> ---
>  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> index cff34d1c74b60..cd34ce5dfd63a 100644
> --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> @@ -792,8 +792,8 @@ soc: soc@0 {
>  
>  		#address-cells = <2>;
>  		#size-cells = <2>;
> -		dma-ranges = <0 0 0 0 0x10 0>;
> -		ranges = <0 0 0 0 0x10 0>;
> +		dma-ranges = <0 0 0 0 0x100 0>;
> +		ranges = <0 0 0 0 0x100 0>;
>  

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both
(somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
we could add your proposed change inside x1-el2.dtso. I'm not sure which
issues we are fixing with that though (besides correctness of the
hardware description). In EL2, all DMA devices should be behind an
IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
addresses (DMA addresses) that are given to the devices. The IOMMU maps
the DMA buffers to arbitrary physical memory addresses (including
outside of the 36-bit range, dma-ranges limits only the DMA address).

I would expect that applying your change effectively just enlarges the
I/O virtual address space, which will then be 40-bit instead of just
36-bit. For most devices, even 32-bit of virtual address space should be
enough. A larger address space will only be applied for drivers that
explicitly request a larger DMA mask (e.g. the nvme driver).

We can make this change for correctness, but given that it is only about
the IOVA space, there shouldn't be much functional difference.

# EL1 with Gunyah hypervisor

For EL1, the hypervisor firmware used on most retail laptops limits the
usable DMA memory in the SMMUs to the physical 36-bit range. You are
right that laptops with 64 GiB memory are essentially unusable in EL1
without disabling the physical memory outside the 36-bit range, but
applying this patch would make it even worse.

There are two separate cases:

 - For devices behind the SMMUv2, the situation should be the same as
   above. Increased IOVA space, but no effect on physical address range.
   This is what is currently causing crashes with 64 GiB RAM in EL1.

 - Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when
   running in EL1. In this case, the 36-bit dma-ranges prevents PCIe
   devices from using memory outside the 36-bit range. They will fall
   back to bounce buffers in that case. Applying your patch will disable
   that, making it even more likely to crash than before.

Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup
with Gunyah hypervisor, I don't think it makes sense to apply this patch
as-is. It will just make it even more likely to crash than before.
I suggest adding these overrides in x1-el2.dtso, with the expected
limited effect I described above.

Thanks,
Stephan

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Konrad Dybcio]

On 11/28/25 11:26 AM, Stephan Gerhold wrote:
> On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
>> Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
>> The upper address space is used to support more than 32GB of memory.
>>
>> This fixes issues when DMA buffers are allocated outside the 36-bit range.
>>
>> Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
>> Signed-off-by: Jonathan Marek <jonathan@marek.ca>
>> ---
>>  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> index cff34d1c74b60..cd34ce5dfd63a 100644
>> --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> @@ -792,8 +792,8 @@ soc: soc@0 {
>>  
>>  		#address-cells = <2>;
>>  		#size-cells = <2>;
>> -		dma-ranges = <0 0 0 0 0x10 0>;
>> -		ranges = <0 0 0 0 0x10 0>;
>> +		dma-ranges = <0 0 0 0 0x100 0>;
>> +		ranges = <0 0 0 0 0x100 0>;
>>  
> 
> Could you clarify which "issues" (crashes?) you are referring to?
> 
> We need to distinguish two distinct use cases here, which are both
> (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
> with the regular DTB and in EL2 with the x1-el2.dtbo applied.
> 
> # EL2 with x1-el2.dtbo
> 
> For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
> we could add your proposed change inside x1-el2.dtso. I'm not sure which
> issues we are fixing with that though (besides correctness of the
> hardware description). In EL2, all DMA devices should be behind an
> IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
> addresses (DMA addresses) that are given to the devices. The IOMMU maps
> the DMA buffers to arbitrary physical memory addresses (including
> outside of the 36-bit range, dma-ranges limits only the DMA address).

I've been carrying something similar in my working tree for quite
some time too.. The USB4 PCIe controllers have BAR spaces in the >36b
region, so this will be necessary anyway.

As for the broken-firmware laptops, there's only so much we can do.
A fix for this has been *long* released, but it's up to the OEMs to
pull it in.


I'm not fully sure, but I think certain subsystems still have the 36b
address limitation (camera?), so it would be good to know whether that
needs to be accounted for

Konrad

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

On 11/28/25 5:52 AM, Konrad Dybcio wrote:
> On 11/28/25 11:26 AM, Stephan Gerhold wrote:
>> On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
>>> Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
>>> The upper address space is used to support more than 32GB of memory.
>>>
>>> This fixes issues when DMA buffers are allocated outside the 36-bit range.
>>>
>>> Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
>>> Signed-off-by: Jonathan Marek <jonathan@marek.ca>
>>> ---
>>>   arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
>>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>> index cff34d1c74b60..cd34ce5dfd63a 100644
>>> --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>> +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>> @@ -792,8 +792,8 @@ soc: soc@0 {
>>>   
>>>   		#address-cells = <2>;
>>>   		#size-cells = <2>;
>>> -		dma-ranges = <0 0 0 0 0x10 0>;
>>> -		ranges = <0 0 0 0 0x10 0>;
>>> +		dma-ranges = <0 0 0 0 0x100 0>;
>>> +		ranges = <0 0 0 0 0x100 0>;
>>>   
>>
>> Could you clarify which "issues" (crashes?) you are referring to?
>>
>> We need to distinguish two distinct use cases here, which are both
>> (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
>> with the regular DTB and in EL2 with the x1-el2.dtbo applied.
>>
>> # EL2 with x1-el2.dtbo
>>
>> For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
>> we could add your proposed change inside x1-el2.dtso. I'm not sure which
>> issues we are fixing with that though (besides correctness of the
>> hardware description). In EL2, all DMA devices should be behind an
>> IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
>> addresses (DMA addresses) that are given to the devices. The IOMMU maps
>> the DMA buffers to arbitrary physical memory addresses (including
>> outside of the 36-bit range, dma-ranges limits only the DMA address).
> 
> I've been carrying something similar in my working tree for quite
> some time too.. The USB4 PCIe controllers have BAR spaces in the >36b
> region, so this will be necessary anyway.
> 
> As for the broken-firmware laptops, there's only so much we can do.
> A fix for this has been *long* released, but it's up to the OEMs to
> pull it in.
> 
> 
> I'm not fully sure, but I think certain subsystems still have the 36b
> address limitation (camera?), so it would be good to know whether that
> needs to be accounted for
> 
> Konrad
> 

Most devices only support 32-bit address space, and use a 32-bit DMA 
mask (which is the default, I think?) to only get 32-bit virtual 
addresses. Camera driver can set a 36-bit DMA mask if it wants to use 
its whole range.

This patch is about the physical addresses, not virtual. Every device 
can access the full range (without this, the iommu dma driver thinks 
buffers with physical addresses outside 36-bit range are not accessible, 
and tries to use bounce buffers)

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Konrad Dybcio]

On 11/28/25 3:49 PM, Jonathan Marek wrote:
> On 11/28/25 5:52 AM, Konrad Dybcio wrote:
>> On 11/28/25 11:26 AM, Stephan Gerhold wrote:
>>> On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
>>>> Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
>>>> The upper address space is used to support more than 32GB of memory.
>>>>
>>>> This fixes issues when DMA buffers are allocated outside the 36-bit range.
>>>>
>>>> Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
>>>> Signed-off-by: Jonathan Marek <jonathan@marek.ca>
>>>> ---
>>>>   arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
>>>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>>> index cff34d1c74b60..cd34ce5dfd63a 100644
>>>> --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>>> +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>>>> @@ -792,8 +792,8 @@ soc: soc@0 {
>>>>             #address-cells = <2>;
>>>>           #size-cells = <2>;
>>>> -        dma-ranges = <0 0 0 0 0x10 0>;
>>>> -        ranges = <0 0 0 0 0x10 0>;
>>>> +        dma-ranges = <0 0 0 0 0x100 0>;
>>>> +        ranges = <0 0 0 0 0x100 0>;
>>>>   
>>>
>>> Could you clarify which "issues" (crashes?) you are referring to?
>>>
>>> We need to distinguish two distinct use cases here, which are both
>>> (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
>>> with the regular DTB and in EL2 with the x1-el2.dtbo applied.
>>>
>>> # EL2 with x1-el2.dtbo
>>>
>>> For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
>>> we could add your proposed change inside x1-el2.dtso. I'm not sure which
>>> issues we are fixing with that though (besides correctness of the
>>> hardware description). In EL2, all DMA devices should be behind an
>>> IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
>>> addresses (DMA addresses) that are given to the devices. The IOMMU maps
>>> the DMA buffers to arbitrary physical memory addresses (including
>>> outside of the 36-bit range, dma-ranges limits only the DMA address).
>>
>> I've been carrying something similar in my working tree for quite
>> some time too.. The USB4 PCIe controllers have BAR spaces in the >36b
>> region, so this will be necessary anyway.
>>
>> As for the broken-firmware laptops, there's only so much we can do.
>> A fix for this has been *long* released, but it's up to the OEMs to
>> pull it in.
>>
>>
>> I'm not fully sure, but I think certain subsystems still have the 36b
>> address limitation (camera?), so it would be good to know whether that
>> needs to be accounted for
>>
>> Konrad
>>
> 
> Most devices only support 32-bit address space, and use a 32-bit DMA mask (which is the default, I think?) to only get 32-bit virtual addresses. Camera driver can set a 36-bit DMA mask if it wants to use its whole range.

Right

> This patch is about the physical addresses, not virtual. Every device can access the full range (without this, the iommu dma driver thinks buffers with physical addresses outside 36-bit range are not accessible, and tries to use bounce buffers)

Yeah this definitely checks out

With discussions about some broken firmware devices becoming more
obviously broken in mind:

Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>

Konrad

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

On 11/28/25 5:26 AM, Stephan Gerhold wrote:
> On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
>> Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
>> The upper address space is used to support more than 32GB of memory.
>>
>> This fixes issues when DMA buffers are allocated outside the 36-bit range.
>>
>> Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
>> Signed-off-by: Jonathan Marek <jonathan@marek.ca>
>> ---
>>   arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> index cff34d1c74b60..cd34ce5dfd63a 100644
>> --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
>> @@ -792,8 +792,8 @@ soc: soc@0 {
>>   
>>   		#address-cells = <2>;
>>   		#size-cells = <2>;
>> -		dma-ranges = <0 0 0 0 0x10 0>;
>> -		ranges = <0 0 0 0 0x10 0>;
>> +		dma-ranges = <0 0 0 0 0x100 0>;
>> +		ranges = <0 0 0 0 0x100 0>;
>>   
> 
> Could you clarify which "issues" (crashes?) you are referring to?
> 
> We need to distinguish two distinct use cases here, which are both
> (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
> with the regular DTB and in EL2 with the x1-el2.dtbo applied.
> 
> # EL2 with x1-el2.dtbo
> 
> For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
> we could add your proposed change inside x1-el2.dtso. I'm not sure which
> issues we are fixing with that though (besides correctness of the
> hardware description). In EL2, all DMA devices should be behind an
> IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
> addresses (DMA addresses) that are given to the devices. The IOMMU maps
> the DMA buffers to arbitrary physical memory addresses (including
> outside of the 36-bit range, dma-ranges limits only the DMA address).
> 
> I would expect that applying your change effectively just enlarges the
> I/O virtual address space, which will then be 40-bit instead of just
> 36-bit. For most devices, even 32-bit of virtual address space should be
> enough. A larger address space will only be applied for drivers that
> explicitly request a larger DMA mask (e.g. the nvme driver).
> 
> We can make this change for correctness, but given that it is only about
> the IOVA space, there shouldn't be much functional difference.
> 
> # EL1 with Gunyah hypervisor
> 
> For EL1, the hypervisor firmware used on most retail laptops limits the
> usable DMA memory in the SMMUs to the physical 36-bit range. You are
> right that laptops with 64 GiB memory are essentially unusable in EL1
> without disabling the physical memory outside the 36-bit range, but
> applying this patch would make it even worse.
> 
> There are two separate cases:
> 
>   - For devices behind the SMMUv2, the situation should be the same as
>     above. Increased IOVA space, but no effect on physical address range.
>     This is what is currently causing crashes with 64 GiB RAM in EL1.
> 
>   - Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when
>     running in EL1. In this case, the 36-bit dma-ranges prevents PCIe
>     devices from using memory outside the 36-bit range. They will fall
>     back to bounce buffers in that case. Applying your patch will disable
>     that, making it even more likely to crash than before.
> 
> Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup
> with Gunyah hypervisor, I don't think it makes sense to apply this patch
> as-is. It will just make it even more likely to crash than before.
> I suggest adding these overrides in x1-el2.dtso, with the expected
> limited effect I described above.
> 
> Thanks,
> Stephan
> 

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical 
range will try to use bounce buffers. The dma range from the dts is 
compared against the physical address, not the virtual address.

The crash I see is display driver crashes/freezes once a buffer is 
allocated in the upper 36-bit range and it tries to use bounce buffers. 
This can happens very quickly under load.

The same crash would happen for EL1 as well. I wasn't aware of the EL1 
broken firmware when I sent this patch, but instead of display freezing 
I guess the behavior would a hard reset now, which is a bit worse but 
still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of 
broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess 
the Fixes tag can/should be dropped.

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Stephan Gerhold]

On Fri, Nov 28, 2025 at 09:39:52AM -0500, Jonathan Marek wrote:
> On 11/28/25 5:26 AM, Stephan Gerhold wrote:
> > On Thu, Nov 27, 2025 at 04:29:42PM -0500, Jonathan Marek wrote:
> > > Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus.
> > > The upper address space is used to support more than 32GB of memory.
> > > 
> > > This fixes issues when DMA buffers are allocated outside the 36-bit range.
> > > 
> > > Fixes: af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
> > > Signed-off-by: Jonathan Marek <jonathan@marek.ca>
> > > ---
> > >   arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
> > >   1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> > > index cff34d1c74b60..cd34ce5dfd63a 100644
> > > --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> > > +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
> > > @@ -792,8 +792,8 @@ soc: soc@0 {
> > >   		#address-cells = <2>;
> > >   		#size-cells = <2>;
> > > -		dma-ranges = <0 0 0 0 0x10 0>;
> > > -		ranges = <0 0 0 0 0x10 0>;
> > > +		dma-ranges = <0 0 0 0 0x100 0>;
> > > +		ranges = <0 0 0 0 0x100 0>;
> > 
> > Could you clarify which "issues" (crashes?) you are referring to?
> > 
> > We need to distinguish two distinct use cases here, which are both
> > (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor
> > with the regular DTB and in EL2 with the x1-el2.dtbo applied.
> > 
> > # EL2 with x1-el2.dtbo
> > 
> > For EL2, I think the 40-bit dma-ranges should indeed work correctly, so
> > we could add your proposed change inside x1-el2.dtso. I'm not sure which
> > issues we are fixing with that though (besides correctness of the
> > hardware description). In EL2, all DMA devices should be behind an
> > IOMMU. In this case, the dma-ranges limit the size of the I/O virtual
> > addresses (DMA addresses) that are given to the devices. The IOMMU maps
> > the DMA buffers to arbitrary physical memory addresses (including
> > outside of the 36-bit range, dma-ranges limits only the DMA address).
> > 
> > I would expect that applying your change effectively just enlarges the
> > I/O virtual address space, which will then be 40-bit instead of just
> > 36-bit. For most devices, even 32-bit of virtual address space should be
> > enough. A larger address space will only be applied for drivers that
> > explicitly request a larger DMA mask (e.g. the nvme driver).
> > 
> > We can make this change for correctness, but given that it is only about
> > the IOVA space, there shouldn't be much functional difference.
> > 
> > # EL1 with Gunyah hypervisor
> > 
> > For EL1, the hypervisor firmware used on most retail laptops limits the
> > usable DMA memory in the SMMUs to the physical 36-bit range. You are
> > right that laptops with 64 GiB memory are essentially unusable in EL1
> > without disabling the physical memory outside the 36-bit range, but
> > applying this patch would make it even worse.
> > 
> > There are two separate cases:
> > 
> >   - For devices behind the SMMUv2, the situation should be the same as
> >     above. Increased IOVA space, but no effect on physical address range.
> >     This is what is currently causing crashes with 64 GiB RAM in EL1.
> > 
> >   - Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when
> >     running in EL1. In this case, the 36-bit dma-ranges prevents PCIe
> >     devices from using memory outside the 36-bit range. They will fall
> >     back to bounce buffers in that case. Applying your patch will disable
> >     that, making it even more likely to crash than before.
> > 
> > Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup
> > with Gunyah hypervisor, I don't think it makes sense to apply this patch
> > as-is. It will just make it even more likely to crash than before.
> > I suggest adding these overrides in x1-el2.dtso, with the expected
> > limited effect I described above.
> > 
> > Thanks,
> > Stephan
> > 
> 
> I am using EL2.
> 
> Without this patch, DMA buffers allocated in the upper 36-bit physical range
> will try to use bounce buffers. The dma range from the dts is compared
> against the physical address, not the virtual address.

I don't think this is the case for the dma-iommu layer. I debugged a
crash caused by USB in EL1 on a 64 GiB device earlier this year and it
was happily using buffers above the 36-bit physical range without using
bounce buffers. There is some code inside dma-iommu for using swiotlb,
but it's used only for "untrusted" PCI devices and some edge cases with
unaligned/small buffers.

> 
> The crash I see is display driver crashes/freezes once a buffer is allocated
> in the upper 36-bit range and it tries to use bounce buffers. This can
> happens very quickly under load.
> 

You could be right about the MSM display driver though, since that
bypasses dma-iommu and manages the IOMMU itself. I stared at the code a
bit and I'm not immediately seeing where it would end up calling into
swiotlb, but it might be hidden somewhere in the endless nesting.

> The same crash would happen for EL1 as well. I wasn't aware of the EL1
> broken firmware when I sent this patch, but instead of display freezing I
> guess the behavior would a hard reset now, which is a bit worse but still
> unusable unles display/gpu driver is disabled.
> 
> This patch is correct and should be applied regardless of broken-firmware
> EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag
> can/should be dropped.
> 

Please clarify the commit message a bit and mention the two separate use
cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether
to merge it. At the end you are right and using 64 GiB RAM in EL1 is
kind of a lost cause anyway.

Thanks,
Stephan

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

On 11/28/25 11:03 AM, Stephan Gerhold wrote:
> On Fri, Nov 28, 2025 at 09:39:52AM -0500, Jonathan Marek wrote:
>> On 11/28/25 5:26 AM, Stephan Gerhold wrote:

...

>>
>> I am using EL2.
>>
>> Without this patch, DMA buffers allocated in the upper 36-bit physical range
>> will try to use bounce buffers. The dma range from the dts is compared
>> against the physical address, not the virtual address.
> 
> I don't think this is the case for the dma-iommu layer. I debugged a
> crash caused by USB in EL1 on a 64 GiB device earlier this year and it
> was happily using buffers above the 36-bit physical range without using
> bounce buffers. There is some code inside dma-iommu for using swiotlb,
> but it's used only for "untrusted" PCI devices and some edge cases with
> unaligned/small buffers.
> 
>>
>> The crash I see is display driver crashes/freezes once a buffer is allocated
>> in the upper 36-bit range and it tries to use bounce buffers. This can
>> happens very quickly under load.
>>
> 
> You could be right about the MSM display driver though, since that
> bypasses dma-iommu and manages the IOMMU itself. I stared at the code a
> bit and I'm not immediately seeing where it would end up calling into
> swiotlb, but it might be hidden somewhere in the endless nesting.
> 

Looks like you are right about this, MSM driver ends up going through 
dma_direct_map_phys(), which decides to use bounce buffers. I didn't try 
to see if other drivers end up using bounce buffers, but it would make 
sense that only MSM driver is affected.

>> The same crash would happen for EL1 as well. I wasn't aware of the EL1
>> broken firmware when I sent this patch, but instead of display freezing I
>> guess the behavior would a hard reset now, which is a bit worse but still
>> unusable unles display/gpu driver is disabled.
>>
>> This patch is correct and should be applied regardless of broken-firmware
>> EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag
>> can/should be dropped.
>>
> 
> Please clarify the commit message a bit and mention the two separate use
> cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether
> to merge it. At the end you are right and using 64 GiB RAM in EL1 is
> kind of a lost cause anyway.
> 
> Thanks,
> Stephan
> 

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Christopher Obbard]

Hi,

Sorry to hijack this thread, but I've been carrying this patch to
workaround the buggy firmware from Stephan for my Lenovo T14s with
64gb RAM which seems to make Linux boot & work just fine with
systemd-boot.
Of course, it's a hack and not ready for upstream.

On another topic, perhaps we need to add this to
https://gitlab.com/Linaro/arm64-laptops/linux , what do you think
Stephan ?

Chris


commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48
Author: Stephan Gerhold <stephan.gerhold@linaro.org>
Date:   Thu Mar 6 17:41:15 2025 +0000

    HACK: efi-stub: Exclude 40-bit memory

diff --git a/drivers/firmware/efi/libstub/efi-stub.c
b/drivers/firmware/efi/libstub/efi-stub.c
index 874f63b4a383..355a9b2c0818 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -106,6 +106,17 @@ efi_status_t
efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)
 {
        char *cmdline __free(efi_pool) = NULL;
        efi_status_t status;
+       efi_physical_addr_t addr;
+       unsigned long pages;
+
+       addr = 0x0000008800000000;
+       pages = 0x800000000 / EFI_PAGE_SIZE;
+       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,
+                                                EFI_RESERVED_TYPE,
pages, &addr);
+       if (status != EFI_SUCCESS) {
+               efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
+               return EFI_OUT_OF_RESOURCES;
+       }

        /*
         * Get the command line from EFI, using the LOADED_IMAGE




Cheers!

Chris


On Fri, 28 Nov 2025 at 16:36, Jonathan Marek <jonathan@marek.ca> wrote:
>
> On 11/28/25 11:03 AM, Stephan Gerhold wrote:
> > On Fri, Nov 28, 2025 at 09:39:52AM -0500, Jonathan Marek wrote:
> >> On 11/28/25 5:26 AM, Stephan Gerhold wrote:
>
> ...
>
> >>
> >> I am using EL2.
> >>
> >> Without this patch, DMA buffers allocated in the upper 36-bit physical range
> >> will try to use bounce buffers. The dma range from the dts is compared
> >> against the physical address, not the virtual address.
> >
> > I don't think this is the case for the dma-iommu layer. I debugged a
> > crash caused by USB in EL1 on a 64 GiB device earlier this year and it
> > was happily using buffers above the 36-bit physical range without using
> > bounce buffers. There is some code inside dma-iommu for using swiotlb,
> > but it's used only for "untrusted" PCI devices and some edge cases with
> > unaligned/small buffers.
> >
> >>
> >> The crash I see is display driver crashes/freezes once a buffer is allocated
> >> in the upper 36-bit range and it tries to use bounce buffers. This can
> >> happens very quickly under load.
> >>
> >
> > You could be right about the MSM display driver though, since that
> > bypasses dma-iommu and manages the IOMMU itself. I stared at the code a
> > bit and I'm not immediately seeing where it would end up calling into
> > swiotlb, but it might be hidden somewhere in the endless nesting.
> >
>
> Looks like you are right about this, MSM driver ends up going through
> dma_direct_map_phys(), which decides to use bounce buffers. I didn't try
> to see if other drivers end up using bounce buffers, but it would make
> sense that only MSM driver is affected.
>
> >> The same crash would happen for EL1 as well. I wasn't aware of the EL1
> >> broken firmware when I sent this patch, but instead of display freezing I
> >> guess the behavior would a hard reset now, which is a bit worse but still
> >> unusable unles display/gpu driver is disabled.
> >>
> >> This patch is correct and should be applied regardless of broken-firmware
> >> EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag
> >> can/should be dropped.
> >>
> >
> > Please clarify the commit message a bit and mention the two separate use
> > cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether
> > to merge it. At the end you are right and using 64 GiB RAM in EL1 is
> > kind of a lost cause anyway.
> >
> > Thanks,
> > Stephan
> >
>

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Christopher Obbard]

Hi,

On Fri, 28 Nov 2025 at 22:10, Christopher Obbard
<christopher.obbard@linaro.org> wrote:
>
> Hi,
>
> Sorry to hijack this thread, but I've been carrying this patch to
> workaround the buggy firmware from Stephan for my Lenovo T14s with
> 64gb RAM which seems to make Linux boot & work just fine with
> systemd-boot.
> Of course, it's a hack and not ready for upstream.
>
> On another topic, perhaps we need to add this to
> https://gitlab.com/Linaro/arm64-laptops/linux , what do you think
> Stephan ?

Small update, I create an issue to track this bug:
https://gitlab.com/Linaro/arm64-laptops/linux/-/issues/23
and opened a merge request with Stephan's patch:
https://gitlab.com/Linaro/arm64-laptops/linux/-/merge_requests/1


Cheers!

Chris

>
> Chris
>
>
> commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48
> Author: Stephan Gerhold <stephan.gerhold@linaro.org>
> Date:   Thu Mar 6 17:41:15 2025 +0000
>
>     HACK: efi-stub: Exclude 40-bit memory
>
> diff --git a/drivers/firmware/efi/libstub/efi-stub.c
> b/drivers/firmware/efi/libstub/efi-stub.c
> index 874f63b4a383..355a9b2c0818 100644
> --- a/drivers/firmware/efi/libstub/efi-stub.c
> +++ b/drivers/firmware/efi/libstub/efi-stub.c
> @@ -106,6 +106,17 @@ efi_status_t
> efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)
>  {
>         char *cmdline __free(efi_pool) = NULL;
>         efi_status_t status;
> +       efi_physical_addr_t addr;
> +       unsigned long pages;
> +
> +       addr = 0x0000008800000000;
> +       pages = 0x800000000 / EFI_PAGE_SIZE;
> +       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,
> +                                                EFI_RESERVED_TYPE,
> pages, &addr);
> +       if (status != EFI_SUCCESS) {
> +               efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
> +               return EFI_OUT_OF_RESOURCES;
> +       }
>
>         /*
>          * Get the command line from EFI, using the LOADED_IMAGE
>
>
>
>
> Cheers!
>
> Chris
>
>
> On Fri, 28 Nov 2025 at 16:36, Jonathan Marek <jonathan@marek.ca> wrote:
> >
> > On 11/28/25 11:03 AM, Stephan Gerhold wrote:
> > > On Fri, Nov 28, 2025 at 09:39:52AM -0500, Jonathan Marek wrote:
> > >> On 11/28/25 5:26 AM, Stephan Gerhold wrote:
> >
> > ...
> >
> > >>
> > >> I am using EL2.
> > >>
> > >> Without this patch, DMA buffers allocated in the upper 36-bit physical range
> > >> will try to use bounce buffers. The dma range from the dts is compared
> > >> against the physical address, not the virtual address.
> > >
> > > I don't think this is the case for the dma-iommu layer. I debugged a
> > > crash caused by USB in EL1 on a 64 GiB device earlier this year and it
> > > was happily using buffers above the 36-bit physical range without using
> > > bounce buffers. There is some code inside dma-iommu for using swiotlb,
> > > but it's used only for "untrusted" PCI devices and some edge cases with
> > > unaligned/small buffers.
> > >
> > >>
> > >> The crash I see is display driver crashes/freezes once a buffer is allocated
> > >> in the upper 36-bit range and it tries to use bounce buffers. This can
> > >> happens very quickly under load.
> > >>
> > >
> > > You could be right about the MSM display driver though, since that
> > > bypasses dma-iommu and manages the IOMMU itself. I stared at the code a
> > > bit and I'm not immediately seeing where it would end up calling into
> > > swiotlb, but it might be hidden somewhere in the endless nesting.
> > >
> >
> > Looks like you are right about this, MSM driver ends up going through
> > dma_direct_map_phys(), which decides to use bounce buffers. I didn't try
> > to see if other drivers end up using bounce buffers, but it would make
> > sense that only MSM driver is affected.
> >
> > >> The same crash would happen for EL1 as well. I wasn't aware of the EL1
> > >> broken firmware when I sent this patch, but instead of display freezing I
> > >> guess the behavior would a hard reset now, which is a bit worse but still
> > >> unusable unles display/gpu driver is disabled.
> > >>
> > >> This patch is correct and should be applied regardless of broken-firmware
> > >> EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag
> > >> can/should be dropped.
> > >>
> > >
> > > Please clarify the commit message a bit and mention the two separate use
> > > cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether
> > > to merge it. At the end you are right and using 64 GiB RAM in EL1 is
> > > kind of a lost cause anyway.
> > >
> > > Thanks,
> > > Stephan
> > >
> >

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

FYI, for a less hacky solution you can use CONFIG_ARM64_VA_BITS_39, 
arm64 linux as-is refuses to use any memory it can't identity map (and 
the extra 32GB of memory is outside the 39-bit range).

On 11/28/25 5:10 PM, Christopher Obbard wrote:
> Hi,
> 
> Sorry to hijack this thread, but I've been carrying this patch to
> workaround the buggy firmware from Stephan for my Lenovo T14s with
> 64gb RAM which seems to make Linux boot & work just fine with
> systemd-boot.
> Of course, it's a hack and not ready for upstream.
> 
> On another topic, perhaps we need to add this to
> https://gitlab.com/Linaro/arm64-laptops/linux , what do you think
> Stephan ?
> 
> Chris
> 
> 
> commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48
> Author: Stephan Gerhold <stephan.gerhold@linaro.org>
> Date:   Thu Mar 6 17:41:15 2025 +0000
> 
>      HACK: efi-stub: Exclude 40-bit memory
> 
> diff --git a/drivers/firmware/efi/libstub/efi-stub.c
> b/drivers/firmware/efi/libstub/efi-stub.c
> index 874f63b4a383..355a9b2c0818 100644
> --- a/drivers/firmware/efi/libstub/efi-stub.c
> +++ b/drivers/firmware/efi/libstub/efi-stub.c
> @@ -106,6 +106,17 @@ efi_status_t
> efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)
>   {
>          char *cmdline __free(efi_pool) = NULL;
>          efi_status_t status;
> +       efi_physical_addr_t addr;
> +       unsigned long pages;
> +
> +       addr = 0x0000008800000000;
> +       pages = 0x800000000 / EFI_PAGE_SIZE;
> +       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,
> +                                                EFI_RESERVED_TYPE,
> pages, &addr);
> +       if (status != EFI_SUCCESS) {
> +               efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
> +               return EFI_OUT_OF_RESOURCES;
> +       }
> 
>          /*
>           * Get the command line from EFI, using the LOADED_IMAGE
> 
> 
> 
> 
> Cheers!
> 
> Chris
> 
> 

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Steev Klimaszewski]

Hi All,

On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot
the laptop anymore.  It doesn't matter whether I have mem=30GB, nor if I set
CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the
device tree, the laptop just resets.  

I have tried over 20 boots with it applied, as well as trying both booting a
working kernel, and then rebooting as well as cold boots.


-- steev

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Jonathan Marek]

I was curious about this so I tried booting EL1
- 32GB CRD boots fine
- 64GB T14S boots with CONFIG_ARM64_VA_BITS_39 (except display driver 
didn't probe, because it depends on adsp firmware, lol). without 
CONFIG_ARM64_VA_BITS_39 it resets

Anyway making things worse for "64GB+EL1+broken firmware" shouldn't 
block this patch, because "64GB+EL1+broken firmware" needs workarounds 
anyway and this change is needed to use 64GB without issues otherwise.

On 11/30/25 7:13 PM, Steev Klimaszewski wrote:
> Hi All,
> 
> On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot
> the laptop anymore.  It doesn't matter whether I have mem=30GB, nor if I set
> CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the
> device tree, the laptop just resets.
> 
> I have tried over 20 boots with it applied, as well as trying both booting a
> working kernel, and then rebooting as well as cold boots.
> 
> 
> -- steev
> 
.

Re: [PATCH] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

[Steev Klimaszewski]

Hi Jonathan,

On Sun, Nov 30, 2025 at 8:07 PM Jonathan Marek <jonathan@marek.ca> wrote:
>
> I was curious about this so I tried booting EL1
> - 32GB CRD boots fine
> - 64GB T14S boots with CONFIG_ARM64_VA_BITS_39 (except display driver
> didn't probe, because it depends on adsp firmware, lol). without
> CONFIG_ARM64_VA_BITS_39 it resets

Weird, I definitely have CONFIG_ARM64_VA_BITS_39 set here.

>
> Anyway making things worse for "64GB+EL1+broken firmware" shouldn't
> block this patch, because "64GB+EL1+broken firmware" needs workarounds
> anyway and this change is needed to use 64GB without issues otherwise.

I'm not sure I follow this logic; most users when they first get the
system will be stuck in el1 until they get el2 set up, which also
requires workarounds in order to set up.  I don't at all disagree that
the fix is more correct, but I don't like users having broken things
just because the vendor hasn't fixed their firmware yet.
Additionally, distro kernels will default to BITS_52 for arm64
kernels, not 39 because most of them run more places than just
Qualcomm laptops.

>
> On 11/30/25 7:13 PM, Steev Klimaszewski wrote:
> > Hi All,
> >
> > On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot
> > the laptop anymore.  It doesn't matter whether I have mem=30GB, nor if I set
> > CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the
> > device tree, the laptop just resets.
> >
> > I have tried over 20 boots with it applied, as well as trying both booting a
> > working kernel, and then rebooting as well as cold boots.
> >
> >
> > -- steev
> >
> .