Re: [PATCH v2 2/5] iommu: sun50i: allocate page tables from below 4 GiB

[Parthiban <parthiban@linumiz.com>]

On 6/17/24 4:10 AM, Andre Przywara wrote:
> The Allwinner IOMMU is a strict 32-bit device, with its input addresses,
> the page table root pointer as well as both level's page tables and also
> the target addresses all required to be below 4GB.
> The Allwinner H6 SoC only supports 32-bit worth of physical addresses
> anyway, so this isn't a problem so far, but the H616 and later SoCs extend
> the PA space beyond 32 bit to accommodate more DRAM.
> To make sure we stay within the 32-bit PA range required by the IOMMU,
> force the memory for the page tables to come from below 4GB. by using
> allocations with the DMA32 flag.
> Also reject any attempt to map target addresses beyond 4GB, and print a
> warning to give users a hint while this fails.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/iommu/sun50i-iommu.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/iommu/sun50i-iommu.c b/drivers/iommu/sun50i-iommu.c
> index dd3f07384624c..20a07f829085d 100644
> --- a/drivers/iommu/sun50i-iommu.c
> +++ b/drivers/iommu/sun50i-iommu.c
> @@ -602,6 +602,14 @@ static int sun50i_iommu_map(struct iommu_domain *domain, unsigned long iova,
>  	u32 *page_table, *pte_addr;
>  	int ret = 0;
>  
> +	/* the IOMMU can only handle 32-bit addresses, both input and output */
> +	if ((uint64_t)paddr >> 32) {
> +		ret = -EINVAL;
> +		dev_warn_once(iommu->dev,
> +			      "attempt to map address beyond 4GB\n");

A133 uses the same IOMMU block and works fine when claimed by the display engine under 1GB
RAM. When used with 4GB,

root@helper-board-a133:~# head /proc/meminfo 
MemTotal:        4021212 kB

root@helper-board-a133:~# dmesg | grep -i -e drm -e iommu
[    0.465613] iommu: Default domain type: Translated
[    0.470892] iommu: DMA domain TLB invalidation policy: strict mode
[    0.914097] sun8i-mixer 6100000.mixer: Adding to iommu group 0
[    1.092400] sun4i-drm display-engine: Adding to iommu group 0
[    1.100044] sun4i-drm display-engine: bound 6100000.mixer (ops 0xffff800080b05dc0)
[    1.108399] sun4i-drm display-engine: bound 6510000.tcon-top (ops 0xffff800080b0a628)
[    1.116694] sun4i-drm display-engine: bound 6511000.lcd-controller (ops 0xffff800080b04220)
[    1.126415] [drm] Initialized sun4i-drm 1.0.0 for display-engine on minor 0
[    1.135110] sun50i-iommu 30f0000.iommu: attempt to map address beyond 4GB
[    1.142559] sun4i-drm display-engine: [drm] *ERROR* fbdev-dma: Failed to setup generic emulation (ret=-12)
[    1.154210] sun4i-drm display-engine: [drm] *ERROR* fbdev-dma: Failed to setup generic emulation (ret=-12)

allocation fails. Limiting the memory mem=2G to the kernel works.

Thanks,
Parthiban

> +		goto out;
> +	}
> +
>  	page_table = sun50i_dte_get_page_table(sun50i_domain, iova, gfp);
>  	if (IS_ERR(page_table)) {
>  		ret = PTR_ERR(page_table);
> @@ -682,7 +690,8 @@ sun50i_iommu_domain_alloc_paging(struct device *dev)
>  	if (!sun50i_domain)
>  		return NULL;
>  
> -	sun50i_domain->dt = iommu_alloc_pages(GFP_KERNEL, get_order(DT_SIZE));
> +	sun50i_domain->dt = iommu_alloc_pages(GFP_KERNEL | GFP_DMA32,
> +					      get_order(DT_SIZE));
>  	if (!sun50i_domain->dt)
>  		goto err_free_domain;
>  
> @@ -997,7 +1006,7 @@ static int sun50i_iommu_probe(struct platform_device *pdev)
>  
>  	iommu->pt_pool = kmem_cache_create(dev_name(&pdev->dev),
>  					   PT_SIZE, PT_SIZE,
> -					   SLAB_HWCACHE_ALIGN,
> +					   SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA32,
>  					   NULL);
>  	if (!iommu->pt_pool)
>  		return -ENOMEM;