Re: [PATCH 5.19 319/365] swiotlb: panic if nslabs is too small

Re: [PATCH 5.19 319/365] swiotlb: panic if nslabs is too small

[Dongli Zhang]

Adding Robin, Yu and swiotlb list.

Hi Greg,

There is an on-going discussion whether to revert this patch, because it breaks
a corner case in MIPS when many kernel CONFIGs are not enabled (related to PCI
and device). As a result, MIPS pre-allocates only PAGE_SIZE buffer as swiotlb.

https://lore.kernel.org/all/20220820012031.1285979-1-yuzhao@google.com/

However, the core idea of the patch is to panic on purpose if the swiotlb is
configured with <1MB memory, in order to sync with the remap failure handler in
swiotlb_init_remap().

Therefore, I am waiting for suggestion from Christoph whether (1) to revert this
patch, or (2) enforce the restriction to disallow <1MB allocation.

Thank you very much!

Dongli Zhang

On 8/23/22 1:03 AM, Greg Kroah-Hartman wrote:
> From: Dongli Zhang <dongli.zhang@oracle.com>
> 
> [ Upstream commit 0bf28fc40d89b1a3e00d1b79473bad4e9ca20ad1 ]
> 
> Panic on purpose if nslabs is too small, in order to sync with the remap
> retry logic.
> 
> In addition, print the number of bytes for tlb alloc failure.
> 
> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Sasha Levin <sashal@kernel.org>
> ---
>  kernel/dma/swiotlb.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index 5830dce6081b..f5304e2f6a35 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -242,6 +242,9 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
>  	if (swiotlb_force_disable)
>  		return;
>  
> +	if (nslabs < IO_TLB_MIN_SLABS)
> +		panic("%s: nslabs = %lu too small\n", __func__, nslabs);
> +
>  	/*
>  	 * By default allocate the bounce buffer memory from low memory, but
>  	 * allow to pick a location everywhere for hypervisors with guest
> @@ -254,7 +257,8 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
>  	else
>  		tlb = memblock_alloc_low(bytes, PAGE_SIZE);
>  	if (!tlb) {
> -		pr_warn("%s: failed to allocate tlb structure\n", __func__);
> +		pr_warn("%s: Failed to allocate %zu bytes tlb structure\n",
> +			__func__, bytes);
>  		return;
>  	}
>  
> 

Re: [PATCH 5.19 319/365] swiotlb: panic if nslabs is too small

[Yu Zhao]

On Tue, Aug 23, 2022 at 11:25 AM Dongli Zhang <dongli.zhang@oracle.com> wrote:
>
> Adding Robin, Yu and swiotlb list.

Thanks.

> There is an on-going discussion whether to revert this patch, because it breaks
> a corner case in MIPS

I wouldn't call it a corner case. Cavium Octeon is the major platform
we use to test Debian MIPS ports [1], and 4 out of 5 best-selling
Wi-Fi routers are MIPS-based [2].

[1] https://wiki.debian.org/MIPSPort
[2] https://www.amazon.com/bestsellers/pc/300189

> when many kernel CONFIGs are not enabled (related to PCI
> and device). As a result, MIPS pre-allocates only PAGE_SIZE buffer as swiotlb.
>
> https://lore.kernel.org/all/20220820012031.1285979-1-yuzhao@google.com/
>
> However, the core idea of the patch is to panic on purpose if the swiotlb is
> configured with <1MB memory, in order to sync with the remap failure handler in
> swiotlb_init_remap().
>
> Therefore, I am waiting for suggestion from Christoph whether (1) to revert this
> patch, or (2) enforce the restriction to disallow <1MB allocation.

There are other archs (arm, ppc, riscv, s390, etc.) that call
swiotlb_init(). Have you verified them all?

Re: [PATCH 5.19 319/365] swiotlb: panic if nslabs is too small

[Dongli Zhang]

On 8/23/22 10:51 AM, Yu Zhao wrote:
> On Tue, Aug 23, 2022 at 11:25 AM Dongli Zhang <dongli.zhang@oracle.com> wrote:
>>
>> Adding Robin, Yu and swiotlb list.
> 
> Thanks.
> 
>> There is an on-going discussion whether to revert this patch, because it breaks
>> a corner case in MIPS
> 
> I wouldn't call it a corner case. Cavium Octeon is the major platform
> we use to test Debian MIPS ports [1], and 4 out of 5 best-selling
> Wi-Fi routers are MIPS-based [2].
> 
> [1] https://urldefense.com/v3/__https://wiki.debian.org/MIPSPort__;!!ACWV5N9M2RV99hQ!NDVwU_XfPmfl_OSGxbroJXOjYmdrb-Vmbnx-zq0UIxkYNCetx4ZWdl6KlftLS9F5ORGT4t8F5YapMSSBiA$  
> [2] https://urldefense.com/v3/__https://www.amazon.com/bestsellers/pc/300189__;!!ACWV5N9M2RV99hQ!NDVwU_XfPmfl_OSGxbroJXOjYmdrb-Vmbnx-zq0UIxkYNCetx4ZWdl6KlftLS9F5ORGT4t8F5YacDz0Zlg$  
> 
>> when many kernel CONFIGs are not enabled (related to PCI
>> and device). As a result, MIPS pre-allocates only PAGE_SIZE buffer as swiotlb.
>>
>> https://urldefense.com/v3/__https://lore.kernel.org/all/20220820012031.1285979-1-yuzhao@google.com/__;!!ACWV5N9M2RV99hQ!NDVwU_XfPmfl_OSGxbroJXOjYmdrb-Vmbnx-zq0UIxkYNCetx4ZWdl6KlftLS9F5ORGT4t8F5YbWJyEn2A$  
>>
>> However, the core idea of the patch is to panic on purpose if the swiotlb is
>> configured with <1MB memory, in order to sync with the remap failure handler in
>> swiotlb_init_remap().
>>
>> Therefore, I am waiting for suggestion from Christoph whether (1) to revert this
>> patch, or (2) enforce the restriction to disallow <1MB allocation.
> 
> There are other archs (arm, ppc, riscv, s390, etc.) that call
> swiotlb_init(). Have you verified them all?
> 

The issue is not about swiotlb_init(). It is about swiotlb_adjust_size() where
the 'default_nslabs' is configured to a very small value (e.g., equivalent to
swiotlb=2). I do not see any arch can directly configure 'default_nslabs'.

There are only two callers (archs) of swiotlb_adjust_size(): amd/sev and
mips/cavium-octeon.

About amd/sev, it uses at least IO_TLB_DEFAULT_SIZE so that there is not any issue.

244         size = total_mem * 6 / 100;
245         size = clamp_val(size, IO_TLB_DEFAULT_SIZE, SZ_1G);
246         swiotlb_adjust_size(size);

In this case, only swiotlb=2 is allocated if PAGE_SIZE is 4K.

Thank you very much!

Dongli Zhang

Re: [PATCH 5.19 319/365] swiotlb: panic if nslabs is too small

[Greg Kroah-Hartman]

On Tue, Aug 23, 2022 at 10:25:27AM -0700, Dongli Zhang wrote:
> Adding Robin, Yu and swiotlb list.
> 
> Hi Greg,
> 
> There is an on-going discussion whether to revert this patch, because it breaks
> a corner case in MIPS when many kernel CONFIGs are not enabled (related to PCI
> and device). As a result, MIPS pre-allocates only PAGE_SIZE buffer as swiotlb.
> 
> https://lore.kernel.org/all/20220820012031.1285979-1-yuzhao@google.com/
> 
> However, the core idea of the patch is to panic on purpose if the swiotlb is
> configured with <1MB memory, in order to sync with the remap failure handler in
> swiotlb_init_remap().

Ok, I've dropped it from the 5.19 queue now.  If you want it added to a
future kernel release, just resend it.

thanks,

greg k-h