[RFC 1/2] powerpc/swiotlb: Dont free up allocated SWIOTLB slab on POWER

[RFC 1/2] powerpc/swiotlb: Dont free up allocated SWIOTLB slab on POWER

[Anshuman Khandual]

Even though SWIOTLB slab gets allocated and initialized on powerpc with
swiotlb_init() called during mem_init(), it gets released away again on
POWER platform because 'ppc_swiotlb_enable' never gets set. The function
swiotlb_detect_4g() checks for 4GB memory and then sets the variable
'ppc_swiotlb_enable' which prevents freeing up the SWIOTLB slab. Lets
make POWER platform call swiotlb_detect_4g() during setup_arch() which
will keep the SWIOTLB slab through out the runtime.

A previous commit cf5621032f ("powerpc/64: Limit ZONE_DMA32 to 4GiB in
swiotlb_detect_4g()") enforced 4GB limit on ZONE_DMA32 which is is not
applicable on POWER (CONFIG_PPC_BOOK3S_64) platform. Lets remove this
unnecessary restriction.

After the patch, SWIOTLB slab does not get released.

[0.410992] software IO TLB [mem 0xfbff0000-0xffff0000] (64MB) mapped
at [00000000767f6cb3-000000004a10114f]

Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 arch/powerpc/kernel/dma-swiotlb.c  | 2 +-
 arch/powerpc/kernel/setup-common.c | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/dma-swiotlb.c b/arch/powerpc/kernel/dma-swiotlb.c
index 88f3963ca30f..2255c6dc89db 100644
--- a/arch/powerpc/kernel/dma-swiotlb.c
+++ b/arch/powerpc/kernel/dma-swiotlb.c
@@ -110,7 +110,7 @@ void __init swiotlb_detect_4g(void)
 {
 	if ((memblock_end_of_DRAM() - 1) > 0xffffffff) {
 		ppc_swiotlb_enable = 1;
-#ifdef CONFIG_ZONE_DMA32
+#if defined(CONFIG_ZONE_DMA32) && !defined(CONFIG_PPC_BOOK3S_64)
 		limit_zone_pfn(ZONE_DMA32, (1ULL << 32) >> PAGE_SHIFT);
 #endif
 	}
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index d73ec518ef80..c4db844e0b0d 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -944,6 +944,7 @@ void __init setup_arch(char **cmdline_p)
 	/* Initialize the MMU context management stuff. */
 	mmu_context_init();
 
+	swiotlb_detect_4g();
 #ifdef CONFIG_PPC64
 	/* Interrupt code needs to be 64K-aligned. */
 	if ((unsigned long)_stext & 0xffff)
-- 
2.14.1

[RFC 2/2] powerpc/swiotlb: Make swiotlb_dma_ops available on POWER

[Anshuman Khandual]

Generic swiotlb_dma_ops structure is available under CONFIG_DMA_DIRECT_OPS
configuration. Hence select it on POWER platform.

Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 73ce5dd07642..61a55f9928d9 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -235,6 +235,7 @@ config PPC
 	select SPARSE_IRQ
 	select SYSCTL_EXCEPTION_TRACE
 	select VIRT_TO_BUS			if !PPC64
+	select DMA_DIRECT_OPS			if PPC_BOOK3S_64
 	#
 	# Please keep this list sorted alphabetically.
 	#
-- 
2.14.1

Re: [RFC 1/2] powerpc/swiotlb: Dont free up allocated SWIOTLB slab on POWER

[Michael Ellerman]

Anshuman Khandual <khandual@linux.vnet.ibm.com> writes:
> Even though SWIOTLB slab gets allocated and initialized on powerpc with
> swiotlb_init() called during mem_init(), it gets released away again on
> POWER platform because 'ppc_swiotlb_enable' never gets set. The function
> swiotlb_detect_4g() checks for 4GB memory and then sets the variable
> 'ppc_swiotlb_enable' which prevents freeing up the SWIOTLB slab. Lets
> make POWER platform call swiotlb_detect_4g() during setup_arch() which
> will keep the SWIOTLB slab through out the runtime.
>
> A previous commit cf5621032f ("powerpc/64: Limit ZONE_DMA32 to 4GiB in
> swiotlb_detect_4g()") enforced 4GB limit on ZONE_DMA32 which is is not
> applicable on POWER (CONFIG_PPC_BOOK3S_64) platform. Lets remove this
> unnecessary restriction.

You're using "POWER" to mean Book3S 64-bit, but "POWER" is something
else (the ISA for POWER1/POWER2).

So please just say "Book3S 64-bit" or talk about a specific CPU, eg.
Power9.

> After the patch, SWIOTLB slab does not get released.
>
> [0.410992] software IO TLB [mem 0xfbff0000-0xffff0000] (64MB) mapped
> at [00000000767f6cb3-000000004a10114f]

But we don't want SWIOTLB on any existing Book3S 64-bit platforms, so
leaving it enabled is just wasting memory.

> diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
> index d73ec518ef80..c4db844e0b0d 100644
> --- a/arch/powerpc/kernel/setup-common.c
> +++ b/arch/powerpc/kernel/setup-common.c
> @@ -944,6 +944,7 @@ void __init setup_arch(char **cmdline_p)
>  	/* Initialize the MMU context management stuff. */
>  	mmu_context_init();
>  
> +	swiotlb_detect_4g();

You shouldn't be calling this, your use case has nothing to do with 4GB.

Instead when you detect that you're running under the ultravisor then
you should set ppc_swiotlb_enable = 1.

cheers

Re: [RFC 1/2] powerpc/swiotlb: Dont free up allocated SWIOTLB slab on POWER

[Ram Pai]

On Wed, Apr 04, 2018 at 10:48:31PM +1000, Michael Ellerman wrote:
> Anshuman Khandual <khandual@linux.vnet.ibm.com> writes:
> > Even though SWIOTLB slab gets allocated and initialized on powerpc with
> > swiotlb_init() called during mem_init(), it gets released away again on
> > POWER platform because 'ppc_swiotlb_enable' never gets set. The function
> > swiotlb_detect_4g() checks for 4GB memory and then sets the variable
> > 'ppc_swiotlb_enable' which prevents freeing up the SWIOTLB slab. Lets
> > make POWER platform call swiotlb_detect_4g() during setup_arch() which
> > will keep the SWIOTLB slab through out the runtime.
> >
> > A previous commit cf5621032f ("powerpc/64: Limit ZONE_DMA32 to 4GiB in
> > swiotlb_detect_4g()") enforced 4GB limit on ZONE_DMA32 which is is not
> > applicable on POWER (CONFIG_PPC_BOOK3S_64) platform. Lets remove this
> > unnecessary restriction.
> 
> You're using "POWER" to mean Book3S 64-bit, but "POWER" is something
> else (the ISA for POWER1/POWER2).
> 
> So please just say "Book3S 64-bit" or talk about a specific CPU, eg.
> Power9.
> 
> > After the patch, SWIOTLB slab does not get released.
> >
> > [0.410992] software IO TLB [mem 0xfbff0000-0xffff0000] (64MB) mapped
> > at [00000000767f6cb3-000000004a10114f]
> 
> But we don't want SWIOTLB on any existing Book3S 64-bit platforms, so
> leaving it enabled is just wasting memory.
> 
> > diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
> > index d73ec518ef80..c4db844e0b0d 100644
> > --- a/arch/powerpc/kernel/setup-common.c
> > +++ b/arch/powerpc/kernel/setup-common.c
> > @@ -944,6 +944,7 @@ void __init setup_arch(char **cmdline_p)
> >  	/* Initialize the MMU context management stuff. */
> >  	mmu_context_init();
> >  
> > +	swiotlb_detect_4g();
> 
> You shouldn't be calling this, your use case has nothing to do with 4GB.
> 
> Instead when you detect that you're running under the ultravisor then
> you should set ppc_swiotlb_enable = 1.

Please assume that you will have a interface to detect if you are
running in a protected-environment(AKA secure-environment).

So maybe you can rename swiotlb_detect_4g() to swiotlb_detect() 
and modify that function to enable or disable ppc_swiotlb_enable
depending on the availability of 4g or availability of 
protected-environment?

RP