[PATCH] Force 4K pages for IO addresses.

[PATCH] Force 4K pages for IO addresses.

[Tony Breeds]

Currently HEA requires 4K pages for IO resources.  Just set the pages size to
IO to 4K.

Signed-off-by: Tony Breeds <tony@bakeyournoodle.com>
---
Paul, please consider this for 2.6.25

 arch/powerpc/mm/hash_utils_64.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

Fixes the following ooops:
Unable to handle kernel paging request for data at address 0xd00008008006c040
Faulting instruction address: 0xc000000000336b08
Oops: Kernel access of bad area, sig: 7 [#1]
SMP NR_CPUS=32 pSeries
Modules linked in:
NIP: c000000000336b08 LR: c000000000336aac CTR: 8000000000138240
REGS: c0000000fcef3190 TRAP: 0300   Not tainted  (2.6.25-rc5.20080317-143135)
MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 24000442  XER: 00000020
DAR: d00008008006c040, DSISR: 0000000042000000
TASK = c0000000fc979190[2781] 'ifconfig' THREAD: c0000000fcef0000 CPU: 7
GPR00: 8000000000000000 c0000000fcef3410 c0000000009cf350 0000000000000000
GPR04: 0000000000000005 0000000000000000 c0000000fc470e50 0000000000000000
GPR08: c0000000fc11fc00 d00008008006c040 f000ffffffffffff 0000000000000000
GPR12: 0000000000000000 c0000000008aed80 0000000028004422 0000000000000000
GPR16: 0000000010120588 00000000100d0000 00000000100b0000 00000000100d0000
GPR20: 0000000010010474 c0000000fea08000 c0000000fcef3cd0 0000000000008914
GPR24: 0000000000000000 0000000000000000 c0000000fe02e000 0000000000000060
GPR28: 0000000000000060 0000000000000000 c00000000094afa8 c0000000fc470e40
NIP [c000000000336b08] .ehea_create_cq+0x1c4/0x26c
LR [c000000000336aac] .ehea_create_cq+0x168/0x26c
Call Trace:
[c0000000fcef3410] [c000000000336aac] .ehea_create_cq+0x168/0x26c (unreliable)
[c0000000fcef34b0] [c00000000032efc4] .ehea_init_port_res+0xd0/0x4c0
[c0000000fcef3570] [c000000000331ba8] .ehea_up+0x134/0x754
[c0000000fcef3670] [c0000000003323c4] .ehea_open+0x84/0x104
[c0000000fcef3710] [c0000000004d55a0] .dev_open+0xb8/0x128
[c0000000fcef37a0] [c0000000004d53e0] .dev_change_flags+0x110/0x218
[c0000000fcef3840] [c000000000528db0] .devinet_ioctl+0x2d4/0x76c
[c0000000fcef3940] [c0000000005293a8] .inet_ioctl+0xe0/0x130
[c0000000fcef39c0] [c0000000004c4590] .sock_ioctl+0x2c0/0x314
[c0000000fcef3a60] [c0000000000fb970] .vfs_ioctl+0x5c/0xf0
[c0000000fcef3b00] [c0000000000fbe34] .do_vfs_ioctl+0x430/0x460
[c0000000fcef3bb0] [c0000000000fbeb0] .sys_ioctl+0x4c/0x88
[c0000000fcef3c60] [c00000000012c364] .dev_ifsioc+0x138/0x324
[c0000000fcef3d40] [c00000000012fdbc] .compat_sys_ioctl+0x398/0x428
[c0000000fcef3e30] [c000000000008748] syscall_exit+0x0/0x40

diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index 32f4161..f399b1d 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -351,9 +351,7 @@ static void __init htab_init_page_sizes(void)
 		mmu_vmalloc_psize = MMU_PAGE_64K;
 		if (mmu_linear_psize == MMU_PAGE_4K)
 			mmu_linear_psize = MMU_PAGE_64K;
-		if (cpu_has_feature(CPU_FTR_CI_LARGE_PAGE))
-			mmu_io_psize = MMU_PAGE_64K;
-		else
+		if (!cpu_has_feature(CPU_FTR_CI_LARGE_PAGE))
 			mmu_ci_restrictions = 1;
 	}
 #endif /* CONFIG_PPC_64K_PAGES */
-- 
1.5.4.3

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!

[PATCH v2] Force 4K pages for IO addresses.

[Tony Breeds]

Currently HEA requires 4K pages for IO resources.  Just set the pages size to
IO to 4K.

Signed-off-by: Tony Breeds <tony@bakeyournoodle.com>
---
* Updated the comment aswell, sorry.
* Paul please consider this for 2.6.25

 arch/powerpc/mm/hash_utils_64.c |   13 +++++--------
 1 files changed, 5 insertions(+), 8 deletions(-)

Fixes the following ooops:
Unable to handle kernel paging request for data at address 0xd00008008006c040
Faulting instruction address: 0xc000000000336b08
Oops: Kernel access of bad area, sig: 7 [#1]
SMP NR_CPUS=32 pSeries
Modules linked in:
NIP: c000000000336b08 LR: c000000000336aac CTR: 8000000000138240
REGS: c0000000fcef3190 TRAP: 0300   Not tainted  (2.6.25-rc5.20080317-143135)
MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 24000442  XER: 00000020
DAR: d00008008006c040, DSISR: 0000000042000000
TASK = c0000000fc979190[2781] 'ifconfig' THREAD: c0000000fcef0000 CPU: 7
GPR00: 8000000000000000 c0000000fcef3410 c0000000009cf350 0000000000000000
GPR04: 0000000000000005 0000000000000000 c0000000fc470e50 0000000000000000
GPR08: c0000000fc11fc00 d00008008006c040 f000ffffffffffff 0000000000000000
GPR12: 0000000000000000 c0000000008aed80 0000000028004422 0000000000000000
GPR16: 0000000010120588 00000000100d0000 00000000100b0000 00000000100d0000
GPR20: 0000000010010474 c0000000fea08000 c0000000fcef3cd0 0000000000008914
GPR24: 0000000000000000 0000000000000000 c0000000fe02e000 0000000000000060
GPR28: 0000000000000060 0000000000000000 c00000000094afa8 c0000000fc470e40
NIP [c000000000336b08] .ehea_create_cq+0x1c4/0x26c
LR [c000000000336aac] .ehea_create_cq+0x168/0x26c
Call Trace:
[c0000000fcef3410] [c000000000336aac] .ehea_create_cq+0x168/0x26c (unreliable)
[c0000000fcef34b0] [c00000000032efc4] .ehea_init_port_res+0xd0/0x4c0
[c0000000fcef3570] [c000000000331ba8] .ehea_up+0x134/0x754
[c0000000fcef3670] [c0000000003323c4] .ehea_open+0x84/0x104
[c0000000fcef3710] [c0000000004d55a0] .dev_open+0xb8/0x128
[c0000000fcef37a0] [c0000000004d53e0] .dev_change_flags+0x110/0x218
[c0000000fcef3840] [c000000000528db0] .devinet_ioctl+0x2d4/0x76c
[c0000000fcef3940] [c0000000005293a8] .inet_ioctl+0xe0/0x130
[c0000000fcef39c0] [c0000000004c4590] .sock_ioctl+0x2c0/0x314
[c0000000fcef3a60] [c0000000000fb970] .vfs_ioctl+0x5c/0xf0
[c0000000fcef3b00] [c0000000000fbe34] .do_vfs_ioctl+0x430/0x460
[c0000000fcef3bb0] [c0000000000fbeb0] .sys_ioctl+0x4c/0x88
[c0000000fcef3c60] [c00000000012c364] .dev_ifsioc+0x138/0x324
[c0000000fcef3d40] [c00000000012fdbc] .compat_sys_ioctl+0x398/0x428
[c0000000fcef3e30] [c000000000008748] syscall_exit+0x0/0x40

diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index 32f4161..8b2c978 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -340,20 +340,17 @@ static void __init htab_init_page_sizes(void)
 	/*
 	 * Pick a size for the ordinary pages. Default is 4K, we support
 	 * 64K for user mappings and vmalloc if supported by the processor.
-	 * We only use 64k for ioremap if the processor
-	 * (and firmware) support cache-inhibited large pages.
-	 * If not, we use 4k and set mmu_ci_restrictions so that
-	 * hash_page knows to switch processes that use cache-inhibited
-	 * mappings to 4k pages.
+	 * We always use 4k for ioremap, however if the processor
+	 * (and firmware) support cache-inhibited large pages we set
+	 * mmu_ci_restrictions so that hash_page knows to switch processes that
+	 * use cache-inhibited mappings to 4k pages.
 	 */
 	if (mmu_psize_defs[MMU_PAGE_64K].shift) {
 		mmu_virtual_psize = MMU_PAGE_64K;
 		mmu_vmalloc_psize = MMU_PAGE_64K;
 		if (mmu_linear_psize == MMU_PAGE_4K)
 			mmu_linear_psize = MMU_PAGE_64K;
-		if (cpu_has_feature(CPU_FTR_CI_LARGE_PAGE))
-			mmu_io_psize = MMU_PAGE_64K;
-		else
+		if (!cpu_has_feature(CPU_FTR_CI_LARGE_PAGE))
 			mmu_ci_restrictions = 1;
 	}
 #endif /* CONFIG_PPC_64K_PAGES */
-- 
1.5.4.3

Re: [PATCH v2] Force 4K pages for IO addresses.

[Olof Johansson]

On Mon, Mar 17, 2008 at 02:54:19PM +1100, Tony Breeds wrote:
> Currently HEA requires 4K pages for IO resources.  Just set the pages size to
> IO to 4K.

Well, that's too bad. Why penalize all platforms for it?

I.e.: Nack, we use 64K iopages on pa6t and it works well. No need to
waste tlb and erat space.


-Olof

Re: [PATCH v2] Force 4K pages for IO addresses.

[Benjamin Herrenschmidt]

On Mon, 2008-03-17 at 19:34 -0500, Olof Johansson wrote:
> On Mon, Mar 17, 2008 at 02:54:19PM +1100, Tony Breeds wrote:
> > Currently HEA requires 4K pages for IO resources.  Just set the pages size to
> > IO to 4K.
> 
> Well, that's too bad. Why penalize all platforms for it?
> 
> I.e.: Nack, we use 64K iopages on pa6t and it works well. No need to
> waste tlb and erat space.

We would have to make that pSeries specific for now I suppose...

We don't have a way to know that there "can" be an EHEA right ? It may
not be in the device-tree at boot and dynamically added to the
partition... And we can't dynamically "demote" the ioremap area like we
do for userspace as we have no tracking of bolted entries hashed in
early during boot (I plan to fix that but that will take a while).

Cheers,
Ben.

Re: [PATCH v2] Force 4K pages for IO addresses.

[Nathan Lynch]

Benjamin Herrenschmidt wrote:
> 
> On Mon, 2008-03-17 at 19:34 -0500, Olof Johansson wrote:
> > On Mon, Mar 17, 2008 at 02:54:19PM +1100, Tony Breeds wrote:
> > > Currently HEA requires 4K pages for IO resources.  Just set the pages size to
> > > IO to 4K.
> > 
> > Well, that's too bad. Why penalize all platforms for it?
> > 
> > I.e.: Nack, we use 64K iopages on pa6t and it works well. No need to
> > waste tlb and erat space.
> 
> We would have to make that pSeries specific for now I suppose...
> 
> We don't have a way to know that there "can" be an EHEA right ? It may
> not be in the device-tree at boot and dynamically added to the
> partition...

The ibm,drc-names property of the root node should have "HEA" strings
in it on systems where EHEA can potentially be present.

Re: [PATCH v2] Force 4K pages for IO addresses.

[Benjamin Herrenschmidt]

On Tue, 2008-03-18 at 09:45 -0500, Nathan Lynch wrote:
> Benjamin Herrenschmidt wrote:
> > 
> > On Mon, 2008-03-17 at 19:34 -0500, Olof Johansson wrote:
> > > On Mon, Mar 17, 2008 at 02:54:19PM +1100, Tony Breeds wrote:
> > > > Currently HEA requires 4K pages for IO resources.  Just set the pages size to
> > > > IO to 4K.
> > > 
> > > Well, that's too bad. Why penalize all platforms for it?
> > > 
> > > I.e.: Nack, we use 64K iopages on pa6t and it works well. No need to
> > > waste tlb and erat space.
> > 
> > We would have to make that pSeries specific for now I suppose...
> > 
> > We don't have a way to know that there "can" be an EHEA right ? It may
> > not be in the device-tree at boot and dynamically added to the
> > partition...
> 
> The ibm,drc-names property of the root node should have "HEA" strings
> in it on systems where EHEA can potentially be present.

Ah good, I didn't know. That will force us to stick an ugly wart
somewhere to test it though, maybe we can just force mmu_ci_restrictions
to 1 ? Paul what do you think ?

Ben.

Re: [PATCH v2] Force 4K pages for IO addresses.

[Tony Breeds]

On Wed, Mar 19, 2008 at 07:31:25AM +1100, Benjamin Herrenschmidt wrote:
> On Tue, 2008-03-18 at 09:45 -0500, Nathan Lynch wrote:
> > The ibm,drc-names property of the root node should have "HEA" strings
> > in it on systems where EHEA can potentially be present.
> 
> Ah good, I didn't know. That will force us to stick an ugly wart
> somewhere to test it though, maybe we can just force mmu_ci_restrictions
> to 1 ? Paul what do you think ?

That's not enough, mmu_io_psize also needs to be 4K.  I'll poke at doing
something platform specific today.

Yours Tony

  linux.conf.au    http://www.marchsouth.org/
  Jan 19 - 24 2009 The Australian Linux Technical Conference!