Re: Kernel crash in 2.6.32.6 / bcm1480 with 16k page size

[David Daney <ddaney@caviumnetworks.com>]

Guenter Roeck wrote:
> On Fri, Jan 29, 2010 at 01:06:20PM -0500, Ralf Baechle wrote:
>> On Fri, Jan 29, 2010 at 09:11:46AM -0800, David Daney wrote:
>>
>>>> So first question would be: Has anyone successfully loaded a 64
>>>> bit mips kernel with 2.6.32 and a page size of 16k or 64k ? This
>>>> would at least help me reducing the problem to sb1.
>>> Yes, I routinely run with both 64K and 16K page sizes on 2.6.32 and
>>> 2.6.33-rc*.  I have not seen any crashes that can not be easily
>>> explained.
>> I can reproduce it with today's 14b7baff3eb4b1b46a592630e6f85ded9264798a.
>> 4K page size works ok, 16K without IPv6 works ok and 16K with IPv6 crashes.
>> Note, I was testing with a non-16K capable userland so ok means userland is
>> reached.
>>
>> Either way, that's good enought to look into things.
>>
> 16k page size works for me with the patch below. Not that I have any idea why;
> this was just a blind test.
> 
> It seems to me that the notes in arch/mips/include/asm/pgtable-64.h regarding
> available virtual memory per page size may contradict with the definition
> of VMALLOC_END. VMALLOC_END-VMALLOC_START increases as the page size increases,
> but the comments indicate that a system with 16k pages should have _less_
> virtual memory available than a system with 4k pages because it only uses
> a 2 level page table.
> 
> Guenter
> 
> ---------------
> 
> git diff arch/mips/include/asm/pgtable-64.h
> diff --git a/arch/mips/include/asm/pgtable-64.h b/arch/mips/include/asm/pgtable-64.h
> index 9cd5089..bd61030 100644
> --- a/arch/mips/include/asm/pgtable-64.h
> +++ b/arch/mips/include/asm/pgtable-64.h
> @@ -110,7 +110,7 @@
>  #define VMALLOC_START          MAP_BASE
>  #define VMALLOC_END    \
>         (VMALLOC_START + \
> -        PTRS_PER_PGD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE - (1UL << 32))
> +        (PTRS_PER_PGD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE / 16) - (1UL << 32))
>  #if defined(CONFIG_MODULES) && defined(KBUILD_64BIT_SYM32) && \
>         VMALLOC_START != CKSSEG
>  /* Load modules into 32bit-compatible segment. */

Although it may fix it, I think something along the lines of this:

In asm/mach-generic/spaces.h:
#define MAX_VIRTUAL_SIZE (1 << some number)

In asm/mach-sibyte/paces.h:
#define MAX_VIRTUAL_SIZE (1 << some other umber)

In arch/mips/include/asm/pgtable-64.h

#define VIRTUAL_SIZE (PTRS_PER_PGD * PTRS_PER_PMD * PTRS_PER_PTE * 
PAGE_SIZE)

#define VMALLOC_END (VMALLOC_START + min(MAX_VIRTUAL_SIZE, VIRTUAL_SIZE) 
- (1UL << 32))