for-next/pgtable branch: OOM error with 16K page size

for-next/pgtable branch: OOM error with 16K page size

[Jaggi, Manish]

Hi All,

I am trying to use the for-next/pgtable branch with the top commit, 

commit 068a17a5805dfbca4bbf03e664ca6b19709cc7a8
Author: Mark Rutland <mark.rutland@arm.com>
Date: ? Mon Jan 25 11:45:12 2016 +0000

? ? arm64: mm: create new fine-grained mappings at boot? ??

Setup: ARMv8, Cavium ThunderX, 48Cores 64G ram.
linux config: defconfig + 16k page size.

Observations:
a)?During linux boot I see some oom messages?
http://pastebin.com/8jSMFhqf

b) cat /proc/meminfo

MemTotal:??????   66613760 kB  
MemFree:????????? 307328 kB       
MemAvailable:????? 22416 kB  

Available memory is very less compared to total.

c) With 64K pagesize?
MemTotal:?????? 66831552 kB       
MemFree:???????? 3541184 kB       
MemAvailable:??? 3298752 kB    

So it seems there is some issue with 16k only. 
Need some inputs how to fix this

 Regards, 
Manish Jaggi    

for-next/pgtable branch: OOM error with 16K page size

[Jaggi, Manish]

>>From: linux-arm-kernel <linux-arm-kernel-bounces@lists.infradead.org> on behalf of Jaggi, Manish <Manish.Jaggi@caviumnetworks.com>
>>Sent: Monday, February 29, 2016 5:56 PM
>>To: linux-arm-kernel at lists.infradead.org
>>Subject: for-next/pgtable branch: OOM error with 16K page size
>>
>>Hi All,
>>
>>I am trying to use the for-next/pgtable branch with the top commit,

The same issue is observed with for-next/core branch

>>
>>
>>commit 068a17a5805dfbca4bbf03e664ca6b19709cc7a8
>>Author: Mark Rutland <mark.rutland@arm.com>
>>Date:   Mon Jan 25 11:45:12 2016 +0000
>>
>>   arm64: mm: create new fine-grained mappings at boot
>>
>>Setup: ARMv8, Cavium ThunderX, 48Cores 64G ram.
>>linux config: defconfig + 16k page size.
>>
>>Observations:
>>a) During linux boot I see some oom messages
>>http://pastebin.com/8jSMFhqf
>>
>>b) cat /proc/meminfo
>>
>>MemTotal:         66613760 kB
>>MemFree:          307328 kB
>>MemAvailable:      22416 kB
>>
>>Available memory is very less compared to total.
>>
>>c) With 64K pagesize
>>MemTotal:       66831552 kB
>>MemFree:         3541184 kB
>>MemAvailable:    3298752 kB
>>
>>So it seems there is some issue with 16k only.
>>Need some inputs how to fix this
>>
>> Regards,
>>Manish Jaggi

for-next/pgtable branch: OOM error with 16K page size

[Suzuki K. Poulose]

On 01/03/16 10:48, Jaggi, Manish wrote:
>
>>> From: linux-arm-kernel <linux-arm-kernel-bounces@lists.infradead.org> on behalf of Jaggi, Manish <Manish.Jaggi@caviumnetworks.com>
>>> Sent: Monday, February 29, 2016 5:56 PM
>>> To: linux-arm-kernel at lists.infradead.org
>>> Subject: for-next/pgtable branch: OOM error with 16K page size
>>>
>>> Hi All,
>>>
>>> I am trying to use the for-next/pgtable branch with the top commit,
>
> The same issue is observed with for-next/core branch
>

>>> Observations:
>>> a) During linux boot I see some oom messages
>>> http://pastebin.com/8jSMFhqf
>>>
>>> b) cat /proc/meminfo
>>>
>>> MemTotal:         66613760 kB
>>> MemFree:          307328 kB
>>> MemAvailable:      22416 kB
>>>


Could you please provide the complete o/p of the following
with 16K ?

1) /proc/meminfo
2) /proc/buddyinfo
3) /proc/slabinfo

Thanks
Suzuki

for-next/pgtable branch: OOM error with 16K page size

[Jaggi, Manish]

>>From: Suzuki K. Poulose <Suzuki.Poulose@arm.com>
>>Sent: Tuesday, March 1, 2016 4:53 PM
>>To: Jaggi, Manish
>>Cc: linux-arm-kernel at lists.infradead.org; mark.rutland at arm.com
>>Subject: Re: for-next/pgtable branch: OOM error with 16K page size
>>
>>On 01/03/16 10:48, Jaggi, Manish wrote:
>>>
>>>>> From: linux-arm-kernel <linux-arm-kernel-bounces@lists.infradead.org> on behalf of Jaggi, Manish <Manish.Jaggi@caviumnetworks.com>
>>>>> Sent: Monday, February 29, 2016 5:56 PM
>>>>> To: linux-arm-kernel at lists.infradead.org
>>>>> Subject: for-next/pgtable branch: OOM error with 16K page size
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I am trying to use the for-next/pgtable branch with the top commit,
>>>
>>> The same issue is observed with for-next/core branch
>>>
>>
>>>>> Observations:
>>>>> a) During linux boot I see some oom messages
>>>>> http://pastebin.com/8jSMFhqf
>>>>>
>>>>> b) cat /proc/meminfo
>>>>>
>>>>> MemTotal:         66613760 kB
>>>>> MemFree:          307328 kB
>>>>> MemAvailable:      22416 kB
>>>>>
>>
>>
>>Could you please provide the complete o/p of the following
>>with 16K ?
>>
>>1) /proc/meminfo
>>2) /proc/buddyinfo
>>3) /proc/slabinfo

http://pastebin.com/Eszwp1cS

>>Thanks
>>Suzuki

for-next/pgtable branch: OOM error with 16K page size

[Suzuki K. Poulose]

On 01/03/16 12:03, Jaggi, Manish wrote:
>>> From: Suzuki K. Poulose <Suzuki.Poulose@arm.com>
>>> Sent: Tuesday, March 1, 2016 4:53 PM
>>> To: Jaggi, Manish
>>> Cc: linux-arm-kernel at lists.infradead.org; mark.rutland at arm.com
>>> Subject: Re: for-next/pgtable branch: OOM error with 16K page size
>>>

>>> Could you please provide the complete o/p of the following
>>> with 16K ?
>>>
>>> 1) /proc/meminfo

> http://pastebin.com/Eszwp1cS

> HugePages_Total:    2019
> HugePages_Free:     2019
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:      32768 kB

That clearly tells you whats wrong. You have 2019x32M in hugepages.
Could you try releasing them and see if it solves the memory issue ?

$ echo 0 > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages


btw, whats your root file system(to be more precise, the init) ?

Cheers
Suzuki

for-next/pgtable branch: OOM error with 16K page size

[Suzuki K. Poulose]

On 01/03/16 12:17, Suzuki K. Poulose wrote:
>
>> http://pastebin.com/Eszwp1cS
>
>> HugePages_Total:    2019
>> HugePages_Free:     2019
>> HugePages_Rsvd:        0
>> HugePages_Surp:        0
>> Hugepagesize:      32768 kB
>
> That clearly tells you whats wrong. You have 2019x32M in hugepages.
> Could you try releasing them and see if it solves the memory issue ?
>
> $ echo 0 > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages
>
>
> btw, whats your root file system(to be more precise, the init) ?

Also, could you get the number of HugePages_Total with 4K and 64K ?

May be your init (or whatever allocates the hugepages) requires some
hints about 16K and do something similar to what it does for 64K.

Thanks
Suzuki

for-next/pgtable branch: OOM error with 16K page size

[Mark Rutland]

Hi,

On Tue, Mar 01, 2016 at 10:48:02AM +0000, Jaggi, Manish wrote:
> 
> >>From: linux-arm-kernel <linux-arm-kernel-bounces@lists.infradead.org> on behalf of Jaggi, Manish <Manish.Jaggi@caviumnetworks.com>
> >>Sent: Monday, February 29, 2016 5:56 PM
> >>To: linux-arm-kernel at lists.infradead.org
> >>Subject: for-next/pgtable branch: OOM error with 16K page size
> >>
> >>Hi All,
> >>
> >>I am trying to use the for-next/pgtable branch with the top commit,
> 
> The same issue is observed with for-next/core branch

I take it that you do not see this issue on v4.5-rc4?

> >>commit 068a17a5805dfbca4bbf03e664ca6b19709cc7a8
> >>Author: Mark Rutland <mark.rutland@arm.com>
> >>Date:   Mon Jan 25 11:45:12 2016 +0000
> >>
> >>   arm64: mm: create new fine-grained mappings at boot
> >>
> >>Setup: ARMv8, Cavium ThunderX, 48Cores 64G ram.
> >>linux config: defconfig + 16k page size.
> >>
> >>Observations:
> >>a) During linux boot I see some oom messages
> >>http://pastebin.com/8jSMFhqf
> >>

Could you try again with "efi=debug"? If possible, also
"memblock=debug", though the latter will hugely increase the dmesg
output.

> >>b) cat /proc/meminfo
> >>
> >>MemTotal:         66613760 kB
> >>MemFree:          307328 kB
> >>MemAvailable:      22416 kB
> >>
> >>Available memory is very less compared to total.
> >>
> >>c) With 64K pagesize
> >>MemTotal:       66831552 kB
> >>MemFree:         3541184 kB
> >>MemAvailable:    3298752 kB

The former does look very odd.

Could you dump the full contents of /proc/meminfo for both of these
cases?

Thanks,
Mark.