1M hugepage size being registered on Linux

1M hugepage size being registered on Linux

[victora]

Hi Alistair/Jeremy,

I am working on a bug related to 1M hugepage size being registered on 
Linux (Power 8 Baremetal - Garrison).

I was checking dmesg and it seems that 1M page size is coming from 
firmware to Linux.

[    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000, 
tlbiel=0, penc=2
[    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 pages

Should Linux support this page size? As afar as I know, this was an 
unsupported page size in the past isn't it? If this should be supported 
now, is there any specific reason for that?

Thanks,

Victor Aoqui
Software Engineer: Linux Kernel Backports
Linux Technology Center
IBM Systems

Re: 1M hugepage size being registered on Linux

[Michael Ellerman]

victora <victora@linux.vnet.ibm.com> writes:

> Hi Alistair/Jeremy,
>
> I am working on a bug related to 1M hugepage size being registered on 
> Linux (Power 8 Baremetal - Garrison).

Wasn't that caused by a firmware bug?

> I was checking dmesg and it seems that 1M page size is coming from 
> firmware to Linux.
>
> [    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000, 
> tlbiel=0, penc=2
> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 pages
>
> Should Linux support this page size?

Does it work? :)

The user manual says it's a supported size, but I thought it didn't work
(in hardware) for some reason.

cheers

Re: 1M hugepage size being registered on Linux

[Mauricio Faria de Oliveira]

On 06/21/2017 07:33 AM, Michael Ellerman wrote:
>> I am working on a bug related to 1M hugepage size being registered on
>> Linux (Power 8 Baremetal - Garrison).

> Wasn't that caused by a firmware bug?

Ben/Stewart, does that ring a bell, something new, intended or not? :- )

Thanks,

>> I was checking dmesg and it seems that 1M page size is coming from
>> firmware to Linux.
>>
>> [    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000,
>> tlbiel=0, penc=2
>> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 pages
>>
>> Should Linux support this page size?
> Does it work?:)
> 
> The user manual says it's a supported size, but I thought it didn't work
> (in hardware) for some reason.

-- 
Mauricio Faria de Oliveira
IBM Linux Technology Center

Re: 1M hugepage size being registered on Linux

[Balbir Singh]

On Wed, 2017-06-21 at 12:32 -0300, Mauricio Faria de Oliveira wrote:
> On 06/21/2017 07:33 AM, Michael Ellerman wrote:
> > > I am working on a bug related to 1M hugepage size being registered on
> > > Linux (Power 8 Baremetal - Garrison).
> > Wasn't that caused by a firmware bug?
> 
> Ben/Stewart, does that ring a bell, something new, intended or not? :- )
> 
> 

Aneesh changed the code to support just 16M and 16G hugepages for hash even
if the firmware exported that size. What is your end goal to support those
pages or to prevent them from being used?

Balbir Singh.

Re: 1M hugepage size being registered on Linux

[Benjamin Herrenschmidt]

On Wed, 2017-06-21 at 20:33 +1000, Michael Ellerman wrote:
> victora <victora@linux.vnet.ibm.com> writes:
> 
> > Hi Alistair/Jeremy,
> > 
> > I am working on a bug related to 1M hugepage size being registered on 
> > Linux (Power 8 Baremetal - Garrison).
> 
> Wasn't that caused by a firmware bug?
> 
> > I was checking dmesg and it seems that 1M page size is coming from 
> > firmware to Linux.
> > 
> > [    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000, 
> > tlbiel=0, penc=2
> > [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 pages
> > 
> > Should Linux support this page size?
> 
> Does it work? :)
> 
> The user manual says it's a supported size, but I thought it didn't work
> (in hardware) for some reason.

Right, I thought there was some config bit to switch between 1M and 16M
and that had never been fully verified ? I don't remember the details.

Re: 1M hugepage size being registered on Linux

[Michael Ellerman]

Hi Victor,

Someone refreshed my memory on this, coffee was involved ...

victora <victora@linux.vnet.ibm.com> writes:
> Hi Alistair/Jeremy,
>
> I am working on a bug related to 1M hugepage size being registered on 
> Linux (Power 8 Baremetal - Garrison).

On those machines the property in the device tree comes straight from
hostboot, and it includes 1M:

# lsprop ibm,segment-page-sizes 
ibm,segment-page-sizes
		 0000000c  00000000  00000003  0000000c
                 baseshift slbenc    lpnum     shift
		 00000000  00000010  00000007  00000018
                 penc      shift     penc      shift
		 00000038  00000010  00000110  00000002
                 penc      baseshift slbenc    lpnum
		 00000010  00000001  00000018  00000008
                 shift     penc      shift     penc
		 00000014  00000130  00000001  00000014	<--- 1MB = 2^0x14
                 baseshift slbenc    lpnum     shift
		 00000002  00000018  00000100  00000001
                 penc      baseshift slbenc    lpnum
		 00000018  00000000  00000022  00000120
                 shift     penc      baseshift slbenc
		 00000001  00000022  00000003
                 lpnum     shift     penc


> I was checking dmesg and it seems that 1M page size is coming from 
> firmware to Linux.
>
> [    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000, tlbiel=0, penc=2
> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 pages

Which is why you see that message.

> Should Linux support this page size? As afar as I know, this was an 
> unsupported page size in the past isn't it? If this should be supported 
> now, is there any specific reason for that?

It's unsupported in Linux because it doesn't match the page table
geometry.

We merged a patch from Aneesh to filter it out in 4.12-rc1:

  a525108cf1cc ("powerpc/mm/hugetlb: Filter out hugepage size not supported by page table layout")

I guess we should probably send that patch to stable et. al.

cheers

Re: 1M hugepage size being registered on Linux

[Benjamin Herrenschmidt]

On Thu, 2017-06-22 at 13:59 +1000, Michael Ellerman wrote:
> It's unsupported in Linux because it doesn't match the page table
> geometry.
> 
> We merged a patch from Aneesh to filter it out in 4.12-rc1:
> 
>   a525108cf1cc ("powerpc/mm/hugetlb: Filter out hugepage size not supported by page table layout")
> 
> I guess we should probably send that patch to stable et. al.

It's also not supported in HW afaik, why is it there ?

Cheers,
Ben.

Re: 1M hugepage size being registered on Linux

[Michael Ellerman]

Benjamin Herrenschmidt <benh@au1.ibm.com> writes:

> On Thu, 2017-06-22 at 13:59 +1000, Michael Ellerman wrote:
>> It's unsupported in Linux because it doesn't match the page table
>> geometry.
>>=20
>> We merged a patch from Aneesh to filter it out in 4.12-rc1:
>>=20
>> =C2=A0 a525108cf1cc ("powerpc/mm/hugetlb: Filter out hugepage size not s=
upported by page table layout")
>>=20
>> I guess we should probably send that patch to stable et. al.
>
> It's also not supported in HW afaik, why is it there ?

Because hostboot said so AIUI.

cheers

Re: 1M hugepage size being registered on Linux

[victora]

Em 2017-06-22 00:59, Michael Ellerman escreveu:
> Hi Victor,
> 
> Someone refreshed my memory on this, coffee was involved ...
> 
> victora <victora@linux.vnet.ibm.com> writes:
>> Hi Alistair/Jeremy,
>> 
>> I am working on a bug related to 1M hugepage size being registered on
>> Linux (Power 8 Baremetal - Garrison).
> 
> On those machines the property in the device tree comes straight from
> hostboot, and it includes 1M:
> 
> # lsprop ibm,segment-page-sizes
> ibm,segment-page-sizes
> 		 0000000c  00000000  00000003  0000000c
>                  baseshift slbenc    lpnum     shift
> 		 00000000  00000010  00000007  00000018
>                  penc      shift     penc      shift
> 		 00000038  00000010  00000110  00000002
>                  penc      baseshift slbenc    lpnum
> 		 00000010  00000001  00000018  00000008
>                  shift     penc      shift     penc
> 		 00000014  00000130  00000001  00000014	<--- 1MB = 2^0x14
>                  baseshift slbenc    lpnum     shift
> 		 00000002  00000018  00000100  00000001
>                  penc      baseshift slbenc    lpnum
> 		 00000018  00000000  00000022  00000120
>                  shift     penc      baseshift slbenc
> 		 00000001  00000022  00000003
>                  lpnum     shift     penc
> 
> 
>> I was checking dmesg and it seems that 1M page size is coming from
>> firmware to Linux.
>> 
>> [    0.000000] base_shift=20: shift=20, sllp=0x0130, avpnm=0x00000000, 
>> tlbiel=0, penc=2
>> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0 
>> pages
> 
> Which is why you see that message.
> 
>> Should Linux support this page size? As afar as I know, this was an
>> unsupported page size in the past isn't it? If this should be 
>> supported
>> now, is there any specific reason for that?
> 
> It's unsupported in Linux because it doesn't match the page table
> geometry.
> 
> We merged a patch from Aneesh to filter it out in 4.12-rc1:
> 
>   a525108cf1cc ("powerpc/mm/hugetlb: Filter out hugepage size not
> supported by page table layout")
> 
> I guess we should probably send that patch to stable et. al.
> 
> cheers

Hi Michael,

Sorry for the delay. Thanks for merging that patch.
Was that patch also sent to stable et. al.?

Thanks
Victor

Re: 1M hugepage size being registered on Linux

[Michael Ellerman]

victora <victora@linux.vnet.ibm.com> writes:
> Em 2017-06-22 00:59, Michael Ellerman escreveu:
>> 
>> We merged a patch from Aneesh to filter it out in 4.12-rc1:
>> 
>>   a525108cf1cc ("powerpc/mm/hugetlb: Filter out hugepage size not
>> supported by page table layout")
>> 
>> I guess we should probably send that patch to stable et. al.
              ^
              :)
>
> Sorry for the delay. Thanks for merging that patch.
> Was that patch also sent to stable et. al.?

No it wasn't.

cheers

Re: 1M hugepage size being registered on Linux

[Michal Suchánek]

On Tue, 20 Jun 2017 10:47:47 -0300
victora <victora@linux.vnet.ibm.com> wrote:

> Hi Alistair/Jeremy,
> 
> I am working on a bug related to 1M hugepage size being registered on 
> Linux (Power 8 Baremetal - Garrison).
> 
> I was checking dmesg and it seems that 1M page size is coming from 
> firmware to Linux.
> 
> [    0.000000] base_shift=20: shift=20, sllp=0x0130,
> avpnm=0x00000000, tlbiel=0, penc=2
> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0
> pages
> 
> Should Linux support this page size? As afar as I know, this was an 
> unsupported page size in the past isn't it? If this should be
> supported now, is there any specific reason for that?

Hello,

a525108cf1cc powerpc/mm/hugetlb: Filter out hugepage size not supported
by page table layout

Should reject unsupported huge page sizes, right?

Thanks

Michal

Re: 1M hugepage size being registered on Linux

[Michael Ellerman]

Michal Such=C3=A1nek <msuchanek@suse.de> writes:

> On Tue, 20 Jun 2017 10:47:47 -0300
> victora <victora@linux.vnet.ibm.com> wrote:
>
>> Hi Alistair/Jeremy,
>>=20
>> I am working on a bug related to 1M hugepage size being registered on=20
>> Linux (Power 8 Baremetal - Garrison).
>>=20
>> I was checking dmesg and it seems that 1M page size is coming from=20
>> firmware to Linux.
>>=20
>> [    0.000000] base_shift=3D20: shift=3D20, sllp=3D0x0130,
>> avpnm=3D0x00000000, tlbiel=3D0, penc=3D2
>> [    1.528867] HugeTLB registered 1 MB page size, pre-allocated 0
>> pages
>>=20
>> Should Linux support this page size? As afar as I know, this was an=20
>> unsupported page size in the past isn't it? If this should be
>> supported now, is there any specific reason for that?
>
> Hello,
>
> a525108cf1cc powerpc/mm/hugetlb: Filter out hugepage size not supported
> by page table layout
>
> Should reject unsupported huge page sizes, right?

Yes, see my reply.

cheers