Re: Page Allocation Failure and Page allocation stalls

Re: Page Allocation Failure and Page allocation stalls

[Valdis Klētnieks]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1603 bytes --]

On Thu, 02 May 2019 04:56:05 +0530, Pankaj Suryawanshi said:

> Please help me to decode the error messages and reason for this errors.

> [ 3205.818891] HwBinder:1894_6: page allocation failure: order:7, mode:0x14040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null)

Order 7 - so it wants 2**7 contiguous pages.  128 4K pages.

> [ 3205.967748] [<802186cc>] (__alloc_from_contiguous) from [<80218854>] (cma_allocator_alloc+0x44/0x4c)

And that 3205.nnn tells me the system has been running for almost an hour. Going
to be hard finding that much contiguous free memory.

Usually CMA is called right at boot to avoid this problem - why is this
triggering so late?

> [  671.925663] kworker/u8:13: page allocation stalls for 10090ms, order:1, mode:0x15080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), nodemask=(null)

That's.... a *really* long stall.

> [  672.031702] [<8021e800>] (copy_process.part.5) from [<802203b0>] (_do_fork+0xd0/0x464)
> [  672.039617]  r10:00000000 r9:00000000 r8:9d008400 r7:00000000 r6:81216588 r5:9b62f840
> [  672.047441]  r4:00808111
> [  672.049972] [<802202e0>] (_do_fork) from [<802207a4>] (kernel_thread+0x38/0x40)
> [  672.057281]  r10:00000000 r9:81422554 r8:9d008400 r7:00000000 r6:9d004500 r5:9b62f840
> [  672.065105]  r4:81216588
> [  672.067642] [<8022076c>] (kernel_thread) from [<802399b4>] (call_usermodehelper_exec_work+0x44/0xe0)

First possibility that comes to mind is that a usermodehelper got launched, and
it then tried to fork with a very large active process image.  Do we have any
clues what was going on?  Did a device get hotplugged?

[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

Re: Page Allocation Failure and Page allocation stalls

[Pankaj Suryawanshi]

On Thu, May 2, 2019 at 10:51 AM Valdis Klētnieks
<valdis.kletnieks@vt.edu> wrote:
>
> On Thu, 02 May 2019 04:56:05 +0530, Pankaj Suryawanshi said:
>
> > Please help me to decode the error messages and reason for this errors.
>
> > [ 3205.818891] HwBinder:1894_6: page allocation failure: order:7, mode:0x14040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null)
>
> Order 7 - so it wants 2**7 contiguous pages.  128 4K pages.
>
kmalloc fails to allocate 2**7

> > [ 3205.967748] [<802186cc>] (__alloc_from_contiguous) from [<80218854>] (cma_allocator_alloc+0x44/0x4c)
>
> And that 3205.nnn tells me the system has been running for almost an hour. Going
> to be hard finding that much contiguous free memory.
>
> Usually CMA is called right at boot to avoid this problem - why is this
> triggering so late?
>
The use case for late triggering is someone try to play video after an
hour, and video memory from CMA area, maybe its due to fragmentation.
> > [  671.925663] kworker/u8:13: page allocation stalls for 10090ms, order:1, mode:0x15080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), nodemask=(null)
>
> That's.... a *really* long stall.
>
Yes very long any pointers to block this warnings/errors.

> > [  672.031702] [<8021e800>] (copy_process.part.5) from [<802203b0>] (_do_fork+0xd0/0x464)
> > [  672.039617]  r10:00000000 r9:00000000 r8:9d008400 r7:00000000 r6:81216588 r5:9b62f840
> > [  672.047441]  r4:00808111
> > [  672.049972] [<802202e0>] (_do_fork) from [<802207a4>] (kernel_thread+0x38/0x40)
> > [  672.057281]  r10:00000000 r9:81422554 r8:9d008400 r7:00000000 r6:9d004500 r5:9b62f840
> > [  672.065105]  r4:81216588
> > [  672.067642] [<8022076c>] (kernel_thread) from [<802399b4>] (call_usermodehelper_exec_work+0x44/0xe0)
>
> First possibility that comes to mind is that a usermodehelper got launched, and
> it then tried to fork with a very large active process image.  Do we have any
> clues what was going on?  Did a device get hotplugged?

Yes,The system is android and it tries to allocate memory for video
player from CMA reserved memory using custom octl call for dma apis.

Please let me know how to overcome this issues, or how to reduce
fragmentation of memory so that higher order allocation get suuceed ?

Thanks

Re: Page Allocation Failure and Page allocation stalls

[Vlastimil Babka]

On 5/3/19 7:44 PM, Pankaj Suryawanshi wrote:
>> First possibility that comes to mind is that a usermodehelper got launched, and
>> it then tried to fork with a very large active process image.  Do we have any
>> clues what was going on?  Did a device get hotplugged?
> 
> Yes,The system is android and it tries to allocate memory for video
> player from CMA reserved memory using custom octl call for dma apis.

The stacktrace doesn't look like a CMA allocation though. That would be
doing alloc_contig_range(), not kmalloc(). Could be some CMA area setup
issue?

> Please let me know how to overcome this issues, or how to reduce
> fragmentation of memory so that higher order allocation get suuceed ?
> 
> Thanks
> 

Re: Page Allocation Failure and Page allocation stalls

[Pankaj Suryawanshi]

On Mon, May 6, 2019 at 2:35 PM Vlastimil Babka <vbabka@suse.cz> wrote:
>
> On 5/3/19 7:44 PM, Pankaj Suryawanshi wrote:
> >> First possibility that comes to mind is that a usermodehelper got launched, and
> >> it then tried to fork with a very large active process image.  Do we have any
> >> clues what was going on?  Did a device get hotplugged?
> >
> > Yes,The system is android and it tries to allocate memory for video
> > player from CMA reserved memory using custom octl call for dma apis.
>
> The stacktrace doesn't look like a CMA allocation though. That would be
> doing alloc_contig_range(), not kmalloc(). Could be some CMA area setup
> issue?
>
I know cma uses alloc_contig_range() but using dma api it will uses
many functions.
the failure is coming from dma_common_contiguous_remap() for kmalloc ,
and which is called by dma_alloc_attr for cma allocation.

Please let me know, how to avoid page allocation stalls. any reason ?
Cpu Utilization issue ? or I am running out of memory ?

My System configuration is
2GB RAM
Memory Spilt 2G/2G
vmalloc=1024M
CMA=1024
Max contiguous memory required 390M

> > Please let me know how to overcome this issues, or how to reduce
> > fragmentation of memory so that higher order allocation get suuceed ?
> >
> > Thanks
> >
>