Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Joakim Tjernlund]

Could this patch be backported to stable? I have seen similar OOMs with
reserved_highatomic:4096KB

Upstream commit: 04c8716f7b0075def05dc05646e2408f318167d2

  Jocke 

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Greg KH]

On Thu, Jan 18, 2024 at 05:23:58PM +0000, Joakim Tjernlund wrote:
> Could this patch be backported to stable? I have seen similar OOMs with
> reserved_highatomic:4096KB
> 
> Upstream commit: 04c8716f7b0075def05dc05646e2408f318167d2

Backported to exactly where?  This commit is in the 4.20 kernel and
newer, please tell me you aren't relying on the 4.19.y kernel anymore...

thanks,

greg k-h

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Joakim Tjernlund]

Seems like I pasted the wrong commit(sorry), should be: ac3f3b0a55518056bc80ed32a41931c99e1f7d81
I only see that one in master.

________________________________________
From: Greg KH <gregkh@linuxfoundation.org>
Sent: 22 January 2024 18:41
To: Joakim Tjernlund
Cc: quic_charante@quicinc.com; david@redhat.com; mhocko@suse.com; akpm@linux-foundation.org; linux-mm@kvack.org; hannes@cmpxchg.org; mgorman@techsingularity.net; linux-kernel@vger.kernel.org; quic_pkondeti@quicinc.com; vbabka@suse.cz; stable@vger.kernel.org; quic_cgoldswo@quicinc.com
Subject: Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

On Thu, Jan 18, 2024 at 05:23:58PM +0000, Joakim Tjernlund wrote:
> Could this patch be backported to stable? I have seen similar OOMs with
> reserved_highatomic:4096KB
>
> Upstream commit: 04c8716f7b0075def05dc05646e2408f318167d2

Backported to exactly where?  This commit is in the 4.20 kernel and
newer, please tell me you aren't relying on the 4.19.y kernel anymore...

thanks,

greg k-h

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Greg KH]

A: http://en.wikipedia.org/wiki/Top_post
Q: Were do I find info about this thing called top-posting?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Mon, Jan 22, 2024 at 10:49:32PM +0000, Joakim Tjernlund wrote:
> Seems like I pasted the wrong commit(sorry), should be: ac3f3b0a55518056bc80ed32a41931c99e1f7d81
> I only see that one in master.

And what kernels have you tested this on?  How far back should it go?

For mm patches like this, that are not explicitly tagged by the
maintainers to be included in the stable tree, we need their ack to be
able to apply them based on their requests.  So can you get that for
this change and provide tested patches, we will be glad to queue them
up.

thanks,

greg k-h

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Joakim Tjernlund]

On Mon, 2024-01-22 at 15:04 -0800, Greg KH wrote:
> A: http://en.wikipedia.org/wiki/Top_post
> Q: Were do I find info about this thing called top-posting?
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing in e-mail?
>
> A: No.
> Q: Should I include quotations after my reply?

haha, serves me right :) (note to self, do not use webmail for Linux stuff ...)
>
> http://daringfireball.net/2007/07/on_top
>
> On Mon, Jan 22, 2024 at 10:49:32PM +0000, Joakim Tjernlund wrote:
> > Seems like I pasted the wrong commit(sorry), should be: ac3f3b0a55518056bc80ed32a41931c99e1f7d81
> > I only see that one in master.
>
> And what kernels have you tested this on?  How far back should it go?

I am testing it now in latest 5.15.x but the jury is still out. No
OOM since a few days but the error does not happen often.

>
> For mm patches like this, that are not explicitly tagged by the
> maintainers to be included in the stable tree, we need their ack to be
> able to apply them based on their requests.  So can you get that for
> this change and provide tested patches, we will be glad to queue them
> up.

I asked the author and he acknowledged it could be backported. Charan, please chim in.

 Jocke

>
> thanks,
>
> greg k-h

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Michal Hocko]

On Mon 22-01-24 23:14:59, Joakim Tjernlund wrote:
> On Mon, 2024-01-22 at 15:04 -0800, Greg KH wrote:
[...]
> > For mm patches like this, that are not explicitly tagged by the
> > maintainers to be included in the stable tree, we need their ack to be
> > able to apply them based on their requests.  So can you get that for
> > this change and provide tested patches, we will be glad to queue them
> > up.
> 
> I asked the author and he acknowledged it could be backported. Charan, please chim in.

The patch itself is safe to backport but it would be great to here what
kind of problem you are trying to deal with. The issue fixed by this
patch is more on a corner case side than something that many users
should see. Could you share oom report you are seeing?
-- 
Michal Hocko
SUSE Labs

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Joakim Tjernlund]

On Tue, 2024-01-23 at 11:05 +0100, Michal Hocko wrote:
> On Mon 22-01-24 23:14:59, Joakim Tjernlund wrote:
> > On Mon, 2024-01-22 at 15:04 -0800, Greg KH wrote:
> [...]
> > > For mm patches like this, that are not explicitly tagged by the
> > > maintainers to be included in the stable tree, we need their ack to be
> > > able to apply them based on their requests.  So can you get that for
> > > this change and provide tested patches, we will be glad to queue them
> > > up.
> > 
> > I asked the author and he acknowledged it could be backported. Charan, please chim in.
> 
> The patch itself is safe to backport but it would be great to here what
> kind of problem you are trying to deal with. The issue fixed by this
> patch is more on a corner case side than something that many users
> should see. Could you share oom report you are seeing?

Yes, here it is:

Mar  9 12:52:39 xr kern.warn kernel: [ 1065.896824] xr-swm-install- invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.907321] CPU: 0 PID: 2428 Comm: xr-swm-install- Not tainted 5.15.129-xr-linux #1
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.914974] Hardware name: infinera,xr (DT)
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.919158] Call trace:
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.921600]  dump_backtrace+0x0/0x148
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.925310]  show_stack+0x14/0x1c
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.928628]  dump_stack_lvl+0x64/0x7c
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.932314]  dump_stack+0x14/0x2c
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.935633]  dump_header+0x64/0x1fc
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.939123]  oom_kill_process+0xc0/0x28c
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.943063]  out_of_memory+0x2c8/0x2e0
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.946811]  __alloc_pages_slowpath.constprop.0+0x4f4/0x5b0
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.952388]  __alloc_pages+0xcc/0xdc
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.955962]  __page_cache_alloc+0x18/0x20
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.959993]  pagecache_get_page+0x14c/0x1bc
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.964174]  filemap_fault+0x1f4/0x390
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.967920]  __do_fault+0x48/0x78
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.971253]  __handle_mm_fault+0x35c/0x7c0
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.975351]  handle_mm_fault+0x2c/0xc4
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.979101]  do_page_fault+0x224/0x350
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.982857]  do_translation_fault+0x3c/0x58
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.987038]  do_mem_abort+0x40/0xa4
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.990526]  el0_ia+0x74/0xc8
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.993498]  el0t_32_sync_handler+0xa8/0xe8
Mar  9 12:52:40 xr kern.warn kernel: [ 1065.997679]  el0t_32_sync+0x15c/0x160
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.001379] Mem-Info:
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665] active_anon:18 inactive_anon:2435 isolated_anon:0
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  active_file:1 inactive_file:0 isolated_file:0
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  unevictable:267 dirty:0 writeback:0
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  slab_reclaimable:384 slab_unreclaimable:2223
Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  mapped:1 shmem:0 pagetables:309 bounce:0
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  kernel_misc_reclaimable:0
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  free:1054 free_pcp:24 free_cma:0
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.037938] Node 0 active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:9
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.061723] Normal free:4216kB min:128kB low:160kB high:192kB reserved_highatomic:4096KB active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB writepending:0kB present:36864kB managed:2944
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.087595] lowmem_reserve[]: 0 0
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.090918] Normal: 206*4kB (MEH) 198*8kB (UMEH) 79*16kB (UMEH) 13*32kB (H) 2*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4216kB
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.104104] 269 total pagecache pages
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.107768] 9216 pages RAM
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.110469] 0 pages HighMem/MovableOnly
Mar  9 12:52:41 xr kern.warn kernel: [ 1066.114305] 1855 pages reserved
Mar  9 12:52:41 xr kern.info kernel: [ 1066.117448] Tasks state (memory values in pages):
Mar  9 12:52:41 xr kern.info kernel: [ 1066.122150] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Mar  9 12:52:41 xr kern.info kernel: [ 1066.130764] [     77]     0    77      380       24    24576        0             0 mdev
Mar  9 12:52:41 xr kern.info kernel: [ 1066.138857] [    224]     0   224      364        9    24576        0             0 syslogd
Mar  9 12:52:41 xr kern.info kernel: [ 1066.147208] [    227]     0   227      364        9    28672        0             0 klogd
Mar  9 12:52:41 xr kern.info kernel: [ 1066.155385] [    242]     0   242      301       10    28672        0             0 dropbear
Mar  9 12:52:41 xr kern.info kernel: [ 1066.163824] [    248]     0   248      337       38    24576        0             0 dhcpcd
Mar  9 12:52:41 xr kern.info kernel: [ 1066.172090] [    259]     0   259      899       34    32768        0             0 watchdog
Mar  9 12:52:41 xr kern.info kernel: [ 1066.180528] [    261]     0   261      741       33    32768        0             0 rpmsg_broker
Mar  9 12:52:41 xr kern.info kernel: [ 1066.189311] [    263]     0   263    75466      965   622592        0             0 waactrl-main
Mar  9 12:52:41 xr kern.info kernel: [ 1066.198095] [    320]     0   320     1127       90    32768        0             0 xr-fm-agent
Mar  9 12:52:41 xr kern.info kernel: [ 1066.206794] [    324]     0   324      797       40    24576        0             0 factory_reset
Mar  9 12:52:41 xr kern.info kernel: [ 1066.215668] [    328]     0   328     1674      242    40960        0             0 xr-cm-agent
Mar  9 12:52:41 xr kern.info kernel: [ 1066.224628] [    331]     0   331      707       31    32768        0             0 mmcu-agent
Mar  9 12:52:41 xr kern.info kernel: [ 1066.233245] [    334]     0   334     4054      427    49152        0             0 xr-swm-agent
Mar  9 12:52:41 xr kern.info kernel: [ 1066.242031] [    338]     0   338     1129       94    32768        0             0 xr-pm-agent
Mar  9 12:52:41 xr kern.info kernel: [ 1066.250729] [    364]     0   364      734       34    32768        0             0 process_supervi
Mar  9 12:52:41 xr kern.info kernel: [ 1066.259774] [    399]     0   399     1327      215    32768        0             0 swupdate
Mar  9 12:52:41 xr kern.info kernel: [ 1066.268212] [    413]     0   413      370       13    28672        0             0 sh
Mar  9 12:52:41 xr kern.info kernel: [ 1066.276131] [   2072]     0  2072      306       13    28672        0             0 dropbear
Mar  9 12:52:42 xr kern.info kernel: [ 1066.284569] [   2081]     0  2081      366       11    24576        0             0 sh
Mar  9 12:52:42 xr kern.info kernel: [ 1066.292492] [   2427]     0  2427      876       93    28672        0             0 xr-swm-install-
Mar  9 12:52:42 xr kern.info kernel: [ 1066.301536] [   2468]     0  2468      546       25    28672        0             0 ip
Mar  9 12:52:42 xr kern.info kernel: [ 1066.309453] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=waactrl-main,pid=263,uid=0
Mar  9 12:52:42 xr kern.err kernel: [ 1066.318197] Out of memory: Killed process 263 (waactrl-main) total-vm:301864kB, anon-rss:3860kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:608kB oom_score_adj:0

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[Michal Hocko]

On Tue 23-01-24 11:08:28, Joakim Tjernlund wrote:
[...]
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.001379] Mem-Info:
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665] active_anon:18 inactive_anon:2435 isolated_anon:0
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  active_file:1 inactive_file:0 isolated_file:0
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  unevictable:267 dirty:0 writeback:0
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  slab_reclaimable:384 slab_unreclaimable:2223
> Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  mapped:1 shmem:0 pagetables:309 bounce:0
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  kernel_misc_reclaimable:0
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  free:1054 free_pcp:24 free_cma:0
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.037938] Node 0 active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:9
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.061723] Normal free:4216kB min:128kB low:160kB high:192kB reserved_highatomic:4096KB active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB writepending:0kB present:36864kB managed:2944
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.087595] lowmem_reserve[]: 0 0
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.090918] Normal: 206*4kB (MEH) 198*8kB (UMEH) 79*16kB (UMEH) 13*32kB (H) 2*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4216kB
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.104104] 269 total pagecache pages
> Mar  9 12:52:41 xr kern.warn kernel: [ 1066.107768] 9216 pages RAM

OK, so this is really a tiny system (36MB) and having 4MB in highatomic
reserves is clearly too much. So this matches the patch you are
referencing and it migh help you indeed.

Thanks!

-- 
Michal Hocko
SUSE Labs

Re: [RESEND PATCH V2] mm: page_alloc: unreserve highatomic page blocks before oom

[gregkh]

On Tue, Jan 23, 2024 at 12:24:22PM +0100, Michal Hocko wrote:
> On Tue 23-01-24 11:08:28, Joakim Tjernlund wrote:
> [...]
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.001379] Mem-Info:
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665] active_anon:18 inactive_anon:2435 isolated_anon:0
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  active_file:1 inactive_file:0 isolated_file:0
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  unevictable:267 dirty:0 writeback:0
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  slab_reclaimable:384 slab_unreclaimable:2223
> > Mar  9 12:52:40 xr kern.warn kernel: [ 1066.003665]  mapped:1 shmem:0 pagetables:309 bounce:0
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  kernel_misc_reclaimable:0
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.003665]  free:1054 free_pcp:24 free_cma:0
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.037938] Node 0 active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:9
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.061723] Normal free:4216kB min:128kB low:160kB high:192kB reserved_highatomic:4096KB active_anon:72kB inactive_anon:9740kB active_file:4kB inactive_file:0kB unevictable:1068kB writepending:0kB present:36864kB managed:2944
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.087595] lowmem_reserve[]: 0 0
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.090918] Normal: 206*4kB (MEH) 198*8kB (UMEH) 79*16kB (UMEH) 13*32kB (H) 2*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4216kB
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.104104] 269 total pagecache pages
> > Mar  9 12:52:41 xr kern.warn kernel: [ 1066.107768] 9216 pages RAM
> 
> OK, so this is really a tiny system (36MB) and having 4MB in highatomic
> reserves is clearly too much. So this matches the patch you are
> referencing and it migh help you indeed.

Ok, now queued up, thanks.

greg k-h