* ABI compatibility for /proc/zoneinfo @ 2020-08-10 21:24 Sonny Rao 2020-08-10 21:46 ` Yu Zhao 0 siblings, 1 reply; 7+ messages in thread From: Sonny Rao @ 2020-08-10 21:24 UTC (permalink / raw) To: linux-mm; +Cc: Yu Zhao, bhe, David Rientjes, Andrew Morton [-- Attachment #1: Type: text/plain, Size: 437 bytes --] We (Chrome OS) noticed recently one of our tests started failing on upstream kernels while parsing /proc/zoneinfo I think this patch is the cause: 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection information of empty zone Maybe our parser was being overly strict by looking for the protection line, and it's not hard to fix but raised the question of whether there's any ABI compatibility guarantees about these files? [-- Attachment #2: Type: text/html, Size: 992 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-10 21:24 ABI compatibility for /proc/zoneinfo Sonny Rao @ 2020-08-10 21:46 ` Yu Zhao 2020-08-11 2:37 ` Baoquan He 0 siblings, 1 reply; 7+ messages in thread From: Yu Zhao @ 2020-08-10 21:46 UTC (permalink / raw) To: Sonny Rao; +Cc: linux-mm, bhe, David Rientjes, Andrew Morton On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > We (Chrome OS) noticed recently one of our tests started failing on > upstream kernels while parsing /proc/zoneinfo > I think this patch is the cause: > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > information of empty zone > > Maybe our parser was being overly strict by looking for the protection > line, and it's not hard to fix but raised the question of whether there's > any ABI compatibility guarantees about these files? According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has an array of protection pages". I'm not sure if this is the guarantee, but the doc should reflect the actual format. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-10 21:46 ` Yu Zhao @ 2020-08-11 2:37 ` Baoquan He 2020-08-11 3:18 ` Sonny Rao 0 siblings, 1 reply; 7+ messages in thread From: Baoquan He @ 2020-08-11 2:37 UTC (permalink / raw) To: Yu Zhao; +Cc: Sonny Rao, linux-mm, David Rientjes, Andrew Morton On 08/10/20 at 03:46pm, Yu Zhao wrote: > On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > > We (Chrome OS) noticed recently one of our tests started failing on > > upstream kernels while parsing /proc/zoneinfo > > I think this patch is the cause: > > > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > > information of empty zone > > > > Maybe our parser was being overly strict by looking for the protection > > line, and it's not hard to fix but raised the question of whether there's > > any ABI compatibility guarantees about these files? > > According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has > an array of protection pages". I'm not sure if this is the guarantee, > but the doc should reflect the actual format. The current code will list all zones in one memory node, even though that node only has one existing zone. E.g in below node 1, it only has NORMAL zone, but we will list zone DMA, DMA32, MOVABLE, DEVICE which are all empty zone, namely doesn't exist. So, each zone has an array of protection pages, it should not include the nonexistent zone. I thought nobody would check the protection line of an empty zone, seems I was wrong. (Note that below /proc/zoneinfo is from an old system, protection line is still existing for empty zone) Node 1, zone DMA pages free 0 min 0 low 0 high 0 spanned 0 present 0 managed 0 protection: (0, 0, 61854, 61854, 61854) Node 1, zone DMA32 pages free 0 min 0 low 0 high 0 spanned 0 present 0 managed 0 protection: (0, 0, 61854, 61854, 61854) Node 1, zone Normal per-node stats nr_inactive_anon 259 nr_active_anon 11926 ... nr_written 0 nr_kernel_misc_reclaimable 0 pages free 16206452 pages free 16206452 min 11280 low 27114 high 42948 spanned 16777216 present 16777216 managed 15834637 protection: (0, 0, 0, 0, 0) ... Node 1, zone Movable pages free 0 min 0 low 0 high 0 spanned 0 present 0 managed 0 protection: (0, 0, 0, 0, 0) Node 1, zone Device pages free 0 min 0 low 0 high 0 spanned 0 present 0 managed 0 protection: (0, 0, 0, 0, 0) > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-11 2:37 ` Baoquan He @ 2020-08-11 3:18 ` Sonny Rao 2020-08-11 3:58 ` Baoquan He 2020-08-11 4:01 ` Andrew Morton 0 siblings, 2 replies; 7+ messages in thread From: Sonny Rao @ 2020-08-11 3:18 UTC (permalink / raw) To: Baoquan He; +Cc: Yu Zhao, linux-mm, David Rientjes, Andrew Morton On Mon, Aug 10, 2020 at 7:37 PM Baoquan He <bhe@redhat.com> wrote: > > On 08/10/20 at 03:46pm, Yu Zhao wrote: > > On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > > > We (Chrome OS) noticed recently one of our tests started failing on > > > upstream kernels while parsing /proc/zoneinfo > > > I think this patch is the cause: > > > > > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > > > information of empty zone > > > > > > Maybe our parser was being overly strict by looking for the protection > > > line, and it's not hard to fix but raised the question of whether there's > > > any ABI compatibility guarantees about these files? > > > > According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has > > an array of protection pages". I'm not sure if this is the guarantee, > > but the doc should reflect the actual format. > > The current code will list all zones in one memory node, even though > that node only has one existing zone. E.g in below node 1, > it only has NORMAL zone, but we will list zone DMA, DMA32, MOVABLE, > DEVICE which are all empty zone, namely doesn't exist. So, each zone > has an array of protection pages, it should not include the nonexistent > zone. I thought nobody would check the protection line of an empty zone, > seems I was wrong. This particular parser was written as a state machine and that line was a convenient thing to look for to mark the end of each zone. AFAICT, there's no explicit documentation on the layout of that file though. > > > (Note that below /proc/zoneinfo is from an old system, protection line > is still existing for empty zone) > Node 1, zone DMA > pages free 0 > min 0 > low 0 > high 0 > spanned 0 > present 0 > managed 0 > protection: (0, 0, 61854, 61854, 61854) > Node 1, zone DMA32 > pages free 0 > min 0 > low 0 > high 0 > spanned 0 > present 0 > managed 0 > protection: (0, 0, 61854, 61854, 61854) > Node 1, zone Normal > per-node stats > nr_inactive_anon 259 > nr_active_anon 11926 > ... > nr_written 0 > nr_kernel_misc_reclaimable 0 > pages free 16206452 > pages free 16206452 > min 11280 > low 27114 > high 42948 > spanned 16777216 > present 16777216 > managed 15834637 > protection: (0, 0, 0, 0, 0) > ... > > Node 1, zone Movable > pages free 0 > min 0 > low 0 > high 0 > spanned 0 > present 0 > managed 0 > protection: (0, 0, 0, 0, 0) > Node 1, zone Device > pages free 0 > min 0 > low 0 > high 0 > spanned 0 > present 0 > managed 0 > protection: (0, 0, 0, 0, 0) > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-11 3:18 ` Sonny Rao @ 2020-08-11 3:58 ` Baoquan He 2020-08-11 4:01 ` Andrew Morton 1 sibling, 0 replies; 7+ messages in thread From: Baoquan He @ 2020-08-11 3:58 UTC (permalink / raw) To: Sonny Rao; +Cc: Yu Zhao, linux-mm, David Rientjes, Andrew Morton On 08/10/20 at 08:18pm, Sonny Rao wrote: > On Mon, Aug 10, 2020 at 7:37 PM Baoquan He <bhe@redhat.com> wrote: > > > > On 08/10/20 at 03:46pm, Yu Zhao wrote: > > > On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > > > > We (Chrome OS) noticed recently one of our tests started failing on > > > > upstream kernels while parsing /proc/zoneinfo > > > > I think this patch is the cause: > > > > > > > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > > > > information of empty zone > > > > > > > > Maybe our parser was being overly strict by looking for the protection > > > > line, and it's not hard to fix but raised the question of whether there's > > > > any ABI compatibility guarantees about these files? > > > > > > According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has > > > an array of protection pages". I'm not sure if this is the guarantee, > > > but the doc should reflect the actual format. > > > > The current code will list all zones in one memory node, even though > > that node only has one existing zone. E.g in below node 1, > > it only has NORMAL zone, but we will list zone DMA, DMA32, MOVABLE, > > DEVICE which are all empty zone, namely doesn't exist. So, each zone > > has an array of protection pages, it should not include the nonexistent > > zone. I thought nobody would check the protection line of an empty zone, > > seems I was wrong. > > > This particular parser was written as a state machine and that line > was a convenient thing to look for to mark the end of each zone. > AFAICT, there's no explicit documentation on the layout of that file > though. Ah, I see. While the protection line can only mark the end of empty zone. For populated zone, it obvisouly can't. One wild idea is not showing those empty zones at all. On NUMA system with multiple memory nodes, the nodes other than node0 won't have DMA, DMA32 zone usually, it doesn't make any sense to list them in /proc/zoneinfo. > > > > > > > (Note that below /proc/zoneinfo is from an old system, protection line > > is still existing for empty zone) > > Node 1, zone DMA > > pages free 0 > > min 0 > > low 0 > > high 0 > > spanned 0 > > present 0 > > managed 0 > > protection: (0, 0, 61854, 61854, 61854) > > Node 1, zone DMA32 > > pages free 0 > > min 0 > > low 0 > > high 0 > > spanned 0 > > present 0 > > managed 0 > > protection: (0, 0, 61854, 61854, 61854) > > Node 1, zone Normal > > per-node stats > > nr_inactive_anon 259 > > nr_active_anon 11926 > > ... > > nr_written 0 > > nr_kernel_misc_reclaimable 0 > > pages free 16206452 > > pages free 16206452 > > min 11280 > > low 27114 > > high 42948 > > spanned 16777216 > > present 16777216 > > managed 15834637 > > protection: (0, 0, 0, 0, 0) > > ... > > > > Node 1, zone Movable > > pages free 0 > > min 0 > > low 0 > > high 0 > > spanned 0 > > present 0 > > managed 0 > > protection: (0, 0, 0, 0, 0) > > Node 1, zone Device > > pages free 0 > > min 0 > > low 0 > > high 0 > > spanned 0 > > present 0 > > managed 0 > > protection: (0, 0, 0, 0, 0) > > > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-11 3:18 ` Sonny Rao 2020-08-11 3:58 ` Baoquan He @ 2020-08-11 4:01 ` Andrew Morton 2020-08-11 4:07 ` Baoquan He 1 sibling, 1 reply; 7+ messages in thread From: Andrew Morton @ 2020-08-11 4:01 UTC (permalink / raw) To: Sonny Rao; +Cc: Baoquan He, Yu Zhao, linux-mm, David Rientjes On Mon, 10 Aug 2020 20:18:11 -0700 Sonny Rao <sonnyrao@chromium.org> wrote: > On Mon, Aug 10, 2020 at 7:37 PM Baoquan He <bhe@redhat.com> wrote: > > > > On 08/10/20 at 03:46pm, Yu Zhao wrote: > > > On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > > > > We (Chrome OS) noticed recently one of our tests started failing on > > > > upstream kernels while parsing /proc/zoneinfo > > > > I think this patch is the cause: > > > > > > > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > > > > information of empty zone > > > > > > > > Maybe our parser was being overly strict by looking for the protection > > > > line, and it's not hard to fix but raised the question of whether there's > > > > any ABI compatibility guarantees about these files? > > > > > > According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has > > > an array of protection pages". I'm not sure if this is the guarantee, > > > but the doc should reflect the actual format. > > > > The current code will list all zones in one memory node, even though > > that node only has one existing zone. E.g in below node 1, > > it only has NORMAL zone, but we will list zone DMA, DMA32, MOVABLE, > > DEVICE which are all empty zone, namely doesn't exist. So, each zone > > has an array of protection pages, it should not include the nonexistent > > zone. I thought nobody would check the protection line of an empty zone, > > seems I was wrong. > > > This particular parser was written as a state machine and that line > was a convenient thing to look for to mark the end of each zone. > AFAICT, there's no explicit documentation on the layout of that file > though. This only affects 5.8 and later, yes? 26e7deadaae17 didn't gain us much at all and we shouldn't break userspace unless it's super important. So I think it's best to revert, and to backport the revert to 5.8.x. Could someone (Baoquan He?) please send a patch with a suitable changelog, cc:stable, reported-by:, etc? Thanks. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ABI compatibility for /proc/zoneinfo 2020-08-11 4:01 ` Andrew Morton @ 2020-08-11 4:07 ` Baoquan He 0 siblings, 0 replies; 7+ messages in thread From: Baoquan He @ 2020-08-11 4:07 UTC (permalink / raw) To: Andrew Morton; +Cc: Sonny Rao, Yu Zhao, linux-mm, David Rientjes On 08/10/20 at 09:01pm, Andrew Morton wrote: > On Mon, 10 Aug 2020 20:18:11 -0700 Sonny Rao <sonnyrao@chromium.org> wrote: > > > On Mon, Aug 10, 2020 at 7:37 PM Baoquan He <bhe@redhat.com> wrote: > > > > > > On 08/10/20 at 03:46pm, Yu Zhao wrote: > > > > On Mon, Aug 10, 2020 at 02:24:03PM -0700, Sonny Rao wrote: > > > > > We (Chrome OS) noticed recently one of our tests started failing on > > > > > upstream kernels while parsing /proc/zoneinfo > > > > > I think this patch is the cause: > > > > > > > > > > 26e7deadaae17 mm/vmstat.c: do not show lowmem reserve protection > > > > > information of empty zone > > > > > > > > > > Maybe our parser was being overly strict by looking for the protection > > > > > line, and it's not hard to fix but raised the question of whether there's > > > > > any ABI compatibility guarantees about these files? > > > > > > > > According to Documentation/admin-guide/sysctl/vm.rst, "Each zone has > > > > an array of protection pages". I'm not sure if this is the guarantee, > > > > but the doc should reflect the actual format. > > > > > > The current code will list all zones in one memory node, even though > > > that node only has one existing zone. E.g in below node 1, > > > it only has NORMAL zone, but we will list zone DMA, DMA32, MOVABLE, > > > DEVICE which are all empty zone, namely doesn't exist. So, each zone > > > has an array of protection pages, it should not include the nonexistent > > > zone. I thought nobody would check the protection line of an empty zone, > > > seems I was wrong. > > > > > > This particular parser was written as a state machine and that line > > was a convenient thing to look for to mark the end of each zone. > > AFAICT, there's no explicit documentation on the layout of that file > > though. > > This only affects 5.8 and later, yes? > > 26e7deadaae17 didn't gain us much at all and we shouldn't break > userspace unless it's super important. So I think it's best to revert, > and to backport the revert to 5.8.x. > > Could someone (Baoquan He?) please send a patch with a suitable > changelog, cc:stable, reported-by:, etc? OK, I will post a patch to revert this. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-08-11 4:07 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-08-10 21:24 ABI compatibility for /proc/zoneinfo Sonny Rao 2020-08-10 21:46 ` Yu Zhao 2020-08-11 2:37 ` Baoquan He 2020-08-11 3:18 ` Sonny Rao 2020-08-11 3:58 ` Baoquan He 2020-08-11 4:01 ` Andrew Morton 2020-08-11 4:07 ` Baoquan He
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.