* [BUG?]Data key in /proc/allocinfo is a multiset
@ 2025-05-09 6:10 David Wang
2025-05-09 15:56 ` Suren Baghdasaryan
0 siblings, 1 reply; 5+ messages in thread
From: David Wang @ 2025-05-09 6:10 UTC (permalink / raw)
To: surenb, kent.overstreet; +Cc: akpm, linux-kernel
Just start a new thread for this[1].
There are duplications in /proc/allocinfo where same [file:line]
shows up several times:
=======================
0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc
0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc
=======================
0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options
0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options
=======================
0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node
=======================
0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node
0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node
...
The duplication make parsing tools a little bit more complicated:
the numbers need to be added up, group by key
81920 20 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 20
1441792 352 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 352
The script for checking:
```
#!/bin/env python
def fetch():
r = {}
with open("/proc/allocinfo") as f:
for l in f:
f = l.strip().split()[2]
if f not in r: r[f]=[]
r[f].append(l)
keys = []
for f, ls in r.items():
if len(ls) > 1: keys.append(f)
keys.sort()
for f in keys:
print "======================="
for l in r[f]: print l,
fetch()
```
Thanks
David
[1]. https://lore.kernel.org/lkml/531adbba.b537.196b0868a8c.Coremail.00107082@163.com/
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [BUG?]Data key in /proc/allocinfo is a multiset 2025-05-09 6:10 [BUG?]Data key in /proc/allocinfo is a multiset David Wang @ 2025-05-09 15:56 ` Suren Baghdasaryan 2025-05-09 16:36 ` David Wang 0 siblings, 1 reply; 5+ messages in thread From: Suren Baghdasaryan @ 2025-05-09 15:56 UTC (permalink / raw) To: David Wang; +Cc: kent.overstreet, akpm, linux-kernel On Thu, May 8, 2025 at 11:10 PM David Wang <00107082@163.com> wrote: > > Just start a new thread for this[1]. > There are duplications in /proc/allocinfo where same [file:line] > shows up several times: > > ======================= > 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > ======================= > 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > ======================= > 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > ======================= > 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node Yep, that happens when an inlined function allocates memory. It ends up inlined in different locations. Usually that's done by allocation helper functions. To fix this we need to wrap these allocator helpers with alloc_hooks: -static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) +static inline void *iommu_alloc_pages_node_noprof(int nid, gfp_t gfp, int order) { - struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, order); struct skcipher_request *req; + struct page *page = alloc_pages_node_noprof(nid, gfp | __GFP_ZERO, order); struct skcipher_request *req; ... } +#define iommu_alloc_pages_node(...) alloc_hooks(iommu_alloc_pages_node_noprof(__VA_ARGS__)) See 2c321f3f70bc "mm: change inlined allocation helpers to account at the call site" for examples of how this was done before. Thanks, Suren. > ... > > The duplication make parsing tools a little bit more complicated: > the numbers need to be added up, group by key > 81920 20 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 20 > 1441792 352 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 352 > > The script for checking: > ``` > #!/bin/env python > def fetch(): > r = {} > with open("/proc/allocinfo") as f: > for l in f: > f = l.strip().split()[2] > if f not in r: r[f]=[] > r[f].append(l) > keys = [] > for f, ls in r.items(): > if len(ls) > 1: keys.append(f) > keys.sort() > for f in keys: > print "=======================" > for l in r[f]: print l, > > fetch() > ``` > > Thanks > David > > [1]. https://lore.kernel.org/lkml/531adbba.b537.196b0868a8c.Coremail.00107082@163.com/ > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG?]Data key in /proc/allocinfo is a multiset 2025-05-09 15:56 ` Suren Baghdasaryan @ 2025-05-09 16:36 ` David Wang 2025-05-09 16:57 ` Suren Baghdasaryan 0 siblings, 1 reply; 5+ messages in thread From: David Wang @ 2025-05-09 16:36 UTC (permalink / raw) To: Suren Baghdasaryan; +Cc: kent.overstreet, akpm, linux-kernel At 2025-05-09 23:56:32, "Suren Baghdasaryan" <surenb@google.com> wrote: >On Thu, May 8, 2025 at 11:10 PM David Wang <00107082@163.com> wrote: >> >> Just start a new thread for this[1]. >> There are duplications in /proc/allocinfo where same [file:line] >> shows up several times: >> >> ======================= >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc >> ======================= >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options >> ======================= >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node >> ======================= >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > >Yep, that happens when an inlined function allocates memory. It ends >up inlined in different locations. Usually that's done by allocation >helper functions. >To fix this we need to wrap these allocator helpers with alloc_hooks: > >-static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) >+static inline void *iommu_alloc_pages_node_noprof(int nid, gfp_t gfp, >int order) >{ >- struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, >order); struct skcipher_request *req; >+ struct page *page = alloc_pages_node_noprof(nid, gfp | >__GFP_ZERO, order); struct skcipher_request *req; >... >} >+#define iommu_alloc_pages_node(...) >alloc_hooks(iommu_alloc_pages_node_noprof(__VA_ARGS__)) > >See 2c321f3f70bc "mm: change inlined allocation helpers to account at >the call site" for examples of how this was done before. >Thanks, >Suren. Thanks for clarifying this, seems like a never-ending work...... >_<||| > >> ... >> >> The duplication make parsing tools a little bit more complicated: >> the numbers need to be added up, group by key >> 81920 20 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 20 >> 1441792 352 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 352 >> >> The script for checking: >> ``` >> #!/bin/env python >> def fetch(): >> r = {} >> with open("/proc/allocinfo") as f: >> for l in f: >> f = l.strip().split()[2] >> if f not in r: r[f]=[] >> r[f].append(l) >> keys = [] >> for f, ls in r.items(): >> if len(ls) > 1: keys.append(f) >> keys.sort() >> for f in keys: >> print "=======================" >> for l in r[f]: print l, >> >> fetch() >> ``` >> >> Thanks >> David >> >> [1]. https://lore.kernel.org/lkml/531adbba.b537.196b0868a8c.Coremail.00107082@163.com/ >> ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG?]Data key in /proc/allocinfo is a multiset 2025-05-09 16:36 ` David Wang @ 2025-05-09 16:57 ` Suren Baghdasaryan 2025-05-09 17:31 ` Kent Overstreet 0 siblings, 1 reply; 5+ messages in thread From: Suren Baghdasaryan @ 2025-05-09 16:57 UTC (permalink / raw) To: David Wang; +Cc: kent.overstreet, akpm, linux-kernel On Fri, May 9, 2025 at 9:36 AM David Wang <00107082@163.com> wrote: > > > At 2025-05-09 23:56:32, "Suren Baghdasaryan" <surenb@google.com> wrote: > >On Thu, May 8, 2025 at 11:10 PM David Wang <00107082@163.com> wrote: > >> > >> Just start a new thread for this[1]. > >> There are duplications in /proc/allocinfo where same [file:line] > >> shows up several times: > >> > >> ======================= > >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > >> ======================= > >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > >> ======================= > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> ======================= > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > > >Yep, that happens when an inlined function allocates memory. It ends > >up inlined in different locations. Usually that's done by allocation > >helper functions. > >To fix this we need to wrap these allocator helpers with alloc_hooks: > > > >-static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) > >+static inline void *iommu_alloc_pages_node_noprof(int nid, gfp_t gfp, > >int order) > >{ > >- struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, > >order); struct skcipher_request *req; > >+ struct page *page = alloc_pages_node_noprof(nid, gfp | > >__GFP_ZERO, order); struct skcipher_request *req; > >... > >} > >+#define iommu_alloc_pages_node(...) > >alloc_hooks(iommu_alloc_pages_node_noprof(__VA_ARGS__)) > > > >See 2c321f3f70bc "mm: change inlined allocation helpers to account at > >the call site" for examples of how this was done before. > >Thanks, > >Suren. > > Thanks for clarifying this, seems like a never-ending work...... >_<||| Like anything else in the kernel :) > > > > >> ... > >> > >> The duplication make parsing tools a little bit more complicated: > >> the numbers need to be added up, group by key > >> 81920 20 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 20 > >> 1441792 352 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node 352 > >> > >> The script for checking: > >> ``` > >> #!/bin/env python > >> def fetch(): > >> r = {} > >> with open("/proc/allocinfo") as f: > >> for l in f: > >> f = l.strip().split()[2] > >> if f not in r: r[f]=[] > >> r[f].append(l) > >> keys = [] > >> for f, ls in r.items(): > >> if len(ls) > 1: keys.append(f) > >> keys.sort() > >> for f in keys: > >> print "=======================" > >> for l in r[f]: print l, > >> > >> fetch() > >> ``` > >> > >> Thanks > >> David > >> > >> [1]. https://lore.kernel.org/lkml/531adbba.b537.196b0868a8c.Coremail.00107082@163.com/ > >> ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG?]Data key in /proc/allocinfo is a multiset 2025-05-09 16:57 ` Suren Baghdasaryan @ 2025-05-09 17:31 ` Kent Overstreet 0 siblings, 0 replies; 5+ messages in thread From: Kent Overstreet @ 2025-05-09 17:31 UTC (permalink / raw) To: Suren Baghdasaryan; +Cc: David Wang, akpm, linux-kernel On Fri, May 09, 2025 at 09:57:27AM -0700, Suren Baghdasaryan wrote: > On Fri, May 9, 2025 at 9:36 AM David Wang <00107082@163.com> wrote: > > > > > > At 2025-05-09 23:56:32, "Suren Baghdasaryan" <surenb@google.com> wrote: > > >On Thu, May 8, 2025 at 11:10 PM David Wang <00107082@163.com> wrote: > > >> > > >> Just start a new thread for this[1]. > > >> There are duplications in /proc/allocinfo where same [file:line] > > >> shows up several times: > > >> > > >> ======================= > > >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > > >> 0 0 ./include/crypto/kpp.h:185 func:kpp_request_alloc > > >> ======================= > > >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > > >> 0 0 ./include/net/tcp.h:2548 func:tcp_v4_save_options > > >> ======================= > > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/amd/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> ======================= > > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > >> 0 0 drivers/iommu/intel/../iommu-pages.h:94 func:iommu_alloc_pages_node > > > > > >Yep, that happens when an inlined function allocates memory. It ends > > >up inlined in different locations. Usually that's done by allocation > > >helper functions. > > >To fix this we need to wrap these allocator helpers with alloc_hooks: > > > > > >-static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) > > >+static inline void *iommu_alloc_pages_node_noprof(int nid, gfp_t gfp, > > >int order) > > >{ > > >- struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, > > >order); struct skcipher_request *req; > > >+ struct page *page = alloc_pages_node_noprof(nid, gfp | > > >__GFP_ZERO, order); struct skcipher_request *req; > > >... > > >} > > >+#define iommu_alloc_pages_node(...) > > >alloc_hooks(iommu_alloc_pages_node_noprof(__VA_ARGS__)) > > > > > >See 2c321f3f70bc "mm: change inlined allocation helpers to account at > > >the call site" for examples of how this was done before. > > >Thanks, > > >Suren. > > > > Thanks for clarifying this, seems like a never-ending work...... >_<||| > > Like anything else in the kernel :) Yeah, I generally end up doing these fixups here and there whenever I'm staring at allocation profiling output - we should probably document the process for that, and there's some nifty tricks you can do. e.g. if you've got "container" data structure, like rhashtable, you don't want allocations accounted to the rhashtable code itself - you really want to know which rhashtable it was for. So, you can create an alloc tag for the rhashtable_init() call (wrap it in alloc_hooks), and then stash a pointer to that alloc tag in 'struct rhashtable', and use that for all the rhashtable internal allocations. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2025-05-09 17:31 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-05-09 6:10 [BUG?]Data key in /proc/allocinfo is a multiset David Wang 2025-05-09 15:56 ` Suren Baghdasaryan 2025-05-09 16:36 ` David Wang 2025-05-09 16:57 ` Suren Baghdasaryan 2025-05-09 17:31 ` Kent Overstreet
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.