Re: [PATCH 07/10] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN

Arnd Bergmann
2022-04-15T11:09:44Z

On Fri, Apr 15, 2022 at 12:25 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> >
> And it could be that if I have 150k of those smallish allocations, a
> server with lots of active users might have millions. Not having
> looked at where they come from, maybe that isn't the case, but it
> *might* be.
>
> Maybe adding something like a
>
> static int warn_every_1k = 0;
> WARN_ON(size < 32 && (1023 & ++warn_every_1k));
>
> to kmalloc() would give us a statistical view of "lots of these small
> allocations" thing, and we could add GFP_NODMA to them. There probably
> aren't that many places that have those small allocations, and it's
> certainly safer to annotate "this is not for DMA" than have the
> requirement that all DMA allocations must be marked.

I think finding out the allocations is one of the most common examples
for ftrace. I followed the instructions from
https://www.kernel.org/doc/Documentation/trace/events.txt to
show me a histogram of all allocations under 256 bytes, which
(one kernel compile later) gives me something like

$echo 'hist:key=call_site.sym-offset:val=bytes_req:sort=bytes_req.descending
if bytes_req<256' > \
/sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
$ make -skj30
...
$ head /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
{ call_site: [ffffffffc04e457f]
btrfs_delete_delayed_dir_index+0xbf/0x1e0 [btrfs] } hitcount:
146914 bytes_req: 16454368
{ call_site: [ffffffffbbe601a3] generic_file_buffered_read+0x463/0x4a0
} hitcount: 98187 bytes_req: 14906232
{ call_site: [ffffffffc0497b81] btrfs_buffered_write+0x131/0x7e0
[btrfs] } hitcount: 156513 bytes_req: 10038544
{ call_site: [ffffffffc05125c9] btrfs_alloc_block_rsv+0x29/0x60
[btrfs] } hitcount: 155044 bytes_req: 8682464
{ call_site: [ffffffffbbfe7272] kernfs_fop_open+0xc2/0x290
} hitcount: 38764 bytes_req: 5892128
{ call_site: [ffffffffbbfb6ea2] load_elf_binary+0x242/0xed0
} hitcount: 58276 bytes_req: 3729664
{ call_site: [ffffffffc04b52d0] __btrfs_map_block+0x1f0/0xb60 [btrfs]
} hitcount: 29289 bytes_req: 3521656
{ call_site: [ffffffffbbf7ac7e] inotify_handle_inode_event+0x7e/0x210
} hitcount: 61688 bytes_req: 2986992
{ call_site: [ffffffffbbf2fa35] alloc_pipe_info+0x65/0x230
} hitcount: 13139 bytes_req: 2312464
{ call_site: [ffffffffbc0cd3ec] security_task_alloc+0x9c/0x100
} hitcount: 60475 bytes_req: 2177100
{ call_site: [ffffffffbc0cd5f6] security_prepare_creds+0x76/0xa0
} hitcount: 266124 bytes_req: 2128992
{ call_site: [ffffffffbbfe710e] kernfs_get_open_node+0x7e/0x120
} hitcount: 38764 bytes_req: 1860672
{ call_site: [ffffffffc04e1fbd] btrfs_alloc_delayed_item+0x1d/0x50
[btrfs] } hitcount: 11859 bytes_req: 1833383
{ call_site: [ffffffffc046595d] split_item+0x8d/0x2e0 [btrfs]
} hitcount: 14049 bytes_req: 1716288
{ call_site: [ffffffffbbfb6dbc] load_elf_binary+0x15c/0xed0
} hitcount: 58276 bytes_req: 1631728
{ call_site: [ffffffffbbf40e79] __d_alloc+0x179/0x1f0
} hitcount: 24814 bytes_req: 1280649
{ call_site: [ffffffffbbf5203f] single_open+0x2f/0xa0
} hitcount: 34541 bytes_req: 1105312
{ call_site: [ffffffffc047ad0a] btrfs_wq_submit_bio+0x4a/0xe0 [btrfs]
} hitcount: 7746 bytes_req: 1053456
{ call_site: [ffffffffbc519e95] xhci_urb_enqueue+0xf5/0x3c0
} hitcount: 5511 bytes_req: 484968
{ call_site: [ffffffffc0482935] btrfs_opendir+0x25/0x70 [btrfs]
} hitcount: 60245 bytes_req: 481960
{ call_site: [ffffffffc04c44ff] overwrite_item+0x1cf/0x5c0 [btrfs]
} hitcount: 7378 bytes_req: 364305
{ call_site: [ffffffffc04c4514] overwrite_item+0x1e4/0x5c0 [btrfs]
} hitcount: 7378 bytes_req: 364305
{ call_site: [ffffffffc04e207f] btrfs_wq_run_delayed_node+0x2f/0x80
[btrfs] } hitcount: 3427 bytes_req: 356408
{ call_site: [ffffffffbbe7e96d] shmem_symlink+0xbd/0x250
} hitcount: 5169 bytes_req: 242943
{ call_site: [ffffffffc03e0526] hid_input_field+0x56/0x290 [hid]
} hitcount: 11004 bytes_req: 175760

I think these are all safe for the GFP_NODMA approach you suggest, maybe
not the xhci_urb_enqueue one.

Arnd