* [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
@ 2023-11-30 15:20 Dongyun Liu
2023-11-30 15:37 ` Jens Axboe
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Dongyun Liu @ 2023-11-30 15:20 UTC (permalink / raw)
To: minchan, senozhatsky, axboe
Cc: linux-kernel, linux-block, lincheng.yang, jiajun.ling, ldys2014,
Dongyun Liu
We encountered a deadlock issue when using zram on kernel version 5.10.
The reason is that the backing_dev_store will first hold the
zram->init_lock, and then it will allocate memory with kvmalloc_node. At
this moment, the system may be in a state of memory shortage, so it will
enter the memory swapping process. In zram_bvec_write, we will trigger
our own thread to move data from zram to the backing device to free up
more available memory, which is the work done in the
try_wakeup_zram_wbd. In this function, zram->init_lock will be acquired
again to read zram->bd_wb_limit, which causes a deadlock as follow call
trace.
The latest version of Linux does not have the bug, but a potential risk
in future.If we try to acquire zram->init_lock again in the zram process
(for example, when we need to read zram->bd_wb_limit), there is a risk
of deadlock. The bug is quite elusive, and it only appears with low
probability after conducting a week-long stress test using 50 devices.
Therefore, I suggest passing the GFP_ATOMIC flag to allocate memory in
the backing_dev_store, to avoid similar issues we encountered. Please
consider it, Thank you.
INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
Call trace:
__switch_to+0x244/0x4e4
__schedule+0x5bc/0xc48
schedule+0x80/0x164
rwsem_down_read_slowpath+0x4fc/0xf9c
__down_read+0x140/0x188
down_read+0x14/0x24
try_wakeup_wbd_thread+0x78/0x1ec [zram]
__zram_bvec_write+0x720/0x878 [zram]
zram_bvec_rw+0xa8/0x234 [zram]
zram_submit_bio+0x16c/0x268 [zram]
submit_bio_noacct+0x128/0x3c8
submit_bio+0x1cc/0x3d0
__swap_writepage+0x5c4/0xd4c
swap_writepage+0x130/0x158
pageout+0x1f4/0x478
shrink_page_list+0x9b4/0x1eb8
shrink_inactive_list+0x2f4/0xaa8
shrink_lruvec+0x184/0x340
shrink_node_memcgs+0x84/0x3a0
shrink_node+0x2c4/0x6c4
shrink_zones+0x16c/0x29c
do_try_to_free_pages+0xe4/0x2b4
try_to_free_pages+0x388/0x7b4
__alloc_pages_direct_reclaim+0x88/0x278
__alloc_pages_slowpath+0x4ec/0xf6c
__alloc_pages_nodemask+0x1f4/0x3dc
kmalloc_order+0x54/0x338
kmalloc_order_trace+0x34/0x1bc
__kmalloc+0x5e8/0x9c0
kvmalloc_node+0xa8/0x264
backing_dev_store+0x1a4/0x818 [zram]
dev_attr_store+0x38/0x8c
sysfs_kf_write+0x64/0xc4
kernfs_fop_write_iter+0x168/0x2ac
vfs_write+0x300/0x37c
ksys_write+0x7c/0xf0
__arm64_sys_write+0x20/0x30
el0_svc_common+0xd4/0x270
el0_svc+0x28/0x98
el0_sync_handler+0x8c/0xf0
el0_sync+0x1b4/0x1c0
Signed-off-by: Dongyun Liu <dongyun.liu@transsion.com>
---
drivers/block/zram/zram_drv.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index d77d3664ca08..ee6c22c50e09 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
nr_pages = i_size_read(inode) >> PAGE_SHIFT;
bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
- bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
+ bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
if (!bitmap) {
err = -ENOMEM;
goto out;
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-11-30 15:20 [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store Dongyun Liu
@ 2023-11-30 15:37 ` Jens Axboe
2023-12-01 6:51 ` Dongyun Liu
2023-12-01 15:39 ` Sergey Senozhatsky
2023-12-01 18:41 ` Minchan Kim
2 siblings, 1 reply; 12+ messages in thread
From: Jens Axboe @ 2023-11-30 15:37 UTC (permalink / raw)
To: Dongyun Liu, minchan, senozhatsky
Cc: linux-kernel, linux-block, lincheng.yang, jiajun.ling, ldys2014,
Dongyun Liu
On 11/30/23 8:20 AM, Dongyun Liu wrote:
> diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
> index d77d3664ca08..ee6c22c50e09 100644
> --- a/drivers/block/zram/zram_drv.c
> +++ b/drivers/block/zram/zram_drv.c
> @@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
>
> nr_pages = i_size_read(inode) >> PAGE_SHIFT;
> bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
> - bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
> + bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
> if (!bitmap) {
> err = -ENOMEM;
> goto out;
Outside of this moving from a zeroed alloc to one that does not, the
change looks woefully incomplete. Why does this allocation need to be
GFP_ATOMIC, and:
1) file_name = kmalloc(PATH_MAX, GFP_KERNEL); does not
2) filp_open() -> getname_kernel() -> __getname() does not
3) filp_open() -> getname_kernel() does not
4) bdev_open_by_dev() does not
IOW, you have a slew of GFP_KERNEL allocations in there, and you
probably just patched the largest one. But the core issue remains.
The whole handling of backing_dev_store() looks pretty broken.
--
Jens Axboe
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-11-30 15:37 ` Jens Axboe
@ 2023-12-01 6:51 ` Dongyun Liu
2023-12-01 14:19 ` Jens Axboe
0 siblings, 1 reply; 12+ messages in thread
From: Dongyun Liu @ 2023-12-01 6:51 UTC (permalink / raw)
To: Jens Axboe, minchan, senozhatsky
Cc: linux-kernel, linux-block, lincheng.yang, jiajun.ling, ldys2014,
Dongyun Liu
On 2023/11/30 23:37, Jens Axboe wrote:
> On 11/30/23 8:20 AM, Dongyun Liu wrote:
>> diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
>> index d77d3664ca08..ee6c22c50e09 100644
>> --- a/drivers/block/zram/zram_drv.c
>> +++ b/drivers/block/zram/zram_drv.c
>> @@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
>>
>> nr_pages = i_size_read(inode) >> PAGE_SHIFT;
>> bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
>> - bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
>> + bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
>> if (!bitmap) {
>> err = -ENOMEM;
>> goto out;
>
> Outside of this moving from a zeroed alloc to one that does not, the
> change looks woefully incomplete. Why does this allocation need to be
> GFP_ATOMIC, and:
By using GFP_ATOMIC, it indicates that the caller cannot reclaim or
sleep, although we can prevent the risk of deadlock when acquiring the
zram->lock again in zram_bvec_write.
>
> 1) file_name = kmalloc(PATH_MAX, GFP_KERNEL); does not
There is no zram->init_lock held here, so there is no need to use
GFP_ATOMIC.
> 2) filp_open() -> getname_kernel() -> __getname() does not
> 3) filp_open() -> getname_kernel() does not
> 4) bdev_open_by_dev() does not
Missing the use of GFP_ATOMIC.
>
> IOW, you have a slew of GFP_KERNEL allocations in there, and you
> probably just patched the largest one. But the core issue remains.
>
> The whole handling of backing_dev_store() looks pretty broken.
>
Indeed, this patch only solves the biggest problem and does not
fundamentally solve it, because there are many processes for holding
zram->init_lock before allocation memory in backing_dev_store that need
to be fully modified, and I did not consider it thoroughly. Obviously,
a larger and better patch is needed to eliminate this risk, but it is
currently not necessary.
Thank you for your kind and patient.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-01 6:51 ` Dongyun Liu
@ 2023-12-01 14:19 ` Jens Axboe
2023-12-01 15:28 ` Sergey Senozhatsky
2023-12-02 13:54 ` Dongyun Liu
0 siblings, 2 replies; 12+ messages in thread
From: Jens Axboe @ 2023-12-01 14:19 UTC (permalink / raw)
To: Dongyun Liu, minchan, senozhatsky
Cc: linux-kernel, linux-block, lincheng.yang, jiajun.ling, ldys2014,
Dongyun Liu
On 11/30/23 11:51 PM, Dongyun Liu wrote:
>
>
> On 2023/11/30 23:37, Jens Axboe wrote:
>> On 11/30/23 8:20 AM, Dongyun Liu wrote:
>>> diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
>>> index d77d3664ca08..ee6c22c50e09 100644
>>> --- a/drivers/block/zram/zram_drv.c
>>> +++ b/drivers/block/zram/zram_drv.c
>>> @@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
>>> nr_pages = i_size_read(inode) >> PAGE_SHIFT;
>>> bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
>>> - bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
>>> + bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
>>> if (!bitmap) {
>>> err = -ENOMEM;
>>> goto out;
>>
>> Outside of this moving from a zeroed alloc to one that does not, the
>> change looks woefully incomplete. Why does this allocation need to be
>> GFP_ATOMIC, and:
>
> By using GFP_ATOMIC, it indicates that the caller cannot reclaim or
> sleep, although we can prevent the risk of deadlock when acquiring
> the zram->lock again in zram_bvec_write.
Yes, I am very much aware of how gfp allocation flags work and how why
it's broken. It was a rhetorical question as to why you think you could
get away with just fixing one of them.
>> 1) file_name = kmalloc(PATH_MAX, GFP_KERNEL); does not
>
> There is no zram->init_lock held here, so there is no need to use
> GFP_ATOMIC.
True
>> 2) filp_open() -> getname_kernel() -> __getname() does not
>> 3) filp_open() -> getname_kernel() does not
>> 4) bdev_open_by_dev() does not
>
> Missing the use of GFP_ATOMIC.
Indeed!
>> IOW, you have a slew of GFP_KERNEL allocations in there, and you
>> probably just patched the largest one. But the core issue remains.
>>
>> The whole handling of backing_dev_store() looks pretty broken.
>>
>
> Indeed, this patch only solves the biggest problem and does not
> fundamentally solve it, because there are many processes for holding
> zram->init_lock before allocation memory in backing_dev_store that
> need to be fully modified, and I did not consider it thoroughly.
> Obviously, a larger and better patch is needed to eliminate this risk,
> but it is currently not necessary.
You agree that it doesn't fix the issue, it just happens to fix the one
that you hit. And then you jump to the conclusion that this is all
that's needed to fix it. Ehm, confused?
--
Jens Axboe
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-01 14:19 ` Jens Axboe
@ 2023-12-01 15:28 ` Sergey Senozhatsky
2023-12-02 13:54 ` Dongyun Liu
1 sibling, 0 replies; 12+ messages in thread
From: Sergey Senozhatsky @ 2023-12-01 15:28 UTC (permalink / raw)
To: Jens Axboe, Dongyun Liu
Cc: minchan, senozhatsky, linux-kernel, linux-block, lincheng.yang,
jiajun.ling, ldys2014, Dongyun Liu
On (23/12/01 07:19), Jens Axboe wrote:
> >> IOW, you have a slew of GFP_KERNEL allocations in there, and you
> >> probably just patched the largest one. But the core issue remains.
> >>
> >> The whole handling of backing_dev_store() looks pretty broken.
> >>
> >
> > Indeed, this patch only solves the biggest problem and does not
> > fundamentally solve it, because there are many processes for holding
> > zram->init_lock before allocation memory in backing_dev_store that
> > need to be fully modified, and I did not consider it thoroughly.
> > Obviously, a larger and better patch is needed to eliminate this risk,
> > but it is currently not necessary.
>
> You agree that it doesn't fix the issue, it just happens to fix the one
> that you hit. And then you jump to the conclusion that this is all
> that's needed to fix it. Ehm, confused?
Yeah.
zram is very sensitive to memory - zsmalloc pool needs physical pages
(allocated on demand) to keep data written to zram. zram probably simply
should not be used on systems under such heavy memory pressure. Throwing
GPF_ATOMICs at zram isn't going to fix anything.
Jens, you said that zram's backing device handling is broken. Got a minute
to elaborate?
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-11-30 15:20 [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store Dongyun Liu
2023-11-30 15:37 ` Jens Axboe
@ 2023-12-01 15:39 ` Sergey Senozhatsky
2023-12-02 15:50 ` Dongyun Liu
2023-12-01 18:41 ` Minchan Kim
2 siblings, 1 reply; 12+ messages in thread
From: Sergey Senozhatsky @ 2023-12-01 15:39 UTC (permalink / raw)
To: Dongyun Liu
Cc: minchan, senozhatsky, axboe, linux-kernel, linux-block,
lincheng.yang, jiajun.ling, ldys2014, Dongyun Liu
On (23/11/30 23:20), Dongyun Liu wrote:
> INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
> Call trace:
> __switch_to+0x244/0x4e4
> __schedule+0x5bc/0xc48
> schedule+0x80/0x164
> rwsem_down_read_slowpath+0x4fc/0xf9c
> __down_read+0x140/0x188
> down_read+0x14/0x24
> try_wakeup_wbd_thread+0x78/0x1ec [zram]
> __zram_bvec_write+0x720/0x878 [zram]
> zram_bvec_rw+0xa8/0x234 [zram]
> zram_submit_bio+0x16c/0x268 [zram]
> submit_bio_noacct+0x128/0x3c8
> submit_bio+0x1cc/0x3d0
> __swap_writepage+0x5c4/0xd4c
> swap_writepage+0x130/0x158
> pageout+0x1f4/0x478
> shrink_page_list+0x9b4/0x1eb8
> shrink_inactive_list+0x2f4/0xaa8
> shrink_lruvec+0x184/0x340
> shrink_node_memcgs+0x84/0x3a0
> shrink_node+0x2c4/0x6c4
> shrink_zones+0x16c/0x29c
> do_try_to_free_pages+0xe4/0x2b4
> try_to_free_pages+0x388/0x7b4
> __alloc_pages_direct_reclaim+0x88/0x278
> __alloc_pages_slowpath+0x4ec/0xf6c
> __alloc_pages_nodemask+0x1f4/0x3dc
> kmalloc_order+0x54/0x338
> kmalloc_order_trace+0x34/0x1bc
> __kmalloc+0x5e8/0x9c0
> kvmalloc_node+0xa8/0x264
> backing_dev_store+0x1a4/0x818 [zram]
> dev_attr_store+0x38/0x8c
> sysfs_kf_write+0x64/0xc4
Hmm, I'm not really following this backtrace. Backing device
configuration is only possible on un-initialized zram device.
If it's uninitialized, then why is it being used for swapout
later in the call stack?
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-11-30 15:20 [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store Dongyun Liu
2023-11-30 15:37 ` Jens Axboe
2023-12-01 15:39 ` Sergey Senozhatsky
@ 2023-12-01 18:41 ` Minchan Kim
2 siblings, 0 replies; 12+ messages in thread
From: Minchan Kim @ 2023-12-01 18:41 UTC (permalink / raw)
To: Dongyun Liu
Cc: senozhatsky, axboe, linux-kernel, linux-block, lincheng.yang,
jiajun.ling, ldys2014, Dongyun Liu
On Thu, Nov 30, 2023 at 11:20:47PM +0800, Dongyun Liu wrote:
> We encountered a deadlock issue when using zram on kernel version 5.10.
> The reason is that the backing_dev_store will first hold the
> zram->init_lock, and then it will allocate memory with kvmalloc_node. At
> this moment, the system may be in a state of memory shortage, so it will
> enter the memory swapping process. In zram_bvec_write, we will trigger
> our own thread to move data from zram to the backing device to free up
> more available memory, which is the work done in the
> try_wakeup_zram_wbd. In this function, zram->init_lock will be acquired
> again to read zram->bd_wb_limit, which causes a deadlock as follow call
> trace.
Isn't it your custom feature instead of upstream?
>
> The latest version of Linux does not have the bug, but a potential risk
> in future.If we try to acquire zram->init_lock again in the zram process
> (for example, when we need to read zram->bd_wb_limit), there is a risk
> of deadlock. The bug is quite elusive, and it only appears with low
It would be very helpful if you show how the zram in the upstream could
cause the deadlock instead of your custom feature.
> probability after conducting a week-long stress test using 50 devices.
> Therefore, I suggest passing the GFP_ATOMIC flag to allocate memory in
> the backing_dev_store, to avoid similar issues we encountered. Please
> consider it, Thank you.
>
> INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
> Call trace:
> __switch_to+0x244/0x4e4
> __schedule+0x5bc/0xc48
> schedule+0x80/0x164
> rwsem_down_read_slowpath+0x4fc/0xf9c
> __down_read+0x140/0x188
> down_read+0x14/0x24
> try_wakeup_wbd_thread+0x78/0x1ec [zram]
> __zram_bvec_write+0x720/0x878 [zram]
> zram_bvec_rw+0xa8/0x234 [zram]
> zram_submit_bio+0x16c/0x268 [zram]
> submit_bio_noacct+0x128/0x3c8
> submit_bio+0x1cc/0x3d0
> __swap_writepage+0x5c4/0xd4c
> swap_writepage+0x130/0x158
> pageout+0x1f4/0x478
> shrink_page_list+0x9b4/0x1eb8
> shrink_inactive_list+0x2f4/0xaa8
> shrink_lruvec+0x184/0x340
> shrink_node_memcgs+0x84/0x3a0
> shrink_node+0x2c4/0x6c4
> shrink_zones+0x16c/0x29c
> do_try_to_free_pages+0xe4/0x2b4
> try_to_free_pages+0x388/0x7b4
> __alloc_pages_direct_reclaim+0x88/0x278
> __alloc_pages_slowpath+0x4ec/0xf6c
> __alloc_pages_nodemask+0x1f4/0x3dc
> kmalloc_order+0x54/0x338
> kmalloc_order_trace+0x34/0x1bc
> __kmalloc+0x5e8/0x9c0
> kvmalloc_node+0xa8/0x264
> backing_dev_store+0x1a4/0x818 [zram]
> dev_attr_store+0x38/0x8c
> sysfs_kf_write+0x64/0xc4
> kernfs_fop_write_iter+0x168/0x2ac
> vfs_write+0x300/0x37c
> ksys_write+0x7c/0xf0
> __arm64_sys_write+0x20/0x30
> el0_svc_common+0xd4/0x270
> el0_svc+0x28/0x98
> el0_sync_handler+0x8c/0xf0
> el0_sync+0x1b4/0x1c0
>
> Signed-off-by: Dongyun Liu <dongyun.liu@transsion.com>
> ---
> drivers/block/zram/zram_drv.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
> index d77d3664ca08..ee6c22c50e09 100644
> --- a/drivers/block/zram/zram_drv.c
> +++ b/drivers/block/zram/zram_drv.c
> @@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
>
> nr_pages = i_size_read(inode) >> PAGE_SHIFT;
> bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
> - bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
> + bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
> if (!bitmap) {
> err = -ENOMEM;
> goto out;
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-01 14:19 ` Jens Axboe
2023-12-01 15:28 ` Sergey Senozhatsky
@ 2023-12-02 13:54 ` Dongyun Liu
1 sibling, 0 replies; 12+ messages in thread
From: Dongyun Liu @ 2023-12-02 13:54 UTC (permalink / raw)
To: Jens Axboe, minchan, senozhatsky
Cc: linux-kernel, linux-block, lincheng.yang, jiajun.ling, ldys2014,
Dongyun Liu
On 2023/12/1 22:19, Jens Axboe wrote:
> On 11/30/23 11:51 PM, Dongyun Liu wrote:
>>
>>
>> On 2023/11/30 23:37, Jens Axboe wrote:
>>> On 11/30/23 8:20 AM, Dongyun Liu wrote:
>>>> diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
>>>> index d77d3664ca08..ee6c22c50e09 100644
>>>> --- a/drivers/block/zram/zram_drv.c
>>>> +++ b/drivers/block/zram/zram_drv.c
>>>> @@ -514,7 +514,7 @@ static ssize_t backing_dev_store(struct device *dev,
>>>> nr_pages = i_size_read(inode) >> PAGE_SHIFT;
>>>> bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
>>>> - bitmap = kvzalloc(bitmap_sz, GFP_KERNEL);
>>>> + bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
>>>> if (!bitmap) {
>>>> err = -ENOMEM;
>>>> goto out;
>>>
>>> Outside of this moving from a zeroed alloc to one that does not, the
>>> change looks woefully incomplete. Why does this allocation need to be
>>> GFP_ATOMIC, and:
>>
>> By using GFP_ATOMIC, it indicates that the caller cannot reclaim or
>> sleep, although we can prevent the risk of deadlock when acquiring
>> the zram->lock again in zram_bvec_write.
>
> Yes, I am very much aware of how gfp allocation flags work and how why
> it's broken. It was a rhetorical question as to why you think you could
> get away with just fixing one of them.
>
>>> 1) file_name = kmalloc(PATH_MAX, GFP_KERNEL); does not
>>
>> There is no zram->init_lock held here, so there is no need to use
>> GFP_ATOMIC.
>
> True
>
>>> 2) filp_open() -> getname_kernel() -> __getname() does not
>>> 3) filp_open() -> getname_kernel() does not
>>> 4) bdev_open_by_dev() does not
>>
>> Missing the use of GFP_ATOMIC.
>
> Indeed!
>
>>> IOW, you have a slew of GFP_KERNEL allocations in there, and you
>>> probably just patched the largest one. But the core issue remains.
>>>
>>> The whole handling of backing_dev_store() looks pretty broken.
>>>
>>
>> Indeed, this patch only solves the biggest problem and does not
>> fundamentally solve it, because there are many processes for holding
>> zram->init_lock before allocation memory in backing_dev_store that
>> need to be fully modified, and I did not consider it thoroughly.
>> Obviously, a larger and better patch is needed to eliminate this risk,
>> but it is currently not necessary.
>
> You agree that it doesn't fix the issue, it just happens to fix the one
> that you hit. And then you jump to the conclusion that this is all
> that's needed to fix it. Ehm, confused?
>
Hi, Jens, Maybe there's something wrong with my expression. You can
think of it this way: I agree with you that it doesn't fix the issue.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-01 15:39 ` Sergey Senozhatsky
@ 2023-12-02 15:50 ` Dongyun Liu
2023-12-03 1:23 ` Sergey Senozhatsky
0 siblings, 1 reply; 12+ messages in thread
From: Dongyun Liu @ 2023-12-02 15:50 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: minchan, axboe, linux-kernel, linux-block, lincheng.yang,
jiajun.ling, ldys2014, Dongyun Liu
On 2023/12/1 23:39, Sergey Senozhatsky wrote:
> On (23/11/30 23:20), Dongyun Liu wrote:
>> INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
>> Call trace:
>> __switch_to+0x244/0x4e4
>> __schedule+0x5bc/0xc48
>> schedule+0x80/0x164
>> rwsem_down_read_slowpath+0x4fc/0xf9c
>> __down_read+0x140/0x188
>> down_read+0x14/0x24
>> try_wakeup_wbd_thread+0x78/0x1ec [zram]
>> __zram_bvec_write+0x720/0x878 [zram]
>> zram_bvec_rw+0xa8/0x234 [zram]
>> zram_submit_bio+0x16c/0x268 [zram]
>> submit_bio_noacct+0x128/0x3c8
>> submit_bio+0x1cc/0x3d0
>> __swap_writepage+0x5c4/0xd4c
>> swap_writepage+0x130/0x158
>> pageout+0x1f4/0x478
>> shrink_page_list+0x9b4/0x1eb8
>> shrink_inactive_list+0x2f4/0xaa8
>> shrink_lruvec+0x184/0x340
>> shrink_node_memcgs+0x84/0x3a0
>> shrink_node+0x2c4/0x6c4
>> shrink_zones+0x16c/0x29c
>> do_try_to_free_pages+0xe4/0x2b4
>> try_to_free_pages+0x388/0x7b4
>> __alloc_pages_direct_reclaim+0x88/0x278
>> __alloc_pages_slowpath+0x4ec/0xf6c
>> __alloc_pages_nodemask+0x1f4/0x3dc
>> kmalloc_order+0x54/0x338
>> kmalloc_order_trace+0x34/0x1bc
>> __kmalloc+0x5e8/0x9c0
>> kvmalloc_node+0xa8/0x264
>> backing_dev_store+0x1a4/0x818 [zram]
>> dev_attr_store+0x38/0x8c
>> sysfs_kf_write+0x64/0xc4
>
> Hmm, I'm not really following this backtrace. Backing device
> configuration is only possible on un-initialized zram device.
> If it's uninitialized, then why is it being used for swapout
> later in the call stack?
Uh, at this moment, zram has finished initializing and is
working. The backing device is an optional zram-based feature.
I think it can be created later.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-02 15:50 ` Dongyun Liu
@ 2023-12-03 1:23 ` Sergey Senozhatsky
2023-12-03 14:45 ` Dongyun Liu
0 siblings, 1 reply; 12+ messages in thread
From: Sergey Senozhatsky @ 2023-12-03 1:23 UTC (permalink / raw)
To: Dongyun Liu
Cc: Sergey Senozhatsky, minchan, axboe, linux-kernel, linux-block,
lincheng.yang, jiajun.ling, ldys2014, Dongyun Liu
On (23/12/02 23:50), Dongyun Liu wrote:
> On 2023/12/1 23:39, Sergey Senozhatsky wrote:
> > On (23/11/30 23:20), Dongyun Liu wrote:
> > > INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
> > > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > > task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
> > > Call trace:
> > > __switch_to+0x244/0x4e4
> > > __schedule+0x5bc/0xc48
> > > schedule+0x80/0x164
> > > rwsem_down_read_slowpath+0x4fc/0xf9c
> > > __down_read+0x140/0x188
> > > down_read+0x14/0x24
> > > try_wakeup_wbd_thread+0x78/0x1ec [zram]
> > > __zram_bvec_write+0x720/0x878 [zram]
> > > zram_bvec_rw+0xa8/0x234 [zram]
> > > zram_submit_bio+0x16c/0x268 [zram]
> > > submit_bio_noacct+0x128/0x3c8
> > > submit_bio+0x1cc/0x3d0
> > > __swap_writepage+0x5c4/0xd4c
> > > swap_writepage+0x130/0x158
> > > pageout+0x1f4/0x478
> > > shrink_page_list+0x9b4/0x1eb8
> > > shrink_inactive_list+0x2f4/0xaa8
> > > shrink_lruvec+0x184/0x340
> > > shrink_node_memcgs+0x84/0x3a0
> > > shrink_node+0x2c4/0x6c4
> > > shrink_zones+0x16c/0x29c
> > > do_try_to_free_pages+0xe4/0x2b4
> > > try_to_free_pages+0x388/0x7b4
> > > __alloc_pages_direct_reclaim+0x88/0x278
> > > __alloc_pages_slowpath+0x4ec/0xf6c
> > > __alloc_pages_nodemask+0x1f4/0x3dc
> > > kmalloc_order+0x54/0x338
> > > kmalloc_order_trace+0x34/0x1bc
> > > __kmalloc+0x5e8/0x9c0
> > > kvmalloc_node+0xa8/0x264
> > > backing_dev_store+0x1a4/0x818 [zram]
> > > dev_attr_store+0x38/0x8c
> > > sysfs_kf_write+0x64/0xc4
> >
> > Hmm, I'm not really following this backtrace. Backing device
> > configuration is only possible on un-initialized zram device.
> > If it's uninitialized, then why is it being used for swapout
> > later in the call stack?
>
> Uh, at this moment, zram has finished initializing and is
> working. The backing device is an optional zram-based feature.
> I think it can be created later.
backing_dev_store() can't be called on an initialized device,
that's what init_done() at the very beginning of backing_dev_store()
is supposed to ensure:
...
down_write(&zram->init_lock);
if (init_done(zram)) {
pr_info("Can't setup backing device for initialized device\n");
err = -EBUSY;
goto out;
}
...
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
2023-12-03 1:23 ` Sergey Senozhatsky
@ 2023-12-03 14:45 ` Dongyun Liu
0 siblings, 0 replies; 12+ messages in thread
From: Dongyun Liu @ 2023-12-03 14:45 UTC (permalink / raw)
To: Sergey Senozhatsky
Cc: minchan, axboe, linux-kernel, linux-block, lincheng.yang,
jiajun.ling, ldys2014, Dongyun Liu
On 2023/12/3 9:23, Sergey Senozhatsky wrote:
> On (23/12/02 23:50), Dongyun Liu wrote:
>> On 2023/12/1 23:39, Sergey Senozhatsky wrote:
>>> On (23/11/30 23:20), Dongyun Liu wrote:
>>>> INFO: task init:331 blocked for more than 120 seconds. "echo 0 >
>>>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>>> task:init state:D stack: 0 pid: 1 ppid: 0 flags:0x04000000
>>>> Call trace:
>>>> __switch_to+0x244/0x4e4
>>>> __schedule+0x5bc/0xc48
>>>> schedule+0x80/0x164
>>>> rwsem_down_read_slowpath+0x4fc/0xf9c
>>>> __down_read+0x140/0x188
>>>> down_read+0x14/0x24
>>>> try_wakeup_wbd_thread+0x78/0x1ec [zram]
>>>> __zram_bvec_write+0x720/0x878 [zram]
>>>> zram_bvec_rw+0xa8/0x234 [zram]
>>>> zram_submit_bio+0x16c/0x268 [zram]
>>>> submit_bio_noacct+0x128/0x3c8
>>>> submit_bio+0x1cc/0x3d0
>>>> __swap_writepage+0x5c4/0xd4c
>>>> swap_writepage+0x130/0x158
>>>> pageout+0x1f4/0x478
>>>> shrink_page_list+0x9b4/0x1eb8
>>>> shrink_inactive_list+0x2f4/0xaa8
>>>> shrink_lruvec+0x184/0x340
>>>> shrink_node_memcgs+0x84/0x3a0
>>>> shrink_node+0x2c4/0x6c4
>>>> shrink_zones+0x16c/0x29c
>>>> do_try_to_free_pages+0xe4/0x2b4
>>>> try_to_free_pages+0x388/0x7b4
>>>> __alloc_pages_direct_reclaim+0x88/0x278
>>>> __alloc_pages_slowpath+0x4ec/0xf6c
>>>> __alloc_pages_nodemask+0x1f4/0x3dc
>>>> kmalloc_order+0x54/0x338
>>>> kmalloc_order_trace+0x34/0x1bc
>>>> __kmalloc+0x5e8/0x9c0
>>>> kvmalloc_node+0xa8/0x264
>>>> backing_dev_store+0x1a4/0x818 [zram]
>>>> dev_attr_store+0x38/0x8c
>>>> sysfs_kf_write+0x64/0xc4
>>>
>>> Hmm, I'm not really following this backtrace. Backing device
>>> configuration is only possible on un-initialized zram device.
>>> If it's uninitialized, then why is it being used for swapout
>>> later in the call stack?
>>
>> Uh, at this moment, zram has finished initializing and is
>> working. The backing device is an optional zram-based feature.
>> I think it can be created later.
>
> backing_dev_store() can't be called on an initialized device,
> that's what init_done() at the very beginning of backing_dev_store()
> is supposed to ensure:
>
We have modified the logic here to allow backing_dev_store() to be
called multiple times. That's why the call trace comes in. On the other
hand, I realized that I should show the upstream's call stack instead
of my custom feature next time. Thank you for your toleration.
> ...
> down_write(&zram->init_lock);
> if (init_done(zram)) {
> pr_info("Can't setup backing device for initialized device\n");
> err = -EBUSY;
> goto out;
> }
> ...
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
@ 2023-12-06 8:08 kernel test robot
0 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2023-12-06 8:08 UTC (permalink / raw)
To: oe-kbuild; +Cc: lkp, Julia Lawall
BCC: lkp@intel.com
CC: oe-kbuild-all@lists.linux.dev
In-Reply-To: <20231130152047.200169-1-dongyun.liu@transsion.com>
References: <20231130152047.200169-1-dongyun.liu@transsion.com>
TO: Dongyun Liu <dongyun.liu3@gmail.com>
TO: minchan@kernel.org
TO: senozhatsky@chromium.org
TO: axboe@kernel.dk
CC: linux-kernel@vger.kernel.org
CC: linux-block@vger.kernel.org
CC: lincheng.yang@transsion.com
CC: jiajun.ling@transsion.com
CC: ldys2014@foxmail.com
CC: Dongyun Liu <dongyun.liu@transsion.com>
Hi Dongyun,
kernel test robot noticed the following build warnings:
[auto build test WARNING on axboe-block/for-next]
[also build test WARNING on linus/master v6.7-rc4 next-20231206]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Dongyun-Liu/zram-Using-GFP_ATOMIC-instead-of-GFP_KERNEL-to-allocate-bitmap-memory-in-backing_dev_store/20231130-233042
base: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-next
patch link: https://lore.kernel.org/r/20231130152047.200169-1-dongyun.liu%40transsion.com
patch subject: [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store
:::::: branch date: 6 days ago
:::::: commit date: 6 days ago
config: i386-randconfig-051-20231201 (https://download.01.org/0day-ci/archive/20231206/202312061559.CDXHSGIQ-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20231206/202312061559.CDXHSGIQ-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Reported-by: Julia Lawall <julia.lawall@inria.fr>
| Closes: https://lore.kernel.org/r/202312061559.CDXHSGIQ-lkp@intel.com/
cocci warnings: (new ones prefixed by >>)
>> drivers/block/zram/zram_drv.c:536:14-15: WARNING kmalloc is used to allocate this memory at line 517
vim +536 drivers/block/zram/zram_drv.c
013bf95a83ec76 Minchan Kim 2017-09-06 459
013bf95a83ec76 Minchan Kim 2017-09-06 460 static ssize_t backing_dev_store(struct device *dev,
013bf95a83ec76 Minchan Kim 2017-09-06 461 struct device_attribute *attr, const char *buf, size_t len)
013bf95a83ec76 Minchan Kim 2017-09-06 462 {
013bf95a83ec76 Minchan Kim 2017-09-06 463 char *file_name;
c8bd134a4bddaf Peter Kalauskas 2018-08-21 464 size_t sz;
013bf95a83ec76 Minchan Kim 2017-09-06 465 struct file *backing_dev = NULL;
013bf95a83ec76 Minchan Kim 2017-09-06 466 struct inode *inode;
013bf95a83ec76 Minchan Kim 2017-09-06 467 struct address_space *mapping;
ee763e2143e79f Christoph Hellwig 2020-11-16 468 unsigned int bitmap_sz;
1363d4662a0d28 Minchan Kim 2017-09-06 469 unsigned long nr_pages, *bitmap = NULL;
eed993a0910338 Jan Kara 2023-09-27 470 struct bdev_handle *bdev_handle = NULL;
013bf95a83ec76 Minchan Kim 2017-09-06 471 int err;
013bf95a83ec76 Minchan Kim 2017-09-06 472 struct zram *zram = dev_to_zram(dev);
013bf95a83ec76 Minchan Kim 2017-09-06 473
013bf95a83ec76 Minchan Kim 2017-09-06 474 file_name = kmalloc(PATH_MAX, GFP_KERNEL);
013bf95a83ec76 Minchan Kim 2017-09-06 475 if (!file_name)
013bf95a83ec76 Minchan Kim 2017-09-06 476 return -ENOMEM;
013bf95a83ec76 Minchan Kim 2017-09-06 477
013bf95a83ec76 Minchan Kim 2017-09-06 478 down_write(&zram->init_lock);
013bf95a83ec76 Minchan Kim 2017-09-06 479 if (init_done(zram)) {
013bf95a83ec76 Minchan Kim 2017-09-06 480 pr_info("Can't setup backing device for initialized device\n");
013bf95a83ec76 Minchan Kim 2017-09-06 481 err = -EBUSY;
013bf95a83ec76 Minchan Kim 2017-09-06 482 goto out;
013bf95a83ec76 Minchan Kim 2017-09-06 483 }
013bf95a83ec76 Minchan Kim 2017-09-06 484
e55e1b4831563e Wolfram Sang 2022-08-18 485 strscpy(file_name, buf, PATH_MAX);
c8bd134a4bddaf Peter Kalauskas 2018-08-21 486 /* ignore trailing newline */
c8bd134a4bddaf Peter Kalauskas 2018-08-21 487 sz = strlen(file_name);
c8bd134a4bddaf Peter Kalauskas 2018-08-21 488 if (sz > 0 && file_name[sz - 1] == '\n')
c8bd134a4bddaf Peter Kalauskas 2018-08-21 489 file_name[sz - 1] = 0x00;
013bf95a83ec76 Minchan Kim 2017-09-06 490
013bf95a83ec76 Minchan Kim 2017-09-06 491 backing_dev = filp_open(file_name, O_RDWR|O_LARGEFILE, 0);
013bf95a83ec76 Minchan Kim 2017-09-06 492 if (IS_ERR(backing_dev)) {
013bf95a83ec76 Minchan Kim 2017-09-06 493 err = PTR_ERR(backing_dev);
013bf95a83ec76 Minchan Kim 2017-09-06 494 backing_dev = NULL;
013bf95a83ec76 Minchan Kim 2017-09-06 495 goto out;
013bf95a83ec76 Minchan Kim 2017-09-06 496 }
013bf95a83ec76 Minchan Kim 2017-09-06 497
013bf95a83ec76 Minchan Kim 2017-09-06 498 mapping = backing_dev->f_mapping;
013bf95a83ec76 Minchan Kim 2017-09-06 499 inode = mapping->host;
013bf95a83ec76 Minchan Kim 2017-09-06 500
013bf95a83ec76 Minchan Kim 2017-09-06 501 /* Support only block device in this moment */
013bf95a83ec76 Minchan Kim 2017-09-06 502 if (!S_ISBLK(inode->i_mode)) {
013bf95a83ec76 Minchan Kim 2017-09-06 503 err = -ENOTBLK;
013bf95a83ec76 Minchan Kim 2017-09-06 504 goto out;
013bf95a83ec76 Minchan Kim 2017-09-06 505 }
013bf95a83ec76 Minchan Kim 2017-09-06 506
eed993a0910338 Jan Kara 2023-09-27 507 bdev_handle = bdev_open_by_dev(inode->i_rdev,
eed993a0910338 Jan Kara 2023-09-27 508 BLK_OPEN_READ | BLK_OPEN_WRITE, zram, NULL);
eed993a0910338 Jan Kara 2023-09-27 509 if (IS_ERR(bdev_handle)) {
eed993a0910338 Jan Kara 2023-09-27 510 err = PTR_ERR(bdev_handle);
eed993a0910338 Jan Kara 2023-09-27 511 bdev_handle = NULL;
013bf95a83ec76 Minchan Kim 2017-09-06 512 goto out;
5547932dc67a48 Minchan Kim 2018-12-28 513 }
013bf95a83ec76 Minchan Kim 2017-09-06 514
1363d4662a0d28 Minchan Kim 2017-09-06 515 nr_pages = i_size_read(inode) >> PAGE_SHIFT;
1363d4662a0d28 Minchan Kim 2017-09-06 516 bitmap_sz = BITS_TO_LONGS(nr_pages) * sizeof(long);
e51f2361fc7fd9 Dongyun Liu 2023-11-30 @517 bitmap = kmalloc(bitmap_sz, GFP_ATOMIC);
1363d4662a0d28 Minchan Kim 2017-09-06 518 if (!bitmap) {
1363d4662a0d28 Minchan Kim 2017-09-06 519 err = -ENOMEM;
1363d4662a0d28 Minchan Kim 2017-09-06 520 goto out;
1363d4662a0d28 Minchan Kim 2017-09-06 521 }
1363d4662a0d28 Minchan Kim 2017-09-06 522
013bf95a83ec76 Minchan Kim 2017-09-06 523 reset_bdev(zram);
013bf95a83ec76 Minchan Kim 2017-09-06 524
eed993a0910338 Jan Kara 2023-09-27 525 zram->bdev_handle = bdev_handle;
013bf95a83ec76 Minchan Kim 2017-09-06 526 zram->backing_dev = backing_dev;
1363d4662a0d28 Minchan Kim 2017-09-06 527 zram->bitmap = bitmap;
1363d4662a0d28 Minchan Kim 2017-09-06 528 zram->nr_pages = nr_pages;
013bf95a83ec76 Minchan Kim 2017-09-06 529 up_write(&zram->init_lock);
013bf95a83ec76 Minchan Kim 2017-09-06 530
013bf95a83ec76 Minchan Kim 2017-09-06 531 pr_info("setup backing device %s\n", file_name);
013bf95a83ec76 Minchan Kim 2017-09-06 532 kfree(file_name);
013bf95a83ec76 Minchan Kim 2017-09-06 533
013bf95a83ec76 Minchan Kim 2017-09-06 534 return len;
013bf95a83ec76 Minchan Kim 2017-09-06 535 out:
1363d4662a0d28 Minchan Kim 2017-09-06 @536 kvfree(bitmap);
1363d4662a0d28 Minchan Kim 2017-09-06 537
eed993a0910338 Jan Kara 2023-09-27 538 if (bdev_handle)
eed993a0910338 Jan Kara 2023-09-27 539 bdev_release(bdev_handle);
013bf95a83ec76 Minchan Kim 2017-09-06 540
013bf95a83ec76 Minchan Kim 2017-09-06 541 if (backing_dev)
013bf95a83ec76 Minchan Kim 2017-09-06 542 filp_close(backing_dev, NULL);
013bf95a83ec76 Minchan Kim 2017-09-06 543
013bf95a83ec76 Minchan Kim 2017-09-06 544 up_write(&zram->init_lock);
013bf95a83ec76 Minchan Kim 2017-09-06 545
013bf95a83ec76 Minchan Kim 2017-09-06 546 kfree(file_name);
013bf95a83ec76 Minchan Kim 2017-09-06 547
013bf95a83ec76 Minchan Kim 2017-09-06 548 return err;
013bf95a83ec76 Minchan Kim 2017-09-06 549 }
013bf95a83ec76 Minchan Kim 2017-09-06 550
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2023-12-06 8:09 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-11-30 15:20 [PATCH] zram: Using GFP_ATOMIC instead of GFP_KERNEL to allocate bitmap memory in backing_dev_store Dongyun Liu
2023-11-30 15:37 ` Jens Axboe
2023-12-01 6:51 ` Dongyun Liu
2023-12-01 14:19 ` Jens Axboe
2023-12-01 15:28 ` Sergey Senozhatsky
2023-12-02 13:54 ` Dongyun Liu
2023-12-01 15:39 ` Sergey Senozhatsky
2023-12-02 15:50 ` Dongyun Liu
2023-12-03 1:23 ` Sergey Senozhatsky
2023-12-03 14:45 ` Dongyun Liu
2023-12-01 18:41 ` Minchan Kim
-- strict thread matches above, loose matches on Subject: below --
2023-12-06 8:08 kernel test robot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.