Re: [PATCH] f2fs: avoid hungtask problem caused by losing wake_up

[He YunLei <heyunlei@huawei.com>]

On 2016/2/23 17:32, Shawn Lin wrote:
> On 2016/2/23 15:02, He YunLei wrote:
>> On 2016/2/23 13:44, Chao Yu wrote:
>>> Hi Yunlei,
>> Hi Chao,
>>>
>>>> -----Original Message-----
>>>> From: Yunlei He [mailto:heyunlei@huawei.com]
>>>> Sent: Tuesday, February 23, 2016 12:08 PM
>>>> To: chao2.yu@samsung.com; jaegeuk@kernel.org; linux-f2fs-devel@lists.sourceforge.net
>>>> Cc: bintian.wang@huawei.com; Yunlei He; Biao He
>>>> Subject: [f2fs-dev] [PATCH] f2fs: avoid hungtask problem caused by losing wake_up
>>>>
>>>> The D state of wait_on_all_pages_writeback should be waken by
>>>> function f2fs_write_end_io when all writeback pages have been
>>>> succesfully written to device. It's possible that wake_up comes
>>>> between get_pages and io_schedule. Maybe in this case it will
>>>> lost wake_up and still in D state even if all pages have been
>>>> write back to device, and finally, the whole system will be into
>>>> the hungtask state.
>>>
>>> I haven't encountered such issue so far, do you suffer this in real
>>> world?
>>>
>> yes, I have encounter it, the whole file system is blocked at function
>> wait_on_all_pages_writeback beyond 120s when write cp, and no error reported
>> by storage device driver.
>
Hi Shawn,
> Hi YunLei
>
> Interesting...
> How to prove the page isn't been locked by storage block layer which
> doesn't finish the related bio?
>
> I suffer that case before, but then I solve it by seting a timer to
> break my mmc driver while missing some important interrupts(chip
> problem) for data-transfer to finish the io queue by mmcqd.
>
Thanks for your review, I has replied Chao about the info of f2fs_sb_info,
and from the info we can make sure the block layer and device driver are
both ok.

> Another thing,
> why making 5HZ for timeout? why not 10HZ ,15HZ...
> io_schedule_timeout(5*HZ);
>

We choose 5HZ in our test, maybe other value is more reasonable.

>>>>
>>>>                   if (!get_pages(sbi, F2FS_WRITEBACK))
>>>>                            break;
>>>>                     <---------  wake_up
>>>
>>> wake_up will put all tasks linked in sbi->cp_wait on run-queue, so
>>> here it should be save to call io_schedule, after being rescheduled,
>>> it will get the chance to check above condition to break out.
>>>
>>> Thanks,
>>
>> Here, we just doubt something weird may cause wait_on_all_pages_writeback
>> could not be waken. Wake_up trigger only one time by last bio's end_io
>> function, if the thread happen to miss it, the thread will be in D state
>> forever. So we change the code to make wait_on_all_pages_writeback awaken
>> periodically, then check the condition.
>>
>>>
>>>>                   io_schedule();
>>>>
>>>> Signed-off-by: Yunlei He <heyunlei@huawei.com>
>>>> Signed-off-by: Biao He <hebiao6@huawei.com>
>>>> ---
>>>>    fs/f2fs/checkpoint.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
>>>> index 2bac8a1..f55355d 100644
>>>> --- a/fs/f2fs/checkpoint.c
>>>> +++ b/fs/f2fs/checkpoint.c
>>>> @@ -920,7 +920,7 @@ static void wait_on_all_pages_writeback(struct f2fs_sb_info *sbi)
>>>>            if (!get_pages(sbi, F2FS_WRITEBACK))
>>>>                break;
>>>>
>>>> -        io_schedule();
>>>> +        io_schedule_timeout(5*HZ);
>>>>        }
>>>>        finish_wait(&sbi->cp_wait, &wait);
>>>>    }
>>>> --
>>>> 1.9.1
>>>
>>>
>>>
>>> .
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> Linux-f2fs-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>
>
>
> .
>


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140