The proper handling of failed IO error?

The proper handling of failed IO error?

[Qu Wenruo]

Hi,

There is a recent (well a year ago) change in btrfs to remove the usage
of page/folio error, which gets me wondering what would happen if we got
a lot of write errors and high memory pressure?

Yes, all file systems calls mapping_set_error() so that fsync call would
return error, but I'm wondering what would happen to those folios that
failed to be written?

Those folios has their DIRTY flag cleared before submission, and and
their endio functions, the WRITEBACK flags is also cleared.

Meaning after such write failure, the page/folio has UPTODATE flag, and
no DIRTY/ERROR/WRITEBACK flags (at least for btrfs and ext4, meanwhile
iomap still set the ERROR flag).

Would any memory pressure just reclaim those pages/folios without them
really reaching the disk?

Thanks,
Qu

Re: The proper handling of failed IO error?

[Matthew Wilcox]

On Mon, Jun 10, 2024 at 06:50:11AM +0930, Qu Wenruo wrote:
> Hi,
> 
> There is a recent (well a year ago) change in btrfs to remove the usage
> of page/folio error, which gets me wondering what would happen if we got
> a lot of write errors and high memory pressure?
> 
> Yes, all file systems calls mapping_set_error() so that fsync call would
> return error, but I'm wondering what would happen to those folios that
> failed to be written?
> 
> Those folios has their DIRTY flag cleared before submission, and and
> their endio functions, the WRITEBACK flags is also cleared.
> 
> Meaning after such write failure, the page/folio has UPTODATE flag, and
> no DIRTY/ERROR/WRITEBACK flags (at least for btrfs and ext4, meanwhile
> iomap still set the ERROR flag).
> 
> Would any memory pressure just reclaim those pages/folios without them
> really reaching the disk?

Yes.

Core code doesn't (and hasn't in some time) checked the page/folio
error flag.  That's why it's being removed.

Also, btrfs was using it incorrectly to indicate a write error.
It was supposed to be used for read errors, not write errors.
Another good reason to remove it.

Re: The proper handling of failed IO error?

[Qu Wenruo]

在 2024/6/10 13:19, Matthew Wilcox 写道:
> On Mon, Jun 10, 2024 at 06:50:11AM +0930, Qu Wenruo wrote:
>> Hi,
>>
>> There is a recent (well a year ago) change in btrfs to remove the usage
>> of page/folio error, which gets me wondering what would happen if we got
>> a lot of write errors and high memory pressure?
>>
>> Yes, all file systems calls mapping_set_error() so that fsync call would
>> return error, but I'm wondering what would happen to those folios that
>> failed to be written?
>>
>> Those folios has their DIRTY flag cleared before submission, and and
>> their endio functions, the WRITEBACK flags is also cleared.
>>
>> Meaning after such write failure, the page/folio has UPTODATE flag, and
>> no DIRTY/ERROR/WRITEBACK flags (at least for btrfs and ext4, meanwhile
>> iomap still set the ERROR flag).
>>
>> Would any memory pressure just reclaim those pages/folios without them
>> really reaching the disk?
> 
> Yes.
> 
> Core code doesn't (and hasn't in some time) checked the page/folio
> error flag.  That's why it's being removed.
> 
> Also, btrfs was using it incorrectly to indicate a write error.
> It was supposed to be used for read errors, not write errors.
> Another good reason to remove it.
> 

Then would it be a good idea to only clear the DIRTY flag when a 
successful writeback happened?

Or MM has some strong requirement to have DIRTY cleared before marking 
WRITEBACK?

And any idea of possible problems if we keep the DIRTY flag?
(I guess if the writeback always fails, we can cause very high memory 
pressure?)

Thanks,
Qu