UBIFS file has zeroes at the end after an unclean reboot

UBIFS file has zeroes at the end after an unclean reboot

[Bean Huo (beanhuo)]

Hi, Richard
Do you have good suggestions about how to prevent this condiciton: http://www.linux-mtd.infradead.org/faq/ubifs.html#L_end_hole ?
Do you know How about usig direct I/O? and set the flush cache with higher frequecy?

I don't know If it is possible for currently UBIFS  to firstly program data inode and then program its metadata assocated. If yes, it is necessary/feasible to acchive it?
Thanks.

//beanhuo

Re: UBIFS file has zeroes at the end after an unclean reboot

[Richard Weinberger]

Bean,

Am Montag, 23. Juli 2018, 13:12:09 CEST schrieb Bean Huo (beanhuo):
> Hi, Richard
> Do you have good suggestions about how to prevent this condiciton: http://www.linux-mtd.infradead.org/faq/ubifs.html#L_end_hole ?

Well, I'd start with making sure that userspaces does the right thing(tm) and to
be very sure what kind of problem the user is facing.
Did you verify whether the affected program is using fsync/fdatasync?

Does your kernel include
commit 1b7fc2c0069f3864a3dda15430b7aded31c0bfcc
Author: Rafał Miłecki <rafal@milecki.pl>
Date:   Tue Sep 20 10:36:15 2016 +0200

    ubifs: Use dirty_writeback_interval value for wbuf timer
    
    Right now wbuf timer has hardcoded timeouts and there is no place for
    manual adjustments. Some projects / cases many need that though. Few
    file systems allow doing that by respecting dirty_writeback_interval
    that can be set using sysctl (dirty_writeback_centisecs).
    
    Lowering dirty_writeback_interval could be some way of dealing with user
    space apps lacking proper fsyncs. This is definitely *not* a perfect
    solution but we don't have ideal (user space) world. There were already
    advanced discussions on this matter, mostly when ext4 was introduced and
    it wasn't behaving as ext3. Anyway, the final decision was to add some
    hacks to the ext4, as trying to fix whole user space or adding new API
    was pointless.
    
    We can't (and shouldn't?) just follow ext4. We can't e.g. sync on close
    as this would cause too many commits and flash wearing. On the other
    hand we still should allow some trade-off between -o sync and default
    wbuf timeout. Respecting dirty_writeback_interval should allow some sane
    cutomizations if used warily.
    
    Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
    Reviewed-by: Boris Brezillon <boris.brezillon@free-electrons.com>
    Signed-off-by: Richard Weinberger <richard@nod.at>
?

Thanks,
//richard

RE: [EXT] Re: UBIFS file has zeroes at the end after an unclean reboot

[Bean Huo (beanhuo)]

Hi,Richard
Thanks.  
it is Async mode ubifs  mounted. After random powerloss testing, found that there is one file contains huge zeroes data.
But UBIFS didn't crash and no ECC/CRC error.

As for the below patch, doesn’t exist, we will try that.
Thanks again.

>Am Montag, 23. Juli 2018, 13:12:09 CEST schrieb Bean Huo (beanhuo):
>> Hi, Richard
>> Do you have good suggestions about how to prevent this condiciton:
>http://www.linux-mtd.infradead.org/faq/ubifs.html#L_end_hole ?
>
>Well, I'd start with making sure that userspaces does the right thing(tm) and
>to be very sure what kind of problem the user is facing.
>Did you verify whether the affected program is using fsync/fdatasync?
>
>Does your kernel include
>commit 1b7fc2c0069f3864a3dda15430b7aded31c0bfcc
>Author: Rafał Miłecki <rafal@milecki.pl>
>Date:   Tue Sep 20 10:36:15 2016 +0200
>
>    ubifs: Use dirty_writeback_interval value for wbuf timer
>
>    Right now wbuf timer has hardcoded timeouts and there is no place for
>    manual adjustments. Some projects / cases many need that though. Few
>    file systems allow doing that by respecting dirty_writeback_interval
>    that can be set using sysctl (dirty_writeback_centisecs).
>
>    Lowering dirty_writeback_interval could be some way of dealing with user
>    space apps lacking proper fsyncs. This is definitely *not* a perfect
>    solution but we don't have ideal (user space) world. There were already
>    advanced discussions on this matter, mostly when ext4 was introduced and
>    it wasn't behaving as ext3. Anyway, the final decision was to add some
>    hacks to the ext4, as trying to fix whole user space or adding new API
>    was pointless.
>
>    We can't (and shouldn't?) just follow ext4. We can't e.g. sync on close
>    as this would cause too many commits and flash wearing. On the other
>    hand we still should allow some trade-off between -o sync and default
>    wbuf timeout. Respecting dirty_writeback_interval should allow some sane
>    cutomizations if used warily.
>
>    Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
>    Reviewed-by: Boris Brezillon <boris.brezillon@free-electrons.com>
>    Signed-off-by: Richard Weinberger <richard@nod.at> ?
>
>Thanks,
>//richard

Re: [EXT] Re: UBIFS file has zeroes at the end after an unclean reboot

[Richard Weinberger]

Bean,

Am Montag, 23. Juli 2018, 15:25:49 CEST schrieb Bean Huo (beanhuo):
> Hi,Richard
> Thanks.  
> it is Async mode ubifs  mounted. After random powerloss testing, found that there is one file contains huge zeroes data.
> But UBIFS didn't crash and no ECC/CRC error.

Well, if the application changed the file while the power-cut happens, such situation can happen on all
kind of filesystems.
That's why I asked whether the application does the right thing. :)

Thanks,
//richard