[Bug 202889] New: fsync on file fails to persist its size, which is changed by pwrite

[Bug 202889] New: fsync on file fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

            Bug ID: 202889
           Summary: fsync on file fails to persist its size, which is
                    changed by pwrite
           Product: File System
           Version: 2.5
    Kernel Version: v5.0
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: f2fs
          Assignee: filesystem_f2fs@kernel-bugs.kernel.org
          Reporter: seulbae@gatech.edu
        Regression: No

Created attachment 281771
  --> https://bugzilla.kernel.org/attachment.cgi?id=281771&action=edit
Proof of Concept

[Kernel version]
This bug can be reproduced on kernel v5.0.


[Reproduce]
* Use a VM, since our PoC simulates a crash by triggering a SysRq!
1. Download a base image (64 MB)
$ wget https://gts3.org/~seulbae/fsimg/f2fs-10.image

2. Mount the image
$ mkdir /tmp/f2fs
$ sudo mount -o loop f2fs-10.image /tmp/f2fs

3. Compile and run PoC
$ gcc poc.c -o poc
$ sudo ./poc /tmp/f2fs
(System reboots)


[Check]
1. Re-mount the crashed image
$ mkdir /tmp/f2fs
$ sudo mount -o loop f2fs-10.image /tmp/f2fs

2. Check inconsistency
$ stat /tmp/f2fs/foo/bar/xattr
Size: 7860


[Description]
In the base image, 2 directories and 7 files exist.

0: 0755 (mount_point)
+--4: 0755 foo
   +--5: 0755 bar
      +--6 : 0644 baz   (size: 12 bytes)
      +--6 : 0644 hln   (size: 12 bytes)
      +--7 : 0644 xattr (size: 0 bytes)
      +--8 : 0644 acl   (size: 0 bytes)
      +--9 : 0644 æøå   (size: 4 bytes)
      +--10: 0644 fifo  (size: 0 bytes)
      +--11: 0777 sln -> mnt/foo/bar/baz

Below is the breakdown of the PoC:
1. Open the existing, empty file "foo/bar/xattr" (inode #7),
(line 24) int fd = syscall(SYS_open, "foo/bar/xattr", O_RDWR, 0);

2. write some data on it (buf is a 8192-byte char array, all bytes initialized
as '\0'),
(line 25) syscall(SYS_write, fd, buf, 4110);
(line 26) syscall(SYS_write, fd, buf, 3750);

3. fdatasync the inode,
(line 27) syscall(SYS_fdatasync, fd);

4. pwrite on the same file to increase its size to 8192
(line 28) syscall(SYS_pwrite64, fd, buf, 1249, 6943);

5. fsync the inode to persist its new size and data, and
(line 29) syscall(SYS_fsync, fd);

7. simulate a crash by rebooting right away without un-mounting.
(line 30) system("echo b > /proc/sysrq-trigger");

Even though we fsync inode #7 ("foo/bar/xattr") after changing its size from
7860 to 8192, this metadata is somehow not persisted, and the size of
"foo/bar/xattr" is still 7860 after recovering from a crash.


Reported by Seulbae Kim (seulbae@gatech.edu) from SSLab, Gatech.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[Bug 202889] fsync on inode fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

Seulbae Kim (seulbae@gatech.edu) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|fsync on file fails to      |fsync on inode fails to
                   |persist its size, which is  |persist its size, which is
                   |changed by pwrite           |changed by pwrite

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 202889] fsync on inode fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

Chao Yu (chao@kernel.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|                            |chao@kernel.org

--- Comment #1 from Chao Yu (chao@kernel.org) ---
Hi Seulbae,

Thanks for the report. :)

Your test program didn't handle any error paths, it should be fixed.

I try to reproduce this bug, but it looks there is no problem in my
environment.

Could you add error handling and try again?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 202889] fsync on inode fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

Seulbae Kim (seulbae@gatech.edu) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #281771|0                           |1
        is obsolete|                            |

--- Comment #2 from Seulbae Kim (seulbae@gatech.edu) ---
Created attachment 281785
  --> https://bugzilla.kernel.org/attachment.cgi?id=281785&action=edit
Proof of Concept - fixed

Fixed the path error in PoC!

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 202889] fsync on inode fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

--- Comment #3 from Seulbae Kim (seulbae@gatech.edu) ---
Hi Chao, Thank you for the quick response again!
And sorry, I missed a line in the PoC.. :(

This bug actually is a little bit tricky, because it's not ALWAYS being
reproduced..
Running the PoC repeatedly on a fresh VM instance, I get the wrong file size at
about 50% chance. However, when I run the same test case on a binary, which
utilizes the Linux Kernel Library for mounting, running PoC, remounting, and
retrieving the stat info from the files, I always get the wrong size for this
file, "foo/bar/xattr".

I've just re-tested the PoC on a new VM with kernel v5.0, and again, this bug
is reproduced 4 times out of 10 trials.

Do you have any idea about what is the cause of this nondeterministic behavior?

(In reply to Chao Yu from comment #1)
> Hi Seulbae,
> 
> Thanks for the report. :)
> 
> Your test program didn't handle any error paths, it should be fixed.
> 
> I try to reproduce this bug, but it looks there is no problem in my
> environment.
> 
> Could you add error handling and try again?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 202889] fsync on inode fails to persist its size, which is changed by pwrite

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=202889

--- Comment #4 from Chao Yu (chao@kernel.org) ---
I checked attachment, but error paths of all syscall still didn't be handled
well?

I doubt the last pwrite64 or fsync failed, actually, it's rare.
Otherwise, I suspect there is a bug when we handle FI_AUTO_RECOVER case,
in where we handle i_size with a tricky way.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.