public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH V4 00/10] Bail out if transaction can cause extent count to overflow
@ 2020-09-18  9:47 Chandan Babu R
  2020-09-18  9:47 ` [PATCH V4 01/10] xfs: Add helper for checking per-inode extent count overflow Chandan Babu R
                   ` (9 more replies)
  0 siblings, 10 replies; 18+ messages in thread
From: Chandan Babu R @ 2020-09-18  9:47 UTC (permalink / raw)
  To: linux-xfs; +Cc: Chandan Babu R, darrick.wong, david

XFS does not check for possible overflow of per-inode extent counter
fields when adding extents to either data or attr fork.

For e.g.
1. Insert 5 million xattrs (each having a value size of 255 bytes) and
   then delete 50% of them in an alternating manner.

2. On a 4k block sized XFS filesystem instance, the above causes 98511
   extents to be created in the attr fork of the inode.

   xfsaild/loop0  2008 [003]  1475.127209: probe:xfs_inode_to_disk: (ffffffffa43fb6b0) if_nextents=98511 i_ino=131

3. The incore inode fork extent counter is a signed 32-bit
   quantity. However, the on-disk extent counter is an unsigned 16-bit
   quantity and hence cannot hold 98511 extents.

4. The following incorrect value is stored in the xattr extent counter,
   # xfs_db -f -c 'inode 131' -c 'print core.naextents' /dev/loop0
   core.naextents = -32561

This patchset adds a new helper function
(i.e. xfs_iext_count_may_overflow()) to check for overflow of the
per-inode data and xattr extent counters and invokes it before
starting an fs operation (e.g. creating a new directory entry). With
this patchset applied, XFS detects counter overflows and returns with
an error rather than causing a silent corruption.

The patchset has been tested by executing xfstests with the following
mkfs.xfs options,
1. -m crc=0 -b size=1k
2. -m crc=0 -b size=4k
3. -m crc=0 -b size=512
4. -m rmapbt=1,reflink=1 -b size=1k
5. -m rmapbt=1,reflink=1 -b size=4k

The patches can also be obtained from
https://github.com/chandanr/linux.git at branch xfs-reserve-extent-count-v4.

I have a patch that defines the newly introduced error injection tag
in xfsprogs
(https://github.com/chandanr/xfsprogs-dev/commit/7fd7aeef1cefbcc9abd6dd5887e710c80e48079d).
I have also written tests (https://paste.debian.net/1164261/) for
verifying the checks introduced in the kernel. The tests have to be
edited to make them suitable for merging with xfstests. But they also
depend on the changes introduced in xfs_bmap_add_extent_hole_real()
and xfs_growfs_rt_alloc(). Hence, I am planning to post the changes
for xfsprogs and xfstests if other developers are fine with the
changes made to the previously listed functions.

Changelog:
V3 -> V4:
  1. Introduce new patch which lets userspace programs to test "extent
     count overflow detection" by injecting an error tag. The new
     error tag reduces the maximum allowed extent count to 10.
  2. Injecting the newly defined error tag prevents
     xfs_bmap_add_extent_hole_real() from merging a new extent with
     its neighbours to allow writing deterministic tests for testing
     extent count overflow for Directories, Xattr and growing realtime
     devices. This is required because the new extent being allocated
     can be contiguous with its neighbours (w.r.t both file and disk
     offsets).
  3. Injecting the newly defined error tag forces block sized extents
     to be allocated for summary/bitmap files when growing a realtime
     device. This is required because xfs_growfs_rt_alloc() allocates
     as large an extent as possible for summary/bitmap files and hence
     it would be impossible to write deterministic tests.
  4. Rename XFS_IEXT_REMOVE_CNT to XFS_IEXT_PUNCH_HOLE_CNT to reflect
     the actual meaning of the fs operation.
  5. Fold XFS_IEXT_INSERT_HOLE_CNT code into that associated with
     XFS_IEXT_PUNCH_HOLE_CNT since both perform the same job.
  6. xfs_swap_extent_rmap(): Check for extent overflow should be made
     on the source file only if the donor file extent has a valid
     on-disk mapping and vice versa.

V2 -> V3:
  1. Move the definition of xfs_iext_count_may_overflow() from
     libxfs/xfs_trans_resv.c to libxfs/xfs_inode_fork.c. Also, I tried
     to make xfs_iext_count_may_overflow() an inline function by
     placing the definition in libxfs/xfs_inode_fork.h. However this
     required that the definition of 'struct xfs_inode' be available,
     since xfs_iext_count_may_overflow() uses a 'struct xfs_inode *'
     type variable.
  2. Handle XFS_COW_FORK within xfs_iext_count_may_overflow() by
     returning a success value.
  3. Rename XFS_IEXT_ADD_CNT to XFS_IEXT_ADD_NOSPLIT_CNT. Thanks to
     Darrick for the suggesting the new name.
  4. Expand comments to make use of 80 columns.

V1 -> V2:
  1. Rename helper function from xfs_trans_resv_ext_cnt() to
     xfs_iext_count_may_overflow().
  2. Define and use macros to represent fs operations and the
     corresponding increase in extent count.
  3. Split the patches based on the fs operation being performed.


Chandan Babu R (10):
  xfs: Add helper for checking per-inode extent count overflow
  xfs: Check for extent overflow when trivally adding a new extent
  xfs: Check for extent overflow when punching a hole
  xfs: Check for extent overflow when adding/removing xattrs
  xfs: Check for extent overflow when adding/removing dir entries
  xfs: Check for extent overflow when writing to unwritten extent
  xfs: Check for extent overflow when moving extent from cow to data
    fork
  xfs: Check for extent overflow when remapping an extent
  xfs: Check for extent overflow when swapping extents
  xfs: Introduce error injection to reduce maximum inode fork extent
    count

 fs/xfs/libxfs/xfs_attr.c       | 13 ++++++
 fs/xfs/libxfs/xfs_bmap.c       | 33 ++++++++++-----
 fs/xfs/libxfs/xfs_bmap.h       |  1 +
 fs/xfs/libxfs/xfs_errortag.h   |  4 +-
 fs/xfs/libxfs/xfs_inode_fork.c | 27 ++++++++++++
 fs/xfs/libxfs/xfs_inode_fork.h | 77 ++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_types.h      |  1 +
 fs/xfs/xfs_bmap_item.c         |  5 +++
 fs/xfs/xfs_bmap_util.c         | 32 ++++++++++++++
 fs/xfs/xfs_dquot.c             |  8 +++-
 fs/xfs/xfs_error.c             |  3 ++
 fs/xfs/xfs_inode.c             | 27 ++++++++++++
 fs/xfs/xfs_iomap.c             | 10 +++++
 fs/xfs/xfs_reflink.c           | 10 +++++
 fs/xfs/xfs_rtalloc.c           | 21 +++++++++-
 fs/xfs/xfs_symlink.c           |  5 +++
 16 files changed, 262 insertions(+), 15 deletions(-)

-- 
2.28.0


^ permalink raw reply	[flat|nested] 18+ messages in thread
end of thread, other threads:[~2020-09-19  9:45 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-09-18  9:47 [PATCH V4 00/10] Bail out if transaction can cause extent count to overflow Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 01/10] xfs: Add helper for checking per-inode extent count overflow Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 02/10] xfs: Check for extent overflow when trivally adding a new extent Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 03/10] xfs: Check for extent overflow when punching a hole Chandan Babu R
2020-09-18 15:54   ` Darrick J. Wong
2020-09-19  9:42     ` Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 04/10] xfs: Check for extent overflow when adding/removing xattrs Chandan Babu R
2020-09-18 15:49   ` Darrick J. Wong
2020-09-18  9:47 ` [PATCH V4 05/10] xfs: Check for extent overflow when adding/removing dir entries Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 06/10] xfs: Check for extent overflow when writing to unwritten extent Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 07/10] xfs: Check for extent overflow when moving extent from cow to data fork Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 08/10] xfs: Check for extent overflow when remapping an extent Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 09/10] xfs: Check for extent overflow when swapping extents Chandan Babu R
2020-09-18 15:44   ` Darrick J. Wong
2020-09-19  9:44     ` Chandan Babu R
2020-09-18  9:47 ` [PATCH V4 10/10] xfs: Introduce error injection to reduce maximum inode fork extent count Chandan Babu R
2020-09-18 15:39   ` Darrick J. Wong
2020-09-19  9:45     ` Chandan Babu R

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox