* [GIT PULL] large atomic writes for xfs @ 2025-05-07 12:00 ` John Garry 2025-05-07 12:14 ` Carlos Maiolino 2025-05-07 21:32 ` Darrick J. Wong 0 siblings, 2 replies; 8+ messages in thread From: John Garry @ 2025-05-07 12:00 UTC (permalink / raw) To: cem; +Cc: djwong, linux-xfs Hi Carlos, Please pull the large atomic writes series for xfs. The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 20:53:52 +0200) are available in the Git repository at: https://github.com/johnpgarry/linux.git tags/large-atomic-writes-xfs for you to fetch changes up to 2c465e8bf4fd45e913a51506d58bd8906e5de0ca: xfs: allow sysadmins to specify a maximum atomic write limit at mount time (2025-05-07 08:40:35 +0100) ---------------------------------------------------------------- large atomic writes for xfs Signed-off-by: John Garry <john.g.garry@oracle.com> ---------------------------------------------------------------- Darrick J. Wong (6): xfs: only call xfs_setsize_buftarg once per buffer target xfs: separate out setting buftarg atomic writes limits xfs: add helpers to compute log item overhead xfs: add helpers to compute transaction reservation for finishing intent items xfs: ignore HW which cannot atomic write a single block xfs: allow sysadmins to specify a maximum atomic write limit at mount time John Garry (11): fs: add atomic write unit max opt to statx xfs: rename xfs_inode_can_atomicwrite() -> xfs_inode_can_hw_atomic_write() xfs: allow block allocator to take an alignment hint xfs: refactor xfs_reflink_end_cow_extent() xfs: refine atomic write size check in xfs_file_write_iter() xfs: add xfs_atomic_write_cow_iomap_begin() xfs: add large atomic writes checks in xfs_direct_write_iomap_begin() xfs: commit CoW-based atomic writes atomically xfs: add xfs_file_dio_write_atomic() xfs: add xfs_calc_atomic_write_unit_max() xfs: update atomic write limits Documentation/admin-guide/xfs.rst | 11 ++ block/bdev.c | 3 +- fs/ext4/inode.c | 2 +- fs/stat.c | 6 +- fs/xfs/libxfs/xfs_bmap.c | 5 + fs/xfs/libxfs/xfs_bmap.h | 6 +- fs/xfs/libxfs/xfs_log_rlimit.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 343 ++++++++++++++++++++++++++++++++++---- fs/xfs/libxfs/xfs_trans_resv.h | 25 +++ fs/xfs/xfs_bmap_item.c | 10 ++ fs/xfs/xfs_bmap_item.h | 3 + fs/xfs/xfs_buf.c | 70 ++++++-- fs/xfs/xfs_buf.h | 4 +- fs/xfs/xfs_buf_item.c | 19 +++ fs/xfs/xfs_buf_item.h | 3 + fs/xfs/xfs_extfree_item.c | 10 ++ fs/xfs/xfs_extfree_item.h | 3 + fs/xfs/xfs_file.c | 87 +++++++++- fs/xfs/xfs_inode.h | 14 +- fs/xfs/xfs_iomap.c | 190 ++++++++++++++++++++- fs/xfs/xfs_iomap.h | 1 + fs/xfs/xfs_iops.c | 76 ++++++++- fs/xfs/xfs_iops.h | 3 + fs/xfs/xfs_log_cil.c | 4 +- fs/xfs/xfs_log_priv.h | 13 ++ fs/xfs/xfs_mount.c | 161 ++++++++++++++++++ fs/xfs/xfs_mount.h | 17 ++ fs/xfs/xfs_refcount_item.c | 10 ++ fs/xfs/xfs_refcount_item.h | 3 + fs/xfs/xfs_reflink.c | 146 ++++++++++++---- fs/xfs/xfs_reflink.h | 6 + fs/xfs/xfs_rmap_item.c | 10 ++ fs/xfs/xfs_rmap_item.h | 3 + fs/xfs/xfs_super.c | 80 ++++++++- fs/xfs/xfs_trace.h | 115 +++++++++++++ include/linux/fs.h | 3 +- include/linux/stat.h | 1 + include/uapi/linux/stat.h | 8 +- 38 files changed, 1351 insertions(+), 127 deletions(-) ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-07 12:00 ` [GIT PULL] large atomic writes for xfs John Garry @ 2025-05-07 12:14 ` Carlos Maiolino 2025-05-07 12:23 ` John Garry 2025-05-07 21:32 ` Darrick J. Wong 1 sibling, 1 reply; 8+ messages in thread From: Carlos Maiolino @ 2025-05-07 12:14 UTC (permalink / raw) To: John Garry; +Cc: djwong, linux-xfs On Wed, May 07, 2025 at 01:00:00PM +0100, John Garry wrote: > Hi Carlos, > > Please pull the large atomic writes series for xfs. > > The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: > > xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 > 20:53:52 +0200) > > are available in the Git repository at: > > https://github.com/johnpgarry/linux.git tags/large-atomic-writes-xfs > > for you to fetch changes up to 2c465e8bf4fd45e913a51506d58bd8906e5de0ca: The last commit has no Reviews into it. > > xfs: allow sysadmins to specify a maximum atomic write limit at mount > time (2025-05-07 08:40:35 +0100) > > ---------------------------------------------------------------- > large atomic writes for xfs > > Signed-off-by: John Garry <john.g.garry@oracle.com> > > ---------------------------------------------------------------- > Darrick J. Wong (6): > xfs: only call xfs_setsize_buftarg once per buffer target > xfs: separate out setting buftarg atomic writes limits > xfs: add helpers to compute log item overhead > xfs: add helpers to compute transaction reservation for finishing > intent items > xfs: ignore HW which cannot atomic write a single block > xfs: allow sysadmins to specify a maximum atomic write limit at > mount time > > John Garry (11): > fs: add atomic write unit max opt to statx > xfs: rename xfs_inode_can_atomicwrite() -> > xfs_inode_can_hw_atomic_write() > xfs: allow block allocator to take an alignment hint > xfs: refactor xfs_reflink_end_cow_extent() > xfs: refine atomic write size check in xfs_file_write_iter() > xfs: add xfs_atomic_write_cow_iomap_begin() > xfs: add large atomic writes checks in > xfs_direct_write_iomap_begin() > xfs: commit CoW-based atomic writes atomically > xfs: add xfs_file_dio_write_atomic() > xfs: add xfs_calc_atomic_write_unit_max() > xfs: update atomic write limits > > Documentation/admin-guide/xfs.rst | 11 ++ > block/bdev.c | 3 +- > fs/ext4/inode.c | 2 +- > fs/stat.c | 6 +- > fs/xfs/libxfs/xfs_bmap.c | 5 + > fs/xfs/libxfs/xfs_bmap.h | 6 +- > fs/xfs/libxfs/xfs_log_rlimit.c | 4 + > fs/xfs/libxfs/xfs_trans_resv.c | 343 > ++++++++++++++++++++++++++++++++++---- > fs/xfs/libxfs/xfs_trans_resv.h | 25 +++ > fs/xfs/xfs_bmap_item.c | 10 ++ > fs/xfs/xfs_bmap_item.h | 3 + > fs/xfs/xfs_buf.c | 70 ++++++-- > fs/xfs/xfs_buf.h | 4 +- > fs/xfs/xfs_buf_item.c | 19 +++ > fs/xfs/xfs_buf_item.h | 3 + > fs/xfs/xfs_extfree_item.c | 10 ++ > fs/xfs/xfs_extfree_item.h | 3 + > fs/xfs/xfs_file.c | 87 +++++++++- > fs/xfs/xfs_inode.h | 14 +- > fs/xfs/xfs_iomap.c | 190 ++++++++++++++++++++- > fs/xfs/xfs_iomap.h | 1 + > fs/xfs/xfs_iops.c | 76 ++++++++- > fs/xfs/xfs_iops.h | 3 + > fs/xfs/xfs_log_cil.c | 4 +- > fs/xfs/xfs_log_priv.h | 13 ++ > fs/xfs/xfs_mount.c | 161 ++++++++++++++++++ > fs/xfs/xfs_mount.h | 17 ++ > fs/xfs/xfs_refcount_item.c | 10 ++ > fs/xfs/xfs_refcount_item.h | 3 + > fs/xfs/xfs_reflink.c | 146 ++++++++++++---- > fs/xfs/xfs_reflink.h | 6 + > fs/xfs/xfs_rmap_item.c | 10 ++ > fs/xfs/xfs_rmap_item.h | 3 + > fs/xfs/xfs_super.c | 80 ++++++++- > fs/xfs/xfs_trace.h | 115 +++++++++++++ > include/linux/fs.h | 3 +- > include/linux/stat.h | 1 + > include/uapi/linux/stat.h | 8 +- > 38 files changed, 1351 insertions(+), 127 deletions(-) > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-07 12:14 ` Carlos Maiolino @ 2025-05-07 12:23 ` John Garry 2025-05-07 21:30 ` Darrick J. Wong 0 siblings, 1 reply; 8+ messages in thread From: John Garry @ 2025-05-07 12:23 UTC (permalink / raw) To: Carlos Maiolino; +Cc: djwong, linux-xfs On 07/05/2025 13:14, Carlos Maiolino wrote: > On Wed, May 07, 2025 at 01:00:00PM +0100, John Garry wrote: >> Hi Carlos, >> >> Please pull the large atomic writes series for xfs. >> >> The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: >> >> xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 >> 20:53:52 +0200) >> >> are available in the Git repository at: >> >> https://urldefense.com/v3/__https://github.com/johnpgarry/ >> linux.git__;!!ACWV5N9M2RV99hQ! >> IVDUFMxfAfmMgnyhV150ZyTdmIuE2vm93RuY0_z92SeHSsReFAeP5gbh3DA- >> iow80_ciEVx3MhZ7gA$ tags/large-atomic-writes-xfs >> >> for you to fetch changes up to 2c465e8bf4fd45e913a51506d58bd8906e5de0ca: > The last commit has no Reviews into it. I'll add it. Thanks ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-07 12:23 ` John Garry @ 2025-05-07 21:30 ` Darrick J. Wong 2025-05-09 7:21 ` Carlos Maiolino 0 siblings, 1 reply; 8+ messages in thread From: Darrick J. Wong @ 2025-05-07 21:30 UTC (permalink / raw) To: John Garry; +Cc: Carlos Maiolino, linux-xfs On Wed, May 07, 2025 at 01:23:51PM +0100, John Garry wrote: > On 07/05/2025 13:14, Carlos Maiolino wrote: > > On Wed, May 07, 2025 at 01:00:00PM +0100, John Garry wrote: > > > Hi Carlos, > > > > > > Please pull the large atomic writes series for xfs. > > > > > > The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: > > > > > > xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 > > > 20:53:52 +0200) > > > > > > are available in the Git repository at: > > > > > > https://urldefense.com/v3/__https://github.com/johnpgarry/ > > > linux.git__;!!ACWV5N9M2RV99hQ! > > > IVDUFMxfAfmMgnyhV150ZyTdmIuE2vm93RuY0_z92SeHSsReFAeP5gbh3DA- > > > iow80_ciEVx3MhZ7gA$ tags/large-atomic-writes-xfs > > > > > > for you to fetch changes up to 2c465e8bf4fd45e913a51506d58bd8906e5de0ca: > > The last commit has no Reviews into it. > > I'll add it. Not sure why John wants me to create a PR, but I'll do that, and with the two RVB tags received since V12 was posted. --D > Thanks > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-07 21:30 ` Darrick J. Wong @ 2025-05-09 7:21 ` Carlos Maiolino 2025-05-09 8:08 ` John Garry 0 siblings, 1 reply; 8+ messages in thread From: Carlos Maiolino @ 2025-05-09 7:21 UTC (permalink / raw) To: Darrick J. Wong; +Cc: John Garry, linux-xfs On Wed, May 07, 2025 at 02:30:47PM -0700, Darrick J. Wong wrote: > On Wed, May 07, 2025 at 01:23:51PM +0100, John Garry wrote: > > On 07/05/2025 13:14, Carlos Maiolino wrote: > > > On Wed, May 07, 2025 at 01:00:00PM +0100, John Garry wrote: > > > > Hi Carlos, > > > > > > > > Please pull the large atomic writes series for xfs. > > > > > > > > The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: > > > > > > > > xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 > > > > 20:53:52 +0200) > > > > > > > > are available in the Git repository at: > > > > > > > > https://urldefense.com/v3/__https://github.com/johnpgarry/ > > > > linux.git__;!!ACWV5N9M2RV99hQ! > > > > IVDUFMxfAfmMgnyhV150ZyTdmIuE2vm93RuY0_z92SeHSsReFAeP5gbh3DA- > > > > iow80_ciEVx3MhZ7gA$ tags/large-atomic-writes-xfs > > > > > > > > for you to fetch changes up to 2c465e8bf4fd45e913a51506d58bd8906e5de0ca: > > > The last commit has no Reviews into it. > > > > I'll add it. > > Not sure why John wants me to create a PR, but I'll do that, and with > the two RVB tags received since V12 was posted. I think it all boils down for what I spoke with John off-list. John would need to send a PR from outside kernel.org, and while I don't think this is a big deal, he also doesn't have a key signed by anybody on kernel.org. This would essentially break the chain-of-trust if I'd go and pull his PR directly from his repository with his current key. So the possible solutions would be for him to send the final series to the list, and/or a PR from you, until we get John's a signed key. Carlos ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-09 7:21 ` Carlos Maiolino @ 2025-05-09 8:08 ` John Garry 0 siblings, 0 replies; 8+ messages in thread From: John Garry @ 2025-05-09 8:08 UTC (permalink / raw) To: Carlos Maiolino, Darrick J. Wong; +Cc: linux-xfs On 09/05/2025 08:21, Carlos Maiolino wrote: >> Not sure why John wants me to create a PR, but I'll do that, and with >> the two RVB tags received since V12 was posted. > I think it all boils down for what I spoke with John off-list. John would need > to send a PR from outside kernel.org, and while I don't think this is a big > deal, he also doesn't have a key signed by anybody on kernel.org. This would > essentially break the chain-of-trust if I'd go and pull his PR directly from > his repository with his current key. I'll get that sorted > > So the possible solutions would be for him to send the final series to the list, > and/or a PR from you, until we get John's a signed key. You have since taken the PR from Darrick, right? Thanks, John ^ permalink raw reply [flat|nested] 8+ messages in thread
* [GIT PULL] large atomic writes for xfs 2025-05-07 12:00 ` [GIT PULL] large atomic writes for xfs John Garry 2025-05-07 12:14 ` Carlos Maiolino @ 2025-05-07 21:32 ` Darrick J. Wong 2025-05-09 11:43 ` Carlos Maiolino 1 sibling, 1 reply; 8+ messages in thread From: Darrick J. Wong @ 2025-05-07 21:32 UTC (permalink / raw) To: cem, djwong; +Cc: hch, john.g.garry, linux-xfs Hi Carlos, Please pull this branch with changes for xfs for 6.16-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit bfecc4091e07a47696ac922783216d9e9ea46c97: xfs: allow ro mounts if rtdev or logdev are read-only (2025-04-30 20:53:52 +0200) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/atomic-writes-6.16_2025-05-07 for you to fetch changes up to 4528b9052731f14c1a9be16b98e33c9401e6d1bc: xfs: allow sysadmins to specify a maximum atomic write limit at mount time (2025-05-07 14:25:33 -0700) ---------------------------------------------------------------- large atomic writes for xfs [v12.1] Currently atomic write support for xfs is limited to writing a single block as we have no way to guarantee alignment and that the write covers a single extent. This series introduces a method to issue atomic writes via a software-based method. The software-based method is used as a fallback for when attempting to issue an atomic write over misaligned or multiple extents. For xfs, this support is based on reflink CoW support. The basic idea of this CoW method is to alloc a range in the CoW fork, write the data, and atomically update the mapping. Initial mysql performance testing has shown this method to perform ok. However, there we are only using 16K atomic writes (and 4K block size), so typically - and thankfully - this software fallback method won't be used often. For other FSes which want large atomics writes and don't support CoW, I think that they can follow the example in [0]. Catherine is currently working on further xfstests for this feature, which we hope to share soon. About 17/17, maybe it can be omitted as there is no strong demand to have it included. Based on bfecc4091e07 (xfs/next-rc, xfs/for-next) xfs: allow ro mounts if rtdev or logdev are read-only [0] https://lore.kernel.org/linux-xfs/20250102140411.14617-1-john.g.garry@oracle.com/ Differences to v12: - add more review tags Differences to v11: - split "xfs: ignore ..." patch - inline sync_blockdev() in xfs_alloc_buftarg() (Christoph) - fix xfs_calc_rtgroup_awu_max() for 0 block count (Darrick) - Add RB tag from Christoph (thanks!) Differences to v10: - add "xfs: only call xfs_setsize_buftarg once ..." by Darrick - symbol renames in "xfs: ignore HW which cannot..." by Darrick Differences to v9: - rework "ignore HW which cannot .." patch by Darrick - Ensure power-of-2 max always for unit min/max when no HW support With a bit of luck, this should all go splendidly. Signed-off-by: "Darrick J. Wong" <djwong@kernel.org> ---------------------------------------------------------------- Darrick J. Wong (6): xfs: only call xfs_setsize_buftarg once per buffer target xfs: separate out setting buftarg atomic writes limits xfs: add helpers to compute log item overhead xfs: add helpers to compute transaction reservation for finishing intent items xfs: ignore HW which cannot atomic write a single block xfs: allow sysadmins to specify a maximum atomic write limit at mount time John Garry (11): fs: add atomic write unit max opt to statx xfs: rename xfs_inode_can_atomicwrite() -> xfs_inode_can_hw_atomic_write() xfs: allow block allocator to take an alignment hint xfs: refactor xfs_reflink_end_cow_extent() xfs: refine atomic write size check in xfs_file_write_iter() xfs: add xfs_atomic_write_cow_iomap_begin() xfs: add large atomic writes checks in xfs_direct_write_iomap_begin() xfs: commit CoW-based atomic writes atomically xfs: add xfs_file_dio_write_atomic() xfs: add xfs_calc_atomic_write_unit_max() xfs: update atomic write limits fs/xfs/libxfs/xfs_bmap.h | 6 +- fs/xfs/libxfs/xfs_trans_resv.h | 25 +++ fs/xfs/xfs_bmap_item.h | 3 + fs/xfs/xfs_buf.h | 4 +- fs/xfs/xfs_buf_item.h | 3 + fs/xfs/xfs_extfree_item.h | 3 + fs/xfs/xfs_inode.h | 14 +- fs/xfs/xfs_iomap.h | 1 + fs/xfs/xfs_iops.h | 3 + fs/xfs/xfs_log_priv.h | 13 ++ fs/xfs/xfs_mount.h | 17 ++ fs/xfs/xfs_refcount_item.h | 3 + fs/xfs/xfs_reflink.h | 6 + fs/xfs/xfs_rmap_item.h | 3 + fs/xfs/xfs_trace.h | 115 +++++++++++++ include/linux/fs.h | 3 +- include/linux/stat.h | 1 + include/uapi/linux/stat.h | 8 +- Documentation/admin-guide/xfs.rst | 11 ++ block/bdev.c | 3 +- fs/ext4/inode.c | 2 +- fs/stat.c | 6 +- fs/xfs/libxfs/xfs_bmap.c | 5 + fs/xfs/libxfs/xfs_log_rlimit.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 343 ++++++++++++++++++++++++++++++++++---- fs/xfs/xfs_bmap_item.c | 10 ++ fs/xfs/xfs_buf.c | 70 ++++++-- fs/xfs/xfs_buf_item.c | 19 +++ fs/xfs/xfs_extfree_item.c | 10 ++ fs/xfs/xfs_file.c | 87 +++++++++- fs/xfs/xfs_iomap.c | 190 ++++++++++++++++++++- fs/xfs/xfs_iops.c | 76 ++++++++- fs/xfs/xfs_log_cil.c | 4 +- fs/xfs/xfs_mount.c | 161 ++++++++++++++++++ fs/xfs/xfs_refcount_item.c | 10 ++ fs/xfs/xfs_reflink.c | 146 ++++++++++++---- fs/xfs/xfs_rmap_item.c | 10 ++ fs/xfs/xfs_super.c | 80 ++++++++- 38 files changed, 1351 insertions(+), 127 deletions(-) ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [GIT PULL] large atomic writes for xfs 2025-05-07 21:32 ` Darrick J. Wong @ 2025-05-09 11:43 ` Carlos Maiolino 0 siblings, 0 replies; 8+ messages in thread From: Carlos Maiolino @ 2025-05-09 11:43 UTC (permalink / raw) To: Darrick J. Wong; +Cc: hch, john.g.garry, linux-xfs On Wed, 07 May 2025 14:32:12 -0700, Darrick J. Wong wrote: > Please pull this branch with changes for xfs for 6.16-rc1. > > As usual, I did a test-merge with the main upstream branch as of a few > minutes ago, and didn't see any conflicts. Please let me know if you > encounter any problems. > > --D > > [...] Merged, thanks! merge commit: 4abb9052a72bc98d521b0535b5deee243a3bbd12 Best regards, -- Carlos Maiolino <cem@kernel.org> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-05-09 11:43 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <Na27AkD0pwxtDXufbKHmtlQIDLpo1dVCpyqoca91pszUTNuR1GqASOvLkmMZysL1IsA7vt-xoCFsf50SlGYzGg==@protonmail.internalid>
2025-05-07 12:00 ` [GIT PULL] large atomic writes for xfs John Garry
2025-05-07 12:14 ` Carlos Maiolino
2025-05-07 12:23 ` John Garry
2025-05-07 21:30 ` Darrick J. Wong
2025-05-09 7:21 ` Carlos Maiolino
2025-05-09 8:08 ` John Garry
2025-05-07 21:32 ` Darrick J. Wong
2025-05-09 11:43 ` Carlos Maiolino
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).