[PATCH v9 00/24] xfs: Delay Ready Attributes

[Allison Collins <allison.henderson@oracle.com>]

Hi all,

This set is a subset of a larger series for delayed attributes. Which is a
subset of an even larger series, parent pointers. Delayed attributes allow
attribute operations (set and remove) to be logged and committed in the same
way that other delayed operations do. This allows more complex operations (like
parent pointers) to be broken up into multiple smaller transactions. To do
this, the existing attr operations must be modified to operate as either a
delayed operation or a inline operation since older filesystems will not be
able to use the new log entries.  This means that they cannot roll, commit, or
finish transactions.  Instead, they return -EAGAIN to allow the calling
function to handle the transaction. In this series, we focus on only the clean
up and refactoring needed to accomplish this. We will introduce delayed attrs
and parent pointers in a later set.

At the moment, I would like people to focus their review efforts on just this
"delay ready" subseries, as I think that is a more conservative use of peoples
review time.  I also think the set is a bit much to manage all at once, and we
need to get the infrastructure ironed out before we focus too much anything
that depends on it. But I do have the extended series for folks that want to
see the bigger picture of where this is going.

To help organize the set, I've arranged the patches to make sort of mini sets.
I thought it would help reviewers break down the reviewing some. For reviewing
purposes, the set could be broken up into 4 different phases:

Error code filtering (patches 1-2):
These two patches are all about finding and catching error codes that need to
be sent back up to user space before starting delayed operations.  Errors that
happen during a delayed operation are treated like internal errors that cause a
shutdown.  But we wouldnt want that for example: when the user tries to rename
a non existent attr.  So the idea is that we need to find all such conditions,
and take care of them before starting a delayed operation.
   xfs: Add xfs_has_attr and subroutines
   xfs: Check for -ENOATTR or -EEXIST

Move transactions upwards (patches 3-10): 
The goal of this subset is to try and move all the transaction specific code up
the call stack much as possible.  The idea being that once we get them to the
top, we can introduce the statemachine to handle the -EAGAIN logic where ever
the transactions used to be.
   xfs: Factor out new helper functions xfs_attr_rmtval_set
   xfs: Pull up trans handling in xfs_attr3_leaf_flipflags
   xfs: Split apart xfs_attr_leaf_addname
   xfs: Refactor xfs_attr_try_sf_addname
   xfs: Pull up trans roll from xfs_attr3_leaf_setflag
   xfs: Factor out xfs_attr_rmtval_invalidate
   xfs: Pull up trans roll in xfs_attr3_leaf_clearflag
   xfs: Add helper function __xfs_attr_rmtval_remove

Modularizing and cleanups (patches 11-18):
Now that we have pulled the transactions up to where we need them, it's time to
start breaking down the top level functions into new subfunctions. The goal
being to work towards a top level function that deals mostly with the
statemachine, and helpers for those states
   xfs: Add helper function xfs_attr_node_shrink
   xfs: Remove unneeded xfs_trans_roll_inode calls
   xfs: Add helpers xfs_attr_is_shortform and xfs_attr_set_shortform
   xfs: Add helper function xfs_attr_leaf_mark_incomplete
   xfs: Add remote block helper functions
   xfs: Add helper function xfs_attr_node_removename_setup
   xfs: Add helper function xfs_attr_node_removename_rmt
   xfs: Simplify xfs_attr_leaf_addname
   xfs: Simplify xfs_attr_node_addname
   xfs: Lift -ENOSPC handler from xfs_attr_leaf_addname

Introduce statemachine (patches 18-20):
Now that we have re-arranged the code such that we can remove the transaction
handling, we proceed to do so.  The behavior of the attr set/remove routines
are now also compatible as a .finish_item callback
   xfs: Add delay ready attr remove routines
   xfs: Add delay ready attr set routines
   xfs: Rename __xfs_attr_rmtval_remove


Changes since v8:
Theres a lot of activity that goes around with each set, so to help people
recall the discussion I've outlined the changes for each patch, which are new,
and which are unchanged:

xfs: Add xfs_has_attr and subroutines
  Fixed ENOATTR check
  Inverted error check handler in xfs_attr_node_hasname
  Indentation fixes

xfs: Check for -ENOATTR or -EEXIST
  Removed whitespace

xfs: Factor out new helper functions xfs_attr_rmtval_set
xfs: Pull up trans handling in xfs_attr3_leaf_flipflags
xfs: Split apart xfs_attr_leaf_addname
xfs: Refactor xfs_attr_try_sf_addname
xfs: Pull up trans roll from xfs_attr3_leaf_setflag
xfs: Factor out xfs_attr_rmtval_invalidate
xfs: Pull up trans roll in xfs_attr3_leaf_clearflag
   No change 

xfs: Add helper function __xfs_attr_rmtval_remove
   __xfs_attr_rmtval_remove becomes a helper function of xfs_attr_rmtval_remove
   instead of a new function
   Retain xfs_defer_finish in the helper so that we can replace it with the
   DEFER_FINISH flag later

xfs: Add helper function xfs_attr_leaf_mark_incomplete
   NEW

xfs: Add helper function xfs_attr_node_shrink
   Removed trailing error = 0

xfs: Remove unneeded xfs_trans_roll_inode calls
   More unneeded xfs_defer_finish calls removed
   Commit message verbage fix

xfs: Add helpers xfs_attr_is_shortform and xfs_attr_set_shortform
   Indentation adjustment
   Fixed comment verbage

xfs: Add helper function xfs_attr_leaf_mark_incomplete
   Simplified error handler

xfs: Add remote block helper functions
   No change

xfs: Add helper function xfs_attr_node_removename_setup
   Move the invalidate addd from new patch 12 into the helper function in this patch

xfs: Add helper function xfs_attr_node_removename_rmt
   Widen comment
   Simplify error handler

xfs: Simplify xfs_attr_leaf_addname
   NEW

xfs: Simplify xfs_attr_node_addname
   NEW

xfs: Lift -ENOSPC handler from xfs_attr_leaf_addname
   NEW

xfs: Add delay ready attr remove routines
  Replaced XFS_DAS_RMTVAL_REMOVE state with XFS_DAC_NODE_RMVNAME_INIT flag
  Extra XFS_DAC_DEFER_FINISH set added to __xfs_attr_rmtval_remove
  Temporary XFS_DAC_DEFER_FINISH hander plumbed into xfs_attr_rmtval_remove until we delete it in the next patch
  Simplify xfs_attr_remove_iter with less states to handle
  New helper function xfs_attr_defer_finish
  Indentation fixes
  Comments added to xfs_delattr_context
  Commit message update
  Flow chart update
  Added flow chart to xfs_attr.h

xfs: Add delay ready attr set routines
  Add fork code moved further up the set into parent pointers
  Replace XFS_DAS_ADD_LEAF state with shortform check
  Replace XFS_DAS_LEAF_TO_NODE state with leaf form check
  Replace XFS_DAS_ALLOC_LEAF with XFS_DAC_LEAF_ADDNAME_INIT flag
  Comments added to xfs_delattr_context, some unused feilds removed
  Rebase adjustments
  Flow chart update

xfs: Rename __xfs_attr_rmtval_remove
   Rebase adjustments


This series can be viewed on github here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v9

As well as the extended delayed attribute and parent pointer series:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v9_extended

And the test cases:
https://github.com/allisonhenderson/xfs_work/tree/pptr_xfstests

In order to run the test cases, you will need have the corresponding xfsprogs
changes as well.  Which can be found here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v9
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v9_extended

To run the xfs attributes tests run:
check -g attr

To run as delayed attributes run:
export MKFS_OPTIONS="-n delattr"
check -g attr

To run parent pointer tests:
check -g parent


I've also made the corresponding updates to the user space side as well, and ported anything
they need to seat correctly.

Questions, comment and feedback appreciated! 

Thanks all!
Allison


Allison Collins (24):
  xfs: Add xfs_has_attr and subroutines
  xfs: Check for -ENOATTR or -EEXIST
  xfs: Factor out new helper functions xfs_attr_rmtval_set
  xfs: Pull up trans handling in xfs_attr3_leaf_flipflags
  xfs: Split apart xfs_attr_leaf_addname
  xfs: Refactor xfs_attr_try_sf_addname
  xfs: Pull up trans roll from xfs_attr3_leaf_setflag
  xfs: Factor out xfs_attr_rmtval_invalidate
  xfs: Pull up trans roll in xfs_attr3_leaf_clearflag
  xfs: Add helper function __xfs_attr_rmtval_remove
  xfs: Pull up xfs_attr_rmtval_invalidate
  xfs: Add helper function xfs_attr_node_shrink
  xfs: Remove unneeded xfs_trans_roll_inode calls
  xfs: Add helpers xfs_attr_is_shortform and xfs_attr_set_shortform
  xfs: Add helper function xfs_attr_leaf_mark_incomplete
  xfs: Add remote block helper functions
  xfs: Add helper function xfs_attr_node_removename_setup
  xfs: Add helper function xfs_attr_node_removename_rmt
  xfs: Simplify xfs_attr_leaf_addname
  xfs: Simplify xfs_attr_node_addname
  xfs: Lift -ENOSPC handler from xfs_attr_leaf_addname
  xfs: Add delay ready attr remove routines
  xfs: Add delay ready attr set routines
  xfs: Rename __xfs_attr_rmtval_remove

 fs/xfs/libxfs/xfs_attr.c        | 1229 ++++++++++++++++++++++++++-------------
 fs/xfs/libxfs/xfs_attr.h        |  199 +++++++
 fs/xfs/libxfs/xfs_attr_leaf.c   |  116 ++--
 fs/xfs/libxfs/xfs_attr_leaf.h   |    3 +
 fs/xfs/libxfs/xfs_attr_remote.c |  259 ++++++---
 fs/xfs/libxfs/xfs_attr_remote.h |    8 +-
 fs/xfs/xfs_attr_inactive.c      |    2 +-
 fs/xfs/xfs_trace.h              |    1 -
 8 files changed, 1281 insertions(+), 536 deletions(-)

-- 
2.7.4