[PATCH RFC 0/2] dm persistent-data: detect btree cycles and record space-map errors

[Ye Bin <yebin@huaweicloud.com>]

From: Ye Bin <yebin10@huawei.com>

This series hardens dm-persistent-data against metadata corruption that
can leave the kernel hung in an uninterruptible state.  The two patches
attack the same class of problem from both sides: symptom detection and
root-cause prevention.

Patch 1 adds a depth limit (DM_BTREE_MAX_DEPTH = 16) to every unbounded
btree traversal loop (lookup, insert, overwrite, remove, find_key).
When metadata is corrupted such that a node's value points back to an
ancestor or to itself, these loops currently never terminate, hanging
the kernel.  With the limit in place the traversal returns -ELOOP
instead, and dm-thin handles this error by aborting the transaction and
downgrading the pool to read-only mode so no further writes hit the
corrupted metadata.

Patch 2 addresses a root cause of such corruption.  The transaction
manager's dm_tm_inc/dm_tm_dec/dm_tm_inc_range/dm_tm_dec_range wrappers
have void return types, so when the underlying space-map operations
fail the error is silently swallowed and the transaction is committed
with inconsistent reference counts.  This can produce permanent space
leaks, premature refcount drops, and -- critically -- self-referencing
btree nodes that trigger the very infinite loops that patch 1 catches.
The fix records the first space-map failure in a sticky error flag
(sm_error in the transaction manager, data_sm_error in dm_pool_metadata)
and refuses to commit while the flag is set, all without changing any
function signatures.

Ye Bin (2):
  dm persistent-data: add btree traversal depth limit to detect metadata
    corruption
  dm persistent-data: record space-map errors in transaction manager

 drivers/md/dm-thin-metadata.c                 | 65 +++++++++++++---
 drivers/md/dm-thin.c                          |  9 +++
 drivers/md/persistent-data/dm-btree-remove.c  | 14 ++++
 drivers/md/persistent-data/dm-btree.c         | 28 +++++++
 drivers/md/persistent-data/dm-btree.h         |  8 ++
 .../persistent-data/dm-transaction-manager.c  | 74 ++++++++++++++++++-
 .../persistent-data/dm-transaction-manager.h  |  6 ++
 7 files changed, 191 insertions(+), 13 deletions(-)

-- 
2.34.1