All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: Dave Chinner <dgc@kernel.org>
Cc: linux-xfs@vger.kernel.org, cem@kernel.org
Subject: Re: [PATCH 02/48] xfs: add log recovery item validation design document
Date: Wed, 14 Oct 2026 16:31:17 -0700	[thread overview]
Message-ID: <20261014233117.GL2705364@frogsfrogsfrogs> (raw)
In-Reply-To: <20261013060204.3138690-3-dgc@kernel.org>

On Tue, Oct 13, 2026 at 05:00:52PM +1100, Dave Chinner wrote:
> Add a design document describing a systematic approach to validating log
> items during journal recovery. Log item data read from the journal cannot
> be fully trusted - corruption from torn writes, hardware errors or
> software bugs can produce invalid type codes, wrong region counts,
> truncated regions or internally inconsistent format structures that the
> current recovery code largely uses unchecked.
> 
> The document describes a three-layer validation scheme (generic region
> header decode, per-region type-specific validation, and full cross-region
> item validation), the treatment of the transaction header as a validated
> region type, and a set of additional safety fixes for the generic region
> assembly code. It also lays out a phased implementation plan.
> 
> Changes:
> - Add Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
> - Add the new document to the XFS documentation index
> 
> Created-by-AI: Claude Opus 4.8 (1M context)
>
> Prompt: Let's start by converting the existing verification doc to kernel
> .rst format and moving it to Documentation/filesystems/xfs/ and adding it
> to the relevant doc index file. The file should be tracked in a new patch.

Eheheh.

> Refinements:
> - Reordered the Phase 1 implementation plan so the ops-struct validation
>   infrastructure (min_regions/max_regions/min_hdr_len fields and the
>   validate_region callback) is added first (Patch 1), followed by the
>   transaction header ops entry that uses it (Patch 2). Downstream patches
>   renumbered accordingly so each patch builds cleanly on its own.
> ---
>  Documentation/filesystems/xfs/index.rst       |   1 +
>  .../xfs-log-recovery-validation-design.rst    | 477 ++++++++++++++++++
>  2 files changed, 478 insertions(+)
>  create mode 100644 Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
> 
> diff --git a/Documentation/filesystems/xfs/index.rst b/Documentation/filesystems/xfs/index.rst
> index ab66c57a5d18..231c1382b27a 100644
> --- a/Documentation/filesystems/xfs/index.rst
> +++ b/Documentation/filesystems/xfs/index.rst
> @@ -9,6 +9,7 @@ XFS Filesystem Documentation
>     :numbered:
>  
>     xfs-delayed-logging-design
> +   xfs-log-recovery-validation-design
>     xfs-maintainer-entry-profile
>     xfs-self-describing-metadata
>     xfs-online-fsck-design
> diff --git a/Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst b/Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst
> new file mode 100644
> index 000000000000..188b1bebf171
> --- /dev/null
> +++ b/Documentation/filesystems/xfs/xfs-log-recovery-validation-design.rst

The document sounds pretty reasonable for a first pass at hardening the
log recovery code.  This is a good start.

<snip>

> +Layer 3: Full item validation (type-specific, cross-region)
> +-----------------------------------------------------------

<snip>

> +Other intent/done items
> +    similar field-level validation

But please be more specific.

> +ICREATE
> +    - Already has validation in
> +      ``xlog_recover_icreate_commit_pass2()``, move the checks to
> +      ``validate_item``
> +
> +QUOTAOFF
> +    - Verify ``qf_flags`` contains only valid quota flags
> +

<snip>

> +Phase 3: Full item validation
> +-----------------------------
> +
> +Patch M: Add validate_item callback infrastructure
> +    - Add the callback to the ops struct
> +    - Call it from ``xlog_recover_reorder_trans()`` after ops verification
> +    - Return ``-EFSCORRUPTED`` on failure, aborting recovery
> +
> +Patches M+1 to M+N: Implement validate_item for each item type
> +    - Move existing validation out of ``commit_pass2`` into
> +      ``validate_item`` where possible (e.g. ICREATE field checks)
> +    - Add new cross-region validation
> +    - Order: simple types first, complex types last
> +    - INODE validation is the most complex (fork format vs region size)
> +
> +Phase 4: Cleanup
> +----------------
> +
> +- Remove redundant validation from ``commit_pass1``/``commit_pass2`` that
> +  is now covered by ``validate_region``/``validate_item``
> +- Add ``ASSERT()``\ s in commit handlers to verify validation has run
> +- Review and update error messages for consistency

Why is the plan for splitting scope across M+N patches relevant for a
design document that will live in the kernel?

--D

  reply	other threads:[~2026-10-14 23:31 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-13  6:00 [RFC] [PATCH 00/48] Validate journal log items during log recovery Dave Chinner
2026-10-13  6:00 ` [PATCH 01/48] xfs: set the unmount record operation header length Dave Chinner
2026-10-13  6:00 ` [PATCH 02/48] xfs: add log recovery item validation design document Dave Chinner
2026-10-14 23:31   ` Darrick J. Wong [this message]
2026-10-13  6:00 ` [PATCH 03/48] xfs: lift transaction header parsing out of the region assembly Dave Chinner
2026-10-13  6:00 ` [PATCH 04/48] xfs: track transaction header decode state explicitly Dave Chinner
2026-10-13  6:00 ` [PATCH 05/48] xfs: track the log item being rebuilt with an explicit pointer Dave Chinner
2026-10-13  6:00 ` [PATCH 06/48] xfs: complete log items explicitly during recovery Dave Chinner
2026-10-13  6:00 ` [PATCH 07/48] xfs: add log item validation infrastructure to log recovery Dave Chinner
2026-10-13  6:00 ` [PATCH 08/48] xfs: recover the transaction header as a validated item type Dave Chinner
2026-10-13  6:00 ` [PATCH 09/48] xfs: factor out xlog_recover_validate_ophdr() Dave Chinner
2026-10-13  6:01 ` [PATCH 10/48] xfs: validate operation header length and alignment during recovery Dave Chinner
2026-10-13  6:01 ` [PATCH 11/48] xfs: validate the operation header padding " Dave Chinner
2026-10-13  6:01 ` [PATCH 12/48] xfs: validate the operation header flags " Dave Chinner
2026-10-13  6:01 ` [PATCH 13/48] xfs: enforce zero length control records " Dave Chinner
2026-10-13  6:01 ` [PATCH 14/48] xfs: validate the unmount record " Dave Chinner
2026-10-13  6:01 ` [PATCH 15/48] xfs: handle zero length continuation op headers " Dave Chinner
2026-10-13  6:01 ` [PATCH 16/48] xfs: bound the assembled region size " Dave Chinner
2026-10-13  6:01 ` [PATCH 17/48] xfs: dump the corrupt region when log item validation fails Dave Chinner
2026-10-13  6:01 ` [PATCH 18/48] xfs: validate fixed size log items during recovery Dave Chinner
2026-10-13  6:01 ` [PATCH 19/48] xfs: validate extent free " Dave Chinner
2026-10-13  6:01 ` [PATCH 20/48] xfs: validate and convert extent free intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 21/48] xfs: validate and convert extent free done items " Dave Chinner
2026-10-13  6:01 ` [PATCH 22/48] xfs: validate rmap update intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 23/48] xfs: validate rmap update done items " Dave Chinner
2026-10-13  6:01 ` [PATCH 24/48] xfs: validate refcount update intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 25/48] xfs: validate refcount update done items " Dave Chinner
2026-10-13  6:01 ` [PATCH 26/48] xfs: validate bmap update intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 27/48] xfs: validate bmap update done items " Dave Chinner
2026-10-13  6:01 ` [PATCH 28/48] xfs: validate mapping exchange intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 29/48] xfs: validate mapping exchange done items " Dave Chinner
2026-10-13  6:01 ` [PATCH 30/48] xfs: validate dquot " Dave Chinner
2026-10-13  6:01 ` [PATCH 31/48] xfs: validate attr done " Dave Chinner
2026-10-13  6:01 ` [PATCH 32/48] xfs: validate attr intents " Dave Chinner
2026-10-13  6:01 ` [PATCH 33/48] xfs: convert inode items to host format " Dave Chinner
2026-10-13  6:01 ` [PATCH 34/48] xfs: allow xfs_imap() to map without an inode btree lookup Dave Chinner
2026-10-13  6:01 ` [PATCH 35/48] xfs: validate the inode item format header during recovery Dave Chinner
2026-10-13  6:01 ` [PATCH 36/48] xfs: validate the logged inode core " Dave Chinner
2026-10-13  6:01 ` [PATCH 37/48] xfs: validate the logged inode forks " Dave Chinner
2026-10-13  6:01 ` [PATCH 38/48] xfs: validate logged local fork contents " Dave Chinner
2026-10-13  6:01 ` [PATCH 39/48] xfs: validate logged extent " Dave Chinner
2026-10-13  6:01 ` [PATCH 40/48] xfs: allow a long format btree root to be checked without a cursor Dave Chinner
2026-10-13  6:01 ` [PATCH 41/48] xfs: validate logged btree fork roots during recovery Dave Chinner
2026-10-13  6:01 ` [PATCH 42/48] xfs: validate logged metadata " Dave Chinner
2026-10-13  6:01 ` [PATCH 43/48] xfs: validate buffer item region count " Dave Chinner
2026-10-13  6:01 ` [PATCH 44/48] xfs: validate buffer item regions " Dave Chinner
2026-10-13  6:01 ` [PATCH 45/48] xfs: validate the whole buffer item " Dave Chinner
2026-10-13  6:01 ` [PATCH 46/48] xfs: remove redundant buffer checks from recovery replay Dave Chinner
2026-10-13  6:01 ` [PATCH 47/48] xfs: fail recovery on a buffer type mismatch Dave Chinner
2026-10-13  6:01 ` [PATCH 48/48] xfs: validate logged dquot buffers during recovery Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261014233117.GL2705364@frogsfrogsfrogs \
    --to=djwong@kernel.org \
    --cc=cem@kernel.org \
    --cc=dgc@kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.