linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3] btrfs-progs: follow-ups for issue #622
@ 2023-10-22  3:40 Qu Wenruo
  2023-10-22  3:40 ` [PATCH 1/3] btrfs-progs: dump-tree: output the sequence number for inline references Qu Wenruo
                   ` (4 more replies)
  0 siblings, 5 replies; 9+ messages in thread
From: Qu Wenruo @ 2023-10-22  3:40 UTC (permalink / raw)
  To: linux-btrfs

Issue #622 is a very interesting bug report, that ntfs2btrfs has a fixed
bug that it can generate out-of-order inline backref items.

This leads to kernel transaction abort, but btrfs-check failed to detect
it at all.

Although the fix for btrfs-progs is already merged in the latest v6.5.3
release, we still lacks the following thing:

- Better dump-tree support to show the weird inline backref order
  This is very weird, as we have the inline type in ascending order,
  but for the sequence number (hash for EXTENT_DATA_REF, offset for all
  other types) it is descending inside the same type.

  That's why the following output of one data extent item looks
  out-of-order:

	item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16143 itemsize 140
		refs 4 gen 7 flags DATA
		extent data backref root FS_TREE objectid 258 offset 0 count 1
		extent data backref root FS_TREE objectid 257 offset 0 count 1
		extent data backref root FS_TREE objectid 260 offset 0 count 1
		extent data backref root FS_TREE objectid 259 offset 0 count 1

- Lowmem mode support to detect the error

- Test case to make sure we can detect the error

This series would address all the three points above.

Qu Wenruo (3):
  btrfs-progs: dump-tree: output the sequence number for inline
    references
  btrfs-progs: check/lowmem: verify the sequence of inline backref items
  btrfs-progs: fsck-tests: add test image of out-of-order inline backref
    items

 check/mode-lowmem.c                           |  31 ++++++++++++++++++
 check/mode-lowmem.h                           |   1 +
 kernel-shared/print-tree.c                    |  31 +++++++++++-------
 .../btrfs_image.xz                            | Bin 0 -> 2264 bytes
 .../061-out-of-order-inline-backref/test.sh   |  19 +++++++++++
 5 files changed, 70 insertions(+), 12 deletions(-)
 create mode 100644 tests/fsck-tests/061-out-of-order-inline-backref/btrfs_image.xz
 create mode 100755 tests/fsck-tests/061-out-of-order-inline-backref/test.sh

--
2.42.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/3] btrfs-progs: dump-tree: output the sequence number for inline references
  2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
@ 2023-10-22  3:40 ` Qu Wenruo
  2023-10-22  3:40 ` [PATCH 2/3] btrfs-progs: check/lowmem: verify the sequence of inline backref items Qu Wenruo
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 9+ messages in thread
From: Qu Wenruo @ 2023-10-22  3:40 UTC (permalink / raw)
  To: linux-btrfs

Commit 6cf11f3e3815 ("btrfs-progs: check: check order of inline extent
refs") fixes a problem that btrfs check never properly verify the
sequence of inline references.

It's not obvious because by default kernel handles EXTENT_DATA_REF_KEY
using its own hash, resulting some seemingly out-of-order result:

	item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16143 itemsize 140
		refs 4 gen 7 flags DATA
		extent data backref root FS_TREE objectid 258 offset 0 count 1
		extent data backref root FS_TREE objectid 257 offset 0 count 1
		extent data backref root FS_TREE objectid 260 offset 0 count 1
		extent data backref root FS_TREE objectid 259 offset 0 count 1

By a quick glance, no one can see the above inline backref items are in
any order.

To make such sequence more obvious, let dump-tree to output a new prefix
to indicate the type and the internal sequence number:

For above case, the new output would look like this:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16143 itemsize 140
                refs 4 gen 7 flags DATA
                (178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
                (178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
                (178 0xdfb591f9c0534ff) extent data backref root FS_TREE objectid 260 offset 0 count 1
                (178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1

Although still not that obvious, it should show the inline data backrefs
has descending sequence number.

For the type part, it's anti-instinctive in ascending order, which is
not that easy to produce.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 kernel-shared/print-tree.c | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c
index f6e7539eb191..9e3ebddb97b3 100644
--- a/kernel-shared/print-tree.c
+++ b/kernel-shared/print-tree.c
@@ -487,38 +487,45 @@ void print_extent_item(struct extent_buffer *eb, int slot, int metadata)
 	ptr = (unsigned long)iref;
 	end = (unsigned long)ei + item_size;
 	while (ptr < end) {
+		u64 seq;
+
 		iref = (struct btrfs_extent_inline_ref *)ptr;
 		type = btrfs_extent_inline_ref_type(eb, iref);
 		offset = btrfs_extent_inline_ref_offset(eb, iref);
+		seq = offset;
 		switch (type) {
 		case BTRFS_TREE_BLOCK_REF_KEY:
-			printf("\t\ttree block backref root ");
+			printf("\t\t(%u 0x%llx) tree block backref root ",
+			       type, seq);
 			print_objectid(stdout, offset, 0);
 			printf("\n");
 			break;
 		case BTRFS_SHARED_BLOCK_REF_KEY:
-			printf("\t\tshared block backref parent %llu\n",
-			       (unsigned long long)offset);
+			printf("\t\t(%u 0x%llx) shared block backref parent %llu\n",
+			       type, seq, offset);
 			break;
 		case BTRFS_EXTENT_DATA_REF_KEY:
 			dref = (struct btrfs_extent_data_ref *)(&iref->offset);
-			printf("\t\textent data backref root ");
-			print_objectid(stdout,
-		(unsigned long long)btrfs_extent_data_ref_root(eb, dref), 0);
+			seq = hash_extent_data_ref(
+					btrfs_extent_data_ref_root(eb, dref),
+					btrfs_extent_data_ref_objectid(eb, dref),
+					btrfs_extent_data_ref_offset(eb, dref));
+			printf("\t\t(%u 0x%llx) extent data backref root ",
+			       type, seq);
+			print_objectid(stdout, btrfs_extent_data_ref_root(eb, dref), 0);
 			printf(" objectid %llu offset %llu count %u\n",
-			       (unsigned long long)btrfs_extent_data_ref_objectid(eb, dref),
+			       btrfs_extent_data_ref_objectid(eb, dref),
 			       btrfs_extent_data_ref_offset(eb, dref),
 			       btrfs_extent_data_ref_count(eb, dref));
 			break;
 		case BTRFS_SHARED_DATA_REF_KEY:
 			sref = (struct btrfs_shared_data_ref *)(iref + 1);
-			printf("\t\tshared data backref parent %llu count %u\n",
-			       (unsigned long long)offset,
-			       btrfs_shared_data_ref_count(eb, sref));
+			printf("\t\t(%u 0x%llx) shared data backref parent %llu count %u\n",
+			       type, seq, offset, btrfs_shared_data_ref_count(eb, sref));
 			break;
 		case BTRFS_EXTENT_OWNER_REF_KEY:
-			printf("\t\textent owner root %llu\n",
-			       (unsigned long long)offset);
+			printf("\t\(%u 0x%llx) textent owner root %llu\n",
+			       type, seq, offset);
 			break;
 		default:
 			return;
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 2/3] btrfs-progs: check/lowmem: verify the sequence of inline backref items
  2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
  2023-10-22  3:40 ` [PATCH 1/3] btrfs-progs: dump-tree: output the sequence number for inline references Qu Wenruo
@ 2023-10-22  3:40 ` Qu Wenruo
  2023-10-22  3:40 ` [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order " Qu Wenruo
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 9+ messages in thread
From: Qu Wenruo @ 2023-10-22  3:40 UTC (permalink / raw)
  To: linux-btrfs

Commit 6cf11f3e3815 ("btrfs-progs: check: check order of inline extent
refs") added the ability to detect out-of-order inline extent backref
items.

Meanwhile there is no such ability in lowmem mode, this patch would
introduce such ability to lowmem mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/mode-lowmem.c | 31 +++++++++++++++++++++++++++++++
 check/mode-lowmem.h |  1 +
 2 files changed, 32 insertions(+)

diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
index 3b2807cc5de9..19b7b1a72a1f 100644
--- a/check/mode-lowmem.c
+++ b/check/mode-lowmem.c
@@ -4234,6 +4234,7 @@ static int check_extent_item(struct btrfs_path *path)
 	unsigned long ptr;
 	int slot = path->slots[0];
 	int type;
+	int last_type = 0;
 	u32 nodesize = btrfs_super_nodesize(gfs_info->super_copy);
 	u32 item_size = btrfs_item_size(eb, slot);
 	u64 flags;
@@ -4245,6 +4246,8 @@ static int check_extent_item(struct btrfs_path *path)
 	u64 owner;
 	u64 owner_offset;
 	u64 super_gen;
+	u64 seq;
+	u64 last_seq = U64_MAX;
 	int metadata = 0;
 	/* To handle corrupted values in skinny backref */
 	u64 level;
@@ -4342,6 +4345,32 @@ next:
 	iref = (struct btrfs_extent_inline_ref *)ptr;
 	type = btrfs_extent_inline_ref_type(eb, iref);
 	offset = btrfs_extent_inline_ref_offset(eb, iref);
+	if (type == BTRFS_EXTENT_DATA_REF_KEY) {
+		dref = (struct btrfs_extent_data_ref *)(&iref->offset);
+		seq = hash_extent_data_ref(
+				btrfs_extent_data_ref_root(eb, dref),
+				btrfs_extent_data_ref_objectid(eb, dref),
+				btrfs_extent_data_ref_offset(eb, dref));
+	} else {
+		seq = offset;
+	}
+	/*
+	 * The @type should be ascending, while inside the same type, the
+	 * @seq should be descending.
+	 */
+	if (type < last_type)
+		tmp_err |= BACKREF_OUT_OF_ORDER;
+	else if (type > last_type)
+		last_seq = U64_MAX;
+
+	if (seq > last_seq)
+		tmp_err |= BACKREF_OUT_OF_ORDER;
+
+	if (tmp_err & BACKREF_OUT_OF_ORDER)
+		error(
+"inline extent backref (type %u seq 0x%llx) of extent [%llu %u %llu] is out of order",
+		      type, seq, key.objectid, key.type, key.offset);
+
 	switch (type) {
 	case BTRFS_TREE_BLOCK_REF_KEY:
 		root_objectid = offset;
@@ -4420,6 +4449,8 @@ next:
 
 	err |= tmp_err;
 	ptr_offset += btrfs_extent_inline_ref_size(type);
+	last_type = type;
+	last_seq = seq;
 	goto next;
 
 out:
diff --git a/check/mode-lowmem.h b/check/mode-lowmem.h
index c3cf0878147f..3f84280a89f7 100644
--- a/check/mode-lowmem.h
+++ b/check/mode-lowmem.h
@@ -64,6 +64,7 @@
 #define UNKNOWN_TYPE		(1 << 6) /* Unknown type */
 #define ACCOUNTING_MISMATCH	(1 << 7) /* Used space accounting error */
 #define CHUNK_TYPE_MISMATCH	(1 << 8)
+#define BACKREF_OUT_OF_ORDER	(1 << 9)
 
 int check_fs_roots_lowmem(void);
 int check_chunks_and_extents_lowmem(void);
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order inline backref items
  2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
  2023-10-22  3:40 ` [PATCH 1/3] btrfs-progs: dump-tree: output the sequence number for inline references Qu Wenruo
  2023-10-22  3:40 ` [PATCH 2/3] btrfs-progs: check/lowmem: verify the sequence of inline backref items Qu Wenruo
@ 2023-10-22  3:40 ` Qu Wenruo
  2023-10-24  5:28   ` Anand Jain
  2023-10-23 13:42 ` [PATCH 0/3] btrfs-progs: follow-ups for issue #622 David Sterba
  2023-10-24  6:18 ` Anand Jain
  4 siblings, 1 reply; 9+ messages in thread
From: Qu Wenruo @ 2023-10-22  3:40 UTC (permalink / raw)
  To: linux-btrfs

Issue #622 reported a case where ntfs2btrfs can generate out-of-order
inline backref items, which can lead to kernel transaction abort, but
not detected by btrfs-check.

This patch would add such image, whose extent tree looks like this for
the only data extent:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
                refs 3 gen 7 flags DATA
                (178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
                (178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
                (178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1

While the original good base image has the following backrefs for the
same data extent:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
                refs 3 gen 7 flags DATA
                (178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
                (178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
                (178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1

Notice the sequence (the 2nd number in the round brackets) should be
descending.
(Meanwhie type should be ascending, but it's way harder to create)

For now we don't have a way to fix it, but at least we should detect it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../btrfs_image.xz                            | Bin 0 -> 2264 bytes
 .../061-out-of-order-inline-backref/test.sh   |  19 ++++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 tests/fsck-tests/061-out-of-order-inline-backref/btrfs_image.xz
 create mode 100755 tests/fsck-tests/061-out-of-order-inline-backref/test.sh

diff --git a/tests/fsck-tests/061-out-of-order-inline-backref/btrfs_image.xz b/tests/fsck-tests/061-out-of-order-inline-backref/btrfs_image.xz
new file mode 100644
index 0000000000000000000000000000000000000000..65c10617bc250edce2affb3ed91c359041119345
GIT binary patch
literal 2264
zcmV;}2q*XbH+ooF000E$*0e?f03iVu0001VFXf})5B~_3T>wRyj;C3^v%$$4d1t#p
zhA1?^)9Q?YaslDA79H>>{UD9*Cyo|f=_79MP(Vc_Bso$>WJj6=v5pYC+s}`CWVw3x
zxE<<Z&OfiTEbzVoLX_Wp((^yVG#))u1m&;g4%?g8Bn81%z}1ml)d&0k=A5NqHJBq6
z%wg$T&Bu*4IbKTFYM}gXa*;^`8gE!2|Ff+@B|X)Ff45}-9M1wj=qp0_7cZe?d5H7c
z7sLy=`?K-Y+Hzuv@E{5s0j6IvDRNCAC_bm{NvzJ_@>0_zj#Q!&CYoQV;dIYOjEcZ#
zFHOx5A(x=IhO_&sfmZq(f_atSx^%aJAt$@&5`Q5+Kae^bmtD@uv*%7F^u6@m0iW5)
z>MZE%G3!z@XfNogc=;Ht8<;VKoc?(#z;0!xcgd|3DPL55Eyk);8!*_+Ya_Awo7i!2
zik-{{ab%|@1q-IJp4hR%T+sU=r5($Cp<3tJi08Gaoj2mY&HGB78HmRF%6UL;HL_g@
z4()rR=Vqkx)Oi3aLj44~aj%5k>QlRBVdyg*50%W{=uF<{KO+Xvj3qoNTUO|=N%sMd
z`<lrRv*1v;3gVH}O_x-6-7x@|i(j&_0bFk3<)EcQ9!`<SN1WsMd5)&-l*)p(y1a{X
zQlmBE_nl-Oc*1-`_cI)NcoUeUdX(3QqrQS?F*jcgC&eg*1wzTL{xHU}GA>C8L;II^
zSVrC5wJI(ZHwN3YH;2`{Dddew9<qcl<j54OJFO<x{&QDN?``umRhUQvD9r1tv2fO=
zq!ixNF>?3uNR0d$AratrD5_IO;3Hq}s%gXNFZt-3dC8{+Aky@2!g0BSFb*7=y}m7p
zH`7b>Ueq|aq4`PFz{h_*b@iK-5CK2HE*S(_Ka+($$8cKrmrET*(|8AIIaMi-69~;B
z=yf>&UngW3><zY`WRY*fK#pCA(6u)v$7ScZzieJ)p0_*&p@ZS4nsUr}pP(OrRjLr`
zy;y1%5>~aHla$)+dKC(_s3sQ)JrPR7&%IFB;R~4h_A07)t}l~=-I#vM$sM)-2`wZo
ztND`F#ti2&X~&IRLjT~&m3g?juh25u53e&gbh<ue^GpGJ(SJAUMhxsVklosGc}&H-
zmX`)<x%`eeI5WaBXt90spbR0>3($6v`!+y@{{F37TLBpTA#0v}c*K0}69|zQyt(1h
zg0w^juoIC2%~JE$1fs`-jTVk#v2c<dOXr4$ZbV2Pf_@`tGO0feyUfV;+Y<8K$soLc
zGi?)ey+22aFs|dHeb1OrU6bD7Jr^bqwN$9JF%X8FpyvrE-!o7wS&v8X8N~c+B~}A#
z({WT@a+ORd4N3a}tqV`3p28NP^Cm!tM7irdJgko@F+*W4%=~i;5tq^!-W@I9Lhe6D
zKc{5c9NI@8IG;5Gq7V-d8tnV;n4;_t7of*DOn<*q!(*ytH5}Ef7s{MuVey;6bQcIs
zTj!Ry8o^AvwoCPv=?1+F<gcN!&XDzC;>YR{*?^i_%XnCFZ9F)3W(KCw&<A#MYyz8e
zQf*xzCy6@VNEsZ?7BIT_jcng1YC1kL2>QAoa1jN2Q6ITIiJmC}JvhlI{lq-zLi)Gz
z61}qrCp9`cR4>=o6TH9wSO#w(kkp&{^O^nL2xL@VC6rhM&7`m_odZzDW4v(rhY>s7
z@e@({H=Ps13RM3;CH>~bKEN*>khvxwTRKz7d@t?PwuMq5^G~66!4EMEu=}R0?I)2O
z^vdR#M*5`Hu<38Dc#~V{3Q=0dt`rsBoqXMsM3|5Fu%5kN7&Ym!8iMWX2}S9x=;~^r
zNGaSsc<0mNcn5iA+w}^L=9RG!x`+S3qVkM~UBft@_=%{8Fbl9J#s#@%ryFK5?olIU
zG9RUg8D;O*0bT=a5xZDxjBF&xS~o2L$%*>qB}S}vs5jL$x!<`&C<~`rI?PdEf>F6g
z5xBOR52i*tg@QiUSlv8$xmj>e*m5h;%DF&lQbS~Yb7GfE;t5$TyBGB?2qlMm34&wi
zR!+G7&HN)T3iLZ)JFLYRyom6(+9lf%S!-gK`b&dIQ=86+^q=hR1pK63`0}1AIou(C
z`rKd%Ni&gRf}?4<kEoyq-&w&#uvkFA3)pZsUT*M4+0976^}^idE`M*60=ztyJfGxp
zi!KNajhR=VdsCBJ(yV9Ax5>l3KN4V>;<X#;8q=2S;}f8P_*2ko@DS%FPWG@sM%1Y>
zjSuCZQ6D*dxemFsucgi?s*wy)8bDxm*6M<LI9&ar>T*UJFsTsdE3NqtqQIImzP|*%
zteC9i`^XvP4esR#q53gW;ayfa6y<}enU@x&GUh8?=X}b^HMoe?qIJS00M@L}uJVEY
zbz)EAuqdyrr@)iuk{AAv8V_EL&sI^(`|CG?BNJ1=MG%H$i6By7UD5{P1d-zNE{QD}
z5q^aqWy7#PzZrqKE72thI~Owi#p5D?@@aXX3Ox?2SxnK(m}j*9vH=QEyb}$1H*?m6
zNOsp5*O_yteiZ7BIiMCn!5JJ6iL+TXB2v6!j>+$l@TzZBjHHh2#FklJCBOt45M7*=
z?Jf>Ub|O(c|BOkXh1nl6wvh`xkKm!?61jYG-oSS`^vH`zVJ2P6`i>MvfT9UYI@)gu
ztKT};o&$LJJd>HY8fJ>gPjh6Qinb(svJFeDagU1vGdafKT3QO`*8SoJba!C3rimp6
zX?o402q{VtY}oJTFvpxKT6&25@H6@j=QNW6l@9@J9WTqq1Ixzqoa}*=f!dO&p|gNF
z@%2>k+xs=J#i2iaEap!Q^Lk?y^4}Fn^E=aj9X}hp=v&Kt<jU>}JqflP9ExJn8Io)|
zE?#x?PWc6%<}NRL;5Qu=us?;N3NDAPaCsRr8+nVNoY@;2zl3N_MBn}4jg7I7Ind+)
m0002EWz9KhnPQ&+0kRQ*AOHYLD8n4F#Ao{g000001X)@%f>$&E

literal 0
HcmV?d00001

diff --git a/tests/fsck-tests/061-out-of-order-inline-backref/test.sh b/tests/fsck-tests/061-out-of-order-inline-backref/test.sh
new file mode 100755
index 000000000000..f80d58f416f0
--- /dev/null
+++ b/tests/fsck-tests/061-out-of-order-inline-backref/test.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Verify that check can detect out-of-order inline backref items.
+#
+# There is a report that one ntfs2btrfs generated some out-of-order inline
+# backref items, while btrfs-check failed to detect it at all.
+#
+# Make sure btrfs check can at least detect such error.
+
+source "$TEST_TOP/common"
+
+check_prereq btrfs
+
+check_image() {
+	run_mustfail "out-of-order inline backref items not detected" \
+		"$TOP/btrfs" check "$1"
+}
+
+check_all_images
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH 0/3] btrfs-progs: follow-ups for issue #622
  2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
                   ` (2 preceding siblings ...)
  2023-10-22  3:40 ` [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order " Qu Wenruo
@ 2023-10-23 13:42 ` David Sterba
  2023-10-24  6:18 ` Anand Jain
  4 siblings, 0 replies; 9+ messages in thread
From: David Sterba @ 2023-10-23 13:42 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs

On Sun, Oct 22, 2023 at 02:10:06PM +1030, Qu Wenruo wrote:
> Issue #622 is a very interesting bug report, that ntfs2btrfs has a fixed
> bug that it can generate out-of-order inline backref items.
> 
> This leads to kernel transaction abort, but btrfs-check failed to detect
> it at all.
> 
> Although the fix for btrfs-progs is already merged in the latest v6.5.3
> release, we still lacks the following thing:
> 
> - Better dump-tree support to show the weird inline backref order
>   This is very weird, as we have the inline type in ascending order,
>   but for the sequence number (hash for EXTENT_DATA_REF, offset for all
>   other types) it is descending inside the same type.
> 
>   That's why the following output of one data extent item looks
>   out-of-order:
> 
> 	item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16143 itemsize 140
> 		refs 4 gen 7 flags DATA
> 		extent data backref root FS_TREE objectid 258 offset 0 count 1
> 		extent data backref root FS_TREE objectid 257 offset 0 count 1
> 		extent data backref root FS_TREE objectid 260 offset 0 count 1
> 		extent data backref root FS_TREE objectid 259 offset 0 count 1
> 
> - Lowmem mode support to detect the error
> 
> - Test case to make sure we can detect the error
> 
> This series would address all the three points above.
> 
> Qu Wenruo (3):
>   btrfs-progs: dump-tree: output the sequence number for inline
>     references
>   btrfs-progs: check/lowmem: verify the sequence of inline backref items
>   btrfs-progs: fsck-tests: add test image of out-of-order inline backref
>     items

Thanks, added to misc-next. We can add ntfs2btrfs conversion tests to
btrfs-progs, either with the external utility or on-demand build from git.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order inline backref items
  2023-10-22  3:40 ` [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order " Qu Wenruo
@ 2023-10-24  5:28   ` Anand Jain
  2023-10-24  5:47     ` Qu Wenruo
  0 siblings, 1 reply; 9+ messages in thread
From: Anand Jain @ 2023-10-24  5:28 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs




>   .../btrfs_image.xz                            | Bin 0 -> 2264 bytes

check_all_images() won't find  btrfs_image.xz image.


         for image in $(find "$dir" \( -iname '*.img' -o \
                                 -iname '*.img.xz' -o    \
                                 -iname '*.raw' -o       \
                                 -iname '*.raw.xz' \) | sort)





What's your plan to test lomem mode?



Thanks, Anand


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order inline backref items
  2023-10-24  5:28   ` Anand Jain
@ 2023-10-24  5:47     ` Qu Wenruo
  2023-10-24 11:15       ` David Sterba
  0 siblings, 1 reply; 9+ messages in thread
From: Qu Wenruo @ 2023-10-24  5:47 UTC (permalink / raw)
  To: Anand Jain, Qu Wenruo, linux-btrfs, David Sterba



On 2023/10/24 15:58, Anand Jain wrote:
> 
> 
> 
>>   .../btrfs_image.xz                            | Bin 0 -> 2264 bytes
> 
> check_all_images() won't find  btrfs_image.xz image.

My bad, the filename should be default.img.xz.

David, mind to change that?
> 
> 
>          for image in $(find "$dir" \( -iname '*.img' -o \
>                                  -iname '*.img.xz' -o    \
>                                  -iname '*.raw' -o       \
>                                  -iname '*.raw.xz' \) | sort)
> 
> 
> 
> 
> 
> What's your plan to test lomem mode?

The usual way, you can check tests/common.local to see how lowmem mode 
should be utilized.

Thanks,
Qu
> 
> 
> 
> Thanks, Anand
> 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 0/3] btrfs-progs: follow-ups for issue #622
  2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
                   ` (3 preceding siblings ...)
  2023-10-23 13:42 ` [PATCH 0/3] btrfs-progs: follow-ups for issue #622 David Sterba
@ 2023-10-24  6:18 ` Anand Jain
  4 siblings, 0 replies; 9+ messages in thread
From: Anand Jain @ 2023-10-24  6:18 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs


With the image file prefix fixed.

You can add

Reviewed-by: Anand Jain <anand.jain@oracle.com>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order inline backref items
  2023-10-24  5:47     ` Qu Wenruo
@ 2023-10-24 11:15       ` David Sterba
  0 siblings, 0 replies; 9+ messages in thread
From: David Sterba @ 2023-10-24 11:15 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Anand Jain, Qu Wenruo, linux-btrfs, David Sterba

On Tue, Oct 24, 2023 at 04:17:14PM +1030, Qu Wenruo wrote:
> 
> 
> On 2023/10/24 15:58, Anand Jain wrote:
> > 
> > 
> > 
> >>   .../btrfs_image.xz                            | Bin 0 -> 2264 bytes
> > 
> > check_all_images() won't find  btrfs_image.xz image.
> 
> My bad, the filename should be default.img.xz.
> 
> David, mind to change that?

Renamed and pushed.

> > 
> > 
> >          for image in $(find "$dir" \( -iname '*.img' -o \
> >                                  -iname '*.img.xz' -o    \
> >                                  -iname '*.raw' -o       \
> >                                  -iname '*.raw.xz' \) | sort)
> > 
> > 
> > 
> > 
> > 
> > What's your plan to test lomem mode?
> 
> The usual way, you can check tests/common.local to see how lowmem mode 
> should be utilized.

There are more ways how to run the lowmem mode tests, e.g.
'make test-check-lowmem' that sets up the variables s needed.

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2023-10-24 11:22 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-22  3:40 [PATCH 0/3] btrfs-progs: follow-ups for issue #622 Qu Wenruo
2023-10-22  3:40 ` [PATCH 1/3] btrfs-progs: dump-tree: output the sequence number for inline references Qu Wenruo
2023-10-22  3:40 ` [PATCH 2/3] btrfs-progs: check/lowmem: verify the sequence of inline backref items Qu Wenruo
2023-10-22  3:40 ` [PATCH 3/3] btrfs-progs: fsck-tests: add test image of out-of-order " Qu Wenruo
2023-10-24  5:28   ` Anand Jain
2023-10-24  5:47     ` Qu Wenruo
2023-10-24 11:15       ` David Sterba
2023-10-23 13:42 ` [PATCH 0/3] btrfs-progs: follow-ups for issue #622 David Sterba
2023-10-24  6:18 ` Anand Jain

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).