[PATCH 0/2] btrfs-progs: check: Detect overlapping csum item

[PATCH 0/2] btrfs-progs: check: Detect overlapping csum item

[Qu Wenruo]

There is one report about tree-checker rejecting overlapping csum item.

I haven't yet seen another report, thus the problem doesn't look
widespread, thus maybe some regression in older kernels.

At least let btrfs check to detect such problem.
If we had another report, I'll spending extra time for the repair
functionality (it's not that simple, as it involves a lot of csum item
operation, and unexpected overlapping range).

Qu Wenruo (2):
  btrfs-progs: check: Detect overlap csum items
  btrfs-progs: fsck-tests: Add test image for overlapping csum item

 check/main.c                                  |   9 +++++++++
 .../overlap_csum_item.img.xz                  | Bin 0 -> 2172 bytes
 .../fsck-tests/045-overlap-csum-item/test.sh  |  19 ++++++++++++++++++
 3 files changed, 28 insertions(+)
 create mode 100644 tests/fsck-tests/045-overlap-csum-item/overlap_csum_item.img.xz
 create mode 100755 tests/fsck-tests/045-overlap-csum-item/test.sh

-- 
2.25.1

[PATCH 1/2] btrfs-progs: check: Detect overlap csum items

[Qu Wenruo]

There is a report about csum item overlap, which makes newer btrfs
kernel to reject it due to tree-checker.

Now let btrfs-progs have the same ability to detect such problem.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/check/main.c b/check/main.c
index d02dd1636852..c526c72d158e 100644
--- a/check/main.c
+++ b/check/main.c
@@ -5893,6 +5893,7 @@ static int check_csums(struct btrfs_root *root)
 	struct btrfs_path path;
 	struct extent_buffer *leaf;
 	struct btrfs_key key;
+	u64 last_data_end = 0;
 	u64 offset = 0, num_bytes = 0;
 	u16 csum_size = btrfs_super_csum_size(root->fs_info->super_copy);
 	int errors = 0;
@@ -5952,6 +5953,13 @@ static int check_csums(struct btrfs_root *root)
 			continue;
 		}
 
+		if (key.offset < last_data_end) {
+			error(
+	"csum overlap, current bytenr=%llu prev_end=%llu, eb=%llu slot=%u",
+				key.offset, last_data_end, leaf->start,
+				path.slots[0]);
+			errors++;
+		}
 		data_len = (btrfs_item_size_nr(leaf, path.slots[0]) /
 			      csum_size) * root->fs_info->sectorsize;
 		if (!verify_csum)
@@ -5982,6 +5990,7 @@ skip_csum_check:
 			num_bytes = 0;
 		}
 		num_bytes += data_len;
+		last_data_end = key.offset + data_len;
 		path.slots[0]++;
 	}
 
-- 
2.25.1

[PATCH 2/2] btrfs-progs: fsck-tests: Add test image for overlapping csum item

[Qu Wenruo]

The image has a csum item overlap with next csum item, make sure btrfs
check can detect it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../overlap_csum_item.img.xz                  | Bin 0 -> 2172 bytes
 .../fsck-tests/045-overlap-csum-item/test.sh  |  19 ++++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 tests/fsck-tests/045-overlap-csum-item/overlap_csum_item.img.xz
 create mode 100755 tests/fsck-tests/045-overlap-csum-item/test.sh

diff --git a/tests/fsck-tests/045-overlap-csum-item/overlap_csum_item.img.xz b/tests/fsck-tests/045-overlap-csum-item/overlap_csum_item.img.xz
new file mode 100644
index 0000000000000000000000000000000000000000..9bcf152c7c590d4dca915f5d1ecb15fb4a58af84
GIT binary patch
literal 2172
zcmV-?2!r?iH+ooF000E$*0e?f03iV!0000G&sfah5B~@|T>wRyj;C3^v%$$4d1t#p
zhA1?_T+OH))zRFdMm2c&pnO1wG%HdgqI}w%X?J7m77UG=uUbZE^{Xv=U?o*n&ukHJ
z0b3B>5Ui4qA{eq9)|q*K|9wB5H4fyt4jl()>6hOTdc`kC)*Xeb9d9xC+HniKLB{Xd
z*4D|3NF#40L&yo>s0zoXV-yPOo{HLNDT>F5D5~|0s|`Ib`+O*TVUe}OeW#L<f?r7n
z9%@#>inXNoXGAEX8w{s<=@appY}Gf4uNY2DvFr94hx8d>LE*E%W17TzSz~G$a!@+s
zf-k14+prl-s9Qf(WIo_$os3JbCdfQCV$+CV%yg!#4BDYxc!B%OLDTi6ZhxgiiauGR
zV``d$m7{3UR19>&BI*LNJ(!-|QGiy-9u$k}E1`?^ihp*D&cl<wYffJxFW&o8^Q+RE
zNXM2=O(&FS^q(zq>AMU9<gHo#IUN3M!}RBksrCGZoUD4amrgJUOfVwlQ7C!MCASbW
zzHFhAWBM|uE8~SVt8t!BjbYX84&%e{P^_3H4p{6$f0PQx-<{Jl0+X+cA-x$E9AQ>9
zezal;)TJ%+=@3bnbf?N*nD6gUEg3@G4!&*k9;2^^8YmM1Fsaw`7s&nMRzRtKo_={{
zx-AGCk1g1^%ITHu3`O_-;?2e3@#6=Nyuh1Mda0s)q2PkMQ>?wD^GMWAqEE<|d%M6{
zL<5f;x<Qr<=9{vc*dPJx2QlU3J?4GJgwo3iWfSs{uZ+m<1px7E0K!P@-W)3F#t>mm
z`ltDJr`Geh$yfwM*H(lU2{qOT3!_Es0fR60bWih0z#M!O)U$b3{rR1sa>Tx_E$^bU
zsL+sid4hYuWpz61828`93}OYDo!B_Pdg&Fhb*AYg;p&hEyl@*ZutP)=A;CIHU+QHG
zkBl(-s#9)sW-091w^IOmh=KgaOqx^&m0IK0sNj*3Qjl^AC^T#}O~7w7m7l^ej-D&f
zJT^+JYI7S}k-JkoXXV>AKnh``rg<K4X`M6-iy*=tpaz1W5&YKHs*X^=(1%bBI@K$G
zWk#<hIfe=5MegY+Y(>MHqW_H`jVV_<!CrsR7o0&yAX@Y8>l-LsXNNra3zNc>8wETo
z#dA{RVyngC<L_}|V9@h}YQdLK_V?EAd>+yVpYnl9CDJ|L-o<>w`qF<p5q|jI&%w-p
zI9jKkn9YSgZAwuo{u265B*I{=ueZH6Qk{mskzFlAy=)D(3+0P}1oiGVl8yZhD;a_z
zO+tobjxsvSABZl<dKnD4U+%qDx?=t$b!zdB?Dr+7|NF}Zg6Zv;WJu@9fRt*lKI9Na
zo%wPC{Q7E;?o0VgIt}1(d|R%ZWP195Rdc(yE-vP&6~K%x06DO)=r>C-N}~UJf;YA9
z6w&S=9}78~#FAjNnIW20kY8DE0pSS@$9w_LVgV&}Op1Byv>)fLd$VZkF&S|jxS=e5
za<IEi@v=ey^O<ELJASWv6o~{I==anE1-7_a{+L%5Cig9U;D4Wx40b&gekRljGD+W=
zMkP6Y<Z#Iv=A;I!w2U02^3mAhS**UB&sB54`aYe*XH8{pj4f@75m-FN7~i1xt6+l4
zA&LSJfIEI63vjj?@jiMRYwSmdO;M9^j_)t+24T2gFTb;!vR8Bc#>E+p4oj0P5X)_0
z57t)zhdl%A)lLv}rQg~H5WXTcv&`$x=(Zt%snCWGCDB(X_c1>0lHH+Mz=R2da3n^-
zX@Pvp%}=7lq14FHCiJcno)fm@lOsTo3Cqh4aj{;|ccl21Z--!BhIpZ`mAFlvOWxft
zSl<i;9bL6MqeehiFBmOz5L71YCd4bf_{xswRGVzJui}2Sm*D0aH8k>P+_P0-EF4vf
zA5<|anCLY-0i0-?LH|9YcQM;S$gf0*1YC}3>5xR(YOtR0)+CiOgm>_+$?sjL*}Dzo
zcAFzZxSr@yAx{uT7&X*kgjJnk=aQj>nDX19_OHpF;-nwdr6#kSe}BZQflT{<FlDd`
z;_wES+%h+k7kqyN1J2@S`&rmMKp=6oT8j--QZ(e`#QpqLe)T;^VFMi&ZWVrwlNi{6
zUS`Pllhkp8W;+|-8JEY4c4xYmtv-WUaOQ&SRZ`}=Yt6M%TY6}`dV+r(V9z#T9yu5T
zFw`}bdJ&b`wsHA#FNTkGmam6Jo+Q@Zj2J->X4bS1_-cFCYC4!Om><=|UIaS(If=w+
z1X!_%J5^j^$)jJ&(z^|IHbrP^>lx4oo%9uxxkc%m__wsDR#}APB@PKC>uvlM1{<V2
zc4}|uOnJaAMY|i8ObV;kwvE%K{Ls%rLdf?e^t8KK14~irH%<d5Dj2F|1VJ}yXFQ@+
zxz^{A40g4PpQ9re@cu4@NJVx76Vr!c(t{R0PmJvwKTxnNGe~e46A2tVcEq(OR&uBm
z)IlD@-R!qAYIjR{U<qCi^c#8rP-UcS^tp<0>HWpr1(lu!P{^2q4aoZd<Xy|<CGU$=
zwFbou9Ay1^8OnntXZ_vWCx&ivn7ZoGXx^S`b2RLauHSZ3Rc-KwJr;QD(~;+ZZgGAj
zlw}*^Fw07W&YmNnZ14%gVf{?XBf)8PX6elV7(l#8!<+Gqc+JVINdDW-Tfz>$;0J)U
zzgVo*)X}c_X$7#t^l5w<P6%S{Y9Py&7>=^9(5FAz<YQmF&!>Y$N|c_~JwjPeuioO%
zn~xHVMzcPVxdd>l<LeSlCK#ZcOuh45zuR03K0l%f_6Jkv`AA0vGi#>Pct>4heaj&P
y1gdz|yzZwQDw|p~00012=iJ<?O9`j|0oM?KAOHXppPG5G#Ao{g000001X)^7D=rHF

literal 0
HcmV?d00001

diff --git a/tests/fsck-tests/045-overlap-csum-item/test.sh b/tests/fsck-tests/045-overlap-csum-item/test.sh
new file mode 100755
index 000000000000..06649ed9331c
--- /dev/null
+++ b/tests/fsck-tests/045-overlap-csum-item/test.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Verify that check can detect overlapped csum items
+#
+# There is one report of overlap csum items, causing tree-checker to reject
+# the csum tree.
+#
+# Make sure btrfs check can at least detect such error
+
+source "$TEST_TOP/common"
+
+check_prereq btrfs
+
+check_image() {
+	run_mustfail "overlapping csum item not detected" \
+		"$TOP/btrfs" check "$1"
+}
+
+check_all_images
-- 
2.25.1

Re: [PATCH 0/2] btrfs-progs: check: Detect overlapping csum item

[David Sterba]

On Wed, Mar 04, 2020 at 03:26:59PM +0800, Qu Wenruo wrote:
> There is one report about tree-checker rejecting overlapping csum item.

Do you have link of the report?

I think the bug has been fixed by "btrfs: fix corrupt log due to
concurrent fsync of inodes with shared extents".

> I haven't yet seen another report, thus the problem doesn't look
> widespread, thus maybe some regression in older kernels.
> 
> At least let btrfs check to detect such problem.
> If we had another report, I'll spending extra time for the repair
> functionality (it's not that simple, as it involves a lot of csum item
> operation, and unexpected overlapping range).
> 
> Qu Wenruo (2):
>   btrfs-progs: check: Detect overlap csum items
>   btrfs-progs: fsck-tests: Add test image for overlapping csum item

Added to devel.