public inbox for stable@vger.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: stable@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	patches@lists.linux.dev, Wu Bo <bo.wu@vivo.com>,
	Mikulas Patocka <mpatocka@redhat.com>,
	Mike Snitzer <snitzer@kernel.org>
Subject: [PATCH 4.14 23/30] dm verity: dont perform FEC for failed readahead IO
Date: Tue,  5 Dec 2023 12:16:30 +0900	[thread overview]
Message-ID: <20231205031512.867706368@linuxfoundation.org> (raw)
In-Reply-To: <20231205031511.476698159@linuxfoundation.org>

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Wu Bo <bo.wu@vivo.com>

commit 0193e3966ceeeef69e235975918b287ab093082b upstream.

We found an issue under Android OTA scenario that many BIOs have to do
FEC where the data under dm-verity is 100% complete and no corruption.

Android OTA has many dm-block layers, from upper to lower:
dm-verity
dm-snapshot
dm-origin & dm-cow
dm-linear
ufs

DM tables have to change 2 times during Android OTA merging process.
When doing table change, the dm-snapshot will be suspended for a while.
During this interval, many readahead IOs are submitted to dm_verity
from filesystem. Then the kverity works are busy doing FEC process
which cost too much time to finish dm-verity IO. This causes needless
delay which feels like system is hung.

After adding debugging it was found that each readahead IO needed
around 10s to finish when this situation occurred. This is due to IO
amplification:

dm-snapshot suspend
erofs_readahead     // 300+ io is submitted
	dm_submit_bio (dm_verity)
		dm_submit_bio (dm_snapshot)
		bio return EIO
		bio got nothing, it's empty
	verity_end_io
	verity_verify_io
	forloop range(0, io->n_blocks)    // each io->nblocks ~= 20
		verity_fec_decode
		fec_decode_rsb
		fec_read_bufs
		forloop range(0, v->fec->rsn) // v->fec->rsn = 253
			new_read
			submit_bio (dm_snapshot)
		end loop
	end loop
dm-snapshot resume

Readahead BIOs get nothing while dm-snapshot is suspended, so all of
them will cause verity's FEC.
Each readahead BIO needs to verify ~20 (io->nblocks) blocks.
Each block needs to do FEC, and every block needs to do 253
(v->fec->rsn) reads.
So during the suspend interval(~200ms), 300 readahead BIOs trigger
~1518000 (300*20*253) IOs to dm-snapshot.

As readahead IO is not required by userspace, and to fix this issue,
it is best to pass readahead errors to upper layer to handle it.

Cc: stable@vger.kernel.org
Fixes: a739ff3f543a ("dm verity: add support for forward error correction")
Signed-off-by: Wu Bo <bo.wu@vivo.com>
Reviewed-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/md/dm-verity-target.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -588,7 +588,9 @@ static void verity_end_io(struct bio *bi
 	struct dm_verity_io *io = bio->bi_private;
 
 	if (bio->bi_status &&
-	    (!verity_fec_is_enabled(io->v) || verity_is_system_shutting_down())) {
+	    (!verity_fec_is_enabled(io->v) ||
+	     verity_is_system_shutting_down() ||
+	     (bio->bi_opf & REQ_RAHEAD))) {
 		verity_finish_io(io, bio->bi_status);
 		return;
 	}



  parent reply	other threads:[~2023-12-05  3:19 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-05  3:16 [PATCH 4.14 00/30] 4.14.332-rc1 review Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 01/30] RDMA/irdma: Prevent zero-length STAG registration Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 02/30] drm/panel: simple: Fix Innolux G101ICE-L01 timings Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 03/30] ata: pata_isapnp: Add missing error check for devm_ioport_map() Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 04/30] drm/rockchip: vop: Fix color for RGB888/BGR888 format on VOP full Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 05/30] ipv4: Correct/silence an endian warning in __ip_do_redirect Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 06/30] net: usb: ax88179_178a: fix failed operations during ax88179_reset Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 07/30] arm/xen: fix xen_vcpu_info allocation alignment Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 08/30] amd-xgbe: handle corner-case during sfp hotplug Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 09/30] amd-xgbe: propagate the correct speed and duplex status Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 10/30] net: axienet: Fix check for partial TX checksum Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 11/30] mtd: rawnand: brcmnand: Fix ecc chunk calculation for erased page bitfips Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 12/30] s390/dasd: protect device queue against concurrent access Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 13/30] USB: serial: option: add Luat Air72*U series products Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 14/30] bcache: check return value from btree_node_alloc_replacement() Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 15/30] bcache: prevent potential division by zero error Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 16/30] USB: serial: option: add Fibocom L7xx modules Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 17/30] USB: serial: option: fix FM101R-GL defines Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 18/30] USB: serial: option: dont claim interface 4 for ZTE MF290 Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 19/30] usb: dwc3: set the dma max_seg_size Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 20/30] pinctrl: avoid reload of p state in list iteration Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 21/30] firewire: core: fix possible memory leak in create_units() Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 22/30] dm-verity: align struct dm_verity_fec_io properly Greg Kroah-Hartman
2023-12-05  3:16 ` Greg Kroah-Hartman [this message]
2023-12-05  3:16 ` [PATCH 4.14 24/30] powerpc: Dont clobber f0/vs0 during fp|altivec register save Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 25/30] btrfs: fix off-by-one when checking chunk map includes logical address Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 26/30] btrfs: send: ensure send_fd is writable Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 27/30] ipv4: igmp: fix refcnt uaf issue when receiving igmp query packet Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 28/30] ravb: Fix races between ravb_tx_timeout_work() and net related ops Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 29/30] net: ravb: Start TX queues after HW initialization succeeded Greg Kroah-Hartman
2023-12-05  3:16 ` [PATCH 4.14 30/30] driver core: Release all resources during unbind before updating device links Greg Kroah-Hartman
2023-12-05  9:10 ` [PATCH 4.14 00/30] 4.14.332-rc1 review Harshit Mogalapalli
2023-12-05 10:37 ` Pavel Machek
2023-12-05 11:09 ` Jon Hunter
2023-12-05 14:01 ` Naresh Kamboju
2023-12-05 16:44 ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231205031512.867706368@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=bo.wu@vivo.com \
    --cc=mpatocka@redhat.com \
    --cc=patches@lists.linux.dev \
    --cc=snitzer@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox