From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E8BD296BCB for ; Tue, 30 Jun 2026 08:39:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782808778; cv=none; b=eUIYO4jXxJetODZK4ypFt4wjFfhoyT0asnnSdL8ICGamJcF7zOFOeyL4yosXUmSr//XkCgLgv9J38/pjRr2IgMVEXr5SbwusjkzeExiHden/pyub3w2iAi5DiZ6bdmuhYmxC882+kCos7Og1hru92so4Eu5LrUpNCrfMyKUqFZs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782808778; c=relaxed/simple; bh=nk1SuMr8/7XigAlohRNbSpX0vVJqn/+ae9Y2cBmpQ84=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID: MIME-Version:Content-Type; b=UVnFKghln5w39vwZnZgeICMSv8KNSxbW2w9TCbeHmE07k0/iixbdDzg4JWLKDaudukRcrHKFbZw3TueYOypW7G1MpclKJxuBO2HokVIQgNnj/eKvpUlrGkutg+pRRWniJAlG3EgoPziUt2qdy2zYjPfNjAW4g9yt1D4uxnmVk6o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OSUMW8os; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OSUMW8os" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-493a5d32e8cso24675065e9.1 for ; Tue, 30 Jun 2026 01:39:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782808776; x=1783413576; darn=vger.kernel.org; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=07Zjugs28X7mQA/ysZ3DBSDjZ3etII5V3AX1NHliDV4=; b=OSUMW8oszm6AFTLIXQmy/7v5vbs987C4Wye5ODarxtcgRVR7jyCrcCxGTpaYhalEKF V9daJbIJFkIU2yLDJiaabGjvXL2OmQqvWtnzSELaaw7JbCnYhagu9KTU19Kh2VKyivnG jVLyj5+DxkXyMw1rBivU2RqVtfVsXhu2w5fzfE9a7CwNVAKXAmNwgQslBWvd4Rax20m5 rJbDRWZQqj4+Epsdrkz3lON1S4+3Q3A4ZKnv92/W8+7yuQznX6iO6/IkQtxEY8tWXEaZ PLmeStQLoj0EV+femvnNxU3znt5aG8em4K46YZxrh30MZ5cHqUv17xrcfbPmKj9mmCVH D/BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782808776; x=1783413576; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=07Zjugs28X7mQA/ysZ3DBSDjZ3etII5V3AX1NHliDV4=; b=ZUrjnmqHWrkeCeOr9i0ch+h3oVYpKzHSapxj8VOW5dlGFk2lJY5syoGBQGJtn8j93p 6xzhq9YPH7kYzfU9GY8ByuyN6d9gLGMCg+CsNOW5vVdyRpOdcLQ2YVFVQxZDJiS1Inec 9lebEsgmjPzmVDQYB1O1ACSnnqAtCdlfU6W8GzrIvDQp5hnqa1ry7U7uiFPkte0IQoo3 YTs7F/W4EZybwW5qqsWT2t6hUmp8tAjhw10+P3LK1mK+O1IMUiX7hdQW4tIMTbWVRdRA pHi86yNXBS6rPqkuhNXsk/x8x/ML8o1JrS3sf/FSJyPmsNlY3D1ZhgW8Bd2Y3ALw155I LEVg== X-Forwarded-Encrypted: i=1; AFNElJ++HulnihI5kXGhBQ6c+1Xm6b2olEjnOiL4ubhtjvgODlNa2h79HWc/XMorYmD9uoangzpgeJ6IyLdryKk=@vger.kernel.org X-Gm-Message-State: AOJu0YyMQ7SLv6lxliKFV51uPKkifxyy5Ol/hcdhj5y7+lpizlxIUR26 LQoUVLOMW6DfEEZdICuYyG5OAtiZ9Sa00V83qdDFVVS7ERsMNoGohAXF X-Gm-Gg: AfdE7ck2pmrSq7zVXYKnGuPQiivS6IqEI1VnErkl9chSfFH5CwrJtUpcxHrYISWCENH UGe/20KetUSzGwG5tQwaZDKbtH4EDwoAVmoqMesBEoI8X2rE1NxXA2fJF4opy4znkIY3L9lsPPt Yqlh0UdoctBeyFRpCZbOeHHbeQoGr0LT4xWC9gknLiEVo6dHrQ96MC+iqvZ3My/Ba4ZbZb1qaq6 P+dEH1yhyxKEhLGyTYUQDXMspajDoz7R80yt1HAI37G6X98swmIYJL5IiPPWSEStoFPFCcBKcbe 8h1NzOJwXpNVpM23RL//CngpGvFHcokCxR2hygvYf+amMJoMvLSPgZfRigk2qXlAjQ2LtVNmqhh 6TX2/hfEhDwOYCtQT/pHfMmiGDGDtgzpl2al/sTbKNSN2bTqAsiV+CIq4nsnb1bQCQ6zaplFFyL 8jx2WZY/URkZ69XfHkcK718SnYe//h1PJMiJBqzGteEOVNEQNLwJy59Z7p X-Received: by 2002:a05:600c:4f90:b0:490:e974:e006 with SMTP id 5b1f17b1804b1-493b82b625dmr38535625e9.29.1782808775623; Tue, 30 Jun 2026 01:39:35 -0700 (PDT) Received: from Abds-MacBook-Air.local ([141.2.113.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-493b8d0bbbcsm59964435e9.13.2026.06.30.01.39.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2026 01:39:35 -0700 (PDT) From: Abd-Alrhman Masalkhi To: John Garry , song@kernel.org, yukuai@fygo.io, magiclinan@didiglobal.com, xiao@kernel.org, axboe@kernel.dk, vverma@digitalocean.com, martin.petersen@oracle.com, linux-kernel@vger.kernel.org Cc: linux-raid@vger.kernel.org Subject: Re: [PATCH v2 2/7] md/raid1: advertise atomic write limits and handle runtime constraints In-Reply-To: <7cbaaca3-4eb5-434d-a13f-f9574c9f977b@oracle.com> References: <20260628142420.1051027-1-abd.masalkhi@gmail.com> <20260628142420.1051027-3-abd.masalkhi@gmail.com> <7cbaaca3-4eb5-434d-a13f-f9574c9f977b@oracle.com> Date: Tue, 30 Jun 2026 10:39:34 +0200 Message-ID: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain Hi John, On Mon, Jun 29, 2026 at 15:48 +0100, John Garry wrote: > On 28/06/2026 15:24, Abd-Alrhman Masalkhi wrote: >> Atomic writes in RAID1 must fit within a single barrier unit. Advertise >> this restriction through the queue limits by setting >> atomic_write_hw_unit_max to BARRIER_UNIT_SECTOR_SIZE so that bios which >> would cross a barrier-unit boundary are rejected by the block layer >> before reaching MD. >> >> A bio that passes block-layer validation may still become unserviceable >> within RAID1 due to bad blocks or write-behind constraints. In the former >> case, complete the bio with EIO. In the latter case, disable >> write-behind rather than failing the bio with EIO. >> >> Fixes: f2a38abf5f1c ("md/raid1: Atomic write support") >> Fixes: a4c55c902670 ("md/raid1: simplify raid1_write_request() error handling") >> Signed-off-by: Abd-Alrhman Masalkhi >> --- >> Changes in v2: >> - Drop the early atomic write split check from raid1_write_request(). >> - Advertise the atomic write size limit via queue limits. >> - Disable write-behind instead of failing atomic writes when the >> BIO_MAX_VECS limit is encountered. >> - Link to v1: https://urldefense.com/v3/__https://lore.kernel.org/linux-raid/20260623072456.333437-3-abd.masalkhi@gmail.com/__;!!ACWV5N9M2RV99hQ!LbMSGSClRi0PNBqQti5ZNWGDVjDd34-7saYEAwNyBNjpNTjEA7veqM5RHG8KB1QiscarW4UaIefjm19ywSImtIgh$ >> --- >> drivers/md/raid1.c | 36 +++++++++++++++++++----------------- >> 1 file changed, 19 insertions(+), 17 deletions(-) >> >> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c >> index afe2ca96ad8c..f322048ab3c2 100644 >> --- a/drivers/md/raid1.c >> +++ b/drivers/md/raid1.c >> @@ -1522,6 +1522,7 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio, >> int first_clone; >> bool write_behind = false; >> bool nowait = bio->bi_opf & REQ_NOWAIT; >> + bool atomic = bio->bi_opf & REQ_ATOMIC; >> bool is_discard = op_is_discard(bio->bi_opf); >> sector_t sector = bio->bi_iter.bi_sector; >> >> @@ -1603,20 +1604,6 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio, >> } >> if (is_bad) { >> int good_sectors; >> - >> - /* >> - * We cannot atomically write this, so just >> - * error in that case. It could be possible to >> - * atomically write other mirrors, but the >> - * complexity of supporting that is not worth >> - * the benefit. >> - */ >> - if (bio->bi_opf & REQ_ATOMIC) { >> - bio->bi_status = BLK_STS_NOTSUPP; >> - bio_endio(bio); >> - goto err_dec_pending; >> - } >> - >> good_sectors = first_bad - sector; >> if (good_sectors < max_sectors) >> max_sectors = good_sectors; >> @@ -1633,10 +1620,24 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio, >> * at a time and thus needs a new bio that can fit the whole payload >> * this bio in page sized chunks. >> */ >> - if (write_behind && mddev->bitmap) >> - max_sectors = min_t(int, max_sectors, >> - BIO_MAX_VECS * (PAGE_SIZE >> 9)); >> + if (write_behind && mddev->bitmap) { >> + if (atomic && max_sectors > BIO_MAX_VECS * (PAGE_SIZE >> 9)) > > where does BIO_MAX_VECS * (PAGE_SIZE >> 9) even come from? > BIO_MAX_VECS * (PAGE_SIZE >> 9) defines the maximum size supported by write-behind. The write-behind copy (alloc_behind_master_bio) uses a single bio, which can hold at most BIO_MAX_VECS pages, making this the largest payload it can carry. With a 4 KiB PAGE_SIZE, that corresponds to 256 pages, or 1 MiB (2048 sectors). This patch changes the behavior for atomic writes that exceed this limit. Instead of failing the write with -EIO when the number of sectors must be reduced, it disables write-behind and proceeds with the atomic write. >> + /* >> + * Atomic writes cannot be split, so disable >> + * write-behind. >> + */ >> + write_behind = false; >> + else >> + max_sectors = min_t(int, max_sectors, >> + BIO_MAX_VECS * (PAGE_SIZE >> 9)); >> + } >> + >> if (max_sectors < bio_sectors(bio)) { >> + if (atomic) { >> + bio_io_error(bio); >> + goto err_dec_pending; >> + } >> + >> bio = bio_submit_split_bioset(bio, max_sectors, >> &conf->bio_split); >> if (!bio) >> @@ -3229,6 +3230,7 @@ static int raid1_set_limits(struct mddev *mddev) >> lim.max_write_zeroes_sectors = 0; >> lim.max_hw_wzeroes_unmap_sectors = 0; >> lim.logical_block_size = mddev->logical_block_size; >> + lim.atomic_write_hw_unit_max = BARRIER_UNIT_SECTOR_SIZE; > > This BARRIER_UNIT_SECTOR_SIZE is a bit like chunk sectors, no? I am just > wondering if we just should set it to chunk sectors = > BARRIER_UNIT_SECTOR_SIZE > > I assume that it affects more than Reads and writes, e.g. discard also. > BARRIER_UNIT_SECTOR_SIZE is the resync barrier-bucket, not the layout chunk size. Unless I'm missing something, using atomic_write_hw_unit_max seems more appropriate than using the chunk size. That way, the limit only applies to atomic writes instead of affecting other operations such. >> lim.features |= BLK_FEAT_ATOMIC_WRITES; >> lim.features |= BLK_FEAT_PCI_P2PDMA; >> err = mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY); > -- Best Regards, Abd-Alrhman