From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21527431E74 for ; Fri, 10 Jul 2026 13:23:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783689833; cv=none; b=YkVdAcvfDzXqNumuvmoA3/t70L+o16QCAlD8epCkj1k5DsnK5PQR8YvG4V4bi8uCM2rNPlHN0TTLNQ/4yx42UypRvrJzoaRkL/MGzdwDOx1F1aquiPgQRwPO+dUDO+ZII8kw454lqml1XgAQZ8/aFlMget433Ez5zsp6k0PWlt0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783689833; c=relaxed/simple; bh=h5a1Su9PjrEKsF0LuXoSVIdSjt3j29OkNIH0r9t+t9M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oX87cmJT18WtJH3D1CfxG53pok/JLp4Eg5Cs1eqtJBi6uiucFi5Zso3q1E8hc+UTErNGcM+ScvByQc8mUXhF3+W0Ibp7kgs0/VcsM92SGawRTJQHd+NFdd0tTuYTUEJPs7G0N7I/ZT8/kG67oh+VyusJywdUEIrTIpJbI8jYXu0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=oU4TnXBG; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oU4TnXBG" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2cc6dd436c6so9342195ad.2 for ; Fri, 10 Jul 2026 06:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783689831; x=1784294631; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=sDVSdr7r152gQiXZ3Vozdie+Fv1rp8S/dmZQcBepj3A=; b=oU4TnXBG4tlyGC9o7IVozmq7RPH6k18hxtBvKtisIaqj1q+XvqRs1FDiqQ29HcZLa5 3fhXNgbCxBdmwyQSTJ/PWHPeZ0jTYQZqi3qd+LZ2p77bRXCcmFGzqQA1KfefuT4Bga11 VJM5zxO1+RLOy2KN6FQLKUFFqv1U9uaweh0jaxMkutZ6pXMjwYFFZRHMYbW/6D17HGEA 2SJhJzv2mwChfM1R5c34CKPx0QeegFad3AsVvXgF5v1KKNNPPr7rFUPqjCxEUK1oDoJR ConSQ8jssLSqpkjs96xWw9QLXs8lK6iQPQI/bQAkz3WXOQc/i6KemBNHepFp8RxGpmMa jg9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783689831; x=1784294631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=sDVSdr7r152gQiXZ3Vozdie+Fv1rp8S/dmZQcBepj3A=; b=X5aqE2slI0z8T01OzrfCnEag+cCj7SNZgt83wlPPxQ2t75FMBBC+fXutZi2c2FYtsN KVs5GiJj+Ss81F+U6Wq+qYQxkpE+OWj8XobjnqbJdC/rkWU3ScZ/qG746JgLxPyvj7Nr BHSZx+zA6YC+19Of7DZzcv7Rfvqk89mKSk2QEBo3voaO+v1AvM9+1ZdjDQrSPOcYF6cd JTcTduFSsapXTrCBvU7h2sb+YSPSFfZ0r9Sbv45Pfzjrh4oBB9XQQyDpu+QF3AJtT8zh SL20kP3XdqWaavR9IpmJj+V+64UG3/i9PfKEC1Kuu5Yf60q0jtWapG65teo9gSDbZxfL rx4w== X-Forwarded-Encrypted: i=1; AHgh+Rpz78mgiYSKV99q6TBz8KzuV+kdO0QqAG8GXcCtTWJEccLKvLU6B4oMUPPuSJ7ft7ZD+D5vqJ6jabLi2MY=@vger.kernel.org X-Gm-Message-State: AOJu0YyWphVhfnaWuZTNyGiz+CQ9Sxoe8yfUoPVyBvE8IzsNBZ7cFGI5 4qoCKuIXego4DJSK7UB1fYfQBDw8kerlHCRMltlYnDZesVOHTPLxRtom X-Gm-Gg: AfdE7cmkQNPClkgUxPXXYv/Nx3nPMX58qwgpF//Qq/d+uEaVU9YIrJvUpydbRzbCge1 iJ9zai14CB3Y4JDGsYLfkuUyzpvXKvNDt0gNRRZND6x/9YOCMauLxykFg2zrO/n0UzvH/mHsz2a CDUC24pAa17T6tzlS7sMKxtSA9KSPXF0FRSWXrq111e1XDHqrJZMWdtNWYrM8sHm9xnCXHe5KcB jQ6ZhM3Kit+a7C4dqL9S7ElEOhHiICvd5q32pmuISsuIJCJcY45Z6ZWusRFadkCHIEgp0fNZ2tH YQVcuKeobsnuDVJb28qvKE6sz/Nq5NV1ONw9ik7B1oibQibM1BaH3tzw69qfTCO9dUDx7foqu0K Q9LGPWRlIe82QC75PlGqkO0JvNhaDkkVcqfNBL2KGzim38VmDW0VMlTfK6OmRplZxFOmhHT8foR aYJ1CFz6r0OuTNKY84HZ+w49hBXaPdwGBQCSMpALsM2yX7vP4Jyre8IS9kzWLKajaqkApJIdtMf gNevbo= X-Received: by 2002:a05:6300:67c7:b0:3c0:9c1a:8952 with SMTP id adf61e73a8af0-3c0bd311c69mr13059970637.74.1783689831379; Fri, 10 Jul 2026 06:23:51 -0700 (PDT) Received: from research02.. ([2601:1c1:8700:f5b:fe34:97ff:fea3:c147]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-ca5b3b251eesm4903776a12.31.2026.07.10.06.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jul 2026 06:23:51 -0700 (PDT) From: Hiroshi Nishida To: Song Liu , Yu Kuai Cc: Li Nan , Xiao Ni , linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, Hiroshi Nishida Subject: [PATCH 3/6] md/raid5: scale the stripe_cache_size limit with system memory Date: Fri, 10 Jul 2026 06:23:43 -0700 Message-ID: <20260710132346.7295-4-nishidafmly@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260710132346.7295-1-nishidafmly@gmail.com> References: <20260710132346.7295-1-nishidafmly@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit raid5_set_cache_size() caps how far the per-array stripe_cache_size sysfs knob may be raised at a fixed 32768 stripes. The stripe cache costs roughly max_nr_stripes * (sizeof(stripe_head) + pool_size * (sizeof(bio) + PAGE_SIZE)), so 32768 stripes is about 1.5GB on a 12-disk array. That fixed limit is wrong at both ends: a host with hundreds of GB of RAM backing a wide array cannot grow the cache past ~1.5GB even though it has the memory to spare, while on a small box 32768 stripes can already exceed total RAM. Derive the limit from memory instead. When the new stripe_cache_size_max module parameter is 0 (the default), the ceiling is the stripe count that fits in at most 1/8 of RAM for this array's width, but never less than the historical 32768 -- so the limit only ever grows relative to today and small systems are unchanged. A non-zero stripe_cache_size_max pins a fixed ceiling for administrators who want one. The default stripe count (256), and thus the default memory footprint, is unchanged; this only changes how high an administrator may raise stripe_cache_size. Signed-off-by: Hiroshi Nishida --- drivers/md/raid5.c | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 7f72981121fd..e41d3fc92dd0 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include @@ -72,6 +73,11 @@ module_param(nr_stripe_hash_locks, uint, 0644); MODULE_PARM_DESC(nr_stripe_hash_locks, "Number of spinlocks the stripe cache hash is striped across, rounded up to a power of two and capped at 32. 0 (the default) auto-sizes it from the online CPU count (never below 8); a non-zero value overrides that. Larger values reduce lock contention on many-core systems at a small per-array memory cost. Read when an array is created"); +static unsigned int stripe_cache_size_max; +module_param(stripe_cache_size_max, uint, 0644); +MODULE_PARM_DESC(stripe_cache_size_max, + "Maximum the per-array stripe_cache_size may be raised to. 0 (the default) derives the limit from system memory (never below the historical 32768), so large-memory hosts can grow the stripe cache without a recompile while small ones are not offered a limit above what RAM can back. A non-zero value sets a fixed limit"); + static bool devices_handle_discard_safely = false; module_param(devices_handle_discard_safely, bool, 0644); MODULE_PARM_DESC(devices_handle_discard_safely, @@ -6922,13 +6928,42 @@ raid5_show_stripe_cache_size(struct mddev *mddev, char *page) return ret; } +/* + * Upper bound that the per-array stripe_cache_size may be raised to. The + * stripe cache costs roughly max_nr_stripes * (sizeof(stripe_head) + + * pool_size * (sizeof(bio) + PAGE_SIZE)). The limit was historically a fixed + * 32768 stripes, which both under-serves large-memory hosts backing wide + * arrays and, on a small box, still permits a cache larger than RAM. Derive + * it from memory instead -- at most 1/8 of RAM -- but never below the + * historical 32768, so the limit only ever grows relative to today. A + * non-zero stripe_cache_size_max module parameter overrides the heuristic. + */ +#define RAID5_CACHE_SIZE_FLOOR 32768 +#define RAID5_CACHE_SIZE_RAM_SHIFT 3 /* cap the cache at 1/8 of RAM */ + +static unsigned long raid5_max_cache_size(struct r5conf *conf) +{ + unsigned long per_stripe, limit; + + if (stripe_cache_size_max) { + limit = stripe_cache_size_max; + } else { + per_stripe = sizeof(struct stripe_head) + + conf->pool_size * (sizeof(struct bio) + PAGE_SIZE); + limit = ((totalram_pages() << PAGE_SHIFT) >> + RAID5_CACHE_SIZE_RAM_SHIFT) / per_stripe; + limit = max_t(unsigned long, limit, RAID5_CACHE_SIZE_FLOOR); + } + return min_t(unsigned long, limit, INT_MAX); +} + int raid5_set_cache_size(struct mddev *mddev, int size) { int result = 0; struct r5conf *conf = mddev->private; - if (size <= 16 || size > 32768) + if (size <= 16 || size > raid5_max_cache_size(conf)) return -EINVAL; WRITE_ONCE(conf->min_nr_stripes, size); -- 2.43.0