From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E388F16D32A for ; Thu, 20 Mar 2025 16:13:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.25 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742487224; cv=none; b=ncyi/g6PUcrvTNkfXvR2g4MpEBUHU13DMpx0ygqHVZqkkpve8Sz+LLum5oveWjjpz2+KVhf2TmSxFhFD7IcgtxPhWWt2UaAsElZFBhRd1m8xOsC5P7KGyF2S5jSHX3+lNsXMnnsnP7v/cARrlWLy6Ns6zkR6teiAaBCVd1rZpu8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742487224; c=relaxed/simple; bh=BqjdSgiXP93170tlNOugnnP/KUOYumTk/f1yaoXR0XU=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:From:In-Reply-To: Content-Type:References; b=tO0eRveFe27VX6e2F3PElk3Jbd17TsTioBXV5343qQImnrkFzIG+wskdYD7jrSNuifZEtj2AaptUI5ZC8Ga68r+GaVvZH68GdYkNmMs6uBlwNW3PWHjCfMbMKgJYmFru/JUd2RuljiJz6xtPY76HrioezfX5Ex6zE53Zcjen+dI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=mKTXmvAW; arc=none smtp.client-ip=203.254.224.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="mKTXmvAW" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20250320161339epoutp025db637eb04332a889c0f81d393a430a3~ujuTvJMtA3263832638epoutp02s for ; Thu, 20 Mar 2025 16:13:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20250320161339epoutp025db637eb04332a889c0f81d393a430a3~ujuTvJMtA3263832638epoutp02s DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1742487219; bh=1QhXPUc+UhsbA3QNDxXoX2eKXdOw0cgYCugrDd2js8Y=; h=Date:Subject:To:Cc:From:In-Reply-To:References:From; b=mKTXmvAW2l6BMRPv6hVoOS7GlBcBv3c58LsOpkCGvuVHLLusUxnpRooDLf1EYqvSD HuLEanTr/hEErOJnpJT4t3PEGueerPXNxe7OiR8gOruUZrSg1/6cWy6rBS4rNwCVp4 Iv9g5yRFW/xz0AroyEnSvL2/X6bCi0/DcEqLES30= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20250320161338epcas5p3300497d14102a14142b36f46515e39a6~ujuSqt-RI0817008170epcas5p3l; Thu, 20 Mar 2025 16:13:38 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4ZJVxX4mxsz4x9Pq; Thu, 20 Mar 2025 16:13:36 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 34.3B.19710.0BE3CD76; Fri, 21 Mar 2025 01:13:36 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20250320161335epcas5p2709afc3ff29ffec350164003b511f92d~ujuQEy8Ch0265102651epcas5p2M; Thu, 20 Mar 2025 16:13:35 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20250320161335epsmtrp241cc853355b1a879f392060c68c9cbd8~ujuQDsIZq3146031460epsmtrp2o; Thu, 20 Mar 2025 16:13:35 +0000 (GMT) X-AuditID: b6c32a44-36bdd70000004cfe-8e-67dc3eb0b364 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 99.43.18729.FAE3CD76; Fri, 21 Mar 2025 01:13:35 +0900 (KST) Received: from [107.122.11.51] (unknown [107.122.11.51]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20250320161332epsmtip13339e115f8abf0a704f8b543d976a2d5~ujuNIqF5u1448214482epsmtip1d; Thu, 20 Mar 2025 16:13:32 +0000 (GMT) Message-ID: <796adeb4-2b7e-4d01-866b-1f47b6eb10e5@samsung.com> Date: Thu, 20 Mar 2025 21:43:31 +0530 Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64 To: Keith Busch , Bart Van Assche Cc: Christoph Hellwig , Luis Chamberlain , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, lsf-pc@lists.linux-foundation.org, david@fromorbit.com, leon@kernel.org, sagi@grimberg.me, axboe@kernel.dk, joro@8bytes.org, brauner@kernel.org, hare@suse.de, willy@infradead.org, djwong@kernel.org, john.g.garry@oracle.com, ritesh.list@gmail.com, p.raghav@samsung.com, gost.dev@samsung.com, da.gomez@samsung.com Content-Language: en-US From: Kanchan Joshi In-Reply-To: Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA01TWVBTVxjm5N4sMAWvEeXIQ4mZWgYsmGCgBwuhM2XayzI2M07t8tA0E24D Q0hiFitahgyboIWwaWkUtSClQKsYGAQhZVFphYqUFIS0QZDwUBYR6JQCbWk2W96+8/3f969z WBjbwAxkpSm0lFohkXMZPnjbnZCQsGbhrzLextmXUJPNwEDzd1YAuvBsHUNLljYaau2fBMhi 90MTPR001FVTTkMNTfdoaHijHaCii81MVN43BlDlRh2GzNYDqMt8H0eT32zR0XfzZgyNl84C tFqXx0S9A4VMdH1+CUebf15ivL6HtPdeppGWn5PIgRpIdhhtTPLh5E2cbPk6lLQ80JGmxiIG aVopZ5IV4/WA7JzQM8jlWStOlrQ2ArJl8DS5anpRtOOD9JhUSpJCqTmUQqpMSVPIYrlJR8Vv iCOjePwwfjR6lctRSDKoWG58sijszTS5Y3gu54RErnNQIolGwz0ojFErdVqKk6rUaGO5lCpF rhKowjWSDI1OIQtXUNrDfB4vItIh/Cg9daN6mK4q9T7ZvKXWg2vMs8CbBQkB3DTZHdiHxSY6 AXxa00VzP1YANCzn0JwqNvEHgGsFp547nlU+YLh5M4CN1TK3YRHAHz6bAM6ALyGEU0UGlxkn 9sPf6to8/E54/ws77sS7iSD42FrlamMXkQhHlrpdSf2JBDhx/nOGMylG2DDYPmZ0BTAiAFrt VxxJWSwGEQKHK3RO2puIg+sr/R5JELy1eAlzeiFxyxvaegY9c8bDheLbdDfeBee+b/XwgXD1 qZnhxulw6skU7safwvaWEo8+Dur/Gqc762KOujduH3TX8oPFm3ZXO5DwhYUFbLd6H5wsn/U4 A+B01TUPJuHgeAPu3lUZDdZZG/BSwDFuW4tx25TGbeMY/698FeCNYC+l0mTIKGmkiq+gPvnv 3FJlhgm4fkZofDsYv/JPeB+gsUAfgCyM6+/rX/iLjO2bIsk8RamVYrVOTmn6QKTjPmVY4G6p 0vG1FFoxXxDNE0RFRQmiD0XxuQG+uR15MjYhk2ipdIpSUernPhrLO1BPe/u99+dDE4ePliRn q/Yc95qeDlurb5qdGUX7Z6bezQ0ePSSViRNLbEf8KmQDaWcsq1vFufvMP5q+xLUvJH21UHLM z8JMWV8z/TRU2FMVcr2GOl2SMK+tjRtQj2gr946Kk7O2kvINd4948YRNj8/bOBF3O+nczpz8 LP9HAoZ16BXGWztu5L7sN/f3OcPv5Qn9tctPgmNeuynN9lm5ly/uvjBTsMotG8iJGDrcLQpo 0l82aKz5+vCR8AUChB4vzTv54VxQjf0YZ5SdJZ66esZWtHFAeO5EptejwrHa1I+rlY3dskyR fKctuHYxuLR2hJ5fUfmw91uzaFnX8M5Fq7BeIsrm4ppUCT8UU2sk/wKQrvlbogQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0hTYRjuO+d4drZaHpfpl9ptUHihlSXxhWUXiI5alFlEQpeRp1XqlM1p WcGsEGeXqU3LaSss081VtnmlTFslalTSbDVzdmFRZqUtArWL5Vbgv4fnfW4/XgoXfCYCqP3S dFYmFScLSR5Rf084a8GNqF7Jorq6MFTtUJNo4J4LoOKhERwNWusxVNvWB5DVORXZW5swdLu8 EEP66gcY6hptBEhVWsNBhRYbQJrRChw194Sh280dBOozjnmhOwPNOHqR/w6gbxUnOOhuZy4H XR8YJNCP4TJylR/jvKvDGGt3LNNZDpkmrYPDPOm7STDmqlDG+kjBmAwqkjG5CjnM2ReVgLll V5LM13c9BHOm1gAY88PDzDfTrE3eCbzliWzy/gxWtjBqN2/f6IUur7R87sGaMZkSXOHkAS4F 6Qg4pHlE5gEeJaBvAXhsqBd4Dv7wuG34n2ga1P9+z/GIBgB8XnXGLeLTUfC1So2NY4KeB/sr 6v/xPrCjxEmM4+n0bPiq57w7aBodA58OtpDj2JeOhvaic+5mnHbg0G5wEJ6GAgwatGq3G/87 o8d58W8DRZF0COw6qxinufRKOOJqIz2SpTCvLg948GzY8LkMzwcC7YQd2glJ2gkW7QTLJUAY wAw2TZ4iSZGHp4VL2UyRXJwiV0gloj2pKSbg/ojQkEbQYBgSWQBGAQuAFC705fvmvpQI+Ini Q1msLHWXTJHMyi0gkCKE/nz/D6cTBbREnM4msWwaK/t/xShugBITqURv5pQGvsnNiinxrlow PS4us9Gp10fsCihQFIflt228tjaoLjgzNGOzjctV2x8e6HiZ2YLV2HRrlWa1wCeeyjqnB5Fq jaV92dWA4tb+x8YtJ3f4roteUVqyl9nmCBmNm1xr+tQfZAkyrl5h//nlY2dgzMaMZ92BcxNO 8S7nGBtcpV5v7yjnz0zlpSt/FmwfWemzxng/tiyn/COxQTcWuzc5J3sJX3fJ3L0t+H78jqQo v8U3NdGFP37Ft2eQp13V+iO6JRxnhKNp+ffr6VMqW9tbZ5zK3jMpyft75FTpVrReg/sdshGD TS0l2qNFsb3mnerhyGxVTeTj4MqYyX0JViEh3ycOD8VlcvEfOW2zHoADAAA= X-CMS-MailID: 20250320161335epcas5p2709afc3ff29ffec350164003b511f92d X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20250320155856eucas1p1e98451f76fb93feebc7e4bb497e8155f References: <20250320141846.GA11512@lst.de> On 3/20/2025 9:28 PM, Keith Busch wrote: > On Thu, Mar 20, 2025 at 08:37:05AM -0700, Bart Van Assche wrote: >> On 3/20/25 7:18 AM, Christoph Hellwig wrote: >>> On Thu, Mar 20, 2025 at 04:41:11AM -0700, Luis Chamberlain wrote: >>>> We've been constrained to a max single 512 KiB IO for a while now on x86_64. >>> No, we absolutely haven't. I'm regularly seeing multi-MB I/O on both >>> SCSI and NVMe setup. >> Is NVME_MAX_KB_SZ the current maximum I/O size for PCIe NVMe >> controllers? From drivers/nvme/host/pci.c: > Yes, this is the driver's limit. The device's limit may be lower or > higher. > > I allocate out of hugetlbfs to reliably send direct IO at this size > because the nvme driver's segment count is limited to 128. The driver > doesn't impose a segment size limit, though. If each segment is only 4k > (a common occurance), I guess that's where Luis is getting the 512K > limit? Even if we hit that segment count limit (128), the I/O can go fine as block layer will split that, while application still thinks it's single I/O. But if we don't want this internal split (for LBS) or using passthrough path, we will see failure.