From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 545BDC52D7C for ; Thu, 15 Aug 2024 17:26:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB7D16B017C; Thu, 15 Aug 2024 13:26:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D67606B018C; Thu, 15 Aug 2024 13:26:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE1916B018D; Thu, 15 Aug 2024 13:26:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9C5A16B017C for ; Thu, 15 Aug 2024 13:26:11 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5D17A1215B6 for ; Thu, 15 Aug 2024 17:26:11 +0000 (UTC) X-FDA: 82455158142.11.15DE9D8 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf06.hostedemail.com (Postfix) with ESMTP id 5F63218000E for ; Thu, 15 Aug 2024 17:26:09 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TO+JOwYH; spf=pass (imf06.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723742696; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9m9sXMIDc7Stw58EnRpTLc98lXl6hbDfeMyJIcE5wAI=; b=oh9ZCB/jIWuMiCRRMo0KZpRth3XI4uERNnFrR/k8ocM9DgBeegpTeVCaq4ifYX86m0Gh47 kUIT7Hqu/Q0Ra4kZ6/b/dmG7ujTdVCOQLBowt4+bumiSDGUomVFhNCQRcFXNoTn4tBhY/b 4go7zcyK9sA1qaYSir+rvhnmqVOe5ew= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723742696; a=rsa-sha256; cv=none; b=o9mhrPnDpjhOt5svo3FHOzIBRRa86ZlsiIAxuGY56UbPjFBIDuQ6YibocOHcvldeqMNcQO SBhPExy9dx9qVfOMW7LPE/I38J86wx4Y477aCBpn1QaK6tpaBhkjH4qGxQg4uQgUjJb4Bo gYnXwoymTl1qRbaCvOViFAsDEXO+OJU= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TO+JOwYH; spf=pass (imf06.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a7aa212c1c9so148610866b.2 for ; Thu, 15 Aug 2024 10:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723742768; x=1724347568; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=9m9sXMIDc7Stw58EnRpTLc98lXl6hbDfeMyJIcE5wAI=; b=TO+JOwYH9du5vU0ootk61TY1a+DMjenqqoG9liMDpNYZ4hORHRTTNnVKAjFrdzxWjT OAC/wZn5FoFxjb4n8tdu4ZMO66dwKpbyZdD6lKVhxqh/rlG1dpeT2b+1t12nbEfaxEoT 7kSpegOX6i1B/ndAcUKpfY/atGwSPslxFvXj/VpQXeeybgaFnhnOaK3eHyQ0/n439ild SOYvpDoQdN4KvHiVuXAxRvFdxiqbqN4oJXLtJ5lLPxTyNqLx37dKmLoMsXt0B0qFN9l2 dv24JGIxIIQbLjKqf42OQrfVioJzHVgwephJLlWGmnZ+wpxMKWYc//SJWIFY2gV0+S9N oImQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723742768; x=1724347568; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9m9sXMIDc7Stw58EnRpTLc98lXl6hbDfeMyJIcE5wAI=; b=RtG/wdgZVfKkEnNqlb5IwYRzM4sO5/Sj1wd7FsHQvC6w61uTluNv5YlTtQLEMcg3tE QsxhrpZ9M+sq8GiDDtlWkb2ugwwdPMNfklxYmM9z7Wz1UvW2OwGDTuEsodlCw6jqiJRq DfrTbAlBp8d008rpGwqIPk9PsP09wgidl4KaB6cBMb1AcWrulm8N7/FIY+H1UnvJuqv8 o5l1KECBMPCc0BRcmNMIbckVkXE2dpnnoEMBU7mXqBMfF4GgX5jPy4CYx1pp7tQJFrsk fTX6J8AmCEfYKTNSE3GcJpSsGKpM28WW5XFysRKuLdqZqt6bhaSmrYcteKaitYAS6Q0f acug== X-Forwarded-Encrypted: i=1; AJvYcCUaaBiMu6rGSikK4zL2PKMzBVntKahUtth0YyIcdaNbqEYosKfBue69jbK7nQHIHBLL1VGsvXBYT7vUk5fBt5Yxcnk= X-Gm-Message-State: AOJu0Yw+4ZPOn39wPBiBHgxPh55tdj61I8mCHKswPHjQC2sFG00sHRJh SDxgBwiFDISVUX2wBGZeylmuhF1hgJExYXBCJxSg1X0H3y3+aNBXfZK9yYRZ X-Google-Smtp-Source: AGHT+IFm+n1m+h6LlbNc+d2ECCG6xjbcGgBWu8N5vd6RmausgWlVcEkONrOz7KZFXr+AKcHwHCcGnQ== X-Received: by 2002:a17:907:948a:b0:a7a:ae85:f253 with SMTP id a640c23a62f3a-a8392a46fdamr17650066b.57.1723742767519; Thu, 15 Aug 2024 10:26:07 -0700 (PDT) Received: from [192.168.42.192] ([85.255.234.87]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838d023dsm129627066b.64.2024.08.15.10.26.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Aug 2024 10:26:07 -0700 (PDT) Message-ID: <543700ee-7020-4f99-81d3-9fd7228e508e@gmail.com> Date: Thu, 15 Aug 2024 18:26:42 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 0/5] implement asynchronous BLKDISCARD via io_uring To: Jens Axboe , io-uring@vger.kernel.org Cc: Conrad Meyer , linux-block@vger.kernel.org, linux-mm@kvack.org References: <9eac5571-a330-40b1-92ac-c6983be3619c@kernel.dk> Content-Language: en-US From: Pavel Begunkov In-Reply-To: <9eac5571-a330-40b1-92ac-c6983be3619c@kernel.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: 8op8uokw7k1amadbmat6nuph75ykrk9a X-Rspamd-Queue-Id: 5F63218000E X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723742769-661360 X-HE-Meta: U2FsdGVkX197wQv83cITXsPtO8iOVn9t8JZl+Zro0o7uJMo1GiClBnmRAGRMr2qgonTiJc+kt70Bz5JEJIN16RzDiJvzOpkpzDormcgwXIpAFFG22BfgVX77CC0VqY7OdyadU6Wo/LEbIwqCR99qlcdFLtvZVrQBBGF0L+BsE9ChOeyci2jOWufX8J++p8SvkRd/EZxnMf7RmmakstEl2VsLQ3ppFNpQWtqMvLngFc6oZEExkJtD258gFm8CspegD/vbGiuWFD0ymGT4bOjU4Lrst6oey6vir6VfaSQ6wn0Hehfa7R5go0/xzsA48v5/KPwYuhsr4gnuvhrBs08Qs/7BMFQxWG9aq3L6h++osuqnBIdWHKKKh6ObzhEtraSrKTpA1WUg8GvCB/9CiV6pmx0h3vBAbNFD1jJubmBMCB1W0eQFQTqSrMLLTJV4u6S9paWt0zlXU7R7pDI6SmlGHEhg4rheKLhTk51jeVI5Xf0kyG3knA4TUqY1dvC9kT9Gra9Fz48oT5UohecB0kJLyoBTU/Fs7NjK6fAAiIvULQ4SmT0mSEgsxhVQcDN0t//t0e51St+hzFMsPQcPSLo62Jox5KfNIDIX/Mlwzgb3XjHqGHtKWdzSGTDBKMeRersDsnOADfmZNhQFNvfru3udrAuMrr0y78prf2YC+XcExXLGb9gAp0dzGY+q43zwmjQqY75OhdAYHDLqenRAx0TAEXQ3jDdSiFnA7rxe9ON+SLckwakHdZ4NgRERWAded1bqT3/0iWW9I62smhg67lNOFyq7ZUNS9CdlSJSfDtucyVyOAD7nmJ5NlHenXI9zExFI7O6ruqa2ayh9B94+TX7Y+g0+FHc8vCCRoWvFPtNP5LzFD5yyhdvs+3rjxCpc187xyrG68cs9561eiS5tDIcFSqZGdL07pVg2aKFglx22MdhfC4D7gNTy4ZW+ngnWtANdbh9D4/GnXuMQ8NCkJ6f OByDRwUY G4xDHtQXhDU2O7EKYoFg9RMEzJXYoZE9vh934LW1gt/WOOJlUk2HEzI15fe6JSk8aRmOcX4HisVsLDc/sJcBxBkxNNhzeEntVTUQlZYzGdI8XVfgQJtsK3yXLFIDjLPs857/kLDVwwrOvwbDYE4RcabgrjtX0VTrzVW/2DFvOaGT049hbIWdMCluUI3uoaDaLrA1LSqcoBlZYVn15H+4EGwDWWXcNKjjQe3PDNGasrN+iaIRfVNfrNKnOL+adG/mUX3q91leRSN2o0CCKVYqwCElEB8JjM74qvae9/8RpuTlxl1rxsBdnSZ8RnajX4GJqvuy/9Ii9dqf1G5ATLF5VuyJmOKW0Rs8WOk3GGQf8oqGFSc8NHcNUjw3tLuEj14tMpCjskT+2j3DFEL+kaYxEbFIuo5jjyAvCPB8ozSYmJ5ZSFfHvIx8l6Mf0I5r70teGYyNUUFoDBam99LyMYNpZVWQG5qi91IFoGoiH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000037, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 8/15/24 16:50, Jens Axboe wrote: > On 8/14/24 4:45 AM, Pavel Begunkov wrote: >> There is an interest in having asynchronous block operations like >> discard. The patch set implements that as io_uring commands, which is >> an io_uring request type allowing to implement custom file specific >> operations. >> >> First 4 patches are simple preps, and the main part is in Patch 5. >> Not tested with a real drive yet, hence sending as an RFC. >> >> I'm also going to add BLKDISCARDZEROES and BLKSECDISCARD, which should >> reuse structures and helpers from Patch 5. >> >> liburing tests for reference: >> >> https://github.com/isilence/liburing.git discard-cmd-test > > FWIW, did a quick patch to wire it up for fio. Using this > job file: > > [trim] > filename=/dev/nvme31n1 > ioengine=io_uring > iodepth=64 > rw=randtrim > norandommap > bs=4k > > the stock kernel gets: > > trim: IOPS=21.6k, BW=84.4MiB/s (88.5MB/s)(847MiB/10036msec); 0 zone resets > > using ~5% CPU, and with the process predictably stuck in D state all of > the time, waiting on a sync trim. > > With the patches: > > trim: IOPS=75.8k, BW=296MiB/s (310MB/s)(2653MiB/8961msec); 0 zone resets > > using ~11% CPU. Thanks for giving it a run. Can be further improved for particular use cases, e.g. by adding a vectored version as an additional feature, i.e. multiple discard ranges per request, and something like retry of short IO, but for that we'd want getting -EAGAIN directly from submit_bio unlike failing via the callback. > Didn't verify actual functionality, but did check trims are going up and > down. Drive used is: > > Dell NVMe PM1743 RI E3.S 7.68TB > > Outside of async trim being useful for actual workloads rather than the > sync trim we have now, it'll also help characterizing real world mixed > workloads that have trims with reads/writes. -- Pavel Begunkov