From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 040EAD41D44 for ; Tue, 12 Nov 2024 01:04:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ze3l2pViJF1Ke3GJwQHCrNz3T2sv8K+NO5ktGTWIZ3E=; b=NnneGew6C9cSONgISvY3FKnfAB u7WqdcOmzGhw+v5e1bXAcAPLgPL8gMxo/hKYmeOC3tEMZFWrVZDtpXaPYXoRs3wWK6xDa0Pm9MfqC lTh0+b16o4jmLoHNXLQwfJ2n4yqdaQNYKClVRHPDaxgLGSlSsDUrC7Qvf61pFpfRn9LCCjZBCEdVi w/n9rIKhVGgMCX2raoEbVrd5TvFN1rxAsEQ2spSPXbeUZRFz1GGjLgbwMhRHt98PJ04FTnfTUX2BZ OjCtZoeHMDxLWbKYL+EARV00oeP7YkYSxPa1ciOogyIX+V6EQyQ4wiydgQo1+PYBoxuSaPHuhcO+M GzzuOazA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tAfKn-00000001p1n-3Rvx; Tue, 12 Nov 2024 01:04:45 +0000 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tAfAE-00000001nAl-47mM for linux-nvme@lists.infradead.org; Tue, 12 Nov 2024 00:53:52 +0000 Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-37d6a2aa748so3052261f8f.1 for ; Mon, 11 Nov 2024 16:53:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731372829; x=1731977629; darn=lists.infradead.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=ze3l2pViJF1Ke3GJwQHCrNz3T2sv8K+NO5ktGTWIZ3E=; b=A5PJM5/M4cZ1WsPeck3NFahFXZvE8CRBdcGScFpdV+D1PjLKJ+XnbSIhCXwbMdl4XK nGgN99drcI/DXjQcQo/3mcQ3t7sPWgvt67MJXbDcKDflpzOAV6sKFQSlIlJt0aFrOgaF NgsQRDdNLCtIJvNQXLkTkKNSOxOmzTgHdt5S9EAiZF6E2HDc7VtwzvSEKt8VaJ5DHlPk 1jq6P4hxrjIvwuMTehUpA1/OnrWVUdmcu1rR6ye6l/SyGQm2Uf7/N21AOYEjgu0+mbm5 RS37EFSoJIFimJx8hwX2FRp+AmHzVnzSZUMZfVKgyqZGcsNfAgWy+F+/N7rxrTDusJPW DObA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731372829; x=1731977629; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ze3l2pViJF1Ke3GJwQHCrNz3T2sv8K+NO5ktGTWIZ3E=; b=XQY842TLYQ0uE60AXcgc12bOP9bb7y4EBPnpneSJ+78Kmxox6o+/a20JXzdr2pmUra yqOQ8O53CJO0M3NT5kmBbKUSWVGL/f3kViP/IlixjYxDxhBojtJZvHV8EBfQaSMqrPHL YzLy8UDbzjPeVtZp2+Ls3EG9WR2pu6toI4hKZG1zmyqYYScOaOkQsPUyH9zjeK/7GnkD EtTTZfVzHBvZHkmsISGXUWQSRIAuSbvqQSFlhGPGxLAfB6A0sCDyfB8AukBHV8EijMy5 DsokSOCpOvBLKmivpvLCCMujC2BpXCachADy8xcfnsvHoxoqgwI9MCnIPzWpUccpynqB 7rSA== X-Forwarded-Encrypted: i=1; AJvYcCX+EfVZj6Sxkp/RFc1ZdsMax5Tza8XOptZOuaNQJ+BBf6W+ZrBMeS5pj7ltxEmNmk1/qAYLRjYvLicB@lists.infradead.org X-Gm-Message-State: AOJu0Yznn4vkQ+LApGJb4C4NWSBHb4eFPSOuaah2LtJAh3ah/OVzEHqV KdqZ1YXMy/9xNMW2KgRLBHq4ISqRt+9l8R4HqLfpkVdZoe/bjTEW X-Google-Smtp-Source: AGHT+IHK2R7O39rPlG6hNltnOdR+XHDxAnsH0twtPY40zT6Pf5flUZXqdiOD1gkb9VOabYgYNTuldA== X-Received: by 2002:a05:6000:410a:b0:381:f5a7:9baa with SMTP id ffacd0b85a97d-381f5a79bc3mr8812654f8f.0.1731372828800; Mon, 11 Nov 2024 16:53:48 -0800 (PST) Received: from [192.168.42.75] ([85.255.234.98]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-381eda0411csm13818107f8f.95.2024.11.11.16.53.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 11 Nov 2024 16:53:48 -0800 (PST) Message-ID: <86029316-4c73-46fa-8a02-a34103d6c60b@gmail.com> Date: Tue, 12 Nov 2024 00:54:32 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 06/10] io_uring/rw: add support to send metadata along with read/write To: Kanchan Joshi , Keith Busch Cc: axboe@kernel.dk, hch@lst.de, martin.petersen@oracle.com, brauner@kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, gost.dev@samsung.com, vishak.g@samsung.com, anuj1072538@gmail.com, Anuj Gupta References: <20241030180112.4635-1-joshi.k@samsung.com> <20241030180112.4635-7-joshi.k@samsung.com> <914cd186-8d15-4989-ad4e-f7e268cd3266@gmail.com> <47ce9a55-b6fd-47f8-9707-b530f2ea7df5@samsung.com> Content-Language: en-US From: Pavel Begunkov In-Reply-To: <47ce9a55-b6fd-47f8-9707-b530f2ea7df5@samsung.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241111_165351_062114_7683AA29 X-CRM114-Status: GOOD ( 22.22 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On 11/10/24 17:41, Kanchan Joshi wrote: > On 11/7/2024 10:53 PM, Pavel Begunkov wrote: > >>>> 1. SQE128 makes it big for all requests, intermixing with requests that >>>> don't need additional space wastes space. SQE128 is fine to use but at >>>> the same time we should be mindful about it and try to avoid enabling it >>>> if feasible. >>> >>> Right. And initial versions of this series did not use SQE128. But as we >>> moved towards passing more comprehensive PI information, first SQE was >>> not enough. And we thought to make use of SQE128 rather than taking >>> copy_from_user cost. >> >> Do we have any data how expensive it is? I don't think I've ever >> tried to profile it. And where the overhead comes from? speculation >> prevention? > > We did measure this for nvme passthru commands in past (and that was the > motivation for building SQE128). Perf profile showed about 3% overhead > for copy [*]. Interesting. Sounds like the 3% is not accounting spec barriers, and then I'm a bit curious how much of it comes from the generic memcpy what could've been several 64 bit reads. But regardless let's assume it is expensive. >> If it's indeed costly, we can add sth to io_uring like pre-mapping >> memory to optimise it, which would be useful in other places as >> well. > > But why to operate as if SQE128 does not exist? > Reads/Writes, at this point, are clearly not using aboud 20b in first > SQE and entire second SQE. Not using second SQE at all does not seem > like the best way to protect it from being used by future users. You missed the point, if you take another look at the rest of my reply I even mentioned that SQE128 could be used as an optimisation and the only mode for this patchset, but the API has to be nicely extendable with more attributes in the future. You can't fit everything into SQE128. Even if we grow the SQE size further, it's one size for all requests, mixing requests would mean initilising entire SQE256/512/... for all requests, even for those that don't need it. It might be reasonable for some applications but not for a generic case. I know you care about having that particular integrity feature, but it'd be bad for io_uring to lock into a suboptimal API and special-casing PI implementation. Let's shift a discussion about details to the other sub-thread. > Pre-mapping maybe better for opcodes for which copy_for_user has already > been done. For something new (like this), why to start in a suboptimal > way, and later, put the burden of taking hoops on userspace to get to > the same level where it can get by simply passing a flag at the time of > ring setup. -- Pavel Begunkov