From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from fout-b8-smtp.messagingengine.com (fout-b8-smtp.messagingengine.com [202.12.124.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC74D2F3C21 for ; Thu, 18 Sep 2025 08:38:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.151 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758184735; cv=none; b=CaUtZgJbiVXQ3CAjcjXtS3EC5HN9gRTNh85TUbkeNVCy6Tk+q9308/bDEZADczc+tqV0Iqwi+aanKlz0JlAGBbIrqK+jfBFGnPNhZuzaLlwUAISGI1aUvgqVmYjgQU3DVukGSXmc8aCezWQxcM6SFCh2HzomtTL47L+HtPc9n/I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758184735; c=relaxed/simple; bh=G8WzSGFPHxEstLChRXKwgJv1NUV4+915caYraa9HJfE=; h=Message-ID:Date:MIME-Version:Subject:To:References:From:Cc: In-Reply-To:Content-Type; b=A3xBGOFSlPbBBBbadGbEZ1mAhhhYP5myjbEY9+nOcBPG68R02VTb/v5QdeePcVb9BOmjw+vbmAIp9K2MGinDLF6EcwyKoU7o49uiTe768y5nvAB9a9MyuYZzK9OYFmS+sFet3KfgiBY5i1UWRlIsVgQpmPlHYw1uo0zNP0Nk6bM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bsbernd.com; spf=pass smtp.mailfrom=bsbernd.com; dkim=pass (2048-bit key) header.d=bsbernd.com header.i=@bsbernd.com header.b=lYGWU1n9; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jDXhMLqq; arc=none smtp.client-ip=202.12.124.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=bsbernd.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bsbernd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bsbernd.com header.i=@bsbernd.com header.b="lYGWU1n9"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jDXhMLqq" Received: from phl-compute-09.internal (phl-compute-09.internal [10.202.2.49]) by mailfout.stl.internal (Postfix) with ESMTP id CE9D31D00308; Thu, 18 Sep 2025 04:38:50 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-09.internal (MEProxy); Thu, 18 Sep 2025 04:38:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsbernd.com; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1758184730; x=1758271130; bh=FYmQvDTRl040Q1SYSX7rEdiO8HJKUj/rnYo0a6lVRH0=; b= lYGWU1n9M53lph8vec8HYBFJjL+ewVP4ogMcgRKgdlvXXqu1KsslqZeDi+rCzp3V WY1JETp6ZdIwLViAry3Wi5TA2orSq7FGOW8tIGyYVdIM6ULpg15dlERnD1WseJXe CDvoIyBn4lzlhvUZk4r9c2WVp5P9uEOntPo/PhK3sh00py4zEIXlXMD8BNmaeF84 vnzD2/QPkP7XDqumAyG0Sa0XoUvmVsOqXeYg5ZibQSkZIWzg9W/sRYzHlklWGz7d pWNyDlTy2fayEP0vceD6wiej4BJLiffeWKpEXtJCXXxqEgTr5zyddsRf2JB8eNFI OZ96UeJDAWAHN0RiFEO9pQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1758184730; x= 1758271130; bh=FYmQvDTRl040Q1SYSX7rEdiO8HJKUj/rnYo0a6lVRH0=; b=j DXhMLqqRxm0S9Xf6sMy/jDOj82vXlHI1/gfPwCnrYY6fKbZYT0Ef7PQY1Ife9r2A kJIrGA9LYQPR/ZXiEC130mji+V9v3Bqzhvd3XIznGoVUkwE5BO3mOgWTx/ESp/iK jUxZBGjnk1/xg3AR552/X15Tmh8TXFRkcAIvjlXfNnR8dcRLt0VKbkq9z5aNcbUj J52rsXUixEvQodBOqcQ+/sK2McxOBu+zH/q8jaY8koNfarXEeErMYx5uqW7OCFbk DJnVsz1kwSZcCnyzCUPHDG/uutEDiErHiG1Ne4VDmHDtvm7zNz5st+njPyl8IPUC T5g/jQvWxzfLNCmz0QsEA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdegheekjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvfhfhvegjtgfgsehtjeertd dtvdejnecuhfhrohhmpeeuvghrnhguucfutghhuhgsvghrthcuoegsvghrnhgusegsshgs vghrnhgurdgtohhmqeenucggtffrrghtthgvrhhnpeffveekieelleeifefhieekveefve eiteejleejfeetffffhfegheehteethfelhfenucffohhmrghinhepghhithhhuhgsrdgt ohhmpdhlkhhmlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpegsvghrnhgusegsshgsvghrnhgurdgtohhmpdhnsggprhgtphhtthho pedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegutghhghdvtddttdesghhmrg hilhdrtghomhdprhgtphhtthhopehlihhnuhigqdhfshguvghvvghlsehvghgvrhdrkhgv rhhnvghlrdhorhhg X-ME-Proxy: Feedback-ID: i5c2e48a5:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 18 Sep 2025 04:38:50 -0400 (EDT) Message-ID: Date: Thu, 18 Sep 2025 10:38:48 +0200 Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Fuse over io_uring mode cannot handle iodepth > 1 case properly like the default mode To: Gang He References: <5f63c8e3-c246-442a-a3a6-d455c0ee9302@bsbernd.com> From: Bernd Schubert Content-Language: en-US, de-DE, fr Cc: "linux-fsdevel@vger.kernel.org" In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [Added back fsdevel into CC] Hi Gang, On 9/18/25 05:05, Gang He wrote: > Hi Bernd, > > Sorry for interruption again. > About enable fuse over io_uring feature, I can back-port the related > kernel patch set. > But, for user-space libfuse, when will you or the team release a > tag(e.g. 3.18)? then I can upgrade the libfuse directly, rather than > back-port patches. > Second, Fuse over io_uring mode cannot handle iodepth > 1 case > quickly, I feel this is a by-design issue, the iouring uses the own > thread queue to handle the requests from the user space. I write a > kernel patch, which can fix this case, but for the most cases, we > still use the current design. the patch is attached, could you take a > look at it, to see if this patch make sense, or not. I will try to find time to release libfuse-3.18 today, one io-uring related patch is missing (re-init of some fields in struct fuse_req) and then just the release process. Regarding queue balancing, please have a look here: https://github.com/bsbernd/linux/commits/reduced-nr-ring-queues_3.1/ The tricky part and which is why I didn't publish v2 of this series is to keep performance for blocking/sync requests (like DIO, metadata, etc). At DDN we run with an additional workaround patch that disables migration in fuse_request_end(), which results in kind of wake_up_on_current_cpu(). Adding in wake_up_on_current_cpu() to fuse is simple, I have the patch, problem is more that wake_up_on_current_cpu() is not perfect. I'm just in the middle to walk through scheduler code. For example, wake_up_on_current_cpu() gets perfect after increasing /sys/kernel/debug/sched/migration_cost_ns Now I don't want to change system wide migration costs, but it needs to be time limited change when waking up from fuse. Additionally it needs to be a hint, depending on if the queue has more pending work - that is the part that gets even more complex with reduced queues and queue balancing. Also see https://lkml.org/lkml/2023/5/3/646 and related threads for reference. Thanks, Bernd