From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42D81C5AE59 for ; Sat, 31 May 2025 07:16:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAB016B0184; Sat, 31 May 2025 03:16:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B82946B0186; Sat, 31 May 2025 03:16:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABFA66B0189; Sat, 31 May 2025 03:16:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 906FC6B0184 for ; Sat, 31 May 2025 03:16:53 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6F2D2C126A for ; Sat, 31 May 2025 07:16:52 +0000 (UTC) X-FDA: 83502345864.18.A603B54 Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf14.hostedemail.com (Postfix) with ESMTP id C5A2A100003 for ; Sat, 31 May 2025 07:16:50 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bqnCQjCi; spf=pass (imf14.hostedemail.com: domain of mingo@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=mingo@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748675810; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bqTo17aDfd0YvPhz/C/WzHuq2RnNRlMZmTn94f9zkQE=; b=444uWeBU3QJl64e3uQU/vjgaA5AzAeVbprJjB7uHXO43bqH0Md1W4PvbUoAwClmWifJIA8 2SdTowMIogmrUGCwIx7Yc6qjtzbJvitjk37JBzEWeqLSjh1joOOJAHuyARYLrQR2O3MDMN Gy1C+Y9YjvGwhPQmN3e9QdDINIZFu4Q= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bqnCQjCi; spf=pass (imf14.hostedemail.com: domain of mingo@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=mingo@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748675810; a=rsa-sha256; cv=none; b=FA9Vd+Xw4QFQgV8ynz5WhS9MfXjc2WMNCgwByelv6YiKG+BDZquDJxUI0VJdwymAtQ7CzA vWunvw+nARWCgGQCxNMqPVFIgsvbXj5SOzL6Fs2/PLeRSBIzENuk5qm9LJwEgJOPNGzCbt M1YeclfJmdzVNBbglOFsowLrFnUAnY8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id D168D6111F; Sat, 31 May 2025 07:16:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95955C4CEE3; Sat, 31 May 2025 07:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1748675809; bh=zYe1+sXdqUMmr98YnXHwFBRq+aXOFiQG6wruVnbdSlU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bqnCQjCiCD2PbdPMeu84qfvj3Nx7yg7/rzrJ2htLvMM+rP8AmTcuFww7uFsPlBgb6 5BxKRXheiIHLHM8HR27gVsdgqIhowEPm2lDDSt8sGqBtixIkypSI/iLBIu89/MuvXV Mkyc0eQhfUkyLpRFEWir3/XVPEGCuDKPWaC1/5kJIW7HIE3YyRWt6FWIlFT595E/DZ Od8df52QJx0jo/LUPW2H1bJELKIDCwk+WRLadZBn/p16U8yvyjsGvuQJH3ePnn6MAG 5WZzuVvFtAjIgkW59InOd959UEMCxUuSC5FH/5MRtYIt9cyWd0L8SyxKHtlPekWcr+ eFtvjLAdNQXAg== Date: Sat, 31 May 2025 09:16:35 +0200 From: Ingo Molnar To: Bo Li Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, luto@kernel.org, kees@kernel.org, akpm@linux-foundation.org, david@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, peterz@infradead.org, dietmar.eggemann@arm.com, hpa@zytor.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, jannh@google.com, pfalcato@suse.de, riel@surriel.com, harry.yoo@oracle.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, yinhongbo@bytedance.com, dengliang.1214@bytedance.com, xieyongji@bytedance.com, chaiwen.cc@bytedance.com, songmuchun@bytedance.com, yuanzhu@bytedance.com, chengguozhu@bytedance.com, sunjiadong.lff@bytedance.com Subject: Re: [RFC v2 00/35] optimize cost of inter-process communication Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: C5A2A100003 X-Stat-Signature: oam9u8y3nfhnsi3purm5n31k614o573j X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1748675810-987404 X-HE-Meta: U2FsdGVkX18NSP3DPZT/JkbroOlxMIJOubWkF+0R/6ei9OKzKJSbRLGSFEmucPwTilUz+SMkG+nXiop+oQw2dq2uk4GBvl8NYYaP5KU5v32tzqsqDZNUipgy0zw+10apfM5gLedZJRkwZu5FLjCyDFpszq0TF+zA2VM9As6otuAyXgTe0INoQ+iTvuO4OwvuB4SBOTBZJQV2dlCySyxtit4s9LgmHMd8oR4rKHaMfQBPFMqMEHl/4YAyHgLMeRhiFTfrElKtwtbLUosd0SMW54ATZDV0oIardpuIO3wNbz7O386Ez13Er5RVQgOolWANARpmu3LT/GjUywSKvn/ikQLcpuqR0CQYiuvioWKz62wXxUWXp8hAV3LhYjXGB/QmpH8mylaqtuVzOhxxt2Lsp+uz2ER8QE+J+KDjdi4p6D21vGWfAToIiV2A1ow42xTtJ7rKIF3Z5AVd5veUM0aipnRqsbSRNoOoElsKvxqBWYflZMOwjHXHbCEpFOVSy1NNki9ToXIvY3L3COgTpYfff6vtVc6i8oakD8fhPGF+sxZWhBq2bBAogPJErpPELqitXopv64W1OnkeouOmdN/oWy8BoTDmEkt+KxKXlHvvRJVaPoOydvEv+JyeZH27nXP2bmM3oc2IC/8UVAK/z7KaUN5leimrPoR3FifGKhESw5vQfpj/5b+YjC9euCEx0/GMA8ez8mcOUOX//xs8npquv9YiNFetiM+cNPC07+UVH1I3sNXhIagiMPQtoqfDTBhplv927doYGJBFfJ7gd3dqzsZGWc7hu9XLSSiwosQnJi2l9Ee/WYJCkmW/eDh4nGA8OvyWC6xflK6tMd6+koY01+DHc9e7mCDl69nXu7q9e+wpDunboBcwkKbFRefmIvOfmENc6wfc5dLvtsb4QvFJZ0qhemfgyFXwnxmSguUhrSasQicH/KVdrbRC1vH8/fiNsZa6Qj7/FwUXpE+fyPI 5n33ELFl 7S5+zh1RwjFiR0U7NKNvtG21TYfGusF29/gPA1aU5sqkmGNR8oRvDHmqzARZLL6aE6X8Hlc22z7Ifhm3wHSgRBQQ0ZIMPtswhoDyItRhlmXRharQuOdoyiMqABGBhRbaw5ecgY7CVNYGe76JFW7m+kA4I1TgENo2Kjs2JH7VqHR/DLsSFVKQxgVpeQNfZuuANf6tsmI1kS0P/9bkSzforfWxf1pT8C8/6FdbeYLFu1XFPji2aFwq3bOoUyCEQZ7uwr4DQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: * Bo Li wrote: > # Performance > > To quantify the performance improvements driven by RPAL, we measured > latency both before and after its deployment. Experiments were > conducted on a server equipped with two Intel(R) Xeon(R) Platinum > 8336C CPUs (2.30 GHz) and 1 TB of memory. Latency was defined as the > duration from when the client thread initiates a message to when the > server thread is invoked and receives it. > > During testing, the client transmitted 1 million 32-byte messages, and we > computed the per-message average latency. The results are as follows: > > ***************** > Without RPAL: Message length: 32 bytes, Total TSC cycles: 19616222534, > Message count: 1000000, Average latency: 19616 cycles > With RPAL: Message length: 32 bytes, Total TSC cycles: 1703459326, > Message count: 1000000, Average latency: 1703 cycles > ***************** > > These results confirm that RPAL delivers substantial latency > improvements over the current epoll implementation—achieving a > 17,913-cycle reduction (an ~91.3% improvement) for 32-byte messages. No, these results do not necessarily confirm that. 19,616 cycles per message on a vanilla kernel on a 2.3 GHz CPU suggests a messaging performance of 117k messages/second or 8.5 usecs/message, which is *way* beyond typical kernel interprocess communication latencies on comparable CPUs: root@localhost:~# taskset 1 perf bench sched pipe # Running 'sched/pipe' benchmark: # Executed 1000000 pipe operations between two processes Total time: 2.790 [sec] 2.790614 usecs/op 358344 ops/sec And my 2.8 usecs result was from a kernel running inside a KVM sandbox ... ( I used 'taskset' to bind the benchmark to a single CPU, to remove any inter-CPU migration noise from the measurement. ) The scheduler parts of your series simply try to remove much of scheduler and context switching functionality to create a special fast-path with no FPU context switching and TLB flushing AFAICS, for the purposes of message latency benchmarking in essence, and you then compare it against the full scheduling and MM context switching costs of full-blown Linux processes. I'm not convinced, at all, that this many changes are required to speed up the usecase you are trying to optimize: > 61 files changed, 9710 insertions(+), 4 deletions(-) Nor am I convinced that 9,700 lines of *new* code of a parallel facility are needed, crudely wrapped in 1970s technology (#ifdefs), instead of optimizing/improving facilities we already have... So NAK for the scheduler bits, until proven otherwise (and presented in a clean fashion, which the current series is very far from). I'll be the first one to acknowledge that our process and MM context switching overhead is too high and could be improved, and I have no objections against the general goal of improving Linux inter-process messaging performance either, I only NAK this particular implementation/approach. Thanks, Ingo