From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DAA0366541 for ; Tue, 10 Feb 2026 14:01:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770732075; cv=none; b=OVwNHhVLdO0+pTdSjQUs9Iy+Kt2FC6wOhvJvlZzsbO7poPKRZlORWlKA/5eH2vF/iSPrQhtU46ai7aoCnaPfINDg134oEF/7LBgDTB4DoKP6U0Nu1icWeZQ/1thaX1aCwzskVcsPU6OKYUyUTHrK4Ud9vCNAJAIXirsMxayAkx4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770732075; c=relaxed/simple; bh=+50oF9u4n9eTXHH4cVzoVk6iIl3FH4/Tov0ZyJpcWkA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XloUW3tKlg8mkeJQ4QSMDWXbDJIMLkEUz8PLa7ZE5+GRH8ScO7qaYNgYCUbjcBMxDk9jJvM8mrCR1kV7waBR0Gq76ZvOfRoMtu/nduFFSMfsVLgezFdx+QA1LF8b2pXQanxvNww1zxXP/fl8D3FtMdHEMmGOsaS7T2fr6YIHhNc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b=Og1WAsY4; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b="Og1WAsY4" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-4362197d174so602210f8f.3 for ; Tue, 10 Feb 2026 06:01:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1770732073; x=1771336873; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=9FnxXnVFI1K0nd8Coyw5JESfQAuts9vipw7Dj/0GRW4=; b=Og1WAsY4cH7OMJmrKX0jMX4fYSyOXKLWMMsHdbjXE3hHAFUyn2uD0K9PCOuAg41qRx aT+WarygDf4jI+NJQr9qfP062ewgMhUlhUZ1/M05SivHOn7hHqvrbsCcewh3hQ9FO14W Ulj57OvNiYQtlQMAdriYczyJwOE3CAYCyCVpbYGcQN09gNdR7JEp3Ncc5LXVZjUvqLNH kChhg+zRR3LeUKD5mpjjsNEBXpDBEck4bPEJapuvrQDXgoS1DjLlipYS8AF3oArlNszO cYdFsv9Do2hLdKfmnPo65E8PRqFKOeRljB7iSLuNPBb2Mw5BvgLkOpF/zYmPIswTIJ4I OgWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770732073; x=1771336873; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9FnxXnVFI1K0nd8Coyw5JESfQAuts9vipw7Dj/0GRW4=; b=G6IMQg4Y3Awd6/C1D0FTrV1pEUbFLFromJZezIhsXsfaYdSxB5cgB5T7ZwcnpYJIk3 5CPlvfOIpG9VTlrlD/EMWt0cNqza89oJZ0hd5682E30sxT6gRYZAQMOR3LECaRuDwijR LSefZ+hQaCqS/7bCLYqVqpFlXxY7mTYcMaqTkOVSeqlG7kWK3QP4Ywu6ug2tQPB34u0j zr1XkFVjV5gTxiiu0QAu+5ATROU0bTFESe8o5c2EENa/bgoNqxMKPJdXPtUxiPcHP03g wrSauQcN8V/1jAWYlSvn81IEZAw9irCK8CM9/r9wqCvyDTPGfZz1OF8Cby32MeIMI9Iw VryQ== X-Gm-Message-State: AOJu0Yw3FoM8hCza3U5wFNdh+nPRjPR9baa7ppneqNy6s/auO2ZQ6nDV E9Zz+FOIGABkmaqvb0ZjusHPHCTiKmwfRHex9OuFMG8aHEoWxqRnE3O93PKqm1f07iM= X-Gm-Gg: AZuq6aLtSlyyPaX2xUh85GHMFvK/qtKyeGvGlO+1bQkolnXb6x8fHphhLW/kluowVoj 1UnJ2lzSB6+aaM1+0KDM01slNMwM/94VngZpx8PUE3svzT7D8W7YoOKvHtO7asUrwcjb0LGBeS+ 2BgiGn8TXTA4VdNwYllRPOyJN9JjqLu4UMjCSdhcnB8RE4rKtHj0EG387a3KKB7yb004RlvHv2N D/hy3O4klVRexdXzIMoaAIOUa8DAGBChVYGsiouNtN0tKyUV4Zq+DjYblKNprkSpjnRh9gBoOqo CjI6EjwZqO68Rfxm70BAy+6yzxmpM15bB0QitQGAT/HPEX2+5jk/rr3gDOEUVN0kk1T9F0ImG0q gi0jDgqjC8AqU/+KE73bnRZXkOrOzeDSxBdB9Qs0VCK+ps2eZqNpbqvAOXmUrdOvBUMEzM/gVB8 HDZcy79cYSgM+D1havHhc5bPWjHrmgGzaJjtsE X-Received: by 2002:a05:6000:2382:b0:437:678a:5921 with SMTP id ffacd0b85a97d-437678a5b00mr12105418f8f.1.1770732072461; Tue, 10 Feb 2026 06:01:12 -0800 (PST) Received: from localhost (109-81-26-156.rct.o2.cz. [109.81.26.156]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-436296bd4a1sm34344330f8f.17.2026.02.10.06.01.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Feb 2026 06:01:12 -0800 (PST) Date: Tue, 10 Feb 2026 15:01:10 +0100 From: Michal Hocko To: Marcelo Tosatti Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Leonardo Bras , Thomas Gleixner , Waiman Long , Boqun Feng , Frederic Weisbecker Subject: Re: [PATCH 0/4] Introduce QPW for per-cpu operations Message-ID: References: <20260206143430.021026873@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260206143430.021026873@redhat.com> On Fri 06-02-26 11:34:30, Marcelo Tosatti wrote: > The problem: > Some places in the kernel implement a parallel programming strategy > consisting on local_locks() for most of the work, and some rare remote > operations are scheduled on target cpu. This keeps cache bouncing low since > cacheline tends to be mostly local, and avoids the cost of locks in non-RT > kernels, even though the very few remote operations will be expensive due > to scheduling overhead. > > On the other hand, for RT workloads this can represent a problem: getting > an important workload scheduled out to deal with remote requests is > sure to introduce unexpected deadline misses. > > The idea: > Currently with PREEMPT_RT=y, local_locks() become per-cpu spinlocks. > In this case, instead of scheduling work on a remote cpu, it should > be safe to grab that remote cpu's per-cpu spinlock and run the required > work locally. That major cost, which is un/locking in every local function, > already happens in PREEMPT_RT. > > Also, there is no need to worry about extra cache bouncing: > The cacheline invalidation already happens due to schedule_work_on(). > > This will avoid schedule_work_on(), and thus avoid scheduling-out an > RT workload. > > Proposed solution: > A new interface called Queue PerCPU Work (QPW), which should replace > Work Queue in the above mentioned use case. > > If PREEMPT_RT=n this interfaces just wraps the current > local_locks + WorkQueue behavior, so no expected change in runtime. > > If PREEMPT_RT=y, or CONFIG_QPW=y, queue_percpu_work_on(cpu,...) will > lock that cpu's per-cpu structure and perform work on it locally. > This is possible because on functions that can be used for performing > remote work on remote per-cpu structures, the local_lock (which is already > a this_cpu spinlock()), will be replaced by a qpw_spinlock(), which > is able to get the per_cpu spinlock() for the cpu passed as parameter. What about !PREEMPT_RT? We have people running isolated workloads and these sorts of pcp disruptions are really unwelcome as well. They do not have requirements as strong as RT workloads but the underlying fundamental problem is the same. Frederic (now CCed) is working on moving those pcp book keeping activities to be executed to the return to the userspace which should be taking care of both RT and non-RT configurations AFAICS. -- Michal Hocko SUSE Labs