From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C50120C004 for ; Thu, 23 Jan 2025 09:07:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737623260; cv=none; b=sIEUXUI8BtH7E2SlM9QV2Cqs6QcnAg+Ym6CdZgDOZvqUlT/IqNLZrKwmtJfA8IaEGcvpF31/4/2Zs1DuKx2/9rkhjrefSHZcGj4PWBoQk3zBy0rwaTukUG6CfgmS+gMMxXqvcmaeDDhyMqooS80/TDJK5njv7lLm8jvcdEFHCoc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737623260; c=relaxed/simple; bh=9btKLkyRHC2cjijtRjwN1mnFzYmLUlyCM1Z3JxnDT2M=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ImhypNp0AfG/qa2teWmqn6zI/Dl94t1YsK8Sn701OhQjvy5UXfwA8ySZ6U47JzKb6p/RI0lgZbZlInl6VN6HGKVC5/0QTi+s2/XynYKldSvDFrl/Ahfykcyh+0G/6iCWnmVjqBfet2ALy6bfqn47XlfjUj+9vbhOafULtO2v70U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=DsAWb6Fr; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DsAWb6Fr" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=sEevy1J0VAXnRgodm5rMxK1EoTYI9dE+a6pB13YJxTM=; b=DsAWb6FrDXbbqVYyAWAcHgFrcU ygpHgd8phf/4B7SYcOWMgYYi4ehEe2jcgbfz3SNLkzAXiVeAOQd9i6WnTPxOuNcSlKj7uXaP8h0d9 oYB72qy75k/ep7iCS2LKuwqi8yGiZRuwe3ohN3sQESgnOoLbOl8Y6FTjkNYRQqp4s/WbNROn7P3zv s+nLGo6Yhj30utsPY9gRMhQ8Lgp6zRkH0jaqtM9tCRhZ1mrEoaVm/RX22f+EMisWvxF03DdNYuVjt fh3gcO/bB7G3PPDsVZ6ECsu4LwrBtzs6xgKf+fFQSKFLnvWGoHL7hkROyZMYi5evuOmorNu3QsE8G USf5+cFA==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tatBJ-00000008lRx-2Gy6; Thu, 23 Jan 2025 09:07:21 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 117E83006E6; Thu, 23 Jan 2025 10:07:21 +0100 (CET) Date: Thu, 23 Jan 2025 10:07:20 +0100 From: Peter Zijlstra To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Mark Rutland , Will Deacon Subject: Re: [PATCH v6 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes Message-ID: <20250123090720.GI3808@noisy.programming.kicks-ass.net> References: <20250120024104.1924753-1-riel@surriel.com> <20250120024104.1924753-10-riel@surriel.com> <20250122083835.GE7145@noisy.programming.kicks-ass.net> <5820b18ef0ba48c33a62553fcc444c47f963b907.camel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5820b18ef0ba48c33a62553fcc444c47f963b907.camel@surriel.com> On Wed, Jan 22, 2025 at 08:13:03PM -0500, Rik van Riel wrote: > On Wed, 2025-01-22 at 09:38 +0100, Peter Zijlstra wrote: > > > > Looking at this more... I'm left wondering, did 'we' look at any > > other > > architecture code at all? > > > > For example, look at arch/arm64/mm/context.c and see how their reset > > works. Notably, they are not at all limited to reclaiming free'd > > ASIDs, > > but will very aggressively take back all ASIDs except for the current > > running ones. > > > I did look at the ARM64 code, and while their reset > is much nicer, it looks like that comes at a cost on > each process at context switch time. > > In new_context(), there is a call to check_update_reserved_asid(), > which will iterate over all CPUs to check whether this > process's ASID is part of the reserved list that got > carried over during the rollover. > > I don't know if that would scale well enough to work > on systems with thousands of CPUs. So assuming something like 1k CPUs and !PTI, we only have like 4 PCIDs per CPU on average, and rollover could be frequent. While an ARM64 with 1k CPUs and !PTI would have an average of 64 ASIDs per CPU, and rollover would be far less frequent. That is to say, their larger ASID space (16 bits, vs our 12) definitely helps. But at some point yeah, this will become a problem. Notably, I think think a 2 socket Epyc Turin with 192C is one of the larger off-the-shelf systems atm, that gets you 768 CPUs and that is already uncomfortably tight with our PCID space.