From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25B5820F08E for ; Wed, 22 Jan 2025 08:38:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737535139; cv=none; b=cYOMc4C9Bw16T8CYOMUhAzO5+WkQw527NqPi0z/D9FdI9jsj5/VBw0rUp2zok2pxz3Zhh9cuut1SwdiTTn3tBbui8xO+v07FJ281YLwWcds0zPwTcLBD0MIG9oFGm/5LWUKay+6OW3K4b3Elx3atMyrEVPt/eru+AKTwklaVNRE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737535139; c=relaxed/simple; bh=oClxuxtyKzJroWsioZvBsTXFuFHLdsyEeO7ij4k4beU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=HD6zRZnLG6w78ZfyHmB9mAeHGNJriFBCikkAdIm51J4NUt3wmE56J2qDSkp2K1REw5SYvKcTCFR5q8WycuTPJOXAQBJABj+UoImAfClFDPPHTfHgUHLQUsNDKp/Pz0JQBAsWmKUignrg2Oc8BfwDwUZDoOg6UbF4PkdzGOT5L6Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=Vf8J43Sz; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Vf8J43Sz" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=8+DQwNia3kWOeUROafPpORc26/Dzm182xsVdRuaDuec=; b=Vf8J43SzDLrIqCrc4kq9Y5NlbK DRfUoIjR4AlIO1OZpmjdm1cEtaCtRyFPWYOHorZEZl7oyJ3E32n7yZoAfmCJawKLyYpYl1cnIh3y5 ASGPF/o9OZTvShLuxuNcNlPsZvakttDr7IsF+Jq1/477SyhpqsWAi9yVG34vosaumdk8L8wl4yuOn CLOdJmDdPMrqndK43nEOYNEzQvhDYDWl4N6g63EUxj73yy9KFvKbHgQduVye8+HsSsUQJyEmQf3D8 zyNdtNWHUqjBy7Gt5sAhEp1k5Wv72w8ymbDy17XNXU2sbYGoe9cxtdy0FIjhqTlQ9V0xIC3181v6Y g/BZavwQ==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1taWFw-00000000xlm-0aRv; Wed, 22 Jan 2025 08:38:36 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 3E4D9300599; Wed, 22 Jan 2025 09:38:35 +0100 (CET) Date: Wed, 22 Jan 2025 09:38:35 +0100 From: Peter Zijlstra To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com Subject: Re: [PATCH v6 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes Message-ID: <20250122083835.GE7145@noisy.programming.kicks-ass.net> References: <20250120024104.1924753-1-riel@surriel.com> <20250120024104.1924753-10-riel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250120024104.1924753-10-riel@surriel.com> On Sun, Jan 19, 2025 at 09:40:17PM -0500, Rik van Riel wrote: > +#ifdef CONFIG_X86_BROADCAST_TLB_FLUSH > +/* > + * Logic for broadcast TLB invalidation. > + */ > +static DEFINE_RAW_SPINLOCK(global_asid_lock); > +static u16 last_global_asid = MAX_ASID_AVAILABLE; > +static DECLARE_BITMAP(global_asid_used, MAX_ASID_AVAILABLE) = { 0 }; > +static DECLARE_BITMAP(global_asid_freed, MAX_ASID_AVAILABLE) = { 0 }; > +static int global_asid_available = MAX_ASID_AVAILABLE - TLB_NR_DYN_ASIDS - 1; > + > +static void reset_global_asid_space(void) > +{ > + lockdep_assert_held(&global_asid_lock); > + > + /* > + * A global TLB flush guarantees that any stale entries from > + * previously freed global ASIDs get flushed from the TLB > + * everywhere, making these global ASIDs safe to reuse. > + */ > + invlpgb_flush_all_nonglobals(); > + > + /* > + * Clear all the previously freed global ASIDs from the > + * broadcast_asid_used bitmap, now that the global TLB flush > + * has made them actually available for re-use. > + */ > + bitmap_andnot(global_asid_used, global_asid_used, > + global_asid_freed, MAX_ASID_AVAILABLE); > + bitmap_clear(global_asid_freed, 0, MAX_ASID_AVAILABLE); > + > + /* > + * ASIDs 0-TLB_NR_DYN_ASIDS are used for CPU-local ASID > + * assignments, for tasks doing IPI based TLB shootdowns. > + * Restart the search from the start of the global ASID space. > + */ > + last_global_asid = TLB_NR_DYN_ASIDS; > +} > + > +static u16 get_global_asid(void) > +{ > + lockdep_assert_held(&global_asid_lock); > + > + do { > + u16 start = last_global_asid; > + u16 asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, start); > + > + if (asid >= MAX_ASID_AVAILABLE) { > + reset_global_asid_space(); > + continue; > + } > + > + /* Claim this global ASID. */ > + __set_bit(asid, global_asid_used); > + last_global_asid = asid; > + global_asid_available--; > + return asid; > + } while (1); > +} Looking at this more... I'm left wondering, did 'we' look at any other architecture code at all? For example, look at arch/arm64/mm/context.c and see how their reset works. Notably, they are not at all limited to reclaiming free'd ASIDs, but will very aggressively take back all ASIDs except for the current running ones. And IIRC more architectures are like that (at some point in the distant past I read through the tlb and mmu context crap from every architecture we had at that point -- but those memories are vague). If we want to move towards relying on broadcast TBLI, we'll need to go in that direction. Also, as argued in the old thread yesterday, we likely want more PCID bits -- in the interest of competition we can't be having less than ARM64, surely :-) Anyway, please drop the crazy threshold thing, and if you run into falling back to IPIs because you don't have enough ASIDs to go around, we should 'borrow' some of the ARM64 code -- RISC-V seems to have borrowed very heavily from that as well.