From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E4F3C77B7E for ; Wed, 26 Apr 2023 20:00:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235034AbjDZUAk (ORCPT ); Wed, 26 Apr 2023 16:00:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233537AbjDZUAh (ORCPT ); Wed, 26 Apr 2023 16:00:37 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E360D2 for ; Wed, 26 Apr 2023 13:00:36 -0700 (PDT) From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1682539234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VlKUM3Ard0yvYVkNRouDQadczDOl0h3/V1tyd2kzQE4=; b=XQh5cZc+zBpM0TV56z1mOffywR5isafraX73BycznXr2NdYvMba65wjMtqXnoYcYdUCMYr dMtuSrVzQoU86y3fIbBfaCjhzlo6kbvGhS1Btxe3VZPv50venyz4wElCOfOFkSr9uljlwR PqpqGR91UUlq0sJoHRibFHQ4XFWJgOMUQEQ/Hi01uS+fXtuY/VLAi82Xa5iZOCZCl219D1 fTjmMzus60PAgg9V4E1Q4nfevvzD+L4OBp7sAP9DMJvGjSQ+AiKwSsPLcNFFa2Bn/ZE7qp KdC6LJnVQe6g5BJXQGHTlf/mZ8m/i419Q+nPFvRc9mwfIhzLe/t6hOjAmwYt1g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1682539234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VlKUM3Ard0yvYVkNRouDQadczDOl0h3/V1tyd2kzQE4=; b=OsK3BMW+c0X0idT26Fw8YUOSyWNhRVqCZEobdodKgMeAhmKaW1wqQAttlfvB9bp32ZiBxr SdnATjOa2X8jwPCg== To: Tom Lendacky , Tony Battersby , Dave Hansen , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: "H. Peter Anvin" , Mario Limonciello , "linux-kernel@vger.kernel.org" , Andi Kleen Subject: Re: [PATCH RFC] x86/cpu: fix intermittent lockup on poweroff In-Reply-To: <01a44722-931a-7aff-4f4b-75e78855beb1@amd.com> References: <3817d810-e0f1-8ef8-0bbd-663b919ca49b@cybernetics.com> <87o7nbzn8w.ffs@tglx> <5f8a9cb8-70cf-2a17-cfc4-cb31cb658de4@cybernetics.com> <87y1mey503.ffs@tglx> <01a44722-931a-7aff-4f4b-75e78855beb1@amd.com> Date: Wed, 26 Apr 2023 22:00:33 +0200 Message-ID: <87sfcmxvku.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 26 2023 at 12:51, Tom Lendacky wrote: > On 4/26/23 12:37, Tony Battersby wrote: >>> + /* >>> + * native_stop_other_cpus() will write to @stop_cpus_count after >>> + * observing that it went down to zero, which will invalidate the >>> + * cacheline on this CPU. >>> + */ >>> + atomic_dec(&stop_cpus_count); > > This is probably going to pull in a cache line and cause the problem the > native_wbinvd() is trying to avoid. The comment above this atomic_dec() explains why this is _not_ a problem. Here is the counterpart in native_stop_other_cpus(): >>> @@ -216,6 +219,12 @@ static void native_stop_other_cpus(int w >>> disable_local_APIC(); >>> mcheck_cpu_clear(this_cpu_ptr(&cpu_info)); >>> local_irq_restore(flags); >>> + >>> + /* >>> + * Ensure that the cache line is invalidated on the other CPUs. See >>> + * comment vs. SME in stop_this_cpu(). >>> + */ >>> + atomic_set(&stop_cpus_count, INT_MAX); That happens _after_ all the other CPUs did the atomic_dec() as the control CPU waits for it to become 0. As this makes the cacheline exclusive on the control CPU the dirty cacheline on the CPU which did the last atomic_dec() is invalidated. As the atomic_dec() is obviously serialized via the lock prefix there can be only one dirty copy on some other CPU at the time when the control CPU writes to it. After that the only dirty copy is on the control CPU, no? Thanks, tglx