From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8FD5ED149C2 for ; Fri, 25 Oct 2024 16:18:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Subject:Cc:To:From: Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZPD02BSGkv8ElGifQ8Bj9gaTeEhmbG3yYd68KWp/xDo=; b=AmkzvfG5sdaZS1X9kseJcGu5AF N804ueJOOq/iYVZ8QkpdpXSBbNSuXFBLzMiaNBxw18LZyB4A4vvDWWRG0vLVnvjYkOdwpYkRZ0AO3 pM8lVFiS/WE4pJxa2cK2qnwZzMfaUGLx5Wl4+TbQa3SaZdLuTEqfDSENI5s03zGnHwuWZqtXJ7A66 Fwvt3UuINRgc2wrblEVxd5j6zQ5CgLIyadGa9Z9REGIsBJfdU+Xd05hGjA44uxtnLElWo8Z8nK4Bl W5OMtlekdFAwP1OEvVK+6RiuUjmdIdn4iLTwnXtEh52r0J5emtN58NaDX+RDEI2Msa8cbmkn/1+s3 t2UKmUEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t4N11-00000004OLF-2krn; Fri, 25 Oct 2024 16:18:19 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t4Mxq-00000004NUo-1tnH for linux-arm-kernel@lists.infradead.org; Fri, 25 Oct 2024 16:15:03 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 54BC7A42F99; Fri, 25 Oct 2024 16:13:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E1C5C4CEC3; Fri, 25 Oct 2024 16:15:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729872901; bh=xauMgPi9VccclOak+q41k9wam+BeYz0R/HH/+WYNJ20=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ere1JOuJ0Cp0k43C9N+4F8AYElMcYXchbYav/fwlMx9L7Y+J1U0x7ypS3IRS9iOhw /B5XgAhfBHrbG+JY5zrrgWasaga0EU+isSHt0Lk0cPbGeN/bnGGmEDbTPNdB4SFfXw e05ZLxWX2I5HTH9yBLkF94dnnuOaHAlJdqrTt/hlR60AgxwGID6/1AWm1amwT0ubT/ 4x70261adP4tZntRLMYiGe6qScpk+qUl2+M+RUgFzxbw9jI/7zaStmTu+Qx3WfqCvg sq9FwIXdsksXZpQ0dgvSNlcHUtkMzWiR9ICqP687M1/VwwTMPAecqFVyDMTP7aDk8p vbpAPCxkOvF4w== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1t4Mxm-006qyI-Uv; Fri, 25 Oct 2024 17:14:59 +0100 Date: Fri, 25 Oct 2024 17:14:57 +0100 Message-ID: <86h6902m7y.wl-maz@kernel.org> From: Marc Zyngier To: Yu Zhao Cc: Andrew Morton , Catalin Marinas , Muchun Song , Thomas Gleixner , Will Deacon , Douglas Anderson , Mark Rutland , Nanyong Sun , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v1 3/6] irqchip/gic-v3: support SGI broadcast In-Reply-To: References: <20241021042218.746659-1-yuzhao@google.com> <20241021042218.746659-4-yuzhao@google.com> <86a5ew41tp.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: yuzhao@google.com, akpm@linux-foundation.org, catalin.marinas@arm.com, muchun.song@linux.dev, tglx@linutronix.de, will@kernel.org, dianders@chromium.org, mark.rutland@arm.com, sunnanyong@huawei.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241025_091502_674253_F2873578 X-CRM114-Status: GOOD ( 30.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 25 Oct 2024 06:07:45 +0100, Yu Zhao wrote: >=20 > Hi Marc, >=20 > On Tue, Oct 22, 2024 at 9:03=E2=80=AFAM Marc Zyngier wro= te: > > > > On Mon, 21 Oct 2024 05:22:15 +0100, > > Yu Zhao wrote: > > > > > > @@ -1407,6 +1418,13 @@ static void gic_ipi_send_mask(struct irq_data = *d, const struct cpumask *mask) > > > */ > > > dsb(ishst); > > > > > > + cpumask_copy(&broadcast, cpu_present_mask); > > > > Why cpu_present_mask? I'd expect that cpu_online_mask should be the > > correct mask to use -- we don't IPI offline CPUs, in general. >=20 > This is exactly because "we don't IPI offline CPUs, in general", > assuming "we" means the kernel, not GIC. >=20 > My interpretation of what the GIC spec says ("0b1: Interrupts routed > to all PEs in the system, excluding self") is that it broadcasts IPIs to > "cpu_present_mask" (minus the local one). So if the kernel uses > "cpu_online_mask" here, GIC would send IPIs to offline CPUs > (cpu_present_mask ^ cpu_online_mask), which I don't know whether it's > a defined behavior. Offline CPUs are not known to the kernel. Most likely, they are either powered off, or spending quality time in Secure or Realm mode. Either way, this is none of our business. Your approach would make also the kernel perform pretty inconsistently depending on whether CPUs are offline and not. >=20 > But if you actually meant GIC doesn't IPI offline CPUs, then yes, here > the kernel should use "cpu_online_mask". >=20 > > > + cpumask_clear_cpu(smp_processor_id(), &broadcast); > > > + if (cpumask_equal(&broadcast, mask)) { > > > + gic_broadcast_sgi(d->hwirq); > > > + goto done; > > > + } > > > > So the (valid) case where you would IPI *everyone* is not handled as a > > fast path? That seems a missed opportunity. >=20 > You are right: it should handle that case. >=20 > > This also seem an like expensive way to do it. How about something > > like: > > > > int mcnt =3D cpumask_weight(mask); > > int ocnt =3D cpumask_weight(cpu_online_mask); > > if (mcnt =3D=3D ocnt) { > > /* Broadcast to all CPUs including self */ >=20 > Does the comment mean the following two steps? > 1. Broadcasting to everyone else. > 2. Sending to self. Correct. > My understanding of the "Interrupt Routing Mode" is that it can't > broadcast to all CPUs including self, and therefore we need the above > two steps, which still can be a lot faster. Is my understanding > correct? Yes. Thanks, M. --=20 Without deviation from the norm, progress is not possible.