From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from va-1-111.ptr.blmpb.com (va-1-111.ptr.blmpb.com [209.127.230.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9310315B998 for ; Sat, 27 Jun 2026 00:52:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.127.230.111 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782521570; cv=none; b=r19rytKh96rS71cKZ4uTr6LvOVJc7llejDBhvduZJUjgeYoNA16zc8mkqsPieYzB+mTBgZrdOzaAHE/5s4whoLuF2tgGxUay175cHau6udoj7r65XYYSfXvZgjvLyHAGSTdKLftxpG7UC/kVg8rudgnbez+UZ5zRoj+4ipHw5/k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782521570; c=relaxed/simple; bh=DDuZOmbdpRnLnYKkj3sB9CeOJihBDMekQDE1ujVcezo=; h=Mime-Version:To:Cc:Content-Type:From:In-Reply-To:Subject:Date: Message-Id:References; b=V4hBdUReYV9NepzJsn8Uxszumk5AiEudnCORyuTQKD2mheFHhk7GF3Cz6Jr3a4NhRfTP9EhQExU/bYcwmopHuHjF3p++85QaFRpitYsM6mEKLm1OGZBA+hHppC8YG+W6NyFbfPUNm0n9bd56/dt1yfekKFMS9oZmEdFlyQIuhoo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bN9A+Fgl; arc=none smtp.client-ip=209.127.230.111 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bN9A+Fgl" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1782521562; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=v1/zBZ/ukur5mqIEYVW5Ds4mXCgJ1mWU9NGviey2BHc=; b=bN9A+FglQ0rwmp0Rg0X0nGC/kmnw+5G7e6NA5LMmuRTkb42COhpfFc+5ndNsCtnefcl3Bx mD+2BUp4zwZ/XjDbp0Q/yMwxfQCEfeYNMCQaWMdcwnA7dBx6zDy2G8Rw5pmKURrOuOGbcC TooOthspWoO1li45sDnlSancn5UCxe4yH2bWfbr9DS3geAFOUgDYFSERNSVSGCgpMeXOTQ z+6WLqqY6iznhrJLXGp7cNXamhWWJDW/ZmzkI/WuFnLh7/bQ8UWQxwAmm+JR4U5juVhsUv az8eCtJplGP/T8rbFFCBo66B4cYWmO/3XsL4tgk8ZaOnvsFgwxYoVvAMjrGZFA== Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 To: "Thomas Gleixner" , , , , , , , , , , , , , Cc: Content-Type: text/plain; charset=UTF-8 From: "Chuyi Zhou" User-Agent: Mozilla Thunderbird X-Lms-Return-Path: X-Original-From: Chuyi Zhou In-Reply-To: <87a4shi0ix.ffs@fw13> Subject: Re: [PATCH v8 04/14] smp: Use task-local IPI cpumask in smp_call_function_many_cond() Date: Sat, 27 Jun 2026 08:52:20 +0800 Message-Id: Content-Transfer-Encoding: 7bit References: <20260616111127.966468-1-zhouchuyi@bytedance.com> <20260616111127.966468-5-zhouchuyi@bytedance.com> <871pdtjryo.ffs@fw13> <8d3587e6-e3a1-40f5-ba0d-65583a2f1ecb@bytedance.com> <87a4shi0ix.ffs@fw13> On 2026-06-27 3:07 a.m., Thomas Gleixner wrote: > On Fri, Jun 26 2026 at 23:47, Chuyi Zhou wrote: >> On 2026-06-26 10:29 p.m., Thomas Gleixner wrote: >>>> - err = scs_prepare(tsk, node); >>>> + err = smp_task_ipi_mask_alloc(tsk); >>> >>> Hrm. So we unconditionally allocate another per task CPU mask. How many >>> task actually utilize it? >>> >>> We keep making task_struct and the related things larger every other >>> release without actually looking at the resulting overall memory >>> consumption. >>> >> >> Thanks, this is a fair concern. >> >> The task-local cpumask approach came from the earlier discussion with >> Sebastian and Nadav. The problem we tried to solve there was the >> lifetime of the wait mask once the later patch re-enables preemption >> before csd_lock_wait(). At that point the wait mask can no longer be the >> per-CPU cfd->cpumask: the task may be preempted or migrate while it is >> still iterating the mask, and another task running on the original CPU >> could enter smp_call_function_many_cond() and reuse that per-CPU mask. >> >> I agree that the memory cost needs to be called out explicitly. The >> current implementation trades one task-local cpumask for a stable mask >> lifetime and avoids adding allocation/failure handling to the generic >> IPI path. >> >> I considered avoiding the fork-time allocation, but the alternatives do >> not look straightforward: >> >> - stack storage is not suitable for large NR_CPUS/CPUMASK_OFFSTACK >> configurations; >> >> - per-CPU storage is exactly what becomes unsafe once the wait is made >> preemptible; >> >> - allocating the mask in smp_call_function_many_cond() would put an >> allocation in the generic IPI path. It also cannot rely on a sleeping >> allocation because this function is entered from contexts which have >> historically only required preemption to be disabled. Using GFP_ATOMIC >> would need a failure/fallback path, in which case the latency >> improvement becomes opportunistic rather than guaranteed. >> >> For the motivating x86 TLB flush paths, the users are also not a small >> static set of tasks. Ordinary tasks can hit this through exit, unmap, >> reclaim, etc., so I do not see a clean way to allocate this only for a >> pre-identifiable subset of tasks. > > I understand that, but this all wants to be spelled out in the change > log and explained. Understood. Thanks for going through the series and for the detailed review. I will fold this into the changelog and spell out: - why the wait mask needs task-local lifetime once csd_lock_wait() becomes preemptible; - why per-CPU, stack, and in-call allocation are not good fits here; - why this is not limited to a small, pre-identifiable set of tasks. On x86, ordinary tasks can hit smp_call_function_many_cond() through TLB flush paths such as exit, unmap and reclaim; - the memory cost, including the inline case for small CPU counts and the cpumask_size() allocation on larger systems. I will also address your comments on the other patches in the next version. Thanks