From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1962E732DE for ; Thu, 28 Sep 2023 14:44:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230376AbjI1Ook (ORCPT ); Thu, 28 Sep 2023 10:44:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230212AbjI1Ooj (ORCPT ); Thu, 28 Sep 2023 10:44:39 -0400 Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1937A19E for ; Thu, 28 Sep 2023 07:44:37 -0700 (PDT) Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-5032a508e74so4555e87.1 for ; Thu, 28 Sep 2023 07:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695912275; x=1696517075; darn=vger.kernel.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=BFwIpXTwL/lUKsUPWcI8rCYf21e0USndOV3VkIfXzr4=; b=4aQV6YeY6iROW554n7rXB5howK8VsvKRNWX8a3DNmBIYMPkbBacwGWbnQob50OxIPN snU2FUQ/wdeFDsL4rDUwnllZDQOasLaGkntz5J5tGQ117b5b2RUsVjhC5MkCLKyq+blv HxRJsgB7cUxb7K80XelNxlktBfEtOg8KpweSQ5XMRRCQ4VqNff4rw0PsMxN8F/YO2zA/ x2SkKBE5lHVqfOJ97KFYJenZ4uMHX+kr0Ovxdrfj2CK3vX0WzrbDxJXK44B5mMOSQvEE qYsnlWUG0FmvWfgfAorDxSohC8+WSWUsneaCdYuWW5ov7u6r1sWj+pVx/h7rN18O92e/ 3kWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695912275; x=1696517075; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BFwIpXTwL/lUKsUPWcI8rCYf21e0USndOV3VkIfXzr4=; b=pddRIOrC0dptMlUsrSUZ1R61MJpUsHdiI/ll9jBXfAMfIaxW25ZBqIdNaNHpfQjpgX 74mSxtSNqfd4jIDVpwykopujpC305Ta8MNNpUAJoc62AFoxZ2h/bpauo07E59GqiHrBr 4TM8FAKh42f6YeLv2C9GOb+14pqtYqBcyMHsZXlUrkYxQh/4zNrVqZj+Q2n0YT5LOgr8 043WPiOxUTZm7KWL7Zk06h18B2nGO0Wch+xOkyzY7UkWF9oBa6vFQOT+HaIFnoc/snhI OtNy8e7DmSjSkaQBw5eDZKrEETIUsgNeZD4JhoW07/QmYf5c6teCFKjZaEa03NG7G3ES WFFw== X-Gm-Message-State: AOJu0YwDnZyLDr5iE6P0mlVTQBDXvNUUwGWnyxOobEuaLfkj9T/aR2tK JG9Un7OPBJ7KOl2gkyjtGWMcocyN3Ateab+fYH6EOg== X-Google-Smtp-Source: AGHT+IF+k5c+rX4zUNgLUDd2nRwl2MXqWoD/9z4V0hFhg8SGKLIhAegXLUkZVGYDQSttL/WplGoCXnRj0bwajDSqRps= X-Received: by 2002:ac2:47f2:0:b0:502:a55e:fec0 with SMTP id b18-20020ac247f2000000b00502a55efec0mr247781lfp.6.1695912274999; Thu, 28 Sep 2023 07:44:34 -0700 (PDT) MIME-Version: 1.0 References: <2c421e36-a749-7dc3-3562-7a8cf256df3c@efficios.com> <20230926205215.472650-1-dvyukov@google.com> <875y3wp6au.fsf@oldenburg.str.redhat.com> <87bkdmznl6.fsf@oldenburg.str.redhat.com> In-Reply-To: <87bkdmznl6.fsf@oldenburg.str.redhat.com> From: Dmitry Vyukov Date: Thu, 28 Sep 2023 07:44:21 -0700 Message-ID: Subject: Re: [RFC PATCH v2 1/4] rseq: Add sched_state field to struct rseq To: Florian Weimer Cc: mathieu.desnoyers@efficios.com, David.Laight@aculab.com, alexander@mihalicyn.com, andrealmeid@igalia.com, boqun.feng@gmail.com, brauner@kernel.org, carlos@redhat.com, ckennelly@google.com, corbet@lwn.net, dave@stgolabs.net, dvhart@infradead.org, goldstein.w.n@gmail.com, hpa@zytor.com, libc-alpha@sourceware.org, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, longman@redhat.com, mingo@redhat.com, paulmck@kernel.org, peterz@infradead.org, pjt@google.com, posk@posk.io, rostedt@goodmis.org, tglx@linutronix.de Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-api@vger.kernel.org On Thu, 28 Sept 2023 at 01:52, Florian Weimer wrote: > > * Dmitry Vyukov: > > > On Tue, 26 Sept 2023 at 21:51, Florian Weimer wrote: > >> > >> * Dmitry Vyukov: > >> > >> > In reality it's a bit more involved since the field is actually 8 > >> > bytes and only partially overlaps with rseq.cpu_id_start (it's an > >> > 8-byte pointer with high 4 bytes overlap rseq.cpu_id_start): > >> > > >> > https://github.com/google/tcmalloc/blob/229908285e216cca8b844c1781bf16b838128d1b/tcmalloc/internal/percpu.h#L101-L165 > >> > >> This does not compose with other rseq users, as noted in the sources: > >> > >> // Note: this makes __rseq_abi.cpu_id_start unusable for its original purpose. > >> > >> For a core library such a malloc replacement, that is a very bad trap. > > > > I agree. I wouldn't do this if there were other options. That's why I > > am looking for official kernel support for this. > > If we would have a separate 8 bytes that are overwritten with 0 when a > > thread is descheduled, that would be perfect. > > That only solves part of the problem because these fields would still > have to be locked to tcmalloc. I think you'd need a rescheduling > counter, then every library could keep their reference values in > library-private thread-local storage. This unfortunatly won't work for tcmalloc. This data is accessed on the very hot path of malloc/free. We need a ready to use pointer in TLS, which is reset by the kernel to 0 (or some user-space specified value). Doing to separate loads for counters in different cache lines would be too expensive. It may be possible to make several libraries use this feature with an array of notifications (see rseq_desched_notif_t in my previous email).