From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D379C77B6F for ; Thu, 13 Apr 2023 01:56:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229506AbjDMB4j (ORCPT ); Wed, 12 Apr 2023 21:56:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229578AbjDMB4i (ORCPT ); Wed, 12 Apr 2023 21:56:38 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D57DD7280 for ; Wed, 12 Apr 2023 18:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681350951; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QUAJoa4WsPFgOaywimIxgJmb0m1Gj0LU/rbMc/oiFgM=; b=WU229dn6x1HEQ+Cy6vOF9aqrodquqVtFR7dG0QYtHGlK2nmpoK0FjHDoBR2BKVDPu3tfwT +PspGC8+YEFsx6cNb1shDKuQ/QvVIbvldty1C9KXoGz5V1UW5uWan1yFCKtpOCRYijmr8E 0yHNvRbOXXt815Wu6ibutrs4McY/pGk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-SMq2uPKiO0excjeVA5Pqsw-1; Wed, 12 Apr 2023 21:55:47 -0400 X-MC-Unique: SMq2uPKiO0excjeVA5Pqsw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C9473810B1C; Thu, 13 Apr 2023 01:55:47 +0000 (UTC) Received: from [10.22.32.168] (unknown [10.22.32.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 668CDC15BB8; Thu, 13 Apr 2023 01:55:46 +0000 (UTC) Message-ID: <9862da55-5f41-24c3-f3bb-4045ccf24b2e@redhat.com> Date: Wed, 12 Apr 2023 21:55:46 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [RFC PATCH 0/5] cgroup/cpuset: A new "isolcpus" paritition Content-Language: en-US To: Tejun Heo Cc: Zefan Li , Johannes Weiner , Jonathan Corbet , Shuah Khan , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Juri Lelli , Valentin Schneider , Frederic Weisbecker References: <20230412153758.3088111-1-longman@redhat.com> <1ce6a073-e573-0c32-c3d8-f67f3d389a28@redhat.com> <1b8d9128-d076-7d37-767d-11d6af314662@redhat.com> From: Waiman Long In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On 4/12/23 21:17, Tejun Heo wrote: > Hello, Waiman. > > On Wed, Apr 12, 2023 at 08:55:55PM -0400, Waiman Long wrote: >>> Sounds a bit contrived. Does it need to be something defined in the root >>> cgroup? >> Yes, because we need to take away the isolated CPUs from the effective cpus >> of the root cgroup. So it needs to start from the root. That is also why we >> have the partition rule that the parent of a partition has to be a partition >> root itself. With the new scheme, we don't need a special cgroup to hold the > I'm following. The root is already a partition root and the cgroupfs control > knobs are owned by the parent, so the root cgroup would own the first level > cgroups' cpuset.cpus.reserve knobs. If the root cgroup wants to assign some > CPUs exclusively to a first level cgroup, it can then set that cgroup's > reserve knob accordingly (or maybe the better name is > cpuset.cpus.exclusive), which will take those CPUs out of the root cgroup's > partition and give them to the first level cgroup. The first level cgroup > then is free to do whatever with those CPUs that now belong exclusively to > the cgroup subtree. I am OK with the cpuset.cpus.reserve name, but not that much with the cpuset.cpus.exclusive name as it can get confused with cgroup v1's cpuset.cpu_exclusive. Of course, I prefer the cpuset.cpus.isolated name a bit more. Once an isolated CPU gets used in an isolated partition, it is exclusive and it can't be used in another isolated partition. Since we will allow users to set cpuset.cpus.reserve to whatever value they want. The distribution of isolated CPUs is only valid if the cpus are present in its parent's cpuset.cpus.reserve and all the way up to the root. It is a bit expensive, but it should be a relatively rare operation. > >> isolated CPUs. The new root cgroup file will be enough to inform the system >> what CPUs will have to be isolated. >> >> My current thinking is that the root's "cpuset.cpus.isolated" will start >> with whatever have been set in the "isolcpus" or "nohz_full" boot command >> line and can be extended from there but not shrank below that as there can >> be additional isolation attributes with those isolated CPUs. > I'm not sure we wanna tie with those automatically. I think it'd be > confusing than helpful. Yes, I am fine with taking this off for now. Cheers, Longman