From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D865AC77B78 for ; Thu, 4 May 2023 03:02:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229553AbjEDDCc (ORCPT ); Wed, 3 May 2023 23:02:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229612AbjEDDCa (ORCPT ); Wed, 3 May 2023 23:02:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C3C110F4 for ; Wed, 3 May 2023 20:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683169298; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zwXrGnlynmWthuhj861VrvHW2/WpagpsgRCBG9/mDP0=; b=emxXwK1SHSsEENHYdEZ68boZ4nMbm7ltshhIrCxsGV5caqvDNqxdDEWvEhKwOpvewyGBQW hh4v3+jCMVTtykRr2ZKu32KDxRNDxQMZKlx7GJAelrLkXGQfEaa9eNLL834uvLPVQru9OF TCWdWF7KDwkCUvPqoA3jTugQH7QCBRk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-322-Dc2ExmcBNOecvx6fkKjDEw-1; Wed, 03 May 2023 23:01:37 -0400 X-MC-Unique: Dc2ExmcBNOecvx6fkKjDEw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2F6C21C08796; Thu, 4 May 2023 03:01:37 +0000 (UTC) Received: from [10.22.17.228] (unknown [10.22.17.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4186740C2064; Thu, 4 May 2023 03:01:36 +0000 (UTC) Message-ID: Date: Wed, 3 May 2023 23:01:36 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [RFC PATCH 0/5] cgroup/cpuset: A new "isolcpus" paritition Content-Language: en-US To: =?UTF-8?Q?Michal_Koutn=c3=bd?= Cc: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Shuah Khan , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Juri Lelli , Valentin Schneider , Frederic Weisbecker References: <9862da55-5f41-24c3-f3bb-4045ccf24b2e@redhat.com> <226cb2da-e800-6531-4e57-cbf991022477@redhat.com> <60ec12dc-943c-b8f0-8b6f-97c5d332144c@redhat.com> <46d26abf-a725-b924-47fa-4419b20bbc02@redhat.com> From: Waiman Long In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On 5/2/23 18:27, Michal Koutný wrote: > On Tue, May 02, 2023 at 05:26:17PM -0400, Waiman Long wrote: >> In the new scheme, the available cpus are still directly passed down to a >> descendant cgroup. However, isolated CPUs (or more generally CPUs dedicated >> to a partition) have to be exclusive. So what the cpuset.cpus.reserve does >> is to identify those exclusive CPUs that can be excluded from the >> effective_cpus of the parent cgroups before they are claimed by a child >> partition. Currently this is done automatically when a child partition is >> created off a parent partition root. The new scheme will break it into 2 >> separate steps without the requirement that the parent of a partition has to >> be a partition root itself. > new scheme > 1st step: > echo C >p/cpuset.cpus.reserve > # p/cpuset.cpus.effective == A-C (1) > 2nd step (claim): > echo C' >p/c/cpuset.cpus # C'⊆C > echo root >p/c/cpuset.cpus.partition It is something like that. However, the current scheme of automatic reservation is also supported, i.e. cpuset.cpus.reserve will be set automatically when the child cgroup becomes a valid partition as long as the cpuset.cpus.reserve file is not written to. This is for backward compatibility. Once it is written to, automatic mode will end and users have to manually set it afterward. > > current scheme > 1st step (configure): > echo C >p/c/cpuset.cpus > 2nd step (reserve & claim): > echo root >p/c/cpuset.cpus.partition > # p/cpuset.cpus.effective == A-C (2) > > As long as p/c is unpopulated, (1) and (2) are equal situations. > Why is the (different) two step procedure needed? > > Also the relaxation of requirement of a parent being a partition > confuses me -- if the parent is not a partition, i.e. it has no > exclusive ownership of CPUs but it can still "give" it to children -- is > child partition meant to be exclusive? (IOW can parent siblings reserve > some same CPUs?) A valid partition root has exclusive ownership of its CPUs. That is a rule that won't be changed. As a result, an incoming partition root cannot claim CPUs that have been allocated to another partition. To simplify thing, transition to a valid partition root is not possible if any of the CPUs in its cpuset.cpus are not in the cpuset.cpus.reserve of its ancestor or have been allocated to another partition. The partition root simply becomes invalid. The parent can virtually give the reserved CPUs from the root down the hierarchy and a child can claim them once it becomes a partition root. In manual mode, we need to check all the way up the hierarchy to the root to figure out what CPUs in cpuset.cpus.reserve are valid. It has higher overhead, but enabling partition is not a fast operation anyway. Cheers, Longman