From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754598AbbLKWP0 (ORCPT ); Fri, 11 Dec 2015 17:15:26 -0500 Received: from prod-mail-xrelay05.akamai.com ([23.79.238.179]:49951 "EHLO prod-mail-xrelay05.akamai.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754492AbbLKWPX (ORCPT ); Fri, 11 Dec 2015 17:15:23 -0500 Subject: Re: question about cpusets vs sched_setaffinity() To: Chris Friesen , "linux-kernel@vger.kernel.org >> lkml" References: <5669EEFF.20801@mail.usask.ca> Cc: Tejun Heo , lizefan@huawei.com From: Jason Baron X-Enigmail-Draft-Status: N1110 Message-ID: <566B4AF9.10301@akamai.com> Date: Fri, 11 Dec 2015 17:15:21 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <5669EEFF.20801@mail.usask.ca> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/10/2015 04:30 PM, Chris Friesen wrote: > Hi, > > I've got a question about the interaction between cpusets and > sched_setaffinity(). > > If I put a task into a cpuset and then call sched_setaffinity() on it, > it will be affined to the intersection of the two sets of cpus. (Those > specified on the set, and those specified in the syscall.) > > However, if I then change the cpus in the cpuset the process affinity > will simply be overwritten by the new cpuset affinity. It does not seem > to take into account any restrictions from the original > sched_setaffinity() call. > > Wouldn't it make more sense to affine the process to the intersection > between the new set of cpus from the cpuset, and the current process > affinity? That way if I explicitly masked out certain CPUs in the > original sched_setaffinity() call then they would remain masked out > regardless of changes to the set of cpus assigned to the cpuset. > > Thanks, > Chris > > PS: Not subscribed to the list, please CC me on replies. Hi, This behavior seems a bit odd to me as well - if you've done a sched_setaffinity() call to a subset of the cpus of a cpuset that the task in contained within, any change to the cpuset cpus will wipe away the sched_setaffinity() settings even if they continue to be a subset of the cpuset cpus. To add the behavior you are describing, I think requires another cpumask_t field in the task_struct. Where we could store the last requested mask value for sched_setaffinity() and use that when updating the cpus for a cpuset via an intersection as you described. I think adding a task to a cpuset still should wipe out any sched_setaffinity() settings - but that would depend on the desired semantics here. It would also require a knob so as not to break existing behavior by default. You could also create a child cgroup for the process that you don't want to change and set the cpus on that cgroup instead of using sched_setaffinity(). Then you change the cpus for the parent cgroup and that shouldn't affect the child as long as the child cgroup is a subset. But its not entirely clear to me if that addresses your use-case? Thanks, -Jason