From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755100AbbLKXfC (ORCPT ); Fri, 11 Dec 2015 18:35:02 -0500 Received: from smtp-out-so.shaw.ca ([64.59.136.137]:53337 "EHLO smtp-out-so.shaw.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753536AbbLKXfA (ORCPT ); Fri, 11 Dec 2015 18:35:00 -0500 X-Greylist: delayed 487 seconds by postgrey-1.27 at vger.kernel.org; Fri, 11 Dec 2015 18:35:00 EST X-Authority-Analysis: v=2.1 cv=IaYnITea c=1 sm=1 tr=0 a=b4X6fhI4x5MOnsgcuOa5CA==:117 a=b4X6fhI4x5MOnsgcuOa5CA==:17 a=IkcTkHD0fZMA:10 a=DQpvJtcpFZ2zUkm0xJAA:9 a=QEXdDO2ut3YA:10 Message-ID: <566B5BB8.3050505@mail.usask.ca> Date: Fri, 11 Dec 2015 17:26:48 -0600 From: Chris Friesen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Jason Baron , "linux-kernel@vger.kernel.org >> lkml" CC: Tejun Heo , lizefan@huawei.com Subject: Re: question about cpusets vs sched_setaffinity() References: <5669EEFF.20801@mail.usask.ca> <566B4AF9.10301@akamai.com> In-Reply-To: <566B4AF9.10301@akamai.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfEhdcT7EKgyl9tn0s0Ytr1fyymhEaVKN9uXuffrsl3ooeFtLI4iV+TOlVaI4VHpt+ha2r9oVsVdCTn9PPk6JLhe6PtuBh7PhZbd0Uh5YRcrItZe/g3E0 zG7cc9AyVpuyB3eGguWJLtzMY96K5qQc2EwV9uOrQeDiUpMQRTSlJPWovnzVdRKz11bRBcNICfx/fJdkwlsWpEWuzzUZTgm6j4Qm5bLq7hSu/WtUTMcohrPI 4+kZAs5a1ZMOvGasarB4syhmw+bYjB1z68vDgccZJfM= Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/11/2015 04:15 PM, Jason Baron wrote: > On 12/10/2015 04:30 PM, Chris Friesen wrote: >> If I put a task into a cpuset and then call sched_setaffinity() on it, >> it will be affined to the intersection of the two sets of cpus. (Those >> specified on the set, and those specified in the syscall.) >> >> However, if I then change the cpus in the cpuset the process affinity >> will simply be overwritten by the new cpuset affinity. It does not seem >> to take into account any restrictions from the original >> sched_setaffinity() call. >> >> Wouldn't it make more sense to affine the process to the intersection >> between the new set of cpus from the cpuset, and the current process >> affinity? That way if I explicitly masked out certain CPUs in the >> original sched_setaffinity() call then they would remain masked out >> regardless of changes to the set of cpus assigned to the cpuset. > To add the behavior you are describing, I think requires another > cpumask_t field in the task_struct. Where we could store the last > requested mask value for sched_setaffinity() and use that when updating > the cpus for a cpuset via an intersection as you described. I think > adding a task to a cpuset still should wipe out any sched_setaffinity() > settings - but that would depend on the desired semantics here. It would > also require a knob so as not to break existing behavior by default. Agreed, the additional field in the task_struct makes sense. Personally I don't think that adding a task to a cpuset should wipe out any previously-set affinity, I think it should take the intersection for that case as well. In this environment it might make sense to have separate queries to return the requested and actual affinity. > You could also create a child cgroup for the process that you don't want > to change and set the cpus on that cgroup instead of using > sched_setaffinity(). Then you change the cpus for the parent cgroup and > that shouldn't affect the child as long as the child cgroup is a subset. > But its not entirely clear to me if that addresses your use-case? I ended up doing something like this where I had a top-level cpuset and a number of child cpusets, each with an exclusive subset of the CPUs assigned to it. But it meant that I needed more complicated code to figure out which tasks needed to go into which child cpusets, and more complicated code to handle removing a CPU from the top-level cpuset (since you have to remove it from any children first). Chris