* numactl using it's own affinity to determine CPU set @ 2012-10-22 19:35 Jon Stanley 2012-10-22 19:41 ` Andi Kleen 0 siblings, 1 reply; 6+ messages in thread From: Jon Stanley @ 2012-10-22 19:35 UTC (permalink / raw) To: linux-numa, pholasek If you set the affinity of your shell to be a subset of the available CPUs, and then attempt to use numactl to bind to something that is not in that affinity mask, the attempt will fail. I thought that this was fixed in the current numactl 2.0.8, but it doesn't seem to be. A simple reproducer on a 2-node 8-core system is as such: $ numactl --physcpubind=5 ls <see directory goodness> $ taskset -p f $$ $ numactl --physcpubind=5 ls libnuma: Warning: cpu argument 5 is out of range <snip> Let me know if more info is required. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: numactl using it's own affinity to determine CPU set 2012-10-22 19:35 numactl using it's own affinity to determine CPU set Jon Stanley @ 2012-10-22 19:41 ` Andi Kleen 2012-10-22 20:12 ` Jon Stanley 0 siblings, 1 reply; 6+ messages in thread From: Andi Kleen @ 2012-10-22 19:41 UTC (permalink / raw) To: Jon Stanley; +Cc: linux-numa, pholasek On Mon, Oct 22, 2012 at 03:35:42PM -0400, Jon Stanley wrote: > If you set the affinity of your shell to be a subset of the available > CPUs, and then attempt to use numactl to bind to something that is not > in that affinity mask, the attempt will fail. I thought that this was > fixed in the current numactl 2.0.8, but it doesn't seem to be. > > A simple reproducer on a 2-node 8-core system is as such: > > $ numactl --physcpubind=5 ls > <see directory goodness> > $ taskset -p f $$ > $ numactl --physcpubind=5 ls > libnuma: Warning: cpu argument 5 is out of range > <snip> That was done intentional at some point to handle cpusets. numactl 1.0 or so didn't have that problem. However it's unclear if it's really bug. Presumably could have an option to override. -Andi ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: numactl using it's own affinity to determine CPU set 2012-10-22 19:41 ` Andi Kleen @ 2012-10-22 20:12 ` Jon Stanley 2012-10-23 12:21 ` Petr Holasek 0 siblings, 1 reply; 6+ messages in thread From: Jon Stanley @ 2012-10-22 20:12 UTC (permalink / raw) To: Andi Kleen; +Cc: linux-numa, pholasek On Mon, Oct 22, 2012 at 3:41 PM, Andi Kleen <andi@firstfloor.org> wrote: > That was done intentional at some point to handle cpusets. > numactl 1.0 or so didn't have that problem. I'll happily admit to not being an expert here :) However, I thought that the concept of cpusets was that if you attempted to set affinity outside of your cpuset, that would silently fail. I assume that the reason that libnuma didn't want to set outside of it's own affinity mask is to avoid such a silent failure? Also, I question the Real World(TM) prevalence of cpusets. > However it's unclear if it's really bug. Not sure if I'd call it a bug since it was intentional, but sure would call it an unexpected change in behavior :) ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: numactl using it's own affinity to determine CPU set 2012-10-22 20:12 ` Jon Stanley @ 2012-10-23 12:21 ` Petr Holasek 2012-10-23 14:19 ` Cliff Wickman 0 siblings, 1 reply; 6+ messages in thread From: Petr Holasek @ 2012-10-23 12:21 UTC (permalink / raw) To: Jon Stanley; +Cc: Andi Kleen, linux-numa, Cliff Wickman Hi everyone, On Mon, 22 Oct 2012, Jon Stanley wrote: > On Mon, Oct 22, 2012 at 3:41 PM, Andi Kleen <andi@firstfloor.org> wrote: > > That was done intentional at some point to handle cpusets. > > numactl 1.0 or so didn't have that problem. > > I'll happily admit to not being an expert here :) I've added new interface for parsing cpusets/nodesets with _all() suffix in 2.0.8-rc5 libnuma, but numactl utility still uses old ones with cpuset awareness. I didn't want to dig into some libnuma internals and change behaviour of dependent utils in such way. > > However, I thought that the concept of cpusets was that if you > attempted to set affinity outside of your cpuset, that would silently > fail. I assume that the reason that libnuma didn't want to set outside > of it's own affinity mask is to avoid such a silent failure? > > Also, I question the Real World(TM) prevalence of cpusets. > > > However it's unclear if it's really bug. > > Not sure if I'd call it a bug since it was intentional, but sure would > call it an unexpected change in behavior :) I like the idea of some overriding option proposed by Andi. btw, Cliff, can you remember what was the reason for this change in some version after 1.0? Petr ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: numactl using it's own affinity to determine CPU set 2012-10-23 12:21 ` Petr Holasek @ 2012-10-23 14:19 ` Cliff Wickman 2012-10-23 21:17 ` Petr Holasek 0 siblings, 1 reply; 6+ messages in thread From: Cliff Wickman @ 2012-10-23 14:19 UTC (permalink / raw) To: Petr Holasek; +Cc: Jon Stanley, Andi Kleen, linux-numa Hi Petr, On Tue, Oct 23, 2012 at 02:21:40PM +0200, Petr Holasek wrote: > Hi everyone, > > On Mon, 22 Oct 2012, Jon Stanley wrote: > > On Mon, Oct 22, 2012 at 3:41 PM, Andi Kleen <andi@firstfloor.org> wrote: > > > That was done intentional at some point to handle cpusets. > > > numactl 1.0 or so didn't have that problem. > > > > I'll happily admit to not being an expert here :) > > I've added new interface for parsing cpusets/nodesets with _all() suffix > in 2.0.8-rc5 libnuma, but numactl utility still uses old ones with cpuset > awareness. I didn't want to dig into some libnuma internals and change > behaviour of dependent utils in such way. > > > > > However, I thought that the concept of cpusets was that if you > > attempted to set affinity outside of your cpuset, that would silently > > fail. I assume that the reason that libnuma didn't want to set outside > > of it's own affinity mask is to avoid such a silent failure? > > > > Also, I question the Real World(TM) prevalence of cpusets. > > > > > However it's unclear if it's really bug. > > > > Not sure if I'd call it a bug since it was intentional, but sure would > > call it an unexpected change in behavior :) > > I like the idea of some overriding option proposed by Andi. > > btw, Cliff, can you remember what was the reason for this change in some > version after 1.0? > > Petr Memory nodes allowed was made cpuset-aware by the below patch. I'm not quite sure if this is the change you are asking about. patch 0804_ls_patch2 From: Lee Schermerhorn <lee.schermerhorn@hp.com> Date: Wed, 02 Apr 2008 14:34:37 -0400 Depends-on: MPOL_F_MEMS_ALLOWED kernel patch Provide libnuma API for MPOL_F_MEMS_ALLOWED flag. Return nodes allowed by the application's current cpuset context via new API numa_get_mems_allowed(). +.BR numa_get_mems_allowed() +returns the mask of nodes from which the process is allowed to allocate +memory in it's current cpuset context. -Cliff -- Cliff Wickman SGI cpw@sgi.com (651) 683-3824 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: numactl using it's own affinity to determine CPU set 2012-10-23 14:19 ` Cliff Wickman @ 2012-10-23 21:17 ` Petr Holasek 0 siblings, 0 replies; 6+ messages in thread From: Petr Holasek @ 2012-10-23 21:17 UTC (permalink / raw) To: Cliff Wickman; +Cc: Jon Stanley, Andi Kleen, linux-numa On Tue, 23 Oct 2012, Cliff Wickman wrote: > Hi Petr, > > On Tue, Oct 23, 2012 at 02:21:40PM +0200, Petr Holasek wrote: > > btw, Cliff, can you remember what was the reason for this change in some > > version after 1.0? > > > > Petr > > Memory nodes allowed was made cpuset-aware by the below patch. > I'm not quite sure if this is the change you are asking about. > > patch 0804_ls_patch2 > From: Lee Schermerhorn <lee.schermerhorn@hp.com> > Date: Wed, 02 Apr 2008 14:34:37 -0400 > > Depends-on: MPOL_F_MEMS_ALLOWED kernel patch > Provide libnuma API for MPOL_F_MEMS_ALLOWED flag. > Return nodes allowed by the application's current cpuset context > via new API numa_get_mems_allowed(). > > +.BR numa_get_mems_allowed() > +returns the mask of nodes from which the process is allowed to allocate > +memory in it's current cpuset context. Thank you! I'll check it. Petr ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-10-23 21:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-10-22 19:35 numactl using it's own affinity to determine CPU set Jon Stanley 2012-10-22 19:41 ` Andi Kleen 2012-10-22 20:12 ` Jon Stanley 2012-10-23 12:21 ` Petr Holasek 2012-10-23 14:19 ` Cliff Wickman 2012-10-23 21:17 ` Petr Holasek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).