Question about setting affinity in 2.4

Question about setting affinity in 2.4

[Phy Prabab]

Hello everyone,

I am trying to set affinity on a program to make sure I can get the
best use of the cache as possible and to eliminate as much noise as
possible with running my program.  I have tried unsuccessfully to set
affinity using sched_set/getaffinity and the CPU_SET macros.  In
particular, I can not seem to get the process to use affinity I
specify as in:

int main () {
  cpu_set_t mask;
  cpu_set_t setmask;
  long long x;
  double foo;
  printf("Mask set to %08lx\n", sched_getaffinity(0, &mask));
  CPU_SET(1, &mask);
  if (sched_setaffinity(0, &mask) == -1) {
     printf("error setting affinity.");
  }

  printf("Mask is reset to %08lx\n", sched_getaffinity(0, &mask));
  // do some long lasting calculations ....

  return;
}

No mater what I do, the process is never bond to the processor in
question as exposed by this little appy and by cat /proc/<pid>/cpu.
Is this just impossible to do?

Ah, the kernel I am using is RH 2.4.21-37.ELsmp.

TIA!
Phy

Re: Question about setting affinity in 2.4

[Willy Tarreau]

Hi Phy !

On Sun, Feb 18, 2007 at 06:02:17PM -0800, Phy Prabab wrote:
> Hello everyone,
> 
> I am trying to set affinity on a program to make sure I can get the
> best use of the cache as possible and to eliminate as much noise as
> possible with running my program.  I have tried unsuccessfully to set
> affinity using sched_set/getaffinity and the CPU_SET macros.  In
> particular, I can not seem to get the process to use affinity I
> specify as in:
> 
> int main () {
>  cpu_set_t mask;
>  cpu_set_t setmask;
>  long long x;
>  double foo;
>  printf("Mask set to %08lx\n", sched_getaffinity(0, &mask));
>  CPU_SET(1, &mask);
>  if (sched_setaffinity(0, &mask) == -1) {
>     printf("error setting affinity.");
>  }
> 
>  printf("Mask is reset to %08lx\n", sched_getaffinity(0, &mask));
>  // do some long lasting calculations ....
> 
>  return;
> }
> 
> No mater what I do, the process is never bond to the processor in
> question as exposed by this little appy and by cat /proc/<pid>/cpu.
> Is this just impossible to do?
> 
> Ah, the kernel I am using is RH 2.4.21-37.ELsmp.

I don't know for RH kernel, but the affinity syscalls are not in 2.4
mainline. You can apply the patch without much hassle if you want. It's
in Robert Love's directory :

 http://www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity/v2.4/

I believe you'll have to use an -ac patch for RHEL.

> TIA!
> Phy

regards,
Willy

Re: Question about setting affinity in 2.4

[Phy Prabab]

Willy,

Thanks for the heads up and patch location.  It looks like the guys at
RH seemed to have applied a patch to allow one to set affinity,
however, it might be that it is broken.  Guess I will have to contact
the RH people to get an update as to what they did/did not do.

Thanks!
Phy

On 2/18/07, Willy Tarreau <w@1wt.eu> wrote:
> Hi Phy !
>
> On Sun, Feb 18, 2007 at 06:02:17PM -0800, Phy Prabab wrote:
> > Hello everyone,
> >
> > I am trying to set affinity on a program to make sure I can get the
> > best use of the cache as possible and to eliminate as much noise as
> > possible with running my program.  I have tried unsuccessfully to set
> > affinity using sched_set/getaffinity and the CPU_SET macros.  In
> > particular, I can not seem to get the process to use affinity I
> > specify as in:
> >
> > int main () {
> >  cpu_set_t mask;
> >  cpu_set_t setmask;
> >  long long x;
> >  double foo;
> >  printf("Mask set to %08lx\n", sched_getaffinity(0, &mask));
> >  CPU_SET(1, &mask);
> >  if (sched_setaffinity(0, &mask) == -1) {
> >     printf("error setting affinity.");
> >  }
> >
> >  printf("Mask is reset to %08lx\n", sched_getaffinity(0, &mask));
> >  // do some long lasting calculations ....
> >
> >  return;
> > }
> >
> > No mater what I do, the process is never bond to the processor in
> > question as exposed by this little appy and by cat /proc/<pid>/cpu.
> > Is this just impossible to do?
> >
> > Ah, the kernel I am using is RH 2.4.21-37.ELsmp.
>
> I don't know for RH kernel, but the affinity syscalls are not in 2.4
> mainline. You can apply the patch without much hassle if you want. It's
> in Robert Love's directory :
>
>  http://www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity/v2.4/
>
> I believe you'll have to use an -ac patch for RHEL.
>
> > TIA!
> > Phy
>
> regards,
> Willy
>
>

Re: Question about setting affinity in 2.4

[Willy Tarreau]

On Sun, Feb 18, 2007 at 09:14:21PM -0800, Phy Prabab wrote:
> Willy,
> 
> Thanks for the heads up and patch location.  It looks like the guys at
> RH seemed to have applied a patch to allow one to set affinity,
> however, it might be that it is broken.  Guess I will have to contact
> the RH people to get an update as to what they did/did not do.

You should ensure that the syscall has really been added to the syscall
table. Their kernel also has a fully working epoll() implementation, but
the syscall is missing from the table. It's a shame because you have to
add a 10-lines patch to enable it again ! It may be the same problem in
your case.

> Thanks!
> Phy

Regards,
Willy

Re: Question about setting affinity in 2.4

[Arjan van de Ven]

On Sun, 2007-02-18 at 18:02 -0800, Phy Prabab wrote:
> Hello everyone,
> 
> I am trying to set affinity on a program to make sure I can get the
> best use of the cache as possible and to eliminate as much noise as
> possible with running my program.  I have tried unsuccessfully to set
> affinity using sched_set/getaffinity and the CPU_SET macros.  In
> particular, I can not seem to get the process to use affinity I
> specify as in:
> 
> int main () {
>   cpu_set_t mask;
>   cpu_set_t setmask;
>   long long x;
>   double foo;
>   printf("Mask set to %08lx\n", sched_getaffinity(0, &mask));

       int sched_setaffinity(pid_t pid, unsigned int cpusetsize,
                             cpu_set_t *mask);


sched_setaffinity takes 3 not 2 parameters.

Re: Question about setting affinity in 2.4

[Pádraig Brady]

Arjan van de Ven wrote:
> sched_setaffinity takes 3 not 2 parameters.

Yep the interface changed 3 times, hence
it's probably better using the syscall directly.
Search my notes for sched_setaffinity here:
http://www.pixelbeat.org/programming/c_c++_notes.html

Pádraig.