[LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Xiaoguang Wang]

Hi,

When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation fault.
Below is the possible reason.

Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA is an old
distribution, which many people may not use it now, thanks!


Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system call,
the corresponding code is below(The version of glibc I used is glibc-2.5-20061008T1257-RHEL5.11Beta):
I delete some code just for simple.

#######################################################################################

/* Size definition for CPU sets.  */
# define __CPU_SETSIZE  1024
# define __NCPUBITS     (8 * sizeof (__cpu_mask))

/* Type for array elements in 'cpu_set'.  */
typedef unsigned long int __cpu_mask;

/* Basic access functions.  */
# define __CPUELT(cpu)  ((cpu) / __NCPUBITS)

/* Data structure to describe CPU mask.  */
typedef struct
{
  __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
} cpu_set_t;


int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t *cpuset)
{
  if (__builtin_expect (__kernel_cpumask_size == 0, 0))
    {
      int res;

      while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
                                     psize, p),
             INTERNAL_SYSCALL_ERROR_P (res, err)
             && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
     	....

      __kernel_cpumask_size = res;
    }

  /* We now know the size of the kernel cpumask_t.  Make sure the user
     does not request to set a bit beyond that.  */
  for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
    if (((char *) cpuset)[cnt] != '\0')
      {
        /* Found a nonzero byte.  This means the user request cannot be
           fulfilled.  */
        __set_errno (EINVAL);
        return -1;
      }

  return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
}
#######################################################################################

Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
we define one.
#######################################################################################
#ifndef CPU_ALLOC_SIZE
#define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
#endif
#######################################################################################

Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we test EFAULT for sched_setaffinity(2),
the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it first call
raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In RHEL5.10GA,
this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw sched_getaffinity(2) will return 32.
In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that we're testing 
EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate segmentation fault in glibc code,
so this case exits abnormally

As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's because
sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t), which totally depends on CONFIG_NR_CPUS.
In newer kernel, sched_getaffinity(2) returns the smaller one between min_t(size_t, len, cpumask_size()), 
here len is the value passed to sched_getaffinity as cpusetsize, cpumask_size() is the max allowed length.
so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the segmentation fault won't occur.

So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw sched_getaffinity as a workaround in older kernel . See below code:
#########################################################################################

        int ret;
        cpu_set_t cst;

        memset(&cst, 0, sizeof(cst));

        ret = syscall(__NR_sched_getaffinity, getpid(),
                      sizeof(cpu_set_t), &cst);
        if (ret < 0) {
                fprintf(stderr, "sched_getaffinity failed: %s\n",
                        strerror(errno));
                return 1;
        } else {
                printf("length of bit mask the kernel uses to represent the CPU"
                       ": %d\n", ret);
        }
#########################################################################################

Regards,
Xiaoguang Wang

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Stanislav Kholmanskikh]

On 07/25/2014 09:38 AM, Xiaoguang Wang wrote:
> Hi,
>
> When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation fault.
> Below is the possible reason.

Hi!

I though I'd checked this test case on OL5 before submission. Let me 
check it again.

>
> Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA is an old
> distribution, which many people may not use it now, thanks!
>
>
> Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system call,
> the corresponding code is below(The version of glibc I used is glibc-2.5-20061008T1257-RHEL5.11Beta):
> I delete some code just for simple.
>
> #######################################################################################
>
> /* Size definition for CPU sets.  */
> # define __CPU_SETSIZE  1024
> # define __NCPUBITS     (8 * sizeof (__cpu_mask))
>
> /* Type for array elements in 'cpu_set'.  */
> typedef unsigned long int __cpu_mask;
>
> /* Basic access functions.  */
> # define __CPUELT(cpu)  ((cpu) / __NCPUBITS)
>
> /* Data structure to describe CPU mask.  */
> typedef struct
> {
>    __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
> } cpu_set_t;
>
>
> int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t *cpuset)
> {
>    if (__builtin_expect (__kernel_cpumask_size == 0, 0))
>      {
>        int res;
>
>        while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
>                                       psize, p),
>               INTERNAL_SYSCALL_ERROR_P (res, err)
>               && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
>       	....
>
>        __kernel_cpumask_size = res;
>      }
>
>    /* We now know the size of the kernel cpumask_t.  Make sure the user
>       does not request to set a bit beyond that.  */
>    for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
>      if (((char *) cpuset)[cnt] != '\0')
>        {
>          /* Found a nonzero byte.  This means the user request cannot be
>             fulfilled.  */
>          __set_errno (EINVAL);
>          return -1;
>        }
>
>    return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
> }
> #######################################################################################
>
> Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
> we define one.
> #######################################################################################
> #ifndef CPU_ALLOC_SIZE
> #define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
> #endif
> #######################################################################################
>
> Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we test EFAULT for sched_setaffinity(2),
> the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it first call
> raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In RHEL5.10GA,
> this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw sched_getaffinity(2) will return 32.
> In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that we're testing
> EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate segmentation fault in glibc code,
> so this case exits abnormally
>
> As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's because
> sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t), which totally depends on CONFIG_NR_CPUS.
> In newer kernel, sched_getaffinity(2) returns the smaller one between min_t(size_t, len, cpumask_size()),
> here len is the value passed to sched_getaffinity as cpusetsize, cpumask_size() is the max allowed length.
> so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the segmentation fault won't occur.
>
> So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
> refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw sched_getaffinity as a workaround in older kernel . See below code:
> #########################################################################################
>
>          int ret;
>          cpu_set_t cst;
>
>          memset(&cst, 0, sizeof(cst));
>
>          ret = syscall(__NR_sched_getaffinity, getpid(),
>                        sizeof(cpu_set_t), &cst);
>          if (ret < 0) {
>                  fprintf(stderr, "sched_getaffinity failed: %s\n",
>                          strerror(errno));
>                  return 1;
>          } else {
>                  printf("length of bit mask the kernel uses to represent the CPU"
>                         ": %d\n", ret);
>          }
> #########################################################################################
>
> Regards,
> Xiaoguang Wang
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Ltp-list mailing list
> Ltp-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list
>

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Jan Stancek]

----- Original Message -----
> From: "Xiaoguang Wang" <wangxg.fnst@cn.fujitsu.com>
> To: "LTP" <ltp-list@lists.sourceforge.net>
> Cc: "Jan Stancek" <jstancek@redhat.com>
> Sent: Friday, 25 July, 2014 7:38:05 AM
> Subject: Segmentation fault when running sched_setaffinity01  in RHEL5.10GA
> 
> Hi,
> 
> When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation
> fault.
> Below is the possible reason.
> 
> Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA
> is an old
> distribution, which many people may not use it now, thanks!

Hi,

RHEL5.10 GA was January 2013, it's an older codebase, but release is quite recent.

I ran this on RHEL5.3:

(gdb) r
Starting program: /root/ltp/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01 
Detaching after fork from child process 7692.

Program received signal SIGSEGV, Segmentation fault.
__sched_setaffinity_new (pid=<value optimized out>, cpusetsize=<value optimized out>, cpuset=<value optimized out>)
    at ../sysdeps/unix/sysv/linux/sched_setaffinity.c:62
62	    if (((char *) cpuset)[cnt] != '\0')
(gdb) list
57	    }
58	
59	  /* We now know the size of the kernel cpumask_t.  Make sure the user
60	     does not request to set a bit beyond that.  */
61	  for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
62	    if (((char *) cpuset)[cnt] != '\0')
63	      {
64	        /* Found a nonzero byte.  This means the user request cannot be
65		   fulfilled.  */
66		__set_errno (EINVAL);
(gdb) p __kernel_cpumask_size
$1 = 32
(gdb) p cpusetsize
$2 = <value optimized out>


Looking at sources of glibc-2.5 (RHEL5.10) and glibc-2.17 (RHEL7), code leading
up to this place looks identical. So, I agree with your findings. I see 2 options:

1) find cpuset size in same way as glibc does it

2) call syscall directly (and don't use glibc wrapper):

diff --git a/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c b/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
index 0ac4478..0c0488f 100644
--- a/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
+++ b/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
@@ -42,6 +42,7 @@
 #include "usctest.h"
 #include "safe_macros.h"
 #include "sched_setaffinity.h"
+#include "linux_syscall_numbers.h"
 
 char *TCID = "sched_setaffinity01";
 
@@ -151,7 +152,7 @@ int main(int argc, char *argv[])
        for (lc = 0; TEST_LOOPING(lc); lc++) {
                tst_count = 0;
                for (i = 0; i < TST_TOTAL; i++) {
-                       TEST(sched_setaffinity(*(test_cases[i].pid),
+                       TEST(ltp_syscall(__NR_sched_setaffinity, *(test_cases[i].pid),
                                                *(test_cases[i].mask_size),
                                                *(test_cases[i].mask)));


# ./sched_setaffinity01 
sched_setaffinity01    1  TPASS  :  expected failure with 'Bad address'
sched_setaffinity01    2  TPASS  :  expected failure with 'Invalid argument'
sched_setaffinity01    3  TPASS  :  expected failure with 'No such process'
sched_setaffinity01    4  TPASS  :  expected failure with 'Operation not permitted'

# uname -r
2.6.18-128.37.1.el5

Regards,
Jan

> 
> 
> Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system
> call,
> the corresponding code is below(The version of glibc I used is
> glibc-2.5-20061008T1257-RHEL5.11Beta):
> I delete some code just for simple.
> 
> #######################################################################################
> 
> /* Size definition for CPU sets.  */
> # define __CPU_SETSIZE  1024
> # define __NCPUBITS     (8 * sizeof (__cpu_mask))
> 
> /* Type for array elements in 'cpu_set'.  */
> typedef unsigned long int __cpu_mask;
> 
> /* Basic access functions.  */
> # define __CPUELT(cpu)  ((cpu) / __NCPUBITS)
> 
> /* Data structure to describe CPU mask.  */
> typedef struct
> {
>   __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
> } cpu_set_t;
> 
> 
> int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t
> *cpuset)
> {
>   if (__builtin_expect (__kernel_cpumask_size == 0, 0))
>     {
>       int res;
> 
>       while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
>                                      psize, p),
>              INTERNAL_SYSCALL_ERROR_P (res, err)
>              && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
>      	....
> 
>       __kernel_cpumask_size = res;
>     }
> 
>   /* We now know the size of the kernel cpumask_t.  Make sure the user
>      does not request to set a bit beyond that.  */
>   for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
>     if (((char *) cpuset)[cnt] != '\0')
>       {
>         /* Found a nonzero byte.  This means the user request cannot be
>            fulfilled.  */
>         __set_errno (EINVAL);
>         return -1;
>       }
> 
>   return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
> }
> #######################################################################################
> 
> Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp
> testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
> we define one.
> #######################################################################################
> #ifndef CPU_ALLOC_SIZE
> #define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
> #endif
> #######################################################################################
> 
> Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we
> test EFAULT for sched_setaffinity(2),
> the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it
> first call
> raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In
> RHEL5.10GA,
> this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw
> sched_getaffinity(2) will return 32.
> In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that
> we're testing
> EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate
> segmentation fault in glibc code,
> so this case exits abnormally
> 
> As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's
> because
> sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t),
> which totally depends on CONFIG_NR_CPUS.
> In newer kernel, sched_getaffinity(2) returns the smaller one between
> min_t(size_t, len, cpumask_size()),
> here len is the value passed to sched_getaffinity as cpusetsize,
> cpumask_size() is the max allowed length.
> so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the
> segmentation fault won't occur.
> 
> So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in
> testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
> refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw
> sched_getaffinity as a workaround in older kernel . See below code:
> #########################################################################################
> 
>         int ret;
>         cpu_set_t cst;
> 
>         memset(&cst, 0, sizeof(cst));
> 
>         ret = syscall(__NR_sched_getaffinity, getpid(),
>                       sizeof(cpu_set_t), &cst);
>         if (ret < 0) {
>                 fprintf(stderr, "sched_getaffinity failed: %s\n",
>                         strerror(errno));
>                 return 1;
>         } else {
>                 printf("length of bit mask the kernel uses to represent the
>                 CPU"
>                        ": %d\n", ret);
>         }
> #########################################################################################
> 
> Regards,
> Xiaoguang Wang
> 

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Xiaoguang Wang]

Hi Jan,

On 07/25/2014 03:05 PM, Jan Stancek wrote:
> 
> 
> ----- Original Message -----
>> From: "Xiaoguang Wang" <wangxg.fnst@cn.fujitsu.com>
>> To: "LTP" <ltp-list@lists.sourceforge.net>
>> Cc: "Jan Stancek" <jstancek@redhat.com>
>> Sent: Friday, 25 July, 2014 7:38:05 AM
>> Subject: Segmentation fault when running sched_setaffinity01  in RHEL5.10GA
>>
>> Hi,
>>
>> When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation
>> fault.
>> Below is the possible reason.
>>
>> Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA
>> is an old
>> distribution, which many people may not use it now, thanks!
> 
> Hi,
> 
> RHEL5.10 GA was January 2013, it's an older codebase, but release is quite recent.
> 
> I ran this on RHEL5.3:
> 
> (gdb) r
> Starting program: /root/ltp/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01 
> Detaching after fork from child process 7692.
> 
> Program received signal SIGSEGV, Segmentation fault.
> __sched_setaffinity_new (pid=<value optimized out>, cpusetsize=<value optimized out>, cpuset=<value optimized out>)
>     at ../sysdeps/unix/sysv/linux/sched_setaffinity.c:62
> 62	    if (((char *) cpuset)[cnt] != '\0')
> (gdb) list
> 57	    }
> 58	
> 59	  /* We now know the size of the kernel cpumask_t.  Make sure the user
> 60	     does not request to set a bit beyond that.  */
> 61	  for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
> 62	    if (((char *) cpuset)[cnt] != '\0')
> 63	      {
> 64	        /* Found a nonzero byte.  This means the user request cannot be
> 65		   fulfilled.  */
> 66		__set_errno (EINVAL);
> (gdb) p __kernel_cpumask_size
> $1 = 32
> (gdb) p cpusetsize
> $2 = <value optimized out>
> 
> 
> Looking at sources of glibc-2.5 (RHEL5.10) and glibc-2.17 (RHEL7), code leading
> up to this place looks identical. So, I agree with your findings. I see 2 options:
> 
> 1) find cpuset size in same way as glibc does it
> 
> 2) call syscall directly (and don't use glibc wrapper):
> 
> diff --git a/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c b/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
> index 0ac4478..0c0488f 100644
> --- a/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
> +++ b/testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity01.c
> @@ -42,6 +42,7 @@
>  #include "usctest.h"
>  #include "safe_macros.h"
>  #include "sched_setaffinity.h"
> +#include "linux_syscall_numbers.h"
>  
>  char *TCID = "sched_setaffinity01";
>  
> @@ -151,7 +152,7 @@ int main(int argc, char *argv[])
>         for (lc = 0; TEST_LOOPING(lc); lc++) {
>                 tst_count = 0;
>                 for (i = 0; i < TST_TOTAL; i++) {
> -                       TEST(sched_setaffinity(*(test_cases[i].pid),
> +                       TEST(ltp_syscall(__NR_sched_setaffinity, *(test_cases[i].pid),
>                                                 *(test_cases[i].mask_size),
>                                                 *(test_cases[i].mask)));
> 
> 

Thanks very much for your quick response, cool.
Using syscall directly would be OK, but we should also add a note to later reader explaining why we do not use glibc wrapper.

Regards,
Xiaoguang Wang

> # ./sched_setaffinity01 
> sched_setaffinity01    1  TPASS  :  expected failure with 'Bad address'
> sched_setaffinity01    2  TPASS  :  expected failure with 'Invalid argument'
> sched_setaffinity01    3  TPASS  :  expected failure with 'No such process'
> sched_setaffinity01    4  TPASS  :  expected failure with 'Operation not permitted'
> 
> # uname -r
> 2.6.18-128.37.1.el5
> 
> Regards,
> Jan
> 
>>
>>
>> Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system
>> call,
>> the corresponding code is below(The version of glibc I used is
>> glibc-2.5-20061008T1257-RHEL5.11Beta):
>> I delete some code just for simple.
>>
>> #######################################################################################
>>
>> /* Size definition for CPU sets.  */
>> # define __CPU_SETSIZE  1024
>> # define __NCPUBITS     (8 * sizeof (__cpu_mask))
>>
>> /* Type for array elements in 'cpu_set'.  */
>> typedef unsigned long int __cpu_mask;
>>
>> /* Basic access functions.  */
>> # define __CPUELT(cpu)  ((cpu) / __NCPUBITS)
>>
>> /* Data structure to describe CPU mask.  */
>> typedef struct
>> {
>>   __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
>> } cpu_set_t;
>>
>>
>> int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t
>> *cpuset)
>> {
>>   if (__builtin_expect (__kernel_cpumask_size == 0, 0))
>>     {
>>       int res;
>>
>>       while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
>>                                      psize, p),
>>              INTERNAL_SYSCALL_ERROR_P (res, err)
>>              && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
>>      	....
>>
>>       __kernel_cpumask_size = res;
>>     }
>>
>>   /* We now know the size of the kernel cpumask_t.  Make sure the user
>>      does not request to set a bit beyond that.  */
>>   for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
>>     if (((char *) cpuset)[cnt] != '\0')
>>       {
>>         /* Found a nonzero byte.  This means the user request cannot be
>>            fulfilled.  */
>>         __set_errno (EINVAL);
>>         return -1;
>>       }
>>
>>   return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
>> }
>> #######################################################################################
>>
>> Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp
>> testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
>> we define one.
>> #######################################################################################
>> #ifndef CPU_ALLOC_SIZE
>> #define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
>> #endif
>> #######################################################################################
>>
>> Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we
>> test EFAULT for sched_setaffinity(2),
>> the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it
>> first call
>> raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In
>> RHEL5.10GA,
>> this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw
>> sched_getaffinity(2) will return 32.
>> In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that
>> we're testing
>> EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate
>> segmentation fault in glibc code,
>> so this case exits abnormally
>>
>> As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's
>> because
>> sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t),
>> which totally depends on CONFIG_NR_CPUS.
>> In newer kernel, sched_getaffinity(2) returns the smaller one between
>> min_t(size_t, len, cpumask_size()),
>> here len is the value passed to sched_getaffinity as cpusetsize,
>> cpumask_size() is the max allowed length.
>> so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the
>> segmentation fault won't occur.
>>
>> So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in
>> testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
>> refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw
>> sched_getaffinity as a workaround in older kernel . See below code:
>> #########################################################################################
>>
>>         int ret;
>>         cpu_set_t cst;
>>
>>         memset(&cst, 0, sizeof(cst));
>>
>>         ret = syscall(__NR_sched_getaffinity, getpid(),
>>                       sizeof(cpu_set_t), &cst);
>>         if (ret < 0) {
>>                 fprintf(stderr, "sched_getaffinity failed: %s\n",
>>                         strerror(errno));
>>                 return 1;
>>         } else {
>>                 printf("length of bit mask the kernel uses to represent the
>>                 CPU"
>>                        ": %d\n", ret);
>>         }
>> #########################################################################################
>>
>> Regards,
>> Xiaoguang Wang
>>
> .
> 


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Xiaoguang Wang]

Hi,

On 07/25/2014 02:39 PM, Stanislav Kholmanskikh wrote:
> 
> 
> On 07/25/2014 09:38 AM, Xiaoguang Wang wrote:
>> Hi,
>>
>> When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation fault.
>> Below is the possible reason.
> 
> Hi!
> 
> I though I'd checked this test case on OL5 before submission. Let me check it again.

OL5 is based RHEL5?
If it is, please also check your CONFIG_NR_CPUS in kernel config file. If it's 1024 or greater,
this case will not fail. In my environment, this value is 255, thanks!

Regards,
Xiaoguang Wang
> 
>>
>> Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA is an old
>> distribution, which many people may not use it now, thanks!
>>
>>
>> Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system call,
>> the corresponding code is below(The version of glibc I used is glibc-2.5-20061008T1257-RHEL5.11Beta):
>> I delete some code just for simple.
>>
>> #######################################################################################
>>
>> /* Size definition for CPU sets.  */
>> # define __CPU_SETSIZE  1024
>> # define __NCPUBITS     (8 * sizeof (__cpu_mask))
>>
>> /* Type for array elements in 'cpu_set'.  */
>> typedef unsigned long int __cpu_mask;
>>
>> /* Basic access functions.  */
>> # define __CPUELT(cpu)  ((cpu) / __NCPUBITS)
>>
>> /* Data structure to describe CPU mask.  */
>> typedef struct
>> {
>>    __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
>> } cpu_set_t;
>>
>>
>> int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t *cpuset)
>> {
>>    if (__builtin_expect (__kernel_cpumask_size == 0, 0))
>>      {
>>        int res;
>>
>>        while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
>>                                       psize, p),
>>               INTERNAL_SYSCALL_ERROR_P (res, err)
>>               && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
>>           ....
>>
>>        __kernel_cpumask_size = res;
>>      }
>>
>>    /* We now know the size of the kernel cpumask_t.  Make sure the user
>>       does not request to set a bit beyond that.  */
>>    for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
>>      if (((char *) cpuset)[cnt] != '\0')
>>        {
>>          /* Found a nonzero byte.  This means the user request cannot be
>>             fulfilled.  */
>>          __set_errno (EINVAL);
>>          return -1;
>>        }
>>
>>    return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
>> }
>> #######################################################################################
>>
>> Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
>> we define one.
>> #######################################################################################
>> #ifndef CPU_ALLOC_SIZE
>> #define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
>> #endif
>> #######################################################################################
>>
>> Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we test EFAULT for sched_setaffinity(2),
>> the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it first call
>> raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In RHEL5.10GA,
>> this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw sched_getaffinity(2) will return 32.
>> In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that we're testing
>> EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate segmentation fault in glibc code,
>> so this case exits abnormally
>>
>> As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's because
>> sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t), which totally depends on CONFIG_NR_CPUS.
>> In newer kernel, sched_getaffinity(2) returns the smaller one between min_t(size_t, len, cpumask_size()),
>> here len is the value passed to sched_getaffinity as cpusetsize, cpumask_size() is the max allowed length.
>> so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the segmentation fault won't occur.
>>
>> So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
>> refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw sched_getaffinity as a workaround in older kernel . See below code:
>> #########################################################################################
>>
>>          int ret;
>>          cpu_set_t cst;
>>
>>          memset(&cst, 0, sizeof(cst));
>>
>>          ret = syscall(__NR_sched_getaffinity, getpid(),
>>                        sizeof(cpu_set_t), &cst);
>>          if (ret < 0) {
>>                  fprintf(stderr, "sched_getaffinity failed: %s\n",
>>                          strerror(errno));
>>                  return 1;
>>          } else {
>>                  printf("length of bit mask the kernel uses to represent the CPU"
>>                         ": %d\n", ret);
>>          }
>> #########################################################################################
>>
>> Regards,
>> Xiaoguang Wang
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Ltp-list mailing list
>> Ltp-list@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/ltp-list
>>
> .
> 


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] Segmentation fault when running sched_setaffinity01 in RHEL5.10GA

[Stanislav Kholmanskikh]

On 07/25/2014 11:32 AM, Xiaoguang Wang wrote:
> Hi,
>
> On 07/25/2014 02:39 PM, Stanislav Kholmanskikh wrote:
>>
>>
>> On 07/25/2014 09:38 AM, Xiaoguang Wang wrote:
>>> Hi,
>>>
>>> When we run sched_setaffinity01 in RHEL5.10GA, it occurs a segmentation fault.
>>> Below is the possible reason.
>>
>> Hi!
>>
>> I though I'd checked this test case on OL5 before submission. Let me check it again.
>
> OL5 is based RHEL5?
Yes.
> If it is, please also check your CONFIG_NR_CPUS in kernel config file. If it's 1024 or greater,
> this case will not fail. In my environment, this value is 255, thanks!

I initially tested it with 2.6.39-400.212.1.el5uek on OL5, and it passed.
[root@ol5-x64 sched_setaffinity]# grep CONFIG_NR_CPUS 
/boot/config-2.6.39-400.212.1.el5uek.debug
CONFIG_NR_CPUS=4096

Now I checked it with 2.6.18-371.3.1.0.1.el5 and it failed as you said:
[root@ol5-x64 sched_setaffinity]# grep CONFIG_NR_CPUS 
/boot/config-2.6.18-371.3.1.0.1.el5
CONFIG_NR_CPUS=255

Thanks for your findings.

I would also vote for the ltp_syscall(__NR_sched_setaffinity, ...) 
variant proposed by Jan.


>
> Regards,
> Xiaoguang Wang
>>
>>>
>>> Hi Jan, would you please help to confirm this problem. I'm afraid RHEL5.10GA is an old
>>> distribution, which many people may not use it now, thanks!
>>>
>>>
>>> Glibc provides a encapsulation for the raw kernel sched_setaffinity(2) system call,
>>> the corresponding code is below(The version of glibc I used is glibc-2.5-20061008T1257-RHEL5.11Beta):
>>> I delete some code just for simple.
>>>
>>> #######################################################################################
>>>
>>> /* Size definition for CPU sets.  */
>>> # define __CPU_SETSIZE  1024
>>> # define __NCPUBITS     (8 * sizeof (__cpu_mask))
>>>
>>> /* Type for array elements in 'cpu_set'.  */
>>> typedef unsigned long int __cpu_mask;
>>>
>>> /* Basic access functions.  */
>>> # define __CPUELT(cpu)  ((cpu) / __NCPUBITS)
>>>
>>> /* Data structure to describe CPU mask.  */
>>> typedef struct
>>> {
>>>     __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS];
>>> } cpu_set_t;
>>>
>>>
>>> int __sched_setaffinity_new (pid_t pid, size_t cpusetsize, const cpu_set_t *cpuset)
>>> {
>>>     if (__builtin_expect (__kernel_cpumask_size == 0, 0))
>>>       {
>>>         int res;
>>>
>>>         while (res = INTERNAL_SYSCALL (sched_getaffinity, err, 3, getpid (),
>>>                                        psize, p),
>>>                INTERNAL_SYSCALL_ERROR_P (res, err)
>>>                && INTERNAL_SYSCALL_ERRNO (res, err) == EINVAL)
>>>            ....
>>>
>>>         __kernel_cpumask_size = res;
>>>       }
>>>
>>>     /* We now know the size of the kernel cpumask_t.  Make sure the user
>>>        does not request to set a bit beyond that.  */
>>>     for (size_t cnt = __kernel_cpumask_size; cnt < cpusetsize; ++cnt)
>>>       if (((char *) cpuset)[cnt] != '\0')
>>>         {
>>>           /* Found a nonzero byte.  This means the user request cannot be
>>>              fulfilled.  */
>>>           __set_errno (EINVAL);
>>>           return -1;
>>>         }
>>>
>>>     return INLINE_SYSCALL (sched_setaffinity, 3, pid, cpusetsize, cpuset);
>>> }
>>> #######################################################################################
>>>
>>> Glibc in RHEL5.10GA does not provide CPU_ALLOC_SIZE marco, so in ltp testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h,
>>> we define one.
>>> #######################################################################################
>>> #ifndef CPU_ALLOC_SIZE
>>> #define CPU_ALLOC_SIZE(size) sizeof(cpu_set_t)
>>> #endif
>>> #######################################################################################
>>>
>>> Then CPU_ALLOC_SIZE would always return 128 in RHEL5.10GA, that is when we test EFAULT for sched_setaffinity(2),
>>> the passed cpusetsize is 128. But look at __sched_setaffinity_new() above, it first call
>>> raw sched_getaffinity(2) to get  the size of the kernel cpumask_t, In RHEL5.10GA,
>>> this value depends on CONFIG_NR_CPUS, if CONFIG_NR_CPUS is 255, the raw sched_getaffinity(2) will return 32.
>>> In this case, __kernel_cpumask_size would be 32, cpusetsize is 128. Give that we're testing
>>> EFAULT, cpuset is a invalid pointer, if cnt > 32, it will generate segmentation fault in glibc code,
>>> so this case exits abnormally
>>>
>>> As why this test case can run normally in RHEL6.5GA or RHEL7.0GA, it's because
>>> sched_getaffinity(2) in old kernel(RHEL5.10GA) return sizeof(cpumask_t), which totally depends on CONFIG_NR_CPUS.
>>> In newer kernel, sched_getaffinity(2) returns the smaller one between min_t(size_t, len, cpumask_size()),
>>> here len is the value passed to sched_getaffinity as cpusetsize, cpumask_size() is the max allowed length.
>>> so we can ensure __kernel_cpumask_size will never smaller cpusetsize, so the segmentation fault won't occur.
>>>
>>> So I also think CPU_ALLOC and CPU_ALLOC_SIZE is wrong in testcases/kernel/syscalls/sched_setaffinity/sched_setaffinity.h. We should
>>> refer to the implementation in glibc. or we define CPU_ALLOC_SIZE using raw sched_getaffinity as a workaround in older kernel . See below code:
>>> #########################################################################################
>>>
>>>           int ret;
>>>           cpu_set_t cst;
>>>
>>>           memset(&cst, 0, sizeof(cst));
>>>
>>>           ret = syscall(__NR_sched_getaffinity, getpid(),
>>>                         sizeof(cpu_set_t), &cst);
>>>           if (ret < 0) {
>>>                   fprintf(stderr, "sched_getaffinity failed: %s\n",
>>>                           strerror(errno));
>>>                   return 1;
>>>           } else {
>>>                   printf("length of bit mask the kernel uses to represent the CPU"
>>>                          ": %d\n", ret);
>>>           }
>>> #########################################################################################
>>>
>>> Regards,
>>> Xiaoguang Wang
>>>
>>> ------------------------------------------------------------------------------
>>> Want fast and easy access to all the code in your enterprise? Index and
>>> search up to 200,000 lines of code with a free copy of Black Duck
>>> Code Sight - the same software that powers the world's largest code
>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>> http://p.sf.net/sfu/bds
>>> _______________________________________________
>>> Ltp-list mailing list
>>> Ltp-list@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/ltp-list
>>>
>> .
>>
>

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list