[PATCH 0/3] perf: Add more syscalls to benchmark

[PATCH 0/3] perf: Add more syscalls to benchmark

[Tiezhu Yang]

Tiezhu Yang (3):
  perf bench syscall: Introduce bench_syscall_common()
  perf bench syscall: Add close syscall benchmark
  perf bench syscall: Add execve syscall benchmark

 tools/perf/bench/bench.h   |  2 ++
 tools/perf/bench/syscall.c | 76 +++++++++++++++++++++++++++++++++++++++++++---
 tools/perf/builtin-bench.c |  2 ++
 3 files changed, 76 insertions(+), 4 deletions(-)

-- 
2.1.0

[PATCH 1/3] perf bench syscall: Introduce bench_syscall_common()

[Tiezhu Yang]

In the current code, there is only a basic syscall benchmark via getppid,
this is not enough. Introduce bench_syscall_common() so that we can add
more syscalls to benchmark. This is preparation for later patch.

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
---
 tools/perf/bench/syscall.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/tools/perf/bench/syscall.c b/tools/perf/bench/syscall.c
index 9b75101..746fd71 100644
--- a/tools/perf/bench/syscall.c
+++ b/tools/perf/bench/syscall.c
@@ -30,25 +30,41 @@ static const char * const bench_syscall_usage[] = {
 	NULL
 };
 
-int bench_syscall_basic(int argc, const char **argv)
+static int bench_syscall_common(int argc, const char **argv, int syscall)
 {
 	struct timeval start, stop, diff;
 	unsigned long long result_usec = 0;
+	const char *name = NULL;
 	int i;
 
 	argc = parse_options(argc, argv, options, bench_syscall_usage, 0);
 
 	gettimeofday(&start, NULL);
 
-	for (i = 0; i < loops; i++)
-		getppid();
+	for (i = 0; i < loops; i++) {
+		switch (syscall) {
+		case __NR_getppid:
+			getppid();
+			break;
+		default:
+			break;
+		}
+	}
 
 	gettimeofday(&stop, NULL);
 	timersub(&stop, &start, &diff);
 
+	switch (syscall) {
+	case __NR_getppid:
+		name = "getppid()";
+		break;
+	default:
+		break;
+	}
+
 	switch (bench_format) {
 	case BENCH_FORMAT_DEFAULT:
-		printf("# Executed %'d getppid() calls\n", loops);
+		printf("# Executed %'d %s calls\n", loops, name);
 
 		result_usec = diff.tv_sec * 1000000;
 		result_usec += diff.tv_usec;
@@ -79,3 +95,8 @@ int bench_syscall_basic(int argc, const char **argv)
 
 	return 0;
 }
+
+int bench_syscall_basic(int argc, const char **argv)
+{
+	return bench_syscall_common(argc, argv, __NR_getppid);
+}
-- 
2.1.0

[PATCH 2/3] perf bench syscall: Add close syscall benchmark

[Tiezhu Yang]

This commit adds a simple close syscall benchmark, more syscall
benchmarks can be added in the future.

Here are the test results:

  [loongson@linux perf]$ ./perf bench syscall

          # List of available benchmarks for collection 'syscall':

           basic: Benchmark for basic getppid(2) calls
           close: Benchmark for close(2) calls
             all: Run all syscall benchmarks

  [loongson@linux perf]$ ./perf bench syscall basic
  # Running 'syscall/basic' benchmark:
  # Executed 10000000 getppid() calls
       Total time: 1.956 [sec]

         0.195687 usecs/op
          5110201 ops/sec
  [loongson@linux perf]$ ./perf bench syscall close
  # Running 'syscall/close' benchmark:
  # Executed 10000000 close() calls
       Total time: 6.302 [sec]

         0.630297 usecs/op
          1586553 ops/sec
  [loongson@linux perf]$ ./perf bench syscall all
  # Running syscall/basic benchmark...
  # Executed 10000000 getppid() calls
       Total time: 1.956 [sec]

         0.195686 usecs/op
          5110232 ops/sec

  # Running syscall/close benchmark...
  # Executed 10000000 close() calls
       Total time: 6.302 [sec]

         0.630271 usecs/op
          1586619 ops/sec

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
---
 tools/perf/bench/bench.h   |  1 +
 tools/perf/bench/syscall.c | 11 +++++++++++
 tools/perf/builtin-bench.c |  1 +
 3 files changed, 13 insertions(+)

diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
index 6cefb43..916cd47 100644
--- a/tools/perf/bench/bench.h
+++ b/tools/perf/bench/bench.h
@@ -34,6 +34,7 @@ int bench_numa(int argc, const char **argv);
 int bench_sched_messaging(int argc, const char **argv);
 int bench_sched_pipe(int argc, const char **argv);
 int bench_syscall_basic(int argc, const char **argv);
+int bench_syscall_close(int argc, const char **argv);
 int bench_mem_memcpy(int argc, const char **argv);
 int bench_mem_memset(int argc, const char **argv);
 int bench_mem_find_bit(int argc, const char **argv);
diff --git a/tools/perf/bench/syscall.c b/tools/perf/bench/syscall.c
index 746fd71..058394b 100644
--- a/tools/perf/bench/syscall.c
+++ b/tools/perf/bench/syscall.c
@@ -46,6 +46,9 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
 		case __NR_getppid:
 			getppid();
 			break;
+		case __NR_close:
+			close(dup(0));
+			break;
 		default:
 			break;
 		}
@@ -58,6 +61,9 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
 	case __NR_getppid:
 		name = "getppid()";
 		break;
+	case __NR_close:
+		name = "close()";
+		break;
 	default:
 		break;
 	}
@@ -100,3 +106,8 @@ int bench_syscall_basic(int argc, const char **argv)
 {
 	return bench_syscall_common(argc, argv, __NR_getppid);
 }
+
+int bench_syscall_close(int argc, const char **argv)
+{
+	return bench_syscall_common(argc, argv, __NR_close);
+}
diff --git a/tools/perf/builtin-bench.c b/tools/perf/builtin-bench.c
index 334ab89..b63c711 100644
--- a/tools/perf/builtin-bench.c
+++ b/tools/perf/builtin-bench.c
@@ -52,6 +52,7 @@ static struct bench sched_benchmarks[] = {
 
 static struct bench syscall_benchmarks[] = {
 	{ "basic",	"Benchmark for basic getppid(2) calls",		bench_syscall_basic	},
+	{ "close",	"Benchmark for close(2) calls",			bench_syscall_close	},
 	{ "all",	"Run all syscall benchmarks",			NULL			},
 	{ NULL,		NULL,						NULL			},
 };
-- 
2.1.0

[PATCH 3/3] perf bench syscall: Add execve syscall benchmark

[Tiezhu Yang]

This commit adds the execve syscall benchmark, more syscall
benchmarks can be added in the future.

Here are the test results:

  [loongson@linux perf]$ ./perf bench syscall

          # List of available benchmarks for collection 'syscall':

           basic: Benchmark for basic getppid(2) calls
           close: Benchmark for close(2) calls
          execve: Benchmark for execve(2) calls
             all: Run all syscall benchmarks

  [loongson@linux perf]$ ./perf bench syscall execve
  # Running 'syscall/execve' benchmark:
  # Executed 10000 execve() calls
       Total time: 4.827 [sec]

       482.761800 usecs/op
             2071 ops/sec
  [loongson@linux perf]$ ./perf bench syscall all
  # Running syscall/basic benchmark...
  # Executed 10000000 getppid() calls
       Total time: 1.956 [sec]

         0.195692 usecs/op
          5110065 ops/sec

  # Running syscall/close benchmark...
  # Executed 10000000 close() calls
       Total time: 6.306 [sec]

         0.630692 usecs/op
          1585558 ops/sec

  # Running syscall/execve benchmark...
  # Executed 10000 execve() calls
       Total time: 4.842 [sec]

       484.299500 usecs/op
             2064 ops/sec

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
---
 tools/perf/bench/bench.h   |  1 +
 tools/perf/bench/syscall.c | 36 ++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-bench.c |  1 +
 3 files changed, 38 insertions(+)

diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
index 916cd47..6ca3327 100644
--- a/tools/perf/bench/bench.h
+++ b/tools/perf/bench/bench.h
@@ -35,6 +35,7 @@ int bench_sched_messaging(int argc, const char **argv);
 int bench_sched_pipe(int argc, const char **argv);
 int bench_syscall_basic(int argc, const char **argv);
 int bench_syscall_close(int argc, const char **argv);
+int bench_syscall_execve(int argc, const char **argv);
 int bench_mem_memcpy(int argc, const char **argv);
 int bench_mem_memset(int argc, const char **argv);
 int bench_mem_find_bit(int argc, const char **argv);
diff --git a/tools/perf/bench/syscall.c b/tools/perf/bench/syscall.c
index 058394b..c8f8bee 100644
--- a/tools/perf/bench/syscall.c
+++ b/tools/perf/bench/syscall.c
@@ -14,6 +14,7 @@
 #include <sys/time.h>
 #include <sys/syscall.h>
 #include <sys/types.h>
+#include <sys/wait.h>
 #include <unistd.h>
 #include <stdlib.h>
 
@@ -30,6 +31,27 @@ static const char * const bench_syscall_usage[] = {
 	NULL
 };
 
+static void test_execve(void)
+{
+	const char *pathname = "/bin/true";
+	char *const argv[] = { (char *)pathname, NULL };
+	pid_t pid = fork();
+
+	if (pid < 0) {
+		fprintf(stderr, "fork failed\n");
+		exit(1);
+	} else if (pid == 0) {
+		execve(pathname, argv, NULL);
+		fprintf(stderr, "execve /bin/true failed\n");
+		exit(1);
+	} else {
+		if (waitpid(pid, NULL, 0) < 0) {
+			fprintf(stderr, "waitpid failed\n");
+			exit(1);
+		}
+	}
+}
+
 static int bench_syscall_common(int argc, const char **argv, int syscall)
 {
 	struct timeval start, stop, diff;
@@ -49,6 +71,12 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
 		case __NR_close:
 			close(dup(0));
 			break;
+		case __NR_execve:
+			test_execve();
+			/* Only loop 10000 times to save time */
+			if (i == 10000)
+				loops = 10000;
+			break;
 		default:
 			break;
 		}
@@ -64,6 +92,9 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
 	case __NR_close:
 		name = "close()";
 		break;
+	case __NR_execve:
+		name = "execve()";
+		break;
 	default:
 		break;
 	}
@@ -111,3 +142,8 @@ int bench_syscall_close(int argc, const char **argv)
 {
 	return bench_syscall_common(argc, argv, __NR_close);
 }
+
+int bench_syscall_execve(int argc, const char **argv)
+{
+	return bench_syscall_common(argc, argv, __NR_execve);
+}
diff --git a/tools/perf/builtin-bench.c b/tools/perf/builtin-bench.c
index b63c711..6f9ff75 100644
--- a/tools/perf/builtin-bench.c
+++ b/tools/perf/builtin-bench.c
@@ -53,6 +53,7 @@ static struct bench sched_benchmarks[] = {
 static struct bench syscall_benchmarks[] = {
 	{ "basic",	"Benchmark for basic getppid(2) calls",		bench_syscall_basic	},
 	{ "close",	"Benchmark for close(2) calls",			bench_syscall_close	},
+	{ "execve",	"Benchmark for execve(2) calls",		bench_syscall_execve	},
 	{ "all",	"Run all syscall benchmarks",			NULL			},
 	{ NULL,		NULL,						NULL			},
 };
-- 
2.1.0

RE: [PATCH 2/3] perf bench syscall: Add close syscall benchmark

[David Laight]

From: Tiezhu Yang
> Sent: 06 September 2022 04:06
> 
> This commit adds a simple close syscall benchmark, more syscall
> benchmarks can be added in the future.
> 
...
> 
> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
> ---
>  tools/perf/bench/bench.h   |  1 +
>  tools/perf/bench/syscall.c | 11 +++++++++++
>  tools/perf/builtin-bench.c |  1 +
>  3 files changed, 13 insertions(+)
> 
> diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
> index 6cefb43..916cd47 100644
...
> diff --git a/tools/perf/bench/syscall.c b/tools/perf/bench/syscall.c
> index 746fd71..058394b 100644
> --- a/tools/perf/bench/syscall.c
> +++ b/tools/perf/bench/syscall.c
> @@ -46,6 +46,9 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
>  		case __NR_getppid:
>  			getppid();
>  			break;
> +		case __NR_close:
> +			close(dup(0));

Not really a close() test.
The dup(0) call will be significant and may take longer.

I'm also not sure that using the syscall number for the
test number is entirely sensible.

One thing I have measured in the past is the time taken
to read in an iov[] array.
This can be measured quite nicely using writev() on /dev/null.
(No copies ever happen and iov_iter() is never used.)
But you need to test a few different iov lengths.

I'm also not 100% sure how accurate/repeatable/sensible it
is to use the 'wall clock time' for 1000000 iterations.
A lot of modern cpu will dynamically change the clock speed
underneath you and other system code (like ethernet receive)
can badly perturb the results.

What you really want to use is a TSC - but they are now
useless for counting cycles.
The x86 performance counters to have a cycle counter.
I've used that to measure single calls of both library
functions and system calls.
Just 10 iterations give a 'cold cache' value and some
very consistent counts (remove real outliers).
Indeed the fastest value is really the right one.

For functions like the IP checksum you can even
show that the code is executing in the expected number
of clock cycles (usually limited by memory reads).

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: [PATCH 2/3] perf bench syscall: Add close syscall benchmark

[Tiezhu Yang]

On 09/06/2022 11:30 AM, David Laight wrote:
> From: Tiezhu Yang
>> Sent: 06 September 2022 04:06
>>
>> This commit adds a simple close syscall benchmark, more syscall
>> benchmarks can be added in the future.
>>
> ...
>>
>> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
>> ---
>>  tools/perf/bench/bench.h   |  1 +
>>  tools/perf/bench/syscall.c | 11 +++++++++++
>>  tools/perf/builtin-bench.c |  1 +
>>  3 files changed, 13 insertions(+)
>>
>> diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
>> index 6cefb43..916cd47 100644
> ...
>> diff --git a/tools/perf/bench/syscall.c b/tools/perf/bench/syscall.c
>> index 746fd71..058394b 100644
>> --- a/tools/perf/bench/syscall.c
>> +++ b/tools/perf/bench/syscall.c
>> @@ -46,6 +46,9 @@ static int bench_syscall_common(int argc, const char **argv, int syscall)
>>  		case __NR_getppid:
>>  			getppid();
>>  			break;
>> +		case __NR_close:
>> +			close(dup(0));
>
> Not really a close() test.
> The dup(0) call will be significant and may take longer.
>
> I'm also not sure that using the syscall number for the
> test number is entirely sensible.
>
> One thing I have measured in the past is the time taken
> to read in an iov[] array.
> This can be measured quite nicely using writev() on /dev/null.
> (No copies ever happen and iov_iter() is never used.)
> But you need to test a few different iov lengths.
>
> I'm also not 100% sure how accurate/repeatable/sensible it
> is to use the 'wall clock time' for 1000000 iterations.
> A lot of modern cpu will dynamically change the clock speed
> underneath you and other system code (like ethernet receive)
> can badly perturb the results.
>
> What you really want to use is a TSC - but they are now
> useless for counting cycles.
> The x86 performance counters to have a cycle counter.
> I've used that to measure single calls of both library
> functions and system calls.
> Just 10 iterations give a 'cold cache' value and some
> very consistent counts (remove real outliers).
> Indeed the fastest value is really the right one.
>
> For functions like the IP checksum you can even
> show that the code is executing in the expected number
> of clock cycles (usually limited by memory reads).
>
> 	David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
>

Hi David,

Thanks for your reply.

There are some explanations in commit c2a08203052f ("perf bench:
Add basic syscall benchmark"), I presume the current benchmark
framework works well, if not, maybe we should modify the framework
first.

The initial aim of this patch series is to benchmark more syscalls,
some code is similar with the UnixBench syscall test [1].

[1] 
https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench/src/syscall.c

Thanks,
Tiezhu

Re: [PATCH 0/3] perf: Add more syscalls to benchmark

[Tiezhu Yang]

On 09/06/2022 11:06 AM, Tiezhu Yang wrote:
> Tiezhu Yang (3):
>   perf bench syscall: Introduce bench_syscall_common()
>   perf bench syscall: Add close syscall benchmark
>   perf bench syscall: Add execve syscall benchmark
>
>  tools/perf/bench/bench.h   |  2 ++
>  tools/perf/bench/syscall.c | 76 +++++++++++++++++++++++++++++++++++++++++++---
>  tools/perf/builtin-bench.c |  2 ++
>  3 files changed, 76 insertions(+), 4 deletions(-)
>

Hi all,

Any more comments? Do you think this patch series [1] is useful?
If yes, I will send v2 to fix the build error about undeclared
__NR_getppid reported by kernel test robot.

[1] 
https://lore.kernel.org/lkml/1662433577-23266-1-git-send-email-yangtiezhu@loongson.cn/

Thanks,
Tiezhu

Re: [PATCH 0/3] perf: Add more syscalls to benchmark

[Arnaldo Carvalho de Melo]

Em Wed, Sep 28, 2022 at 09:06:03AM +0800, Tiezhu Yang escreveu:
> 
> 
> On 09/06/2022 11:06 AM, Tiezhu Yang wrote:
> > Tiezhu Yang (3):
> >   perf bench syscall: Introduce bench_syscall_common()
> >   perf bench syscall: Add close syscall benchmark
> >   perf bench syscall: Add execve syscall benchmark
> > 
> >  tools/perf/bench/bench.h   |  2 ++
> >  tools/perf/bench/syscall.c | 76 +++++++++++++++++++++++++++++++++++++++++++---
> >  tools/perf/builtin-bench.c |  2 ++
> >  3 files changed, 76 insertions(+), 4 deletions(-)
> > 
> 
> Hi all,
> 
> Any more comments? Do you think this patch series [1] is useful?
> If yes, I will send v2 to fix the build error about undeclared
> __NR_getppid reported by kernel test robot.
> 
> [1] https://lore.kernel.org/lkml/1662433577-23266-1-git-send-email-yangtiezhu@loongson.cn/


Please send the v2.

- Arnaldo