* [LTP] [PATCH 0/2] Update test timeouts in an automated way
@ 2025-01-22 12:48 Andrea Cervesato
2025-01-22 12:48 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Andrea Cervesato
2025-01-22 12:48 ` [LTP] [PATCH 2/2] syscalls: Update test timeouts Andrea Cervesato
0 siblings, 2 replies; 8+ messages in thread
From: Andrea Cervesato @ 2025-01-22 12:48 UTC (permalink / raw)
To: ltp
First patch adds a script that can parse kirk test results, calculate
timeouts and patch tests with the newly calculated timeouts.
Second patch updates tests timeouts using this script with a results
captured on RPi zero.
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
Cyril Hrubis (2):
scripts: Add simple script for calculating timeouts
syscalls: Update test timeouts
scripts/calctimeouts.py | 232 +++++++++++++++++++++
testcases/kernel/syscalls/access/access01.c | 1 +
testcases/kernel/syscalls/add_key/add_key05.c | 1 +
testcases/kernel/syscalls/alarm/alarm05.c | 1 +
testcases/kernel/syscalls/alarm/alarm06.c | 1 +
testcases/kernel/syscalls/alarm/alarm07.c | 1 +
testcases/kernel/syscalls/bind/bind04.c | 1 +
testcases/kernel/syscalls/bpf/bpf_prog05.c | 1 +
testcases/kernel/syscalls/bpf/bpf_prog06.c | 1 +
testcases/kernel/syscalls/bpf/bpf_prog07.c | 1 +
testcases/kernel/syscalls/cachestat/cachestat01.c | 1 +
testcases/kernel/syscalls/cachestat/cachestat04.c | 1 +
testcases/kernel/syscalls/chdir/chdir01.c | 1 +
.../kernel/syscalls/clock_gettime/leapsec01.c | 1 +
.../syscalls/clock_nanosleep/clock_nanosleep01.c | 1 +
.../syscalls/clock_settime/clock_settime03.c | 1 +
.../kernel/syscalls/close_range/close_range01.c | 1 +
testcases/kernel/syscalls/connect/connect02.c | 1 +
testcases/kernel/syscalls/creat/creat05.c | 1 +
testcases/kernel/syscalls/creat/creat09.c | 1 +
testcases/kernel/syscalls/execve/execve05.c | 1 +
testcases/kernel/syscalls/execveat/execveat03.c | 1 +
testcases/kernel/syscalls/fallocate/fallocate04.c | 1 +
testcases/kernel/syscalls/fallocate/fallocate05.c | 1 +
testcases/kernel/syscalls/fallocate/fallocate06.c | 2 +-
testcases/kernel/syscalls/fanotify/fanotify01.c | 1 +
testcases/kernel/syscalls/fanotify/fanotify03.c | 1 +
testcases/kernel/syscalls/fanotify/fanotify05.c | 1 +
testcases/kernel/syscalls/fanotify/fanotify06.c | 1 +
testcases/kernel/syscalls/fanotify/fanotify09.c | 1 +
testcases/kernel/syscalls/fanotify/fanotify10.c | 1 +
testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c | 1 +
testcases/kernel/syscalls/fcntl/fcntl14.c | 1 +
testcases/kernel/syscalls/fcntl/fcntl36.c | 1 +
testcases/kernel/syscalls/fdatasync/fdatasync03.c | 1 +
testcases/kernel/syscalls/fgetxattr/fgetxattr01.c | 1 +
.../kernel/syscalls/fremovexattr/fremovexattr01.c | 1 +
.../kernel/syscalls/fremovexattr/fremovexattr02.c | 1 +
testcases/kernel/syscalls/fsconfig/fsconfig01.c | 1 +
testcases/kernel/syscalls/fsconfig/fsconfig03.c | 1 +
testcases/kernel/syscalls/fsetxattr/fsetxattr01.c | 1 +
testcases/kernel/syscalls/fsmount/fsmount01.c | 1 +
testcases/kernel/syscalls/fsmount/fsmount02.c | 1 +
testcases/kernel/syscalls/fsopen/fsopen01.c | 1 +
testcases/kernel/syscalls/fspick/fspick01.c | 1 +
testcases/kernel/syscalls/fspick/fspick02.c | 1 +
testcases/kernel/syscalls/fstatfs/fstatfs01.c | 1 +
testcases/kernel/syscalls/fsync/fsync01.c | 1 +
testcases/kernel/syscalls/fsync/fsync04.c | 1 +
testcases/kernel/syscalls/getpid/getpid01.c | 1 +
testcases/kernel/syscalls/getxattr/getxattr02.c | 1 +
testcases/kernel/syscalls/getxattr/getxattr03.c | 1 +
testcases/kernel/syscalls/inotify/inotify03.c | 1 +
testcases/kernel/syscalls/inotify/inotify05.c | 1 +
testcases/kernel/syscalls/inotify/inotify07.c | 1 +
testcases/kernel/syscalls/inotify/inotify08.c | 1 +
testcases/kernel/syscalls/inotify/inotify11.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl02.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl04.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl08.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl09.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c | 1 +
.../kernel/syscalls/ioctl/ioctl_ficlonerange01.c | 1 +
.../kernel/syscalls/ioctl/ioctl_ficlonerange02.c | 1 +
testcases/kernel/syscalls/ioctl/ioctl_loop01.c | 1 +
66 files changed, 297 insertions(+), 1 deletion(-)
---
base-commit: d3df587ed1ee3f881f802f5060506a8192c38e0d
change-id: 20250121-cyril_script_update_timeouts-4acc3c00b3dc
Best regards,
--
Andrea Cervesato <andrea.cervesato@suse.com>
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 8+ messages in thread* [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-22 12:48 [LTP] [PATCH 0/2] Update test timeouts in an automated way Andrea Cervesato @ 2025-01-22 12:48 ` Andrea Cervesato 2025-01-22 13:13 ` Cyril Hrubis 2025-01-22 12:48 ` [LTP] [PATCH 2/2] syscalls: Update test timeouts Andrea Cervesato 1 sibling, 1 reply; 8+ messages in thread From: Andrea Cervesato @ 2025-01-22 12:48 UTC (permalink / raw) To: ltp From: Cyril Hrubis <chrubis@suse.cz> This script parses JSON results from kirk and LTP metadata in order calculate timeouts for tests based on the result file. It can also patch tests automatically. The script does: - Take the results and pick all tests that run for longer than 0.5s. Multiplie the time with a constant (currently 1.2) to get a suggested timeout. - Exclude tests that have runtime defined since these are controlled by the runtime (that filters out all fuzzy sync tests). There is a special case for timer tests that defines runtime only dynamically in the timer library code. This should be possibly fixed with special value for the .runtime in tst_test. E.g. TST_RUNTIME_DYNAMIC for tests that only set runtime in the setup. - Normalize the timeout for a single filesystem run if test is running for more than one filesystem. - Verify if tests are build on top of old library by checking at metadata file - Update test with a with newly calculated timeout. By default we only increase timeouts but that can be overridden using the -o option. Signed-off-by: Cyril Hrubis <chrubis@suse.cz> Co-developed-by: Andrea Cervesato <andrea.cervesato@suse.com> Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com> --- scripts/calctimeouts.py | 232 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 232 insertions(+) diff --git a/scripts/calctimeouts.py b/scripts/calctimeouts.py new file mode 100755 index 0000000000000000000000000000000000000000..d5e8fe2c15faad7c8d1ec8c15541f35a97a8c0c4 --- /dev/null +++ b/scripts/calctimeouts.py @@ -0,0 +1,232 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +""" +This script parses JSON results from kirk and LTP metadata in order to +calculate timeouts for tests based on the results file. +It can also patch tests automatically and replace the calculated timeout. +""" + +import re +import os +import json +import argparse + +# The test runtime is multiplied by this to get a timeout +TIMEOUT_MUL = 1.2 + + +def _sed(fname, expr, replace): + """ + Pythonic version of sed command. + """ + content = [] + matcher = re.compile(expr) + + with open(fname, 'r', encoding="utf-8") as data: + for line in data: + match = matcher.search(line) + if not match: + content.append(line) + else: + content.append(replace) + + with open(fname, 'w', encoding="utf-8") as data: + data.writelines(content) + + +def _patch(ltp_dir, fname, new_timeout, override): + """ + If `override` is True, it patches a test file, searching for timeout and + replacing it with `new_timeout`. + """ + orig_timeout = None + file_path = os.path.join(ltp_dir, fname) + + with open(file_path, 'r', encoding="utf-8") as c_source: + matcher = re.compile(r'\s*.timeout\s*=\s*(\d+).') + for line in c_source: + match = matcher.search(line) + if not match: + continue + + timeout = match.group(1) + orig_timeout = int(timeout) + + if orig_timeout: + if orig_timeout < new_timeout and override: + print(f"CHANGE {fname} timeout {orig_timeout} -> {new_timeout}") + _sed(file_path, r".timeout = [0-9]*,\n", + f"\t.timeout = {new_timeout},\n") + else: + print(f"KEEP {fname} timeout {orig_timeout} (new {new_timeout})") + else: + print(f"ADD {fname} timeout {new_timeout}") + _sed(file_path, + "static struct tst_test test = {", + "static struct tst_test test = {\n" + f"\t.timeout = {new_timeout},\n") + + +def _patch_all(ltp_dir, timeouts, override): + """ + Patch all tests. + """ + for timeout in timeouts: + if timeout['path']: + _patch(ltp_dir, timeout['path'], timeout['timeout'], override) + + +def _print_table(timeouts): + """ + Print the timeouts table. + """ + timeouts.sort(key=lambda x: x['timeout'], reverse=True) + + total = 0 + + print("Old library tests\n-----------------\n") + for timeout in timeouts: + if not timeout['newlib']: + print(f"{timeout['name']:30s} {timeout['timeout']}") + total += 1 + + print(f"\n\t{total} tests in total") + + total = 0 + + print("\nNew library tests\n-----------------\n") + for timeout in timeouts: + if timeout['newlib']: + print(f"{timeout['name']:30s} {timeout['timeout']}") + total += 1 + + print(f"\n\t{total} tests in total") + + +def _parse_data(ltp_dir, results_path): + """ + Parse results data and metadata, then it generates timeouts data. + """ + timeouts = [] + results = None + metadata = None + + with open(results_path, 'r', encoding="utf-8") as file: + results = json.load(file) + + metadata_path = os.path.join(ltp_dir, 'metadata', 'ltp.json') + with open(metadata_path, 'r', encoding="utf-8") as file: + metadata = json.load(file) + + for test in results['results']: + name = test['test_fqn'] + duration = test['test']['duration'] + + # if test runs for all_filesystems, normalize runtime to one filesystem + filesystems = max(1, test['test']['log'].count('TINFO: Formatting /')) + + # check if test is new library test + test_is_newlib = name in metadata['tests'] + + # store test file path + path = None + if test_is_newlib: + path = metadata['tests'][name]['fname'] + + test_has_runtime = False + if test_is_newlib: + # filter out tests with runtime + test_has_runtime = 'runtime' in metadata['tests'][name] + + # timer tests define runtime dynamically in timer library + test_has_runtime = 'sample' in metadata['tests'][name] + + # select tests that does not have runtime and which are executed + # for a long time + if not test_has_runtime and duration >= 0.5: + data = {} + data["name"] = name + data["timeout"] = int(TIMEOUT_MUL * duration/filesystems + 0.5) + data["newlib"] = test_is_newlib + data["path"] = path + + timeouts.append(data) + + return timeouts + + +def _file_exists(filepath): + """ + Check if the given file path exists. + """ + if not os.path.isfile(filepath): + raise argparse.ArgumentTypeError( + f"The file '{filepath}' does not exist.") + return filepath + + +def _dir_exists(dirpath): + """ + Check if the given directory path exists. + """ + if not os.path.isdir(dirpath): + raise argparse.ArgumentTypeError( + f"The directory '{dirpath}' does not exist.") + return dirpath + + +def run(): + """ + Entry point of the script. + """ + parser = argparse.ArgumentParser( + description="Script to calculate LTP tests timeouts") + + parser.add_argument( + '-l', + '--ltp-dir', + type=_dir_exists, + help='LTP directory', + default='/opt/ltp') + + parser.add_argument( + '-r', + '--results', + type=_file_exists, + required=True, + help='kirk results.json file location') + + parser.add_argument( + '-o', + '--override', + default=False, + action='store_true', + help='Override test timeouts') + + parser.add_argument( + '-p', + '--patch', + default=False, + action='store_true', + help='Patch tests with updated timeout') + + parser.add_argument( + '-t', + '--print-table', + default=True, + action='store_true', + help='Print table with suggested timeouts') + + args = parser.parse_args() + + timeouts = _parse_data(args.ltp_dir, args.results) + + if args.print_table: + _print_table(timeouts) + + if args.patch: + _patch_all(args.ltp_dir, timeouts, args.override) + + +if __name__ == "__main__": + run() -- 2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-22 12:48 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Andrea Cervesato @ 2025-01-22 13:13 ` Cyril Hrubis 0 siblings, 0 replies; 8+ messages in thread From: Cyril Hrubis @ 2025-01-22 13:13 UTC (permalink / raw) To: Andrea Cervesato; +Cc: ltp Hi! > This script parses JSON results from kirk and LTP metadata in order > calculate timeouts for tests based on the result file. It can also patch > tests automatically. > > The script does: > > - Take the results and pick all tests that run for longer than 0.5s. > Multiplie the time with a constant (currently 1.2) to get a suggested > timeout. > > - Exclude tests that have runtime defined since these are controlled > by the runtime (that filters out all fuzzy sync tests). > > There is a special case for timer tests that defines runtime only > dynamically in the timer library code. This should be possibly fixed > with special value for the .runtime in tst_test. E.g. > TST_RUNTIME_DYNAMIC for tests that only set runtime in the setup. > > - Normalize the timeout for a single filesystem run if test is running for > more than one filesystem. > > - Verify if tests are build on top of old library by checking at > metadata file > > - Update test with a with newly calculated timeout. > By default we only increase timeouts but that can be overridden using > the -o option. > > Signed-off-by: Cyril Hrubis <chrubis@suse.cz> > Co-developed-by: Andrea Cervesato <andrea.cervesato@suse.com> > Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com> > --- > scripts/calctimeouts.py | 232 ++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 232 insertions(+) > > diff --git a/scripts/calctimeouts.py b/scripts/calctimeouts.py > new file mode 100755 > index 0000000000000000000000000000000000000000..d5e8fe2c15faad7c8d1ec8c15541f35a97a8c0c4 > --- /dev/null > +++ b/scripts/calctimeouts.py > @@ -0,0 +1,232 @@ > +#!/usr/bin/env python3 > +# SPDX-License-Identifier: GPL-2.0-or-later > +""" > +This script parses JSON results from kirk and LTP metadata in order to > +calculate timeouts for tests based on the results file. > +It can also patch tests automatically and replace the calculated timeout. > +""" > + > +import re > +import os > +import json > +import argparse > + > +# The test runtime is multiplied by this to get a timeout > +TIMEOUT_MUL = 1.2 > + > + > +def _sed(fname, expr, replace): > + """ > + Pythonic version of sed command. > + """ > + content = [] > + matcher = re.compile(expr) > + > + with open(fname, 'r', encoding="utf-8") as data: > + for line in data: > + match = matcher.search(line) > + if not match: > + content.append(line) > + else: > + content.append(replace) > + > + with open(fname, 'w', encoding="utf-8") as data: > + data.writelines(content) > + > + > +def _patch(ltp_dir, fname, new_timeout, override): > + """ > + If `override` is True, it patches a test file, searching for timeout and > + replacing it with `new_timeout`. The override is for the cases where the test already has a timeout and the timeout in the test is bigger than the one we calculated. By default the function keeps the bigger timeout. > + """ > + orig_timeout = None > + file_path = os.path.join(ltp_dir, fname) > + > + with open(file_path, 'r', encoding="utf-8") as c_source: > + matcher = re.compile(r'\s*.timeout\s*=\s*(\d+).') > + for line in c_source: > + match = matcher.search(line) > + if not match: > + continue > + > + timeout = match.group(1) > + orig_timeout = int(timeout) > + > + if orig_timeout: > + if orig_timeout < new_timeout and override: > + print(f"CHANGE {fname} timeout {orig_timeout} -> {new_timeout}") > + _sed(file_path, r".timeout = [0-9]*,\n", > + f"\t.timeout = {new_timeout},\n") > + else: > + print(f"KEEP {fname} timeout {orig_timeout} (new {new_timeout})") > + else: > + print(f"ADD {fname} timeout {new_timeout}") > + _sed(file_path, > + "static struct tst_test test = {", > + "static struct tst_test test = {\n" > + f"\t.timeout = {new_timeout},\n") > + > + > +def _patch_all(ltp_dir, timeouts, override): > + """ > + Patch all tests. > + """ > + for timeout in timeouts: > + if timeout['path']: > + _patch(ltp_dir, timeout['path'], timeout['timeout'], override) > + > + > +def _print_table(timeouts): > + """ > + Print the timeouts table. > + """ > + timeouts.sort(key=lambda x: x['timeout'], reverse=True) > + > + total = 0 > + > + print("Old library tests\n-----------------\n") > + for timeout in timeouts: > + if not timeout['newlib']: > + print(f"{timeout['name']:30s} {timeout['timeout']}") > + total += 1 > + > + print(f"\n\t{total} tests in total") > + > + total = 0 > + > + print("\nNew library tests\n-----------------\n") > + for timeout in timeouts: > + if timeout['newlib']: > + print(f"{timeout['name']:30s} {timeout['timeout']}") > + total += 1 > + > + print(f"\n\t{total} tests in total") > + > + > +def _parse_data(ltp_dir, results_path): > + """ > + Parse results data and metadata, then it generates timeouts data. > + """ > + timeouts = [] > + results = None > + metadata = None > + > + with open(results_path, 'r', encoding="utf-8") as file: > + results = json.load(file) > + > + metadata_path = os.path.join(ltp_dir, 'metadata', 'ltp.json') > + with open(metadata_path, 'r', encoding="utf-8") as file: > + metadata = json.load(file) > + > + for test in results['results']: > + name = test['test_fqn'] > + duration = test['test']['duration'] > + > + # if test runs for all_filesystems, normalize runtime to one filesystem > + filesystems = max(1, test['test']['log'].count('TINFO: Formatting /')) > + > + # check if test is new library test > + test_is_newlib = name in metadata['tests'] > + > + # store test file path > + path = None > + if test_is_newlib: > + path = metadata['tests'][name]['fname'] > + > + test_has_runtime = False > + if test_is_newlib: > + # filter out tests with runtime > + test_has_runtime = 'runtime' in metadata['tests'][name] > + > + # timer tests define runtime dynamically in timer library > + test_has_runtime = 'sample' in metadata['tests'][name] > + > + # select tests that does not have runtime and which are executed > + # for a long time > + if not test_has_runtime and duration >= 0.5: > + data = {} > + data["name"] = name > + data["timeout"] = int(TIMEOUT_MUL * duration/filesystems + 0.5) > + data["newlib"] = test_is_newlib > + data["path"] = path > + > + timeouts.append(data) > + > + return timeouts > + > + > +def _file_exists(filepath): > + """ > + Check if the given file path exists. > + """ > + if not os.path.isfile(filepath): > + raise argparse.ArgumentTypeError( > + f"The file '{filepath}' does not exist.") > + return filepath > + > + > +def _dir_exists(dirpath): > + """ > + Check if the given directory path exists. > + """ > + if not os.path.isdir(dirpath): > + raise argparse.ArgumentTypeError( > + f"The directory '{dirpath}' does not exist.") > + return dirpath > + > + > +def run(): > + """ > + Entry point of the script. > + """ > + parser = argparse.ArgumentParser( > + description="Script to calculate LTP tests timeouts") > + > + parser.add_argument( > + '-l', > + '--ltp-dir', > + type=_dir_exists, > + help='LTP directory', > + default='/opt/ltp') The script is not supposed to be executed from installed tree, it needs the C source files. So I would argue that '../' or '.' is better default. > + parser.add_argument( > + '-r', > + '--results', > + type=_file_exists, > + required=True, > + help='kirk results.json file location') > + > + parser.add_argument( > + '-o', > + '--override', > + default=False, > + action='store_true', > + help='Override test timeouts') > + > + parser.add_argument( > + '-p', > + '--patch', > + default=False, > + action='store_true', > + help='Patch tests with updated timeout') > + > + parser.add_argument( > + '-t', > + '--print-table', > + default=True, > + action='store_true', > + help='Print table with suggested timeouts') > + > + args = parser.parse_args() > + > + timeouts = _parse_data(args.ltp_dir, args.results) > + > + if args.print_table: > + _print_table(timeouts) > + > + if args.patch: > + _patch_all(args.ltp_dir, timeouts, args.override) > + > + > +if __name__ == "__main__": > + run() > > -- > 2.43.0 > -- Cyril Hrubis chrubis@suse.cz -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] [PATCH 2/2] syscalls: Update test timeouts 2025-01-22 12:48 [LTP] [PATCH 0/2] Update test timeouts in an automated way Andrea Cervesato 2025-01-22 12:48 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Andrea Cervesato @ 2025-01-22 12:48 ` Andrea Cervesato 1 sibling, 0 replies; 8+ messages in thread From: Andrea Cervesato @ 2025-01-22 12:48 UTC (permalink / raw) To: ltp From: Cyril Hrubis <chrubis@suse.cz> This change was genered by: - running syscalls via kirk on RPi zero and saving the json result - running 'cd scripts && calctimeouts.py -l /opt/ltp -r results.json -p' Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- testcases/kernel/syscalls/access/access01.c | 1 + testcases/kernel/syscalls/add_key/add_key05.c | 1 + testcases/kernel/syscalls/alarm/alarm05.c | 1 + testcases/kernel/syscalls/alarm/alarm06.c | 1 + testcases/kernel/syscalls/alarm/alarm07.c | 1 + testcases/kernel/syscalls/bind/bind04.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog05.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog06.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog07.c | 1 + testcases/kernel/syscalls/cachestat/cachestat01.c | 1 + testcases/kernel/syscalls/cachestat/cachestat04.c | 1 + testcases/kernel/syscalls/chdir/chdir01.c | 1 + testcases/kernel/syscalls/clock_gettime/leapsec01.c | 1 + testcases/kernel/syscalls/clock_nanosleep/clock_nanosleep01.c | 1 + testcases/kernel/syscalls/clock_settime/clock_settime03.c | 1 + testcases/kernel/syscalls/close_range/close_range01.c | 1 + testcases/kernel/syscalls/connect/connect02.c | 1 + testcases/kernel/syscalls/creat/creat05.c | 1 + testcases/kernel/syscalls/creat/creat09.c | 1 + testcases/kernel/syscalls/execve/execve05.c | 1 + testcases/kernel/syscalls/execveat/execveat03.c | 1 + testcases/kernel/syscalls/fallocate/fallocate04.c | 1 + testcases/kernel/syscalls/fallocate/fallocate05.c | 1 + testcases/kernel/syscalls/fallocate/fallocate06.c | 2 +- testcases/kernel/syscalls/fanotify/fanotify01.c | 1 + testcases/kernel/syscalls/fanotify/fanotify03.c | 1 + testcases/kernel/syscalls/fanotify/fanotify05.c | 1 + testcases/kernel/syscalls/fanotify/fanotify06.c | 1 + testcases/kernel/syscalls/fanotify/fanotify09.c | 1 + testcases/kernel/syscalls/fanotify/fanotify10.c | 1 + testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c | 1 + testcases/kernel/syscalls/fcntl/fcntl14.c | 1 + testcases/kernel/syscalls/fcntl/fcntl36.c | 1 + testcases/kernel/syscalls/fdatasync/fdatasync03.c | 1 + testcases/kernel/syscalls/fgetxattr/fgetxattr01.c | 1 + testcases/kernel/syscalls/fremovexattr/fremovexattr01.c | 1 + testcases/kernel/syscalls/fremovexattr/fremovexattr02.c | 1 + testcases/kernel/syscalls/fsconfig/fsconfig01.c | 1 + testcases/kernel/syscalls/fsconfig/fsconfig03.c | 1 + testcases/kernel/syscalls/fsetxattr/fsetxattr01.c | 1 + testcases/kernel/syscalls/fsmount/fsmount01.c | 1 + testcases/kernel/syscalls/fsmount/fsmount02.c | 1 + testcases/kernel/syscalls/fsopen/fsopen01.c | 1 + testcases/kernel/syscalls/fspick/fspick01.c | 1 + testcases/kernel/syscalls/fspick/fspick02.c | 1 + testcases/kernel/syscalls/fstatfs/fstatfs01.c | 1 + testcases/kernel/syscalls/fsync/fsync01.c | 1 + testcases/kernel/syscalls/fsync/fsync04.c | 1 + testcases/kernel/syscalls/getpid/getpid01.c | 1 + testcases/kernel/syscalls/getxattr/getxattr02.c | 1 + testcases/kernel/syscalls/getxattr/getxattr03.c | 1 + testcases/kernel/syscalls/inotify/inotify03.c | 1 + testcases/kernel/syscalls/inotify/inotify05.c | 1 + testcases/kernel/syscalls/inotify/inotify07.c | 1 + testcases/kernel/syscalls/inotify/inotify08.c | 1 + testcases/kernel/syscalls/inotify/inotify11.c | 1 + testcases/kernel/syscalls/ioctl/ioctl02.c | 1 + testcases/kernel/syscalls/ioctl/ioctl04.c | 1 + testcases/kernel/syscalls/ioctl/ioctl08.c | 1 + testcases/kernel/syscalls/ioctl/ioctl09.c | 1 + testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c | 1 + testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c | 1 + testcases/kernel/syscalls/ioctl/ioctl_ficlonerange01.c | 1 + testcases/kernel/syscalls/ioctl/ioctl_ficlonerange02.c | 1 + testcases/kernel/syscalls/ioctl/ioctl_loop01.c | 1 + 65 files changed, 65 insertions(+), 1 deletion(-) diff --git a/testcases/kernel/syscalls/access/access01.c b/testcases/kernel/syscalls/access/access01.c index 391c8d44bc7bd5af14713292da988138dec2a16a..1c32c6d046838a2459e9bad09bebde8987fabacc 100644 --- a/testcases/kernel/syscalls/access/access01.c +++ b/testcases/kernel/syscalls/access/access01.c @@ -314,6 +314,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 1, .needs_tmpdir = 1, .needs_root = 1, .forks_child = 1, diff --git a/testcases/kernel/syscalls/add_key/add_key05.c b/testcases/kernel/syscalls/add_key/add_key05.c index 3abd58b833a4f0ac7653cd932f556012f736fc01..c9a2f840e5b97673212dac741f40b2b0b0be0262 100644 --- a/testcases/kernel/syscalls/add_key/add_key05.c +++ b/testcases/kernel/syscalls/add_key/add_key05.c @@ -212,6 +212,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 2, .test = do_test, .tcnt = 2, .needs_root = 1, diff --git a/testcases/kernel/syscalls/alarm/alarm05.c b/testcases/kernel/syscalls/alarm/alarm05.c index 2eeb1c22f3e63cdb6e04d29d21d7306fcfef0050..82bff69cbfd968ec03351769930da0ac3616f8c3 100644 --- a/testcases/kernel/syscalls/alarm/alarm05.c +++ b/testcases/kernel/syscalls/alarm/alarm05.c @@ -44,6 +44,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 2, .test_all = run, .setup = setup, }; diff --git a/testcases/kernel/syscalls/alarm/alarm06.c b/testcases/kernel/syscalls/alarm/alarm06.c index 82c0d44bd2cc0271ea5164b533464a5a9b37e096..2cfa262076e69bbede892f3dd226ad5e86cc3c04 100644 --- a/testcases/kernel/syscalls/alarm/alarm06.c +++ b/testcases/kernel/syscalls/alarm/alarm06.c @@ -41,6 +41,7 @@ static void verify_alarm(void) } static struct tst_test test = { + .timeout = 4, .setup = setup, .test_all = verify_alarm, }; diff --git a/testcases/kernel/syscalls/alarm/alarm07.c b/testcases/kernel/syscalls/alarm/alarm07.c index 64aed507dec7b042545e18fb2cddfbaa5becb1be..c04745f9dfb1ffc9e552f5df54a6161b6c9671bf 100644 --- a/testcases/kernel/syscalls/alarm/alarm07.c +++ b/testcases/kernel/syscalls/alarm/alarm07.c @@ -47,6 +47,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 4, .test_all = verify_alarm, .setup = setup, .forks_child = 1, diff --git a/testcases/kernel/syscalls/bind/bind04.c b/testcases/kernel/syscalls/bind/bind04.c index d8456e739a9460b7201c975cb01d53d1126d1f7e..2a46559dead7ae64d5b056a6c0b07f49d35bdb43 100644 --- a/testcases/kernel/syscalls/bind/bind04.c +++ b/testcases/kernel/syscalls/bind/bind04.c @@ -161,6 +161,7 @@ static void test_bind(unsigned int n) } static struct tst_test test = { + .timeout = 1, .test = test_bind, .tcnt = ARRAY_SIZE(testcase_list), .needs_tmpdir = 1, diff --git a/testcases/kernel/syscalls/bpf/bpf_prog05.c b/testcases/kernel/syscalls/bpf/bpf_prog05.c index 08254ba8930d390b32bd6a525ceb9223036dba3a..0500817128315b333a8ec81cc535fd95e74c0904 100644 --- a/testcases/kernel/syscalls/bpf/bpf_prog05.c +++ b/testcases/kernel/syscalls/bpf/bpf_prog05.c @@ -190,6 +190,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 20, .setup = setup, .test_all = run, .taint_check = TST_TAINT_W | TST_TAINT_D, diff --git a/testcases/kernel/syscalls/bpf/bpf_prog06.c b/testcases/kernel/syscalls/bpf/bpf_prog06.c index cee9616cf34fedd32b0c492bc6151a52e088f6b0..87fdd806033a36162c2367eeaf005c2fa12e1df0 100644 --- a/testcases/kernel/syscalls/bpf/bpf_prog06.c +++ b/testcases/kernel/syscalls/bpf/bpf_prog06.c @@ -131,6 +131,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 20, .setup = setup, .test_all = run, .min_kver = "5.8", diff --git a/testcases/kernel/syscalls/bpf/bpf_prog07.c b/testcases/kernel/syscalls/bpf/bpf_prog07.c index dab5bb8ad16c12b832efc28d9c75417a206a5902..29dfbf2cae4704b8c5a260f28eb5a504e0e3ea39 100644 --- a/testcases/kernel/syscalls/bpf/bpf_prog07.c +++ b/testcases/kernel/syscalls/bpf/bpf_prog07.c @@ -139,6 +139,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 20, .setup = setup, .test_all = run, .min_kver = "5.8", diff --git a/testcases/kernel/syscalls/cachestat/cachestat01.c b/testcases/kernel/syscalls/cachestat/cachestat01.c index 2dca688854d8203886d724cf8d57bb04d9d0d325..22f0071d1e2b68f76986271da24f4501b8eac70b 100644 --- a/testcases/kernel/syscalls/cachestat/cachestat01.c +++ b/testcases/kernel/syscalls/cachestat/cachestat01.c @@ -85,6 +85,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 13, .test = run, .tcnt = 2, .setup = setup, diff --git a/testcases/kernel/syscalls/cachestat/cachestat04.c b/testcases/kernel/syscalls/cachestat/cachestat04.c index a59494451a00c37207a6895771b189eb1a7a713f..7cef9897d416f24b39eebc3d53a0bb3a17007613 100644 --- a/testcases/kernel/syscalls/cachestat/cachestat04.c +++ b/testcases/kernel/syscalls/cachestat/cachestat04.c @@ -47,6 +47,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 2, .test_all = run, .mount_device = 1, .mntpoint = MNTPOINT, diff --git a/testcases/kernel/syscalls/chdir/chdir01.c b/testcases/kernel/syscalls/chdir/chdir01.c index d50a8f50c6c3ccb600a3e91f8b128739f09eb7d5..1c1299332741ab90fa7de552187bc5dadbcbeb54 100644 --- a/testcases/kernel/syscalls/chdir/chdir01.c +++ b/testcases/kernel/syscalls/chdir/chdir01.c @@ -149,6 +149,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 10, .needs_root = 1, .mount_device = 1, .mntpoint = MNTPOINT, diff --git a/testcases/kernel/syscalls/clock_gettime/leapsec01.c b/testcases/kernel/syscalls/clock_gettime/leapsec01.c index e623b4107c980b9e513f01f8acbf2d1e4e941021..4cd469dc277c66b316c9054a3f819270d0a520be 100644 --- a/testcases/kernel/syscalls/clock_gettime/leapsec01.c +++ b/testcases/kernel/syscalls/clock_gettime/leapsec01.c @@ -196,6 +196,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 40, .test_all = run_leapsec, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/clock_nanosleep/clock_nanosleep01.c b/testcases/kernel/syscalls/clock_nanosleep/clock_nanosleep01.c index eef8a59921793bd2329097ab1a7d82c8dec1d92f..55e7b294ee8a33a8d63c8cbdb8287b76cbbe6f5c 100644 --- a/testcases/kernel/syscalls/clock_nanosleep/clock_nanosleep01.c +++ b/testcases/kernel/syscalls/clock_nanosleep/clock_nanosleep01.c @@ -219,6 +219,7 @@ static void do_test(unsigned int i) } static struct tst_test test = { + .timeout = 3, .tcnt = ARRAY_SIZE(tcase), .test = do_test, .test_variants = ARRAY_SIZE(variants), diff --git a/testcases/kernel/syscalls/clock_settime/clock_settime03.c b/testcases/kernel/syscalls/clock_settime/clock_settime03.c index f196a257c75b7c1e6b6264707ecefdd4dd8c36ca..692e63788e5cc25a90041d5d27ee725227622d2c 100644 --- a/testcases/kernel/syscalls/clock_settime/clock_settime03.c +++ b/testcases/kernel/syscalls/clock_settime/clock_settime03.c @@ -104,6 +104,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 4, .test_all = run, .test_variants = ARRAY_SIZE(variants), .setup = setup, diff --git a/testcases/kernel/syscalls/close_range/close_range01.c b/testcases/kernel/syscalls/close_range/close_range01.c index 110ffa9b795f2b4311824dd72ce46945deae3371..941f3e4038dfc5eb4ab771bdde69b6532943caed 100644 --- a/testcases/kernel/syscalls/close_range/close_range01.c +++ b/testcases/kernel/syscalls/close_range/close_range01.c @@ -191,6 +191,7 @@ static void run(unsigned int n) } static struct tst_test test = { + .timeout = 9, .tcnt = 4, .forks_child = 1, .mount_device = 1, diff --git a/testcases/kernel/syscalls/connect/connect02.c b/testcases/kernel/syscalls/connect/connect02.c index e20214e243feded16c8cef57ca5b994f6d57e206..087b514bf48c800c2d47a978b38249dc25dd4c32 100644 --- a/testcases/kernel/syscalls/connect/connect02.c +++ b/testcases/kernel/syscalls/connect/connect02.c @@ -126,6 +126,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 3, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/creat/creat05.c b/testcases/kernel/syscalls/creat/creat05.c index bf409943988bc90afff28643caa02f9321bc7dc0..32074a449ac3b74e4a696f5e75af667afd147140 100644 --- a/testcases/kernel/syscalls/creat/creat05.c +++ b/testcases/kernel/syscalls/creat/creat05.c @@ -74,6 +74,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test_all = verify_creat, .needs_tmpdir = 1, .setup = setup, diff --git a/testcases/kernel/syscalls/creat/creat09.c b/testcases/kernel/syscalls/creat/creat09.c index a5d3740ac6fdb95381168ae00d859c6616e3d7c2..d1816577282532848d57452f312e968841d6db0c 100644 --- a/testcases/kernel/syscalls/creat/creat09.c +++ b/testcases/kernel/syscalls/creat/creat09.c @@ -138,6 +138,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/execve/execve05.c b/testcases/kernel/syscalls/execve/execve05.c index d87d7446d187176740ad1e33a8ee2196f19bb097..530c7cc8903d7f809e6e9a91888654749dcb0c41 100644 --- a/testcases/kernel/syscalls/execve/execve05.c +++ b/testcases/kernel/syscalls/execve/execve05.c @@ -62,6 +62,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 3, .test_all = verify_execve, .options = (struct tst_option[]) { {"n:", &opt_nchild, "Numbers of children"}, diff --git a/testcases/kernel/syscalls/execveat/execveat03.c b/testcases/kernel/syscalls/execveat/execveat03.c index 057d83278af282c56ef93c375aeb05a20be48f0f..684f0d0c68c951f5d32cc33555834e860f5166bc 100644 --- a/testcases/kernel/syscalls/execveat/execveat03.c +++ b/testcases/kernel/syscalls/execveat/execveat03.c @@ -68,6 +68,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 1, .needs_root = 1, .mount_device = 1, .needs_overlay = 1, diff --git a/testcases/kernel/syscalls/fallocate/fallocate04.c b/testcases/kernel/syscalls/fallocate/fallocate04.c index 7b285fb5d10ee45a49937cf3823c4ca9cd576f0d..3a8ea5fa75773d95b791d4f2f0e52232b218d406 100644 --- a/testcases/kernel/syscalls/fallocate/fallocate04.c +++ b/testcases/kernel/syscalls/fallocate/fallocate04.c @@ -282,6 +282,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 9, .options = (struct tst_option[]) { {"v", &verbose, "Turns on verbose mode"}, {} diff --git a/testcases/kernel/syscalls/fallocate/fallocate05.c b/testcases/kernel/syscalls/fallocate/fallocate05.c index 732a2f15da9f40bfe0f1ce151723023a896e9b24..f17cc993e3db992af5edce0a28aa30fa8eb7b87f 100644 --- a/testcases/kernel/syscalls/fallocate/fallocate05.c +++ b/testcases/kernel/syscalls/fallocate/fallocate05.c @@ -172,6 +172,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 42, .needs_root = 1, .mount_device = 1, .mntpoint = MNTPOINT, diff --git a/testcases/kernel/syscalls/fallocate/fallocate06.c b/testcases/kernel/syscalls/fallocate/fallocate06.c index 5e1454c3022977ef904bc481dabd7941d6e3b800..0e1509ace625097929fd5e8dce471c78166e0821 100644 --- a/testcases/kernel/syscalls/fallocate/fallocate06.c +++ b/testcases/kernel/syscalls/fallocate/fallocate06.c @@ -261,7 +261,7 @@ static struct tst_test test = { .tcnt = ARRAY_SIZE(testcase_list), .needs_root = 1, .dev_min_size = 1024, - .timeout = 120, + .timeout = 150, .mount_device = 1, .mntpoint = MNTPOINT, .all_filesystems = 1, diff --git a/testcases/kernel/syscalls/fanotify/fanotify01.c b/testcases/kernel/syscalls/fanotify/fanotify01.c index 5d18fe8512529f348ab497cb463d221f4e8de902..c7e759166d1f5580e3eb482075db0d1e633f1cc1 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify01.c +++ b/testcases/kernel/syscalls/fanotify/fanotify01.c @@ -387,6 +387,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 10, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/fanotify/fanotify03.c b/testcases/kernel/syscalls/fanotify/fanotify03.c index 3ed78e5fab4fcb40603c1a49ec3f5a6cd6feda25..64c933c190081331b3705a7c9181fec836c774a1 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify03.c +++ b/testcases/kernel/syscalls/fanotify/fanotify03.c @@ -343,6 +343,7 @@ static const char *const resource_files[] = { }; static struct tst_test test = { + .timeout = 1, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/fanotify/fanotify05.c b/testcases/kernel/syscalls/fanotify/fanotify05.c index 12c2408818dfcb10915c555ee4a9e8aeedcd8a5a..435a91c49312a879aeb15da05f6e9ea3da4be333 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify05.c +++ b/testcases/kernel/syscalls/fanotify/fanotify05.c @@ -208,6 +208,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 13, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/fanotify/fanotify06.c b/testcases/kernel/syscalls/fanotify/fanotify06.c index 8779e34d95dd704874b0711b256693e4c7c98d89..b4992afa0f95aba9e700f4723eb73db30df42fc1 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify06.c +++ b/testcases/kernel/syscalls/fanotify/fanotify06.c @@ -236,6 +236,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/fanotify/fanotify09.c b/testcases/kernel/syscalls/fanotify/fanotify09.c index 48b198b9415a0c875fb827bc5caf92d5f33adf97..45303c31e76ef6b66b0fbe7173ae3c8f05131e61 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify09.c +++ b/testcases/kernel/syscalls/fanotify/fanotify09.c @@ -507,6 +507,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/fanotify/fanotify10.c b/testcases/kernel/syscalls/fanotify/fanotify10.c index eedd1442f5a5860ba1bd4019110623e9ba55b3d7..23e9554b39da33533f4829c8a0e0fb3306119605 100644 --- a/testcases/kernel/syscalls/fanotify/fanotify10.c +++ b/testcases/kernel/syscalls/fanotify/fanotify10.c @@ -959,6 +959,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 10, .test = test_fanotify, .tcnt = ARRAY_SIZE(tcases), .test_variants = 2, diff --git a/testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c b/testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c index f026b18df570162ff720e6d54095ef44bb32d5d6..e58be31c42991bc8f9a67663d530eaa2d0afb697 100644 --- a/testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c +++ b/testcases/kernel/syscalls/fchmodat2/fchmodat2_01.c @@ -104,6 +104,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 9, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fcntl/fcntl14.c b/testcases/kernel/syscalls/fcntl/fcntl14.c index 367701d68cac191a560aff5aade881b6f60c9eae..21dbede5c31f119542a7181d0e53730dfb103ba5 100644 --- a/testcases/kernel/syscalls/fcntl/fcntl14.c +++ b/testcases/kernel/syscalls/fcntl/fcntl14.c @@ -263,6 +263,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 8, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fcntl/fcntl36.c b/testcases/kernel/syscalls/fcntl/fcntl36.c index fabdbe90591bf1c0603f5053cb62f6f20b6e1f73..19a5a0303bf75435d1388f2bcafd02373c9b1253 100644 --- a/testcases/kernel/syscalls/fcntl/fcntl36.c +++ b/testcases/kernel/syscalls/fcntl/fcntl36.c @@ -387,6 +387,7 @@ static void tests(unsigned int i) } static struct tst_test test = { + .timeout = 9, .needs_tmpdir = 1, .test = tests, .tcnt = ARRAY_SIZE(tcases), diff --git a/testcases/kernel/syscalls/fdatasync/fdatasync03.c b/testcases/kernel/syscalls/fdatasync/fdatasync03.c index 5f3e0c96f52ec44172c40e93af971556705c37b9..f1b5f0fb822cd137e0a0208696d2d1f49ddcdae1 100644 --- a/testcases/kernel/syscalls/fdatasync/fdatasync03.c +++ b/testcases/kernel/syscalls/fdatasync/fdatasync03.c @@ -53,6 +53,7 @@ static void verify_fdatasync(void) } static struct tst_test test = { + .timeout = 15, .needs_root = 1, .mount_device = 1, .all_filesystems = 1, diff --git a/testcases/kernel/syscalls/fgetxattr/fgetxattr01.c b/testcases/kernel/syscalls/fgetxattr/fgetxattr01.c index 52e6e44ab8e3ce7f4ea6fad45ccf3d2a0a60eb45..eefddd829b474e0049779231b3fca6c10396ee4c 100644 --- a/testcases/kernel/syscalls/fgetxattr/fgetxattr01.c +++ b/testcases/kernel/syscalls/fgetxattr/fgetxattr01.c @@ -134,6 +134,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 10, .setup = setup, .test = verify_fgetxattr, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fremovexattr/fremovexattr01.c b/testcases/kernel/syscalls/fremovexattr/fremovexattr01.c index 907d210d1c3d2aa13ae2facc201b5030d5185894..aca2fce7c3d1d551166019a9c57c4a68e8cc4104 100644 --- a/testcases/kernel/syscalls/fremovexattr/fremovexattr01.c +++ b/testcases/kernel/syscalls/fremovexattr/fremovexattr01.c @@ -83,6 +83,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 12, .setup = setup, .test_all = verify_fremovexattr, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fremovexattr/fremovexattr02.c b/testcases/kernel/syscalls/fremovexattr/fremovexattr02.c index eb106a8fa1eca9af5e3f416cf962049e2083d3aa..78af8be326e6f05fe2ef02840a9cc76834896fac 100644 --- a/testcases/kernel/syscalls/fremovexattr/fremovexattr02.c +++ b/testcases/kernel/syscalls/fremovexattr/fremovexattr02.c @@ -105,6 +105,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 10, .setup = setup, .test = verify_fremovexattr, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fsconfig/fsconfig01.c b/testcases/kernel/syscalls/fsconfig/fsconfig01.c index a585daa6d6a714be7b97d6215700c142eed20fb1..678d21815c5c2c546e9deca77899f99852453060 100644 --- a/testcases/kernel/syscalls/fsconfig/fsconfig01.c +++ b/testcases/kernel/syscalls/fsconfig/fsconfig01.c @@ -82,6 +82,7 @@ static void run(void) } static struct tst_test test = { + .timeout = 10, .test_all = run, .setup = fsopen_supported_by_kernel, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fsconfig/fsconfig03.c b/testcases/kernel/syscalls/fsconfig/fsconfig03.c index 0ba5355d3a330ff9702e3348226729402ea85b9f..5037536360aecf64c98054489258ad0306a05f00 100644 --- a/testcases/kernel/syscalls/fsconfig/fsconfig03.c +++ b/testcases/kernel/syscalls/fsconfig/fsconfig03.c @@ -80,6 +80,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 9, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fsetxattr/fsetxattr01.c b/testcases/kernel/syscalls/fsetxattr/fsetxattr01.c index b65b27bdf9ac0774b5d9a61549c77389ac1f37e2..73e1fcfb7292c333e5c05a2e8dc2307e2146ea47 100644 --- a/testcases/kernel/syscalls/fsetxattr/fsetxattr01.c +++ b/testcases/kernel/syscalls/fsetxattr/fsetxattr01.c @@ -214,6 +214,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 10, .setup = setup, .test = verify_fsetxattr, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/fsmount/fsmount01.c b/testcases/kernel/syscalls/fsmount/fsmount01.c index 5f755863f43aee0ee0988f72b4cbd2defcf49379..6dafe25a8fdaa2f71469dd81dd597d2b824053e5 100644 --- a/testcases/kernel/syscalls/fsmount/fsmount01.c +++ b/testcases/kernel/syscalls/fsmount/fsmount01.c @@ -88,6 +88,7 @@ static void run(unsigned int n) } static struct tst_test test = { + .timeout = 10, .tcnt = ARRAY_SIZE(tcases), .test = run, .setup = fsopen_supported_by_kernel, diff --git a/testcases/kernel/syscalls/fsmount/fsmount02.c b/testcases/kernel/syscalls/fsmount/fsmount02.c index a4f42dc188220a7f31807b5eb38563520e06f0aa..55f0e2f28d04938486d3a5855337e6312cfa45fd 100644 --- a/testcases/kernel/syscalls/fsmount/fsmount02.c +++ b/testcases/kernel/syscalls/fsmount/fsmount02.c @@ -68,6 +68,7 @@ static void run(unsigned int n) } static struct tst_test test = { + .timeout = 9, .tcnt = ARRAY_SIZE(tcases), .test = run, .setup = setup, diff --git a/testcases/kernel/syscalls/fsopen/fsopen01.c b/testcases/kernel/syscalls/fsopen/fsopen01.c index c2c719c9678e5860303994a140d0b35146af0f62..9dd87b99f02771a9ce48e9ecce8e95cdf1474007 100644 --- a/testcases/kernel/syscalls/fsopen/fsopen01.c +++ b/testcases/kernel/syscalls/fsopen/fsopen01.c @@ -69,6 +69,7 @@ out: } static struct tst_test test = { + .timeout = 9, .tcnt = ARRAY_SIZE(tcases), .test = run, .setup = fsopen_supported_by_kernel, diff --git a/testcases/kernel/syscalls/fspick/fspick01.c b/testcases/kernel/syscalls/fspick/fspick01.c index d3309a9124cbd538853e38a278ea2ece97cbcba7..d03cacd3d9b07a3c8c49d3664f014befbb0bea09 100644 --- a/testcases/kernel/syscalls/fspick/fspick01.c +++ b/testcases/kernel/syscalls/fspick/fspick01.c @@ -56,6 +56,7 @@ out: } static struct tst_test test = { + .timeout = 9, .tcnt = ARRAY_SIZE(tcases), .test = run, .setup = fsopen_supported_by_kernel, diff --git a/testcases/kernel/syscalls/fspick/fspick02.c b/testcases/kernel/syscalls/fspick/fspick02.c index f9a3697c1f8f577b095b65d3156cdd8a572d168f..89bdd2cc81c51a43bb2106764575037d4de7f697 100644 --- a/testcases/kernel/syscalls/fspick/fspick02.c +++ b/testcases/kernel/syscalls/fspick/fspick02.c @@ -43,6 +43,7 @@ static void run(unsigned int n) } static struct tst_test test = { + .timeout = 9, .tcnt = ARRAY_SIZE(tcases), .test = run, .setup = fsopen_supported_by_kernel, diff --git a/testcases/kernel/syscalls/fstatfs/fstatfs01.c b/testcases/kernel/syscalls/fstatfs/fstatfs01.c index 9d3909aceda671a659a882467964d9a11d8e8d4c..044ab65488727dbd2384111411d00bbc1ecc743d 100644 --- a/testcases/kernel/syscalls/fstatfs/fstatfs01.c +++ b/testcases/kernel/syscalls/fstatfs/fstatfs01.c @@ -56,6 +56,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 9, .setup = setup, .cleanup = cleanup, .tcnt = ARRAY_SIZE(tcases), diff --git a/testcases/kernel/syscalls/fsync/fsync01.c b/testcases/kernel/syscalls/fsync/fsync01.c index 072245fc83ca2bf8e39aa3e0827a3a724844c813..7ae5296c7cf251717cf66f8ee64ea91691b4efd6 100644 --- a/testcases/kernel/syscalls/fsync/fsync01.c +++ b/testcases/kernel/syscalls/fsync/fsync01.c @@ -44,6 +44,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 10, .cleanup = cleanup, .setup = setup, .test_all = verify_fsync, diff --git a/testcases/kernel/syscalls/fsync/fsync04.c b/testcases/kernel/syscalls/fsync/fsync04.c index 9aa1584c1f2b4c6d1e91b90282fa4ed840cfd247..f7553ff58029d1e5cf85132435763af931cdf894 100644 --- a/testcases/kernel/syscalls/fsync/fsync04.c +++ b/testcases/kernel/syscalls/fsync/fsync04.c @@ -53,6 +53,7 @@ static void verify_fsync(void) } static struct tst_test test = { + .timeout = 17, .needs_root = 1, .mount_device = 1, .all_filesystems = 1, diff --git a/testcases/kernel/syscalls/getpid/getpid01.c b/testcases/kernel/syscalls/getpid/getpid01.c index 495002037380fa090585eccc9b001bcb9a5ad400..314c45d0948e8a025f478745577a021aec721e37 100644 --- a/testcases/kernel/syscalls/getpid/getpid01.c +++ b/testcases/kernel/syscalls/getpid/getpid01.c @@ -43,6 +43,7 @@ static void verify_getpid(void) } static struct tst_test test = { + .timeout = 1, .setup = setup, .forks_child = 1, .test_all = verify_getpid, diff --git a/testcases/kernel/syscalls/getxattr/getxattr02.c b/testcases/kernel/syscalls/getxattr/getxattr02.c index 5a84d876c5c00799ee04d74ba9134c6b16c1c569..593ad7b8c70e0680cd0908c7fa1fb8caccb32b7e 100644 --- a/testcases/kernel/syscalls/getxattr/getxattr02.c +++ b/testcases/kernel/syscalls/getxattr/getxattr02.c @@ -105,6 +105,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 10, .all_filesystems = 1, .needs_root = 1, .mntpoint = MNTPOINT, diff --git a/testcases/kernel/syscalls/getxattr/getxattr03.c b/testcases/kernel/syscalls/getxattr/getxattr03.c index 1fe4ba48fc1001a840940a3d5730ff4afee35d5a..85f26402af2e3ff1c7e68de1e0dd11166b67fed9 100644 --- a/testcases/kernel/syscalls/getxattr/getxattr03.c +++ b/testcases/kernel/syscalls/getxattr/getxattr03.c @@ -37,6 +37,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 14, .all_filesystems = 1, .needs_root = 1, .mntpoint = MNTPOINT, diff --git a/testcases/kernel/syscalls/inotify/inotify03.c b/testcases/kernel/syscalls/inotify/inotify03.c index ff025b36025988e8e22bc97abf3ef0294a996321..4ee5e9461356922902ccf4bffa8a726177fc7401 100644 --- a/testcases/kernel/syscalls/inotify/inotify03.c +++ b/testcases/kernel/syscalls/inotify/inotify03.c @@ -166,6 +166,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .needs_root = 1, .format_device = 1, .setup = setup, diff --git a/testcases/kernel/syscalls/inotify/inotify05.c b/testcases/kernel/syscalls/inotify/inotify05.c index a1597217f581314efda1ada5404861e0ac573a88..38b30878c18be9816ef937cee9b4ad50393d1733 100644 --- a/testcases/kernel/syscalls/inotify/inotify05.c +++ b/testcases/kernel/syscalls/inotify/inotify05.c @@ -148,6 +148,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .needs_tmpdir = 1, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/inotify/inotify07.c b/testcases/kernel/syscalls/inotify/inotify07.c index 08ea1e06aaa20ee1735efda8eb46628c2b55be49..bcfc569a8d0f2d65fb44f5a44e1d830c069479e5 100644 --- a/testcases/kernel/syscalls/inotify/inotify07.c +++ b/testcases/kernel/syscalls/inotify/inotify07.c @@ -182,6 +182,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .needs_root = 1, .mount_device = 1, .needs_overlay = 1, diff --git a/testcases/kernel/syscalls/inotify/inotify08.c b/testcases/kernel/syscalls/inotify/inotify08.c index 9c2ecdabdab633f2881681f643c5223eb8c18d0c..149a0a1f817d49322bc8f9ec9e388c74a40ff11c 100644 --- a/testcases/kernel/syscalls/inotify/inotify08.c +++ b/testcases/kernel/syscalls/inotify/inotify08.c @@ -176,6 +176,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .needs_root = 1, .mount_device = 1, .needs_overlay = 1, diff --git a/testcases/kernel/syscalls/inotify/inotify11.c b/testcases/kernel/syscalls/inotify/inotify11.c index 1630477fcc3b2753e02fc96c481d9ff4d5fab975..2e571f3a0c72adb71503e9cff3fad0ce2b608610 100644 --- a/testcases/kernel/syscalls/inotify/inotify11.c +++ b/testcases/kernel/syscalls/inotify/inotify11.c @@ -119,6 +119,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 12, .needs_tmpdir = 1, .forks_child = 1, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl02.c b/testcases/kernel/syscalls/ioctl/ioctl02.c index f3bfb239ab9eae620bf93de1dd4b8804312af082..9337da384ab16840a04224a9a4614061105759e3 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl02.c +++ b/testcases/kernel/syscalls/ioctl/ioctl02.c @@ -242,6 +242,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 9, .needs_root = 1, .needs_checkpoints = 1, .forks_child = 1, diff --git a/testcases/kernel/syscalls/ioctl/ioctl04.c b/testcases/kernel/syscalls/ioctl/ioctl04.c index 262c06e97f887fdb8767556f92fe9b518fb90616..f96b22f0ce83fb3ef5bba6d8164c8398fcdee991 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl04.c +++ b/testcases/kernel/syscalls/ioctl/ioctl04.c @@ -84,6 +84,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .format_device = 1, .needs_root = 1, .setup = setup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl08.c b/testcases/kernel/syscalls/ioctl/ioctl08.c index 834c5ae7461db9864cbf0d64767399f7ebb9559e..14f9354c9b8afdc417498c7031c0b44442472a1a 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl08.c +++ b/testcases/kernel/syscalls/ioctl/ioctl08.c @@ -118,6 +118,7 @@ static void setup(void) static struct tst_test test = { + .timeout = 1, .test = verify_ioctl, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl09.c b/testcases/kernel/syscalls/ioctl/ioctl09.c index 9c79210864b8e932f8d93fe7bc395d0eadddedb7..0d1f1072d3420e3f12677360da307cf78e79c57a 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl09.c +++ b/testcases/kernel/syscalls/ioctl/ioctl09.c @@ -110,6 +110,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .setup = setup, .cleanup = cleanup, .test_all = verify_ioctl, diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c b/testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c index fab0daaeef32a1149d6356d9c583b9afd495e27c..e5d4be96f7b40f7132b6f210a1a542f2becddb95 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c @@ -51,6 +51,7 @@ static void setup(void) } static struct tst_test test = { + .timeout = 10, .test_all = run, .setup = setup, .min_kver = "4.5", diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c b/testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c index f12c4e7fdb28cd59a34d2f0ce3c61fc168eae7dd..b0a0f8d63fdb222d6fd3bbcbb2fa4476d3b25033 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_ficlone03.c @@ -101,6 +101,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test = run, .tcnt = ARRAY_SIZE(tcases), .setup = setup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange01.c b/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange01.c index 8fd1d299a4b2a061c9ea4c0f556f5be665497d10..70727076517cf73a2a61a21c24806632ba7fb529 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange01.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange01.c @@ -137,6 +137,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange02.c b/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange02.c index eb941c3fc17ff445deba388e4b42ea4f7f6c58a8..cc1d0e031920df1836c79653c70aa3b724b483f5 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange02.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_ficlonerange02.c @@ -74,6 +74,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 4, .test_all = run, .setup = setup, .cleanup = cleanup, diff --git a/testcases/kernel/syscalls/ioctl/ioctl_loop01.c b/testcases/kernel/syscalls/ioctl/ioctl_loop01.c index e7b337e4acfef388d508f7a031809ffd53428291..c3fb9f583ffe98ab19d90dcebc2203d0cb6055e3 100644 --- a/testcases/kernel/syscalls/ioctl/ioctl_loop01.c +++ b/testcases/kernel/syscalls/ioctl/ioctl_loop01.c @@ -140,6 +140,7 @@ static void cleanup(void) } static struct tst_test test = { + .timeout = 1, .setup = setup, .cleanup = cleanup, .test_all = verify_ioctl_loop, -- 2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [LTP] [PATCH 0/2] Update test timeouts in an automated way @ 2025-01-21 12:34 Cyril Hrubis 2025-01-21 12:34 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Cyril Hrubis 0 siblings, 1 reply; 8+ messages in thread From: Cyril Hrubis @ 2025-01-21 12:34 UTC (permalink / raw) To: ltp First patch adds a script that can parse kirk test results, calculate timeouts and patch tests with the newly calculated timeouts. Second patch updates tests timeouts using this script with a results captured on RPi zero. Cyril Hrubis (2): scripts: Add simple script for calculating timeouts syscalls: Update test timeouts scripts/calc_timeouts.py | 133 ++++++++++++++++++ testcases/kernel/syscalls/access/access01.c | 1 + testcases/kernel/syscalls/add_key/add_key05.c | 1 + testcases/kernel/syscalls/alarm/alarm05.c | 1 + testcases/kernel/syscalls/alarm/alarm06.c | 1 + testcases/kernel/syscalls/alarm/alarm07.c | 1 + testcases/kernel/syscalls/bind/bind04.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog05.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog06.c | 1 + testcases/kernel/syscalls/bpf/bpf_prog07.c | 1 + .../kernel/syscalls/cachestat/cachestat01.c | 1 + .../kernel/syscalls/cachestat/cachestat04.c | 1 + testcases/kernel/syscalls/chdir/chdir01.c | 1 + .../kernel/syscalls/clock_gettime/leapsec01.c | 1 + .../clock_nanosleep/clock_nanosleep01.c | 1 + .../syscalls/clock_settime/clock_settime03.c | 1 + .../syscalls/close_range/close_range01.c | 1 + testcases/kernel/syscalls/connect/connect02.c | 1 + testcases/kernel/syscalls/creat/creat05.c | 1 + testcases/kernel/syscalls/creat/creat09.c | 1 + testcases/kernel/syscalls/execve/execve05.c | 1 + .../kernel/syscalls/execveat/execveat03.c | 1 + .../kernel/syscalls/fallocate/fallocate04.c | 1 + .../kernel/syscalls/fallocate/fallocate05.c | 1 + .../kernel/syscalls/fallocate/fallocate06.c | 2 +- .../kernel/syscalls/fanotify/fanotify01.c | 1 + .../kernel/syscalls/fanotify/fanotify03.c | 1 + .../kernel/syscalls/fanotify/fanotify05.c | 1 + .../kernel/syscalls/fanotify/fanotify06.c | 1 + .../kernel/syscalls/fanotify/fanotify09.c | 1 + .../kernel/syscalls/fanotify/fanotify10.c | 1 + .../kernel/syscalls/fchmodat2/fchmodat2_01.c | 1 + testcases/kernel/syscalls/fcntl/fcntl14.c | 1 + testcases/kernel/syscalls/fcntl/fcntl36.c | 1 + .../kernel/syscalls/fdatasync/fdatasync03.c | 1 + .../kernel/syscalls/fgetxattr/fgetxattr01.c | 1 + .../syscalls/fremovexattr/fremovexattr01.c | 1 + .../syscalls/fremovexattr/fremovexattr02.c | 1 + .../kernel/syscalls/fsconfig/fsconfig01.c | 1 + .../kernel/syscalls/fsconfig/fsconfig03.c | 1 + .../kernel/syscalls/fsetxattr/fsetxattr01.c | 1 + testcases/kernel/syscalls/fsmount/fsmount01.c | 1 + testcases/kernel/syscalls/fsmount/fsmount02.c | 1 + testcases/kernel/syscalls/fsopen/fsopen01.c | 1 + testcases/kernel/syscalls/fspick/fspick01.c | 1 + testcases/kernel/syscalls/fspick/fspick02.c | 1 + testcases/kernel/syscalls/fstatfs/fstatfs01.c | 1 + testcases/kernel/syscalls/fsync/fsync01.c | 1 + testcases/kernel/syscalls/fsync/fsync04.c | 1 + testcases/kernel/syscalls/getpid/getpid01.c | 1 + .../kernel/syscalls/getxattr/getxattr02.c | 1 + .../kernel/syscalls/getxattr/getxattr03.c | 1 + testcases/kernel/syscalls/inotify/inotify03.c | 1 + testcases/kernel/syscalls/inotify/inotify05.c | 1 + testcases/kernel/syscalls/inotify/inotify07.c | 1 + testcases/kernel/syscalls/inotify/inotify08.c | 1 + testcases/kernel/syscalls/inotify/inotify11.c | 1 + testcases/kernel/syscalls/ioctl/ioctl02.c | 1 + testcases/kernel/syscalls/ioctl/ioctl04.c | 1 + testcases/kernel/syscalls/ioctl/ioctl08.c | 1 + testcases/kernel/syscalls/ioctl/ioctl09.c | 1 + .../kernel/syscalls/ioctl/ioctl_ficlone02.c | 1 + .../kernel/syscalls/ioctl/ioctl_ficlone03.c | 1 + .../syscalls/ioctl/ioctl_ficlonerange01.c | 1 + .../syscalls/ioctl/ioctl_ficlonerange02.c | 1 + .../kernel/syscalls/ioctl/ioctl_loop01.c | 1 + 66 files changed, 198 insertions(+), 1 deletion(-) create mode 100755 scripts/calc_timeouts.py -- 2.45.2 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-21 12:34 [LTP] [PATCH 0/2] Update test timeouts in an automated way Cyril Hrubis @ 2025-01-21 12:34 ` Cyril Hrubis 2025-01-21 13:12 ` Andrea Cervesato via ltp 0 siblings, 1 reply; 8+ messages in thread From: Cyril Hrubis @ 2025-01-21 12:34 UTC (permalink / raw) To: ltp This script parses JSON results from kirk and LTP metadata in order calculate timeouts for tests based on the result file and can even patch tests automatically. The script does: - Take the results and pick all tests that run for longer than 0.5s. Multiplies the time with a constant (currently 1.2) to get a suggested timeout. - Exclude tests that have runtime defined since these are controller by the runtime (that filters out all fuzzy sync tests). There is a special case for timer tests that define runtime only dynamically in the timer library code. This should be possibly fixed with special value for the .runtime in tst_test. E.g. TST_RUNTIME_DYNAMIC for tests that only set runtime in the setup. - Normalize the timeout per a single filesystem run if test is running for more than one filesystem. - Tests that do not have a metadata record are old library tests which which cannot be patched but are printed in a separate table if we request a table to be printed. - If patching option is selected tests are update with newly calculated timeout. By default we only increase timeouts but that can be overrided with the -o option. Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- scripts/calc_timeouts.py | 133 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+) create mode 100755 scripts/calc_timeouts.py diff --git a/scripts/calc_timeouts.py b/scripts/calc_timeouts.py new file mode 100755 index 000000000..c69ab8f57 --- /dev/null +++ b/scripts/calc_timeouts.py @@ -0,0 +1,133 @@ +#!/usr/bin/python +import re +import json +import getopt +import sys +from os import system + +# Top level directory path +top_dir = '../' + +# The test runtime is multiplied by this to get a timeout +timeout_mul = 1.2 + +def patch(fname, new_timeout, patch_override): + + orig_timeout = None + file_path = top_dir + fname + + with open(file_path, 'r') as c_source: + for line in c_source: + timeout = re.search(r'\s*.timeout\s*=\s*(\d+).', line) + if timeout: + orig_timeout = int(timeout.group(1)) + + if orig_timeout: + if orig_timeout < new_timeout or patch_override: + print("CHANGE %s timeout %i -> %i" % (fname, orig_timeout, new_timeout)) + system("sed -i 's/\\.timeout = [0-9]*/\\.timeout = " + str(new_timeout) + "/' " + file_path) + else: + print("KEEP %s timeout %i (new %i)" % (fname, orig_timeout, new_timeout)) + else: + print("ADD %s timeout %i" % (fname, new_timeout)) + system("sed -i '/static struct tst_test test = {/a\\\\t.timeout = " + str(new_timeout) + ",' " + file_path) + +def patch_all(timeouts, patch_override): + for timeout in timeouts: + if timeout[3]: + patch(timeout[3], timeout[1], patch_override) + +def print_table(timeouts): + timeouts.sort(key=lambda x: x[1], reverse=True) + + total = 0; + + print("Old library tests\n-----------------\n"); + for timeout in timeouts: + if not timeout[2]: + print("%-30s %i" % (timeout[0], timeout[1])) + total+=1 + + print("\n\t%i tests in total" % total) + + total = 0; + + print("\nNew library tests\n-----------------\n"); + for timeout in timeouts: + if timeout[2]: + print("%-30s %i" % (timeout[0], timeout[1])) + total+=1 + + print("\n\t%i tests in total" % total) + +def parse_data(results_path): + timeouts = [] + + with open(results_path, 'r') as file: + results = json.load(file) + + with open(top_dir + 'metadata/ltp.json', 'r') as file: + metadata = json.load(file) + + for test in results['results']: + name = test['test_fqn'] + duration = test['test']['duration'] + # If test runs for all_filesystems normalize the runtime per a single filesystem + filesystems = max(1, test['test']['log'].count('TINFO: Formatting /')) + # Check if test is new library test + test_is_newlib = name in metadata['tests'] + # Store test file path + path = None if not test_is_newlib else metadata['tests'][name]['fname'] + # Filter out tests with runtime + test_has_runtime = False if not test_is_newlib else 'runtime' in metadata['tests'][name] + # Timer tests define runtime dynamically in timer library + if test_is_newlib and 'sample' in metadata['tests'][name]: + test_has_runtime = True + # Select tests that does not have runtime and are executed for longer time + if not test_has_runtime and duration >= 0.5: + timeouts.append((name, int(timeout_mul * duration/filesystems + 0.5), test_is_newlib, path)) + + return timeouts + +def print_help(): + print('calc_timeouts.py [OPTION] [RESULT].json') + print('\t-h prints this help') + print('\t-o override test timeouts, by default timeouts are only increased') + print('\t-p patch testcases with updated timeouts') + print('\t-t prints table of tests with suggested timeouts') + +try: + opts, args = getopt.getopt(sys.argv[1:], "hopt") +except: + print_help() + sys.exit(1) + +opt_print_table = False +opt_patch_tests = False +opt_patch_override = False + +for opt,arg in opts: + if opt == '-h': + print_help() + sys.exit(0) + if opt == '-o': + opt_patch_override = True + if opt == '-p': + opt_patch_tests = True + if opt == '-t': + opt_print_table = True + +if not opt_print_table and not opt_patch_tests: + print("No action selected!\n") + print_help() + sys.exit(1) + +results = args[0] if args else 'results.json' + +timeouts = parse_data(results) + +if opt_print_table: + print_table(timeouts) + +if opt_patch_tests: + patch_all(timeouts, opt_patch_override) -- 2.45.2 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-21 12:34 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Cyril Hrubis @ 2025-01-21 13:12 ` Andrea Cervesato via ltp 2025-01-21 14:26 ` Cyril Hrubis 0 siblings, 1 reply; 8+ messages in thread From: Andrea Cervesato via ltp @ 2025-01-21 13:12 UTC (permalink / raw) To: Cyril Hrubis, ltp Hi! I like the general idea, a couple of comments on the python code. On 1/21/25 13:34, Cyril Hrubis wrote: > This script parses JSON results from kirk and LTP metadata in order > calculate timeouts for tests based on the result file and can even patch > tests automatically. > > The script does: > > - Take the results and pick all tests that run for longer than 0.5s. > Multiplies the time with a constant (currently 1.2) to get a suggested > timeout. > > - Exclude tests that have runtime defined since these are controller > by the runtime (that filters out all fuzzy sync tests). > > There is a special case for timer tests that define runtime only > dynamically in the timer library code. This should be possibly fixed > with special value for the .runtime in tst_test. E.g. > TST_RUNTIME_DYNAMIC for tests that only set runtime in the setup. > > - Normalize the timeout per a single filesystem run if test is running for > more than one filesystem. > > - Tests that do not have a metadata record are old library tests which > which cannot be patched but are printed in a separate table if we > request a table to be printed. > > - If patching option is selected tests are update with newly calculated > timeout. By default we only increase timeouts but that can be > overrided with the -o option. > > Signed-off-by: Cyril Hrubis <chrubis@suse.cz> > --- > scripts/calc_timeouts.py | 133 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 133 insertions(+) > create mode 100755 scripts/calc_timeouts.py > > diff --git a/scripts/calc_timeouts.py b/scripts/calc_timeouts.py > new file mode 100755 > index 000000000..c69ab8f57 > --- /dev/null > +++ b/scripts/calc_timeouts.py > @@ -0,0 +1,133 @@ > +#!/usr/bin/python > +import re > +import json > +import getopt > +import sys > +from os import system > + > +# Top level directory path > +top_dir = '../' Better to pass this folder to the script using parameters, also because metadata can be found in the install folder as well. > + > +# The test runtime is multiplied by this to get a timeout > +timeout_mul = 1.2 Global variables in python use TIMEOUT_MUL format. > + > +def patch(fname, new_timeout, patch_override): > + > + orig_timeout = None > + file_path = top_dir + fname os.path.join() > + > + with open(file_path, 'r') as c_source: > + for line in c_source: > + timeout = re.search(r'\s*.timeout\s*=\s*(\d+).', line) This regex should be compiled before loop using re.compile(), otherwise re.search() will compile it for each line. > + if timeout: > + orig_timeout = int(timeout.group(1)) > + > + if orig_timeout: > + if orig_timeout < new_timeout or patch_override: > + print("CHANGE %s timeout %i -> %i" % (fname, orig_timeout, new_timeout)) > + system("sed -i 's/\\.timeout = [0-9]*/\\.timeout = " + str(new_timeout) + "/' " + file_path) This can be substituted with a python version of sed, since system() is never a good idea, unless we really need to use it for external tools which cannot be translated into python. The solution would be: content = [] matcher = re.match(r'\s*.timeout\s*=\s*(\d+).') with open(file_path, 'r') as data: for line in data: # use regex here to find the matching string if matcher.search(line): content.append(f'timeout = {new_timeout}') else: content.append(line) with open(file_path, 'w') as data: data.writelines(content) > + else: > + print("KEEP %s timeout %i (new %i)" % (fname, orig_timeout, new_timeout)) > + else: > + print("ADD %s timeout %i" % (fname, new_timeout)) > + system("sed -i '/static struct tst_test test = {/a\\\\t.timeout = " + str(new_timeout) + ",' " + file_path) Same here. Maybe we can create a generic sed() function for both cases. > + > +def patch_all(timeouts, patch_override): > + for timeout in timeouts: > + if timeout[3]: > + patch(timeout[3], timeout[1], patch_override) > + > +def print_table(timeouts): > + timeouts.sort(key=lambda x: x[1], reverse=True) > + > + total = 0; > + > + print("Old library tests\n-----------------\n"); > + for timeout in timeouts: > + if not timeout[2]: > + print("%-30s %i" % (timeout[0], timeout[1])) > + total+=1 > + > + print("\n\t%i tests in total" % total) > + > + total = 0; > + > + print("\nNew library tests\n-----------------\n"); > + for timeout in timeouts: > + if timeout[2]: > + print("%-30s %i" % (timeout[0], timeout[1])) > + total+=1 > + > + print("\n\t%i tests in total" % total) > + > +def parse_data(results_path): > + timeouts = [] > + > + with open(results_path, 'r') as file: > + results = json.load(file) > + > + with open(top_dir + 'metadata/ltp.json', 'r') as file: > + metadata = json.load(file) > + > + for test in results['results']: > + name = test['test_fqn'] > + duration = test['test']['duration'] > + # If test runs for all_filesystems normalize the runtime per a single filesystem > + filesystems = max(1, test['test']['log'].count('TINFO: Formatting /')) > + # Check if test is new library test > + test_is_newlib = name in metadata['tests'] > + # Store test file path > + path = None if not test_is_newlib else metadata['tests'][name]['fname'] > + # Filter out tests with runtime > + test_has_runtime = False if not test_is_newlib else 'runtime' in metadata['tests'][name] > + # Timer tests define runtime dynamically in timer library > + if test_is_newlib and 'sample' in metadata['tests'][name]: > + test_has_runtime = True > + # Select tests that does not have runtime and are executed for longer time > + if not test_has_runtime and duration >= 0.5: > + timeouts.append((name, int(timeout_mul * duration/filesystems + 0.5), test_is_newlib, path)) > + > + return timeouts In this case it's better to avoid tuples because they make code more difficult to read. Hash is a better approach in general: data["name"] = name data["timeout"] = int(timeout_mul * duration/filesystems + 0.5) data["new_lib"] = test_is_newlib data["path"] = path return data > + > +def print_help(): > + print('calc_timeouts.py [OPTION] [RESULT].json') > + print('\t-h prints this help') > + print('\t-o override test timeouts, by default timeouts are only increased') > + print('\t-p patch testcases with updated timeouts') > + print('\t-t prints table of tests with suggested timeouts') > + > +try: > + opts, args = getopt.getopt(sys.argv[1:], "hopt") > +except: > + print_help() > + sys.exit(1) > + > +opt_print_table = False > +opt_patch_tests = False > +opt_patch_override = False > + > +for opt,arg in opts: > + if opt == '-h': > + print_help() > + sys.exit(0) > + if opt == '-o': > + opt_patch_override = True > + if opt == '-p': > + opt_patch_tests = True > + if opt == '-t': > + opt_print_table = True > + > +if not opt_print_table and not opt_patch_tests: > + print("No action selected!\n") > + print_help() > + sys.exit(1) > + > +results = args[0] if args else 'results.json' > + > +timeouts = parse_data(results) > + > +if opt_print_table: > + print_table(timeouts) > + > +if opt_patch_tests: > + patch_all(timeouts, opt_patch_override) Better to wrap everything into a main() function and to use: if __name__ == "__main__": run() Andrea -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-21 13:12 ` Andrea Cervesato via ltp @ 2025-01-21 14:26 ` Cyril Hrubis 2025-01-21 14:32 ` Andrea Cervesato via ltp 0 siblings, 1 reply; 8+ messages in thread From: Cyril Hrubis @ 2025-01-21 14:26 UTC (permalink / raw) To: Andrea Cervesato; +Cc: ltp Hi! > I like the general idea, a couple of comments on the python code. Feel free to push the script with the changes you suggested, you are the python expert here after all. -- Cyril Hrubis chrubis@suse.cz -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts 2025-01-21 14:26 ` Cyril Hrubis @ 2025-01-21 14:32 ` Andrea Cervesato via ltp 0 siblings, 0 replies; 8+ messages in thread From: Andrea Cervesato via ltp @ 2025-01-21 14:32 UTC (permalink / raw) To: Cyril Hrubis; +Cc: ltp Sure I can do that On 1/21/25 15:26, Cyril Hrubis wrote: > Hi! >> I like the general idea, a couple of comments on the python code. > Feel free to push the script with the changes you suggested, you are the > python expert here after all. > -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-01-22 13:13 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-01-22 12:48 [LTP] [PATCH 0/2] Update test timeouts in an automated way Andrea Cervesato 2025-01-22 12:48 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Andrea Cervesato 2025-01-22 13:13 ` Cyril Hrubis 2025-01-22 12:48 ` [LTP] [PATCH 2/2] syscalls: Update test timeouts Andrea Cervesato -- strict thread matches above, loose matches on Subject: below -- 2025-01-21 12:34 [LTP] [PATCH 0/2] Update test timeouts in an automated way Cyril Hrubis 2025-01-21 12:34 ` [LTP] [PATCH 1/2] scripts: Add simple script for calculating timeouts Cyril Hrubis 2025-01-21 13:12 ` Andrea Cervesato via ltp 2025-01-21 14:26 ` Cyril Hrubis 2025-01-21 14:32 ` Andrea Cervesato via ltp
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox