[Buildroot] [PATCH buildroot-test v2 1/1] scripts/autobuild-run: make hung build timeout configurable

[Buildroot] [PATCH buildroot-test v2 1/1] scripts/autobuild-run: make hung build timeout configurable

[James Hilliard]

A reasonable value for this may vary depending on system load, specs
and how many parallel builds are being run, so we should make it
configurable.

Tweak LoadConfigFile parser to properly convert arg types.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
Changes v1 -> v2:
  - fix LoadConfigFile type conversion
---
 scripts/autobuild-run | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 337644f..e8530f9 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -120,9 +120,6 @@ else:
     decode_bytes = _identity
     encode_str = _identity
 
-# A number of packages can take > 60mins of build time (e.g
-# gst-ffmpeg, qt5webkit, mimic)
-HUNG_BUILD_TIMEOUT = 120 # mins
 VERSION = 1
 
 def rm_ro(f, p, _):
@@ -208,7 +205,7 @@ class Builder:
     def __init__(self, instance, njobs, sysinfo,
                  http_url, http_login, http_password,
                  submitter, make_opts, nice, toolchains_csv,
-                 repo, upload, buildpid, debug):
+                 repo, upload, buildpid, debug, hung_build_timeout):
         self.instance = instance
         self.njobs = njobs
         self.sysinfo = sysinfo
@@ -223,6 +220,7 @@ class Builder:
         self.upload = upload
         self.buildpid = buildpid
         self.debug = debug
+        self.hung_build_timeout = hung_build_timeout
         self.build_parallel = False
 
         # frequently needed directories
@@ -388,7 +386,7 @@ class Builder:
             if os.path.exists(build_time_logfile):
                 mtime = datetime.datetime.fromtimestamp(os.stat(build_time_logfile).st_mtime)
 
-                if mtime < datetime.datetime.now() - datetime.timedelta(minutes=HUNG_BUILD_TIMEOUT):
+                if mtime < datetime.datetime.now() - datetime.timedelta(minutes=self.hung_build_timeout):
                     if sub_proc.poll() is None:
                         monitor_thread_hung_build_flag.set() # Used by do_build() to determine build hang
                         log_write(self.log, "INFO: build hung")
@@ -756,12 +754,10 @@ class LoadConfigFile(argparse.Action):
         config = configparser.RawConfigParser(allow_no_value=True)
         config.read_file(values)
         for k, v in config.items('main'):
-            key = k.replace("-", "_")
-            value = v
-            if key.startswith("no_") and value is None:
-                key = key[3:]
-                value = False
-            setattr(namespace, key, value)
+            args = ["--" + k]
+            if v:
+                args += [v]
+            parsed = parser.parse_args(args, namespace=namespace)
 
 def main():
 
@@ -811,6 +807,11 @@ def main():
     parser.add_argument("--debug", "-d",
                         help="Send log output to stdout instead of log file",
                         type=str)
+    parser.add_argument("--hung-build-timeout",
+                        help="A number of packages can take > 60mins of build "
+                             "time (e.g gst-ffmpeg, qt5webkit, mimic), sets "
+                             "timeout in minutes",
+                        type=int, default=120)
     toolchains_csv = parser.add_mutually_exclusive_group(required=False)
     toolchains_csv.add_argument("--toolchains-csv",
                                 dest="toolchains_csv",
@@ -895,7 +896,8 @@ def main():
             repo = args.repo,
             upload = upload,
             buildpid = buildpid,
-            debug = args.debug)
+            debug = args.debug,
+            hung_build_timeout = args.hung_build_timeout)
         p = multiprocessing.Process(target=builder.run_instance)
         p.start()
         processes.append(p)
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH buildroot-test v2 1/1] scripts/autobuild-run: make hung build timeout configurable

[Thomas Petazzoni via buildroot]

On Sun, 17 Jul 2022 22:39:01 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:

> A reasonable value for this may vary depending on system load, specs
> and how many parallel builds are being run, so we should make it
> configurable.
> 
> Tweak LoadConfigFile parser to properly convert arg types.
> 
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>

I am just wondering if it really makes sense to make this configurable.
Indeed, it's very hard to figure out what is the "right" value.

Have you seen specific problems with the current timeout?

Thoas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH buildroot-test v2 1/1] scripts/autobuild-run: make hung build timeout configurable

[James Hilliard]

On Mon, Jul 18, 2022 at 4:05 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Sun, 17 Jul 2022 22:39:01 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > A reasonable value for this may vary depending on system load, specs
> > and how many parallel builds are being run, so we should make it
> > configurable.
> >
> > Tweak LoadConfigFile parser to properly convert arg types.
> >
> > Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
>
> I am just wondering if it really makes sense to make this configurable.
> Indeed, it's very hard to figure out what is the "right" value.
>
> Have you seen specific problems with the current timeout?

Yeah, I've had to raise it in some cases, mostly when running lots
of single threaded builders on slower systems, there's a few packages
like browsers that seem to often take a very long time to build and
potentially run into timeout issues.

>
> Thoas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH buildroot-test v2 1/1] scripts/autobuild-run: make hung build timeout configurable

[Arnout Vandecappelle via buildroot]

On 18/07/2022 06:39, James Hilliard wrote:
> A reasonable value for this may vary depending on system load, specs
> and how many parallel builds are being run, so we should make it
> configurable.
> 
> Tweak LoadConfigFile parser to properly convert arg types.
> 
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>

  I finally could convince Thomas :-)

  Applied to buildroot-next master, thanks.

  Regards,
  Arnout


> ---
> Changes v1 -> v2:
>    - fix LoadConfigFile type conversion
> ---
>   scripts/autobuild-run | 26 ++++++++++++++------------
>   1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/scripts/autobuild-run b/scripts/autobuild-run
> index 337644f..e8530f9 100755
> --- a/scripts/autobuild-run
> +++ b/scripts/autobuild-run
> @@ -120,9 +120,6 @@ else:
>       decode_bytes = _identity
>       encode_str = _identity
>   
> -# A number of packages can take > 60mins of build time (e.g
> -# gst-ffmpeg, qt5webkit, mimic)
> -HUNG_BUILD_TIMEOUT = 120 # mins
>   VERSION = 1
>   
>   def rm_ro(f, p, _):
> @@ -208,7 +205,7 @@ class Builder:
>       def __init__(self, instance, njobs, sysinfo,
>                    http_url, http_login, http_password,
>                    submitter, make_opts, nice, toolchains_csv,
> -                 repo, upload, buildpid, debug):
> +                 repo, upload, buildpid, debug, hung_build_timeout):
>           self.instance = instance
>           self.njobs = njobs
>           self.sysinfo = sysinfo
> @@ -223,6 +220,7 @@ class Builder:
>           self.upload = upload
>           self.buildpid = buildpid
>           self.debug = debug
> +        self.hung_build_timeout = hung_build_timeout
>           self.build_parallel = False
>   
>           # frequently needed directories
> @@ -388,7 +386,7 @@ class Builder:
>               if os.path.exists(build_time_logfile):
>                   mtime = datetime.datetime.fromtimestamp(os.stat(build_time_logfile).st_mtime)
>   
> -                if mtime < datetime.datetime.now() - datetime.timedelta(minutes=HUNG_BUILD_TIMEOUT):
> +                if mtime < datetime.datetime.now() - datetime.timedelta(minutes=self.hung_build_timeout):
>                       if sub_proc.poll() is None:
>                           monitor_thread_hung_build_flag.set() # Used by do_build() to determine build hang
>                           log_write(self.log, "INFO: build hung")
> @@ -756,12 +754,10 @@ class LoadConfigFile(argparse.Action):
>           config = configparser.RawConfigParser(allow_no_value=True)
>           config.read_file(values)
>           for k, v in config.items('main'):
> -            key = k.replace("-", "_")
> -            value = v
> -            if key.startswith("no_") and value is None:
> -                key = key[3:]
> -                value = False
> -            setattr(namespace, key, value)
> +            args = ["--" + k]
> +            if v:
> +                args += [v]
> +            parsed = parser.parse_args(args, namespace=namespace)
>   
>   def main():
>   
> @@ -811,6 +807,11 @@ def main():
>       parser.add_argument("--debug", "-d",
>                           help="Send log output to stdout instead of log file",
>                           type=str)
> +    parser.add_argument("--hung-build-timeout",
> +                        help="A number of packages can take > 60mins of build "
> +                             "time (e.g gst-ffmpeg, qt5webkit, mimic), sets "
> +                             "timeout in minutes",
> +                        type=int, default=120)
>       toolchains_csv = parser.add_mutually_exclusive_group(required=False)
>       toolchains_csv.add_argument("--toolchains-csv",
>                                   dest="toolchains_csv",
> @@ -895,7 +896,8 @@ def main():
>               repo = args.repo,
>               upload = upload,
>               buildpid = buildpid,
> -            debug = args.debug)
> +            debug = args.debug,
> +            hung_build_timeout = args.hung_build_timeout)
>           p = multiprocessing.Process(target=builder.run_instance)
>           p.start()
>           processes.append(p)
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot