[scarthgap][PATCH 1/2] python3-setuptools-scm: add subprocess timeout control env var

[scarthgap][PATCH 1/2] python3-setuptools-scm: add subprocess timeout control env var

[Hongxu Jia]

Backport patch from upstream to add subprocess timeout control env var

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 .../0001-fix-957-add-subprocess-timeout.patch | 78 +++++++++++++++++++
 .../python/python3-setuptools-scm_8.0.4.bb    |  1 +
 2 files changed, 79 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch

diff --git a/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch
new file mode 100644
index 0000000000..1af1ea0334
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch
@@ -0,0 +1,78 @@
+From 2ac4f084def6db53b454b967b062bed2f945aa03 Mon Sep 17 00:00:00 2001
+From: Ronny Pfannschmidt <opensource@ronnypfannschmidt.de>
+Date: Fri, 5 Jan 2024 13:34:37 +0100
+Subject: [PATCH] fix #957 - add subprocess timeout
+
+Upstream-Status: Backport [https://github.com/pypa/setuptools-scm/commit/2ac4f084def6db53b454b967b062bed2f945aa03]
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ changelog.d/20240105_133254_subprocess_timeout_var.md |  4 ++++
+ docs/overrides.md                                     |  7 +++++++
+ src/setuptools_scm/_run_cmd.py                        | 11 +++++++++--
+ 3 files changed, 20 insertions(+), 2 deletions(-)
+ create mode 100644 changelog.d/20240105_133254_subprocess_timeout_var.md
+
+diff --git a/changelog.d/20240105_133254_subprocess_timeout_var.md b/changelog.d/20240105_133254_subprocess_timeout_var.md
+new file mode 100644
+index 0000000..78ecab2
+--- /dev/null
++++ b/changelog.d/20240105_133254_subprocess_timeout_var.md
+@@ -0,0 +1,4 @@
++
++### Changed
++
++- fix #957 - add subprocess timeout control env var
+diff --git a/docs/overrides.md b/docs/overrides.md
+index 5114a84..22d285e 100644
+--- a/docs/overrides.md
++++ b/docs/overrides.md
+@@ -14,3 +14,10 @@ where the dist name normalization follows adapted PEP 503 semantics.
+ 
+ setuptools_scm parses the environment variable `SETUPTOOLS_SCM_OVERRIDES_FOR_${NORMALIZED_DIST_NAME}`
+ as a toml inline map to override the configuration data from `pyproject.toml`.
++
++## subprocess timeouts
++
++The environment variable `SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT` allows to override the subprocess timeout.
++The default is 40 seconds and should work for most needs. However, users with git lfs + windows reported
++situations where this was not enough.
++
+diff --git a/src/setuptools_scm/_run_cmd.py b/src/setuptools_scm/_run_cmd.py
+index 3dfbda5..a64f904 100644
+--- a/src/setuptools_scm/_run_cmd.py
++++ b/src/setuptools_scm/_run_cmd.py
+@@ -25,7 +25,12 @@ else:
+ # unfortunately github CI for windows sometimes needs
+ # up to 30 seconds to start a command
+ 
+-BROKEN_TIMEOUT: Final[int] = 40
++
++def _get_timeout(env: Mapping[str, str]) -> int:
++    return int(env.get("SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT") or 40)
++
++
++BROKEN_TIMEOUT: Final[int] = _get_timeout(os.environ)
+ 
+ log = _log.log.getChild("run_cmd")
+ 
+@@ -132,7 +137,7 @@ def run(
+     *,
+     strip: bool = True,
+     trace: bool = True,
+-    timeout: int = BROKEN_TIMEOUT,
++    timeout: int | None = None,
+     check: bool = False,
+ ) -> CompletedProcess:
+     if isinstance(cmd, str):
+@@ -141,6 +146,8 @@ def run(
+         cmd = [os.fspath(x) for x in cmd]
+     cmd_4_trace = " ".join(map(_unsafe_quote_for_display, cmd))
+     log.debug("at %s\n    $ %s ", cwd, cmd_4_trace)
++    if timeout is None:
++        timeout = BROKEN_TIMEOUT
+     res = subprocess.run(
+         cmd,
+         capture_output=True,
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
index 67ac407c38..62a15f9d7f 100644
--- a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
+++ b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
@@ -12,6 +12,7 @@ inherit pypi python_setuptools_build_meta
 
 SRC_URI += " \
     file://0001-fix-timeout-while-using-big-git-repo.patch \
+    file://0001-fix-957-add-subprocess-timeout.patch \
 "
 
 UPSTREAM_CHECK_REGEX = "scm-(?P<pver>.*)\.tar"
-- 
2.34.1

[scarthgap][PATCH 2/2] setuptools3-base.bbclass: override default subprocess timeout

[Hongxu Jia]

The environment variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT allows to override
the subprocess timeout. The default is 40 seconds and should work for most
needs.[1] However, it was not enough while using git shallow tarball and starting
multiple Yocto world builds in one host.

|   File "tmp/work/x86_64-linux/python3-scancode-native/32.1.0/recipe-sysroot-
native/usr/lib/python3.13/subprocess.py", line 1263, in _check_timeout
|     raise TimeoutExpired(
|     ...<2 lines>...
|             stderr=b''.join(stderr_seq) if stderr_seq else None)
| subprocess.TimeoutExpired: Command '['git', '--git-dir', 'tmp/work/x86_64-
linux/python3-scancode-native/32.1.0/git/.git', 'status', '--porcelain',
'--untracked-files=no']' timed out after 40 seconds

Explicitly set variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT to 600s in bbclass,
and we could override it in local.conf

[1] https://github.com/pypa/setuptools-scm/blob/main/docs/overrides.md

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes-recipe/setuptools3-base.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes-recipe/setuptools3-base.bbclass b/meta/classes-recipe/setuptools3-base.bbclass
index 27af6abc58..190d9e6e3a 100644
--- a/meta/classes-recipe/setuptools3-base.bbclass
+++ b/meta/classes-recipe/setuptools3-base.bbclass
@@ -23,6 +23,10 @@ export CCSHARED  = "-fPIC -DPIC"
 # the python executable
 export LINKFORSHARED = "${SECURITY_CFLAGS} -Xlinker -export-dynamic"
 
+# The environment variable SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT allows
+# to override the subprocess timeout.
+export SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT ??= "600"
+
 FILES:${PN} += "${PYTHON_SITEPACKAGES_DIR}"
 FILES:${PN}-staticdev += "${PYTHON_SITEPACKAGES_DIR}/*.a"
 FILES:${PN}-dev += "${PYTHON_SITEPACKAGES_DIR}/*.la"
-- 
2.34.1

Re: [OE-core] [scarthgap][PATCH 1/2] python3-setuptools-scm: add subprocess timeout control env var

[Steve Sakoman]

This patch does not apply to current scarthgap HEAD.

I believe you have other changes to this recipe in your working branch.

Steve

On Sun, Feb 16, 2025 at 11:56 PM hongxu via lists.openembedded.org
<hongxu.jia=eng.windriver.com@lists.openembedded.org> wrote:
>
> Backport patch from upstream to add subprocess timeout control env var
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  .../0001-fix-957-add-subprocess-timeout.patch | 78 +++++++++++++++++++
>  .../python/python3-setuptools-scm_8.0.4.bb    |  1 +
>  2 files changed, 79 insertions(+)
>  create mode 100644 meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch
>
> diff --git a/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch
> new file mode 100644
> index 0000000000..1af1ea0334
> --- /dev/null
> +++ b/meta/recipes-devtools/python/python3-setuptools-scm/0001-fix-957-add-subprocess-timeout.patch
> @@ -0,0 +1,78 @@
> +From 2ac4f084def6db53b454b967b062bed2f945aa03 Mon Sep 17 00:00:00 2001
> +From: Ronny Pfannschmidt <opensource@ronnypfannschmidt.de>
> +Date: Fri, 5 Jan 2024 13:34:37 +0100
> +Subject: [PATCH] fix #957 - add subprocess timeout
> +
> +Upstream-Status: Backport [https://github.com/pypa/setuptools-scm/commit/2ac4f084def6db53b454b967b062bed2f945aa03]
> +Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> +---
> + changelog.d/20240105_133254_subprocess_timeout_var.md |  4 ++++
> + docs/overrides.md                                     |  7 +++++++
> + src/setuptools_scm/_run_cmd.py                        | 11 +++++++++--
> + 3 files changed, 20 insertions(+), 2 deletions(-)
> + create mode 100644 changelog.d/20240105_133254_subprocess_timeout_var.md
> +
> +diff --git a/changelog.d/20240105_133254_subprocess_timeout_var.md b/changelog.d/20240105_133254_subprocess_timeout_var.md
> +new file mode 100644
> +index 0000000..78ecab2
> +--- /dev/null
> ++++ b/changelog.d/20240105_133254_subprocess_timeout_var.md
> +@@ -0,0 +1,4 @@
> ++
> ++### Changed
> ++
> ++- fix #957 - add subprocess timeout control env var
> +diff --git a/docs/overrides.md b/docs/overrides.md
> +index 5114a84..22d285e 100644
> +--- a/docs/overrides.md
> ++++ b/docs/overrides.md
> +@@ -14,3 +14,10 @@ where the dist name normalization follows adapted PEP 503 semantics.
> +
> + setuptools_scm parses the environment variable `SETUPTOOLS_SCM_OVERRIDES_FOR_${NORMALIZED_DIST_NAME}`
> + as a toml inline map to override the configuration data from `pyproject.toml`.
> ++
> ++## subprocess timeouts
> ++
> ++The environment variable `SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT` allows to override the subprocess timeout.
> ++The default is 40 seconds and should work for most needs. However, users with git lfs + windows reported
> ++situations where this was not enough.
> ++
> +diff --git a/src/setuptools_scm/_run_cmd.py b/src/setuptools_scm/_run_cmd.py
> +index 3dfbda5..a64f904 100644
> +--- a/src/setuptools_scm/_run_cmd.py
> ++++ b/src/setuptools_scm/_run_cmd.py
> +@@ -25,7 +25,12 @@ else:
> + # unfortunately github CI for windows sometimes needs
> + # up to 30 seconds to start a command
> +
> +-BROKEN_TIMEOUT: Final[int] = 40
> ++
> ++def _get_timeout(env: Mapping[str, str]) -> int:
> ++    return int(env.get("SETUPTOOLS_SCM_SUBPROCESS_TIMEOUT") or 40)
> ++
> ++
> ++BROKEN_TIMEOUT: Final[int] = _get_timeout(os.environ)
> +
> + log = _log.log.getChild("run_cmd")
> +
> +@@ -132,7 +137,7 @@ def run(
> +     *,
> +     strip: bool = True,
> +     trace: bool = True,
> +-    timeout: int = BROKEN_TIMEOUT,
> ++    timeout: int | None = None,
> +     check: bool = False,
> + ) -> CompletedProcess:
> +     if isinstance(cmd, str):
> +@@ -141,6 +146,8 @@ def run(
> +         cmd = [os.fspath(x) for x in cmd]
> +     cmd_4_trace = " ".join(map(_unsafe_quote_for_display, cmd))
> +     log.debug("at %s\n    $ %s ", cwd, cmd_4_trace)
> ++    if timeout is None:
> ++        timeout = BROKEN_TIMEOUT
> +     res = subprocess.run(
> +         cmd,
> +         capture_output=True,
> +--
> +2.25.1
> +
> diff --git a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
> index 67ac407c38..62a15f9d7f 100644
> --- a/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
> +++ b/meta/recipes-devtools/python/python3-setuptools-scm_8.0.4.bb
> @@ -12,6 +12,7 @@ inherit pypi python_setuptools_build_meta
>
>  SRC_URI += " \
>      file://0001-fix-timeout-while-using-big-git-repo.patch \
> +    file://0001-fix-957-add-subprocess-timeout.patch \
>  "
>
>  UPSTREAM_CHECK_REGEX = "scm-(?P<pver>.*)\.tar"
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#211525): https://lists.openembedded.org/g/openembedded-core/message/211525
> Mute This Topic: https://lists.openembedded.org/mt/111228074/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [scarthgap][PATCH 1/2] python3-setuptools-scm: add subprocess timeout control env var

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 56 bytes --]

Sorry, I will resend it later, v2 incoming

//Hongxu

[-- Attachment #2: Type: text/html, Size: 95 bytes --]