* [PATCH] oeqa/selftest/cases/devtool.py: avoid .pyc race
@ 2020-09-29 4:32 Tim Orling
2020-09-30 5:27 ` [OE-core] " Khem Raj
0 siblings, 1 reply; 3+ messages in thread
From: Tim Orling @ 2020-09-29 4:32 UTC (permalink / raw)
To: openembedded-core; +Cc: Tim Orling
From: Tim Orling <timothy.t.orling@linux.intel.com>
In certain conditions, most likely under heavy load on the
AutoBuiler, the prebuilt .pyc files are attempting to be
executed before they have been completely copied. Avoid
this by not copying the .pyc files (nor the __pycache__
directory). The impact of python3-native recreating the .pyc
files should hopefully be negligible.
YOCTO#13421
YOCTO#13803
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index b383ed9c50b..a3d2e9ea7cc 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -56,7 +56,8 @@ def setUpModule():
if pth.startswith(canonical_layerpath):
if relpth.endswith('/'):
destdir = os.path.join(corecopydir, relpth)
- shutil.copytree(pth, destdir)
+ # avoid race condition by not copying .pyc files YPBZ#13421,13803
+ shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
else:
destdir = os.path.join(corecopydir, os.path.dirname(relpth))
bb.utils.mkdirhier(destdir)
--
2.25.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [OE-core] [PATCH] oeqa/selftest/cases/devtool.py: avoid .pyc race
2020-09-29 4:32 [PATCH] oeqa/selftest/cases/devtool.py: avoid .pyc race Tim Orling
@ 2020-09-30 5:27 ` Khem Raj
0 siblings, 0 replies; 3+ messages in thread
From: Khem Raj @ 2020-09-30 5:27 UTC (permalink / raw)
To: Tim Orling; +Cc: Patches and discussions about the oe-core layer, Tim Orling
On Mon, Sep 28, 2020 at 9:32 PM Tim Orling <ticotimo@gmail.com> wrote:
>
> From: Tim Orling <timothy.t.orling@linux.intel.com>
>
> In certain conditions, most likely under heavy load on the
> AutoBuiler, the prebuilt .pyc files are attempting to be
> executed before they have been completely copied. Avoid
> this by not copying the .pyc files (nor the __pycache__
> directory). The impact of python3-native recreating the .pyc
> files should hopefully be negligible.
>
pyc makes difference only when loading, execution is still same. So it
will depend
if we are invoking the script repeatedly or is it invoked once and
executed for longer time.
benefits might be less in later case.
> YOCTO#13421
> YOCTO#13803
>
> Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
> ---
> meta/lib/oeqa/selftest/cases/devtool.py | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
> index b383ed9c50b..a3d2e9ea7cc 100644
> --- a/meta/lib/oeqa/selftest/cases/devtool.py
> +++ b/meta/lib/oeqa/selftest/cases/devtool.py
> @@ -56,7 +56,8 @@ def setUpModule():
> if pth.startswith(canonical_layerpath):
> if relpth.endswith('/'):
> destdir = os.path.join(corecopydir, relpth)
> - shutil.copytree(pth, destdir)
> + # avoid race condition by not copying .pyc files YPBZ#13421,13803
> + shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
> else:
> destdir = os.path.join(corecopydir, os.path.dirname(relpth))
> bb.utils.mkdirhier(destdir)
> --
> 2.25.0
>
>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH] oeqa/selftest/cases/devtool.py: avoid .pyc race
@ 2020-09-30 21:39 Tim Orling
0 siblings, 0 replies; 3+ messages in thread
From: Tim Orling @ 2020-09-30 21:39 UTC (permalink / raw)
To: openembedded-core; +Cc: steve, Tim Orling
From: Tim Orling <timothy.t.orling@linux.intel.com>
In certain conditions, most likely under heavy load on the
AutoBuilder, the prebuilt .pyc files are attempting to be
executed before they have been completely copied. Avoid
this by not copying the .pyc files (nor the __pycache__
directory). The impact of python3-native recreating the .pyc
files should hopefully be negligible.
YOCTO#13421
YOCTO#13803
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
---
This has already been merged to 3.2 tree and as a bug fix
qualifies for backport to dunfell/3.1
meta/lib/oeqa/selftest/cases/devtool.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 5003f08c75f..d8bf4aea081 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -56,7 +56,8 @@ def setUpModule():
if pth.startswith(canonical_layerpath):
if relpth.endswith('/'):
destdir = os.path.join(corecopydir, relpth)
- shutil.copytree(pth, destdir)
+ # avoid race condition by not copying .pyc files YPBZ#13421,13803
+ shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
else:
destdir = os.path.join(corecopydir, os.path.dirname(relpth))
bb.utils.mkdirhier(destdir)
--
2.25.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-09-30 21:39 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-09-29 4:32 [PATCH] oeqa/selftest/cases/devtool.py: avoid .pyc race Tim Orling
2020-09-30 5:27 ` [OE-core] " Khem Raj
-- strict thread matches above, loose matches on Subject: below --
2020-09-30 21:39 Tim Orling
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox