From: "Alexis Lothoré" <alexis.lothore@bootlin.com>
To: <Openembedded-core@lists.openembedded.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
Alexandre Belloni <alexandre.belloni@bootlin.com>
Subject: [PATCH 0/2] oeqa/utils/postactions: fix disk storage consumption
Date: Fri, 5 Jul 2024 16:46:36 +0200 [thread overview]
Message-ID: <20240705144638.441976-1-alexis.lothore@bootlin.com> (raw)
Hello,
this small series aims to fix Bugzilla issue 15536 [1], which results in a
big disk storage consumption and/or inode max count issue. The main root
cause is that in its current form, the artifacts retriever convert symlinks
to the targeted files. This series changes the way those files are
retrieved to make sure the symlinks are preserved. To do so, it replaces
the series of scp performed on each file by a single tar command which
output the generated compressed archive on its stdout. It is captured and
returned as part of an ssh process, and it it then fed to another tar
command run this time on host, to extract the artifacts, at the same place
as before (tmp/log/oeqa/artifacts)
The first commit is a small update to SSH target in ssh.py to allow to
retrieve raw output (ie raw bytes, without any decoding or formatting) from
a command run on the target. This allows to get the compressed archive
directly as an output from the remote command.
The second commit uses this new feature to generate, retrieve and extract
this archive on the fly, without ever storing it onto the target nor the
host.
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15536
Alexis Lothoré (2):
oeqa/ssh: allow to retrieve raw, unformatted ouput
oeqa/utils/postactions: transfer whole archive over ssh instead of
doing individual copies
meta/lib/oeqa/core/target/ssh.py | 16 ++++++++--------
meta/lib/oeqa/utils/postactions.py | 19 +++++++++----------
2 files changed, 17 insertions(+), 18 deletions(-)
--
2.45.2
next reply other threads:[~2024-07-05 14:47 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-07-05 14:46 Alexis Lothoré [this message]
2024-07-05 14:46 ` [PATCH 1/2] oeqa/ssh: allow to retrieve raw, unformatted ouput Alexis Lothoré
2024-07-05 14:46 ` [PATCH 2/2] oeqa/utils/postactions: transfer whole archive over ssh instead of doing individual copies Alexis Lothoré
2024-07-05 15:06 ` Patchtest results for " patchtest
2024-07-07 8:10 ` [OE-core] " Richard Purdie
2024-07-08 21:10 ` Alexis Lothoré
2024-07-08 21:37 ` Richard Purdie
2024-07-09 8:06 ` Alexis Lothoré
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240705144638.441976-1-alexis.lothore@bootlin.com \
--to=alexis.lothore@bootlin.com \
--cc=Openembedded-core@lists.openembedded.org \
--cc=alexandre.belloni@bootlin.com \
--cc=thomas.petazzoni@bootlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox