From: "Jose Quaresma" <quaresma.jose@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Jose Quaresma <quaresma.jose@gmail.com>
Subject: [PATCH 1/5] sstate.bbclass: track found files on mirrors with a counter
Date: Mon, 9 Aug 2021 10:48:49 +0100 [thread overview]
Message-ID: <20210809094853.3619155-1-quaresma.jose@gmail.com> (raw)
We don't need extra python collections to count the found files
on the sstate cache and sstate mirrors.
The main found collections provides all the files that were found,
then we only need to count the files on sstate mirror
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
---
meta/classes/sstate.bbclass | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 2175ace4c4..2575750247 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -871,8 +871,6 @@ BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, **kwargs):
found = set()
- foundLocal = set()
- foundNet = set()
missed = set()
def gethash(task):
@@ -905,12 +903,11 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
if os.path.exists(sstatefile):
bb.debug(2, "SState: Found valid sstate file %s" % sstatefile)
found.add(tid)
- foundLocal.add(tid)
- continue
else:
- missed.add(tid)
bb.debug(2, "SState: Looked for but didn't find file %s" % sstatefile)
+ missed.add(tid)
+ foundMirrors = 0
mirrors = d.getVar("SSTATE_MIRRORS")
if mirrors:
# Copy the data object and override DL_DIR and SRC_URI
@@ -950,8 +947,9 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
connection_cache=thread_worker.connection_cache)
fetcher.checkstatus()
bb.debug(2, "SState: Successful fetch test for %s" % srcuri)
+ foundMirrors += 1
found.add(tid)
- foundNet.add(tid)
+
if tid in missed:
missed.remove(tid)
except:
@@ -1013,7 +1011,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
match = 0
if total:
match = len(found) / total * 100
- bb.plain("Sstate summary: Wanted %d Local %d Network %d Missed %d Current %d (%d%% match, %d%% complete)" % (total, len(foundLocal), len(foundNet),len(missed), currentcount, match, complete))
+ bb.plain("Sstate summary: Wanted %d Local %d Mirrors %d Missed %d Current %d (%d%% match, %d%% complete)" %
+ (total, len(found)-foundMirrors, foundMirrors, len(missed), currentcount, match, complete))
if hasattr(bb.parse.siggen, "checkhashes"):
bb.parse.siggen.checkhashes(sq_data, missed, found, d)
--
2.32.0
next reply other threads:[~2021-08-09 9:49 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-09 9:48 Jose Quaresma [this message]
2021-08-09 9:48 ` [PATCH 2/5] sstate.bbclass: only search on the mirrors for the missing files Jose Quaresma
2021-08-09 9:48 ` [PATCH 3/5] sstate.bbclass: get the number of threads with cpu_count from oe utils Jose Quaresma
2021-08-09 9:48 ` [PATCH 4/5] sstate.bbclass: disable thread lock if we don't have events Jose Quaresma
2021-08-11 19:40 ` [OE-core] " Richard Purdie
2021-08-18 15:51 ` Jose Quaresma
2021-08-09 9:48 ` [PATCH 5/5] sstate.bbclass: sstate mirror progress bar cleanup Jose Quaresma
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210809094853.3619155-1-quaresma.jose@gmail.com \
--to=quaresma.jose@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox