From: Alexander Marques <c137.marques@gmail.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH v2] scripts: Add clean-hashserver-database script
Date: Tue, 11 Mar 2025 16:27:16 +0000 [thread overview]
Message-ID: <20250311162716.448364-1-c137.marques@gmail.com> (raw)
From: Alexandre Marques <c137.marques@gmail.com>
Auxiliary script to clean the hashserver database based on the files
available in the sstate directory.
It makes used of the new "hashclient gc-mark-stream" command to mark all sstate
relevant hashes as "alive" and removes everything else from the
database.
Usage example:
```
./scripts/clean-hashserver-database \
--sstate-dir ~/build/sstate-cache \
--hashclient ./bitbake/bin/bitabke-hashclient \
--hashserver-address "ws://localhost:8688/ws" \
--mark "alive" \
--clean-db
```
Signed-off-by: Alexander Marques <c137.marques@gmail.com>
---
scripts/clean-hashserver-database | 73 +++++++++++++++++++++++++++++++
tmp | 0
2 files changed, 73 insertions(+)
create mode 100755 scripts/clean-hashserver-database
create mode 100644 tmp
diff --git a/scripts/clean-hashserver-database b/scripts/clean-hashserver-database
new file mode 100755
index 0000000000..6eb006758e
--- /dev/null
+++ b/scripts/clean-hashserver-database
@@ -0,0 +1,73 @@
+#!/bin/bash
+set -euo pipefail
+
+SSTATE_DIR=""
+BB_HASHCLIENT=""
+BB_HASHSERVER=""
+
+ALIVE_DB_MARK="alive"
+CLEAN_DB="false"
+
+function help() {
+ cat <<HELP_TEXT
+Usage: $0 --sstate-dir path --hashclient path --hashserver-address address [--mark value] [--clean-db]
+
+Auxiliary script remove unused or no longer relevant entries from the hashequivalence database, based
+on the files available on the sstate directory.
+
+ -h | --help) Show this help message and exit
+ -s | --sstate-dir) Path to the sstate dir
+ -c | --hashclient) Path to bitbake-hashclient
+ -a | --hashserver-adress) bitbake-hashserver address
+ -m | --mark) Marker string to mark database entries
+ --clean-db) Remove all unmarked and unused entries from the database
+HELP_TEXT
+}
+
+function argument_parser() {
+ while [ $# -gt 0 ]; do
+ case "$1" in
+ -h | --help) help; exit 0 ;;
+ -s | --sstate-dir) SSTATE_DIR="$2"; shift ;;
+ -c | --hashclient) BB_HASHCLIENT="$2"; shift ;;
+ -a | --hashserver-address) BB_HASHSERVER="$2"; shift ;;
+ -m | --mark) ALIVE_DB_MARK="$2"; shift ;;
+ --clean-db) CLEAN_DB="true";;
+ *)
+ echo "Argument '$1' is not supported" >&2
+ help >&2
+ exit 1
+ ;;
+ esac
+ shift
+ done
+
+ function validate_mandatory_argument() {
+ local var_value="$1"
+ local error_message="$2"
+
+ if [ -z "$var_value" ]; then
+ echo "$error_message"
+ help >&2
+ exit 1
+ fi
+ }
+
+ validate_mandatory_argument "$SSTATE_DIR" "Please provide the path to the sstate dir."
+ validate_mandatory_argument "$BB_HASHCLIENT" "Please provide the path to bitbake-hashclient."
+ validate_mandatory_argument "$BB_HASHSERVER" "Please provide the address of bitbake-hashserver."
+}
+
+# -- main code --
+argument_parser $@
+
+# Mark all db sstate hashes
+find "$SSTATE_DIR" -name "*.tar.zst" | \
+sed 's/.*:\([^_]*\)_.*/unihash \1/' | \
+$BB_HASHCLIENT --address "$BB_HASHSERVER" gc-mark-stream "${ALIVE_DB_MARK}"
+
+# Remove unmarked and unused entries
+if [ "$CLEAN_DB" = "true" ]; then
+ $BB_HASHCLIENT --address "$BB_HASHSERVER" gc-sweep "${ALIVE_DB_MARK}"
+ $BB_HASHCLIENT --address "$BB_HASHSERVER" clean-unused 0
+fi
diff --git a/tmp b/tmp
new file mode 100644
index 0000000000..e69de29bb2
--
2.34.1
next reply other threads:[~2025-03-11 16:27 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-11 16:27 Alexander Marques [this message]
2025-03-12 11:28 ` [PATCH v2] scripts: Add clean-hashserver-database script Alexandre Marques
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250311162716.448364-1-c137.marques@gmail.com \
--to=c137.marques@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox