From: Joshua Watt <jpewhacker@gmail.com>
To: bitbake-devel@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>
Subject: [bitbake-devel][PATCH v2 0/8] Implement parallel Query API
Date: Sun, 18 Feb 2024 15:59:45 -0700 [thread overview]
Message-ID: <20240218225953.2997239-1-JPEWhacker@gmail.com> (raw)
In-Reply-To: <20240218200743.2982923-1-JPEWhacker@gmail.com>
Implements API to allows querying if a hash exists in the server, and to
do so using multiple connections to the server in parallel. This will be
used to query if an sstate object exists before actually looking for it
in the sstate cache, which is important for e.g. on a CDN where negative
lookups are expensive
V2: Added some patches that were missed in the first series, and also
rebased Tobias patch on top of my changes, as it was easier than
rebasing the other way around.
Joshua Watt (7):
hashserv: Add Unihash Garbage Collection
hashserv: sqlalchemy: Use _execute() helper
hashserv: Add unihash-exists API
asyncrpc: Add Client Pool object
hashserv: Add Client Pool
siggen: Add parallel query API
siggen: Add parallel unihash exist API
Tobias Hagelborn (1):
bitbake: hashserv: Postgres adaptations for ignoring duplicate inserts
bitbake/bin/bitbake-hashclient | 48 +++
bitbake/lib/bb/asyncrpc/__init__.py | 2 +-
bitbake/lib/bb/asyncrpc/client.py | 77 +++++
bitbake/lib/bb/siggen.py | 153 +++++++--
bitbake/lib/hashserv/client.py | 155 ++++++++-
bitbake/lib/hashserv/server.py | 164 +++++----
bitbake/lib/hashserv/sqlalchemy.py | 515 ++++++++++++++++++----------
bitbake/lib/hashserv/sqlite.py | 221 ++++++++++--
bitbake/lib/hashserv/tests.py | 320 +++++++++++++++++
9 files changed, 1336 insertions(+), 319 deletions(-)
--
2.34.1
next prev parent reply other threads:[~2024-02-18 23:00 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-18 20:07 [bitbake-devel][PATCH 0/5] Implement parallel Query API Joshua Watt
2024-02-18 20:07 ` [bitbake-devel][PATCH 1/5] hashserv: sqlalchemy: Use _execute() helper Joshua Watt
2024-02-18 20:07 ` [bitbake-devel][PATCH 2/5] hashserv: Add unihash-exists API Joshua Watt
2024-02-18 20:07 ` [bitbake-devel][PATCH 3/5] asyncrpc: Add Client Pool object Joshua Watt
2024-02-18 20:07 ` [bitbake-devel][PATCH 4/5] hashserv: Add Client Pool Joshua Watt
2024-02-18 20:07 ` [bitbake-devel][PATCH 5/5] siggen: Add parallel query API Joshua Watt
2024-02-18 22:59 ` Joshua Watt [this message]
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 1/8] hashserv: Add Unihash Garbage Collection Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 2/8] hashserv: sqlalchemy: Use _execute() helper Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 3/8] hashserv: Add unihash-exists API Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 4/8] asyncrpc: Add Client Pool object Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 5/8] hashserv: Add Client Pool Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 6/8] siggen: Add parallel query API Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 7/8] siggen: Add parallel unihash exist API Joshua Watt
2024-02-18 22:59 ` [bitbake-devel][PATCH v2 8/8] bitbake: hashserv: Postgres adaptations for ignoring duplicate inserts Joshua Watt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240218225953.2997239-1-JPEWhacker@gmail.com \
--to=jpewhacker@gmail.com \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.