From: Joshua Watt <jpewhacker@gmail.com>
To: bitbake-devel@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>
Subject: [bitbake-devel][PATCH v2 00/10] hashserv: Pipeline Upstream Queries
Date: Thu, 30 Jul 2026 12:30:53 -0600 [thread overview]
Message-ID: <20260730183254.793698-1-JPEWhacker@gmail.com> (raw)
In-Reply-To: <20260724212822.1165552-1-JPEWhacker@gmail.com>
Adds a true asynchronous streaming API to the hash equivalence clients,
then uses that streaming API to pipeline requests to an upstream server.
This means that when a server wants to defer a request to an upstream,
it no longer has to wait for a complete round-trip API call. Instead, it
can use the faster streaming API to send the requests, and continue
processing other requests while a response inbound.
V2: Fixes several bugs related to when the connection between the client
and upstream server was disconnected, and added tests for them.
Joshua Watt (10):
asyncrpc: Add Task Group
asyncrpc: serv: Use Task Group
asyncrpc: serv: Cancel all clients on server stop
hashserv: tests: Improve test logging
hashserv: client: Add asynchronous streaming API
hashserv: server: Add queued streaming API
hashserv: server: Use streaming and queue API for upstream unihash
queries
hashserv: server: Use streaming and queue API for upstream exist
queries
hashserv: tests: Add more upstream tests
hashserv: tests: Add test for upstream pipelining
bin/bitbake-hashclient | 2 +-
lib/bb/asyncrpc/__init__.py | 1 +
lib/bb/asyncrpc/serv.py | 13 +-
lib/bb/asyncrpc/taskgroup.py | 52 ++++++
lib/hashserv/client.py | 349 +++++++++++++++++++++++++++--------
lib/hashserv/server.py | 181 ++++++++++++++----
lib/hashserv/tests.py | 261 ++++++++++++++++++++++++--
7 files changed, 723 insertions(+), 136 deletions(-)
create mode 100644 lib/bb/asyncrpc/taskgroup.py
--
2.54.0
next prev parent reply other threads:[~2026-07-30 18:33 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-24 21:25 [bitbake-devel][PATCH 0/6] hashserv: Pipeline Upstream Queries Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 1/6] hashserv: client: Add asynchronous streaming API Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 2/6] hashserv: server: Fix formatting Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 3/6] hashserv: server: Add queued streaming API Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 4/6] hashserv: server: Use streaming and queue API for upstream unihash queries Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 5/6] hashserv: server: Use streaming and queue API for upstream exist queries Joshua Watt
2026-07-24 21:25 ` [bitbake-devel][PATCH 6/6] hashserv: tests: Add test for upstream pipelining Joshua Watt
2026-07-26 13:50 ` [bitbake-devel][PATCH 0/6] hashserv: Pipeline Upstream Queries Richard Purdie
2026-07-30 18:30 ` Joshua Watt [this message]
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 01/10] asyncrpc: Add Task Group Joshua Watt
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 02/10] asyncrpc: serv: Use " Joshua Watt
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 03/10] asyncrpc: serv: Cancel all clients on server stop Joshua Watt
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 04/10] hashserv: tests: Improve test logging Joshua Watt
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 05/10] hashserv: client: Add asynchronous streaming API Joshua Watt
2026-07-30 18:30 ` [bitbake-devel][PATCH v2 06/10] hashserv: server: Add queued " Joshua Watt
2026-07-30 18:31 ` [bitbake-devel][PATCH v2 07/10] hashserv: server: Use streaming and queue API for upstream unihash queries Joshua Watt
2026-07-30 18:31 ` [bitbake-devel][PATCH v2 08/10] hashserv: server: Use streaming and queue API for upstream exist queries Joshua Watt
2026-07-30 18:31 ` [bitbake-devel][PATCH v2 09/10] hashserv: tests: Add more upstream tests Joshua Watt
2026-07-30 18:31 ` [bitbake-devel][PATCH v2 10/10] hashserv: tests: Add test for upstream pipelining Joshua Watt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260730183254.793698-1-JPEWhacker@gmail.com \
--to=jpewhacker@gmail.com \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.