Re: [bitbake-devel][PATCH v6 00/22] Bitbake Hash Server WebSockets, Alternate Database Backend, and User Management

[Alexandre Belloni <alexandre.belloni@bootlin.com>]

Hello,

I missed it earlier but I got this:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5673/steps/12/logs/stdio

On 03/11/2023 08:26:18-0600, Joshua Watt wrote:
> This patch series reworks the bitbake asyncrpc API to add a WebSockets
> implementation for both the client and server. The hash equivalence
> server is updated to allow using this new API (the PR server can also be
> updated in the future if desired).
> 
> In addition, the database backed for the hash equivalence server is
> abstracted so that sqlalchemy can optionally be used instead of sqlite.
> This allows using "big metal" databases as the backend, which allows the
> hash equivalence server to scale to a large number of queries.
> 
> Note that both websockets and sqlalchemy require 3rd party python
> modules to function. However, these modules are optional unless the user
> desires to use the APIs.
> 
> Also, user management is added. This allows user accounts to be
> registered with the server and users can be given permissions to do
> certain operations on the server. Users are not (necessarily) required
> to login to access the server, as permissions can granted to anonymous
> users. The default permissions will give anonymous users the same
> permissions that they would have before user accounts were added so as
> to retain backward compatibility, but server admins will likely want to
> change this.
> 
> V3: Remove RFC status; patches are ready for review
> V4: Fixed protocol breakage with mixing older and newer clients/servers
> V5: Fixed compatibility with Python 3.8
> V6: Fixed protocol incompatibility when exiting stream state that broke
>     mixing older and new clients/servers
> 
> Joshua Watt (22):
>   asyncrpc: Abstract sockets
>   hashserv: Add websocket connection implementation
>   asyncrpc: Add context manager API
>   hashserv: tests: Add external database tests
>   asyncrpc: Prefix log messages with client info
>   bitbake-hashserv: Allow arguments from environment
>   hashserv: Abstract database
>   hashserv: Add SQLalchemy backend
>   hashserv: Implement read-only version of "report" RPC
>   asyncrpc: Add InvokeError
>   asyncrpc: client: Prevent double closing of loop
>   asyncrpc: client: Add disconnect API
>   hashserv: Add user permissions
>   hashserv: Add become-user API
>   hashserv: Add db-usage API
>   hashserv: Add database column query API
>   hashserv: test: Add bitbake-hashclient tests
>   bitbake-hashclient: Output stats in JSON format
>   bitbake-hashserver: Allow anonymous permissions to be space separated
>   hashserv: tests: Allow authentication for external server tests
>   hashserv: Allow self-service deletion
>   hashserv: server: Add owner if user is logged in
> 
>  bin/bitbake-hashclient        | 145 +++++-
>  bin/bitbake-hashserv          | 132 ++++-
>  lib/bb/asyncrpc/__init__.py   |  33 +-
>  lib/bb/asyncrpc/client.py     | 120 ++---
>  lib/bb/asyncrpc/connection.py | 146 ++++++
>  lib/bb/asyncrpc/exceptions.py |  21 +
>  lib/bb/asyncrpc/serv.py       | 365 ++++++++-----
>  lib/hashserv/__init__.py      | 190 +++----
>  lib/hashserv/client.py        | 147 +++++-
>  lib/hashserv/server.py        | 952 +++++++++++++++++++++-------------
>  lib/hashserv/sqlalchemy.py    | 427 +++++++++++++++
>  lib/hashserv/sqlite.py        | 408 +++++++++++++++
>  lib/hashserv/tests.py         | 736 +++++++++++++++++++++++++-
>  lib/prserv/client.py          |   8 +-
>  lib/prserv/serv.py            |  37 +-
>  15 files changed, 3060 insertions(+), 807 deletions(-)
>  create mode 100644 lib/bb/asyncrpc/connection.py
>  create mode 100644 lib/bb/asyncrpc/exceptions.py
>  create mode 100644 lib/hashserv/sqlalchemy.py
>  create mode 100644 lib/hashserv/sqlite.py
> 
> -- 
> 2.34.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#15421): https://lists.openembedded.org/g/bitbake-devel/message/15421
> Mute This Topic: https://lists.openembedded.org/mt/102364903/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com