Re: [bitbake-devel] hashserv performance issues

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2024-05-23 at 12:11 +0200, Jan-Simon Möller wrote:
> I share your concern ... and also with an eye on PRServ then.
> If we solve this here, we should apply the same lesson there. 
> 
> Can we rethink this maybe:
> - both are "microservices"
> - both are serving data stored in a (maybe even distributed) db
> - both need the concept of an upstream server (or even servers?) to
> 'federate'
> 
> Essentially these bits should be a common codebase and the APIs a
> frontend to  that. Otherwise we will learn lessons twice the hard way
> and have to write more code.

PRServ does not have a problem here, it isn't called in
parsing/runqueue codepaths and isn't tightly looped. 

We've talked about the above issues on calls and we still need to
convince Joshua of the need and come up with a plan, along with someone
to do the development work. I'd like to understand Joshua's concerns
and find a way to alleviate them.

> For the performance issue ... 
> would a local hashserv + upstream hashserv connection help ?
> Or do we need to query things 'always' anyway up to the upstream ... 

I doubt unfortunately since it would still need to pass queries through
to the upstream. Geo specific servers may help and was our original
plan for this but we've been struggling with time for the setup.

> I'm willing to help, but would start on the deployment side as thats
> my  immediate need as we disabled hashserv with scarthgap and want it
> back. Once this is up I can work my way down to code.

See above, I'd be happy to see work in that area but from the project
perspective, the performance issue is effectively blocking builds :(.

Cheers,

Richard