Re: [bitbake-devel] hashserv performance issues

["Jan-Simon Möller" <dl9pf@gmx.de>]

Hi !

I share your concern ... and also with an eye on PRServ then.
If we solve this here, we should apply the same lesson there. 

Can we rethink this maybe:
- both are "microservices"
- both are serving data stored in a (maybe even distributed) db
- both need the concept of an upstream server (or even servers?) to 'federate'

Essentially these bits should be a common codebase and the APIs a frontend to 
that. Otherwise we will learn lessons twice the hard way and have to write 
more code.


For the performance issue ... 
would a local hashserv + upstream hashserv connection help ?
Or do we need to query things 'always' anyway up to the upstream ... 


I'm willing to help, but would start on the deployment side as thats my 
immediate need as we disabled hashserv with scarthgap and want it back. Once 
this is up I can work my way down to code.


My 0,02€ 

Best,
Jan-Simon

Am Mittwoch, 22. Mai 2024, 15:17:02 CEST schrieb Richard Purdie via 
lists.openembedded.org:
> Joshua reminded me of how to get the server stats (thanks).
> 
> For our public server:
> 
> $ bitbake-hashclient --address wss://hashserv.yoctoproject.org/ws stats
> {
>     "requests": {
>         "average": 0.06250257539454776,
>         "max_time": 16.712204183917493,
>         "num": 381569673,
>         "stdev": 0.18345958610620752,
>         "total_time": 23849087.254955433
>     }
> }
> 
> Compared to my local system:
> 
> $ bitbake-hashclient --address ws://localhost:8686 stats
> {
>     "requests": {
>         "average": 0.0004462178160857849,
>         "max_time": 0.04537125899992134,
>         "num": 3121,
>         "stdev": 0.00163338261679226,
>         "total_time": 1.3926458040037346
>     }
> }
> 
> For a world build with 38000 tasks, 38000 * 0.0625 = 2375 seconds or 39
> minutes, so I suspect the hash server performance is a real issue here
> 
> :(
> 
> Michael: Is there anything we can do?
> 
> Cheers,
> 
> Richard