Re: [bitbake-devel] [RFC 8/9] hashserver: Add initial reference server

[Joshua Watt <jpewhacker@gmail.com>]

On Tue, 2018-07-17 at 13:11 +0100, Richard Purdie wrote:
> On Mon, 2018-07-16 at 15:37 -0500, Joshua Watt wrote:
> > Adds an initial reference implementation of the hash server.
> > 
> > NOTE: This is my first dive into HTTP & REST technologies. Feedback
> > is
> > appreciated. Also, I don't think it will be necessary for this
> > reference
> > implementation to live in bitbake, and it can be moved to it's own
> > independent project if necessary?
> > 
> > Also, this server has some concurrency issues that I haven't
> > tracked
> > down and will occasionally fail to record a new POST'd task with an
> > error indicating the database is locked. Based on some reading, I
> > believe this is because the server is using a sqlite backend, and
> > it
> > would go away with a more production worthy backend. Anyway, it is
> > good
> > enough for some preliminary testing.
> > 
> > Starting the server is simple and only requires pipenv to be
> > installed:
> > 
> >  $ pipenv shell
> >  $ ./app.py
> 
> I need to spend some time digesting this series but this patch did
> make
> me a little sad.
> 
> I'm hoping we can make the hash equivalence server something people
> use
> easily and perhaps part of bitbake. The dependencies you've used in
> this code mean it has a significantly higher "barrier to use" than
> most
> of our other code though :(

Ah, I didn't realize there was desire to have it actually be part of
bitbake... That would change the strategy considerably.

I think that is a worth while discussion to have. I obviously wasn't
thinking the server would be part of bitbake, I was more thinking it
would probably be an independent project (I even have a name picked out
already ;), so I'm curious what thoughts there are on the advantages
and disadvantages of doing that.

> On the one hand I can understand people wanting to use dependencies
> and
> new technology. On the other, keeping things simple also has
> advantages.

Sure. Pipenv makes it pretty easy to pull in a lot of shiny modules
that you don't necessarily need, and I may have gone a little
overboard. I do have some justification for *most* of the things I
brought in besides just "it looked cool". We can dig into them deeper.

However, pipenv (or python virtual environments) are designed to amke
these types of decisions easier, so I think that the discussion about
what dependencies we do or don't want should probably start there (i.e.
do we allow pipenv or not?). Basically, it's much easier to pull in the
dependencies you need (and even fix versions for production use) with
pipenv. This of course doesn't mean you can be careless about what you
pull in, but it makes you a lot more independent from the host setup.

> 
> Even the minimum python version is potentially problematic, we don't
> even have working recipes for python 3.6!

Oh right! Pipenv defaults to the version of Python you have installed.
I'm pretty sure you can relax the restriction to just "python 3", but I
forgot to do so before I pushed the patch.

> 
> I appreciate its a reference and means we can test the rest of the
> code
>  so its good but we may need a different implementation of this
> ultimately.

I think perhaps my use of the term "reference implementation" was too
strong. "Example implementation for testing", or "toy implementation"
might be more inline with my intention. 


Thanks,
Joshua Watt

> 
> 
> Cheers,
> 
> Richard
>