Getting bitbake_prserv database contents to persist across power loss

Getting bitbake_prserv database contents to persist across power loss

[Ben Shelton]

Hi all,

We're currently running a shared PR server that is used by all the
OpenEmbedded build machines on our intranet (both for developers'
machines and for nightly build machines).

We recently ran into an issue where the blade that the shared PR server
runs on was shut down unexpectedly and the PR history since the PR
server was last restarted was not committed to disk.  Looking at the
commit 'prserv: Ensure data is committed', it looks like the only times
the transactions are committed is when the PR server process is shut
down.

What would be your guidance in this case?  Should we just shut down /
restart the PR server nightly to save off the data?  If we wrote a patch
to the PR server to commit the data to the database at runtime-specified
intervals, would that be something we could upstream?

Thanks,
Ben

Re: Getting bitbake_prserv database contents to persist across power loss

[Konrad Scherer]

On 09/26/2014 06:01 PM, Ben Shelton wrote:
> Hi all,
>
> We're currently running a shared PR server that is used by all the
> OpenEmbedded build machines on our intranet (both for developers'
> machines and for nightly build machines).
>
> We recently ran into an issue where the blade that the shared PR server
> runs on was shut down unexpectedly and the PR history since the PR
> server was last restarted was not committed to disk.  Looking at the
> commit 'prserv: Ensure data is committed', it looks like the only times
> the transactions are committed is when the PR server process is shut
> down.
>
> What would be your guidance in this case?  Should we just shut down /
> restart the PR server nightly to save off the data?  If we wrote a patch
> to the PR server to commit the data to the database at runtime-specified
> intervals, would that be something we could upstream?

It occurred to me a while ago, that it may be possible to recreate PR server 
data from an sstate-cache. The sstate file for the deploy-rpm task contains the 
rpms and each rpm contains the PR number. The rest of the PR server fields like 
checksum can be retrieved from the sstate filename or siginfo. Unfortunately I 
have not written the script or I would be happy to share it with you.

Has anyone tried this? Am I missing something important? A script that does this 
would help in situations like this.

-- 
Konrad Scherer, MTS, Linux Products Group, Wind River

Re: Getting bitbake_prserv database contents to persist across power loss

[Richard Purdie]

On Fri, 2014-09-26 at 17:01 -0500, Ben Shelton wrote:
> Hi all,
> 
> We're currently running a shared PR server that is used by all the
> OpenEmbedded build machines on our intranet (both for developers'
> machines and for nightly build machines).
> 
> We recently ran into an issue where the blade that the shared PR server
> runs on was shut down unexpectedly and the PR history since the PR
> server was last restarted was not committed to disk.  Looking at the
> commit 'prserv: Ensure data is committed', it looks like the only times
> the transactions are committed is when the PR server process is shut
> down.
> 
> What would be your guidance in this case?  Should we just shut down /
> restart the PR server nightly to save off the data?  If we wrote a patch
> to the PR server to commit the data to the database at runtime-specified
> intervals, would that be something we could upstream?

I've had a few people mention this and I would accept a patch, as long
as it doesn't blindly sync to disk every interval but instead keeps some
notion of "dirty" or not in the server. The sync calls filtering through
the various IO layers can be painful if they happen too often.

Cheers,

Richard