Re: osstest going offline for a bit due to database server move

[Ian Campbell <ian.campbell@citrix.com>]

On Sat, 2015-03-14 at 11:02 +0000, Ian Campbell wrote:
> On Wed, 2015-03-11 at 16:28 +0000, Ian Jackson wrote:
> > The db server for the production osstest instance needs to be
> > physically moved.  This is planned to take place on Friday.
> > 
> > I have dropped a `stop' file in which will stop osstest taking on new
> > work.  The db server will be moved on Friday morning (and anything
> > still running then will be killed).
> > 
> > We expect to resume service some time on Friday.
> 
> TL;DR: The move took place successfully (?) but osstest is still
> unavailable. :-/
> 
> Since the move of the DB server there seems to have been some issues
> with the (unrelated) filer which supplies storage to the server hosting
> the controller VM which is causing the VM's rootfs to go read only about
> once a day. I've rebooted it again this morning but TBH based on recent
> history I don't expect it to survive until tomorrow.
> 
> Separately there seems to be some sort of new issue with the PDU control
> software (which may or may not relate to the db move since PDU control
> is via the DB), so when the VM is up and running it isn't able to
> actually do much which is useful.
> 
> I'll be looking into both of those with some urgency on Monday since
> there isn't much I can do from here. In the meantime I don't think there
> will be much in the way of useful test results.

The PDU issue is fixed.

We've not yet tracked down the source of the mysterious filer reboots
and there was another earlier today, we've fiddled with a few things to
see if we can track them down.

osstest is doing stuff now, fingers crossed.

Ian.