From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Jackson Subject: [OSSTEST PATCH 13/27] cr-ensure-disk-space: Look at referring flights Date: Wed, 16 Sep 2015 14:35:16 +0100 Message-ID: <1442410530-9665-14-git-send-email-ian.jackson@eu.citrix.com> References: <1442410530-9665-1-git-send-email-ian.jackson@eu.citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZcCsR-0000pz-5X for xen-devel@lists.xenproject.org; Wed, 16 Sep 2015 13:35:59 +0000 In-Reply-To: <1442410530-9665-1-git-send-email-ian.jackson@eu.citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xenproject.org Cc: Ian Jackson , Ian Campbell List-Id: xen-devel@lists.xenproject.org Previously the flight to delete was simply the one with the lowest flight number. Now we sort flights not by their own flight number, but by the highest flight number of any referencing flight. This means that flights whose builds are being reused are kept as long as the reusing flights. This almost-entirely fixes a largely-theoretical race in the way cs-bisection-step works (where the flight's logs and build outputs might be deleted between the setup and execution of the referring flight). A smaller race still exists because the stash check in cs-bisection-step occurs before the being-created flight is visible to other db clients. We will have to fix this by taking the flights lock. Signed-off-by: Ian Jackson --- cr-ensure-disk-space | 43 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/cr-ensure-disk-space b/cr-ensure-disk-space index ced9777..0314e7a 100755 --- a/cr-ensure-disk-space +++ b/cr-ensure-disk-space @@ -55,7 +55,17 @@ flock LOCK, LOCK_EX or die $!; $|=1; my $chkq= db_prepare("SELECT * FROM flights WHERE flight=?"); + +my $refq= db_prepare(<= ? +END + our @flights; +our %latestref; for (;;) { open P, "-|", onloghost "df --block-size=1M -P $logdir" or die $!; @@ -69,23 +79,46 @@ for (;;) { last if $space >= logcfg('MinSpaceMby'); if (!@flights) { + %latestref = (); open P, "-|", onloghost "ls -1 $logdir" or die $!; + + my $minflight = undef; while (

) { next unless m/^(\d+)\n$/; - push @flights, $1; + $latestref{$1} = $1; + $minflight //= $1; + $minflight = $1 if $1 < $minflight; } $!=$?=0; close P or die "ls: $? $!"; - @flights = sort { $b <=> $a } @flights; + + print DEBUG "MINFLIGHT $minflight\n"; + + $refq->execute($minflight); + while (my $rr= $refq->fetchrow_hashref) { + my $testflight = $rr->{flight}; + next unless $rr->{val} =~ m/^(\d+)\./; + my $buildflight = $1; + next unless exists $latestref{$buildflight}; + if ($testflight > $latestref{$buildflight}) { + print DEBUG "REF $buildflight <- $testflight\n"; + $latestref{$buildflight} = $testflight; + } + } + + @flights = + sort { $latestref{$b} <=> $latestref{$a} } + keys %latestref; printf "(%d flights) ", scalar @flights; die unless @flights; } my $flight = pop @flights; - printf "selected %s ", $flight; + my $latestref = $latestref{$flight}; + printf "selected %s (latest ref %s) ", $flight, $latestref; - $chkq->execute($flight); + $chkq->execute($latestref); my $row= $chkq->fetchrow_hashref(); $chkq->finish(); - die $flight unless defined $row; + die "$flight $latestref" unless defined $row; my $age= time - $row->{started}; printf "(age %dd) ", $age / 86400; -- 1.7.10.4