[PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Campbell]

Using report_find_push_age_info allows us to provide counts of
attempts since the last baseline on current tip as well as the first
attempt of each of those.

Since everything serialises on the repo lock I didn't bother trying to
parallelise anything.

It's a little terse to keep it in 80 chars.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2: Use new report_find_push_age_info functionality, output condensed.

I did wonder about adding this to some cronjob or something.

Example output:
Branch                       Basis    Tip      #Tip #Tot 1stTip     1stNew
libvirt                      d10a5f58 ebd62eba    0    9 ????-??-?? 2015-06-23
linux-3.0                    e1c63f9f 5dba9ddd    1    8 2015-06-30 2013-02-13
linux-3.10                   b3d78448 b3d78448
linux-3.14                   762167f9 762167f9
linux-3.16                   162d6432 26749e75    0    0 ????-??-?? ????-??-??
linux-3.18                   d048c068 ea5dd38e    2    2 2015-06-29 2015-06-29
linux-3.4                    bb4a05a0 cf1b3dad   14  160 2015-06-20 2014-12-02
linux-4.1                    b953c0d2 6a010c0a    0    - ????-??-?? ????-??-??
linux-arm-xen                64972ceb 64972ceb
linux-linus                  6aaf0da8 05a8256c    0    0 ????-??-?? ????-??-??
linux-mingo-tip-master       d935d0f7 778a1ac5    0   16 ????-??-?? 2012-04-14
linux-next                   2e0a48c9             0  219 ????-??-?? 2014-04-10
osstest                      15d2dd50             0    - ????-??-?? ????-??-??
ovmf                         269e0aeb f83f808c    0    0 ????-??-?? ????-??-??
qemu-mainline                2b464e13 d2966f80    1    1 2015-06-30 2015-06-30
qemu-upstream-4.2-testing    d2382550 d2382550
qemu-upstream-4.3-testing    efae5e0f efae5e0f
qemu-upstream-4.4-testing    32226f42 32226f42
qemu-upstream-4.5-testing    d9552b0a d9552b0a
qemu-upstream-unstable       c4a962ec c4a962ec
rumpuserxen                  30d72f3f 3b91e449   60  112 2015-04-15 2015-02-03
seabios                      f24eb2f8 f24eb2f8
xen-4.0-testing              2692df2a 2692df2a
xen-4.1-testing              40feff87 40feff87
xen-4.2-testing              38fcda22 38fcda22
xen-4.3-testing              e7c02297 e7c02297
xen-4.4-testing              6c1cb3db 6c1cb3db
xen-4.5-testing              e3bd3cef e3bd3cef
xen-unstable                 c40317f1 28e5d9a9    0    2 ????-??-?? 2015-06-29
---
 mg-all-branch-statuses | 113 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)
 create mode 100755 mg-all-branch-statuses

diff --git a/mg-all-branch-statuses b/mg-all-branch-statuses
new file mode 100755
index 0000000..38f3341
--- /dev/null
+++ b/mg-all-branch-statuses
@@ -0,0 +1,113 @@
+#!/bin/bash
+# -*- bash -*-
+#
+# Prints the status of each branch
+#
+# Usage:
+#    ./mg-all-branch-statuses [BRANCH....]
+#
+# If no BRANCHes specified, does all that are normally run by
+# cr-daily-branch or out of crontab.
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2014 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+set -e
+
+. cri-common
+
+mkdir -p tmp
+
+if [ $# = 0 ]; then
+   set `./mg-list-all-branches`
+fi
+
+gather_info()
+{
+    local branch=$1; shift
+    local basis=$1;  shift
+    local tip=$1;    shift
+
+    select_xenbranch
+
+    local info=`perl -we '
+	use Osstest::Executive;
+	use Osstest;
+	use Data::Dumper;
+	open DEBUG, ">/dev/null" or die $!;
+	#open DEBUG, ">&STDERR" or die $!;
+	csreadconfig();
+	my ($branch,$tree,$basis,$tip) = @ARGV;
+	print DEBUG "branch=$branch tree=$tree basis=$basis tip=$tip\n";
+	my $info = report_find_push_age_info([qw(real adhoc play)],
+					      undef, [($branch)],
+					      $tree, $basis, $tip);
+	print DEBUG Dumper $info;
+	my $onevar = sub {
+		my ($var,$dflt) = @_;
+		$dflt //= "";
+		print "export ".uc($var)."=\"";
+		print $info->{$var}//$dflt;
+		print "\";\n";
+	};
+	my $oneflightvar = sub {
+		my ($flight,$var,$dflt) = @_;
+		$dflt //= "";
+		print "export ".uc($flight)."_".uc($var)."=\"";
+		print $info->{$flight}{$var}//$dflt;
+		print "\";\n";
+	};
+	$onevar->($_, "-") foreach qw(CountTip CountAfterBasis);
+	foreach my $flight (qw(Basis FirstAfterBasis FirstTip)) {
+		$oneflightvar->($flight, $_) foreach qw(flight started);
+	}
+' "$branch" "$tree" "$basis" "$tip"`
+    eval $info
+}
+
+printf "%-28s %-8s %-8s %-9s %-10s %-10s\n" \
+	"Branch" "Basis" "Tip" "#Tip #Tot" "1stTip" "1stNew"
+
+for branch in $@; do
+    basis=`./ap-fetch-version-old $branch 2>/dev/null || true`
+    tip=`./ap-fetch-version $branch 2>/dev/null || true`
+
+    gather_info "$branch" "$basis" "$tip"
+
+    basis=${basis:0:8}
+    tip=${tip:0:8}
+    printf "%-28s %8s %8s" ${branch} ${basis} ${tip}
+
+    if [ x$basis != x$tip ] ; then
+
+	if [ -z "$FIRSTTIP_STARTED" ] ; then
+	    tipsince="????-??-??"
+	else
+	    tipsince=$(date -u -I -d @"$FIRSTTIP_STARTED")
+	fi
+
+	if [ -z "$FIRSTAFTERBASIS_STARTED" ] ; then
+	    newsince="????-??-??"
+	else
+	    newsince=$(date -u -I -d @"$FIRSTAFTERBASIS_STARTED")
+	fi
+
+	printf "  %3s  %3s %-10s %-10s" \
+	       $COUNTTIP $COUNTAFTERBASIS $tipsince $newsince
+    fi
+
+    printf "\n"
+done
-- 
2.1.4

Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is"):
> Using report_find_push_age_info allows us to provide counts of
> attempts since the last baseline on current tip as well as the first
> attempt of each of those.
> 
> Since everything serialises on the repo lock I didn't bother trying to
> parallelise anything.
> 
> It's a little terse to keep it in 80 chars.

Thanks.  This is useful.

> I did wonder about adding this to some cronjob or something.
> 
> Example output:
> Branch                       Basis    Tip      #Tip #Tot 1stTip     1stNew
> libvirt                   d10a5f58 ebd62eba    0    9 ????-??-?? 2015-06-23
> linux-3.0                 e1c63f9f 5dba9ddd    1    8 2015-06-30 2013-02-13

These ????-??-?? are quite visually noisy.  And the ISO dates are not
ideal for reading - how about printing a number of days ago instead ?

> linux-next                2e0a48c9             0  219 ????-??-?? 2014-04-10

I'm not sure what this means.  219 in #Tot would normally be a problem
and `1stNew' is from 2014.  But linux-next is not fast-forwarding.
And there's allegedly no tip although I think maybe you mean there is
no basis.

> osstest                   15d2dd50             0    - ????-??-?? ????-??-??

Is the lack of a Tip here a bug ?

> xen-4.5-testing           e3bd3cef e3bd3cef

You could replace one copy of `e3bd3cef' with `same' or soemthing.

Ian.

Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Campbell]

On Wed, 2015-07-01 at 14:37 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is"):
> > Using report_find_push_age_info allows us to provide counts of
> > attempts since the last baseline on current tip as well as the first
> > attempt of each of those.
> > 
> > Since everything serialises on the repo lock I didn't bother trying to
> > parallelise anything.
> > 
> > It's a little terse to keep it in 80 chars.
> 
> Thanks.  This is useful.
> 
> > I did wonder about adding this to some cronjob or something.
> > 
> > Example output:
> > Branch                       Basis    Tip      #Tip #Tot 1stTip     1stNew
> > libvirt                   d10a5f58 ebd62eba    0    9 ????-??-?? 2015-06-23
> > linux-3.0                 e1c63f9f 5dba9ddd    1    8 2015-06-30 2013-02-13
> 
> These ????-??-?? are quite visually noisy.

Agreed, I did consider just omitting them. Or would you prefer
"Unknown"?

>   And the ISO dates are not
> ideal for reading - how about printing a number of days ago instead ?

I was a bit lazy and it was easier to get date(1) to give me a date from
shell. I'll see if I can get the number of days ago out of it by
extending the Perl.

> 
> > linux-next                2e0a48c9             0  219 ????-??-?? 2014-04-10
> 
> I'm not sure what this means.  219 in #Tot would normally be a problem
> and `1stNew' is from 2014.  But linux-next is not fast-forwarding.
> And there's allegedly no tip although I think maybe you mean there is
> no basis.

Lack of quoting on the call to printf may have confused things due to
empty parameters.

Quoting things results in:

linux-next                            2e0a48c9    0  219 ????-??-?? 2014-04-10

Should basis say "None" in this case IYO?

> 
> > osstest                   15d2dd50             0    - ????-??-?? ????-??-??
> 
> Is the lack of a Tip here a bug ?

I think it (ap-fetch-version) is looking in my $HOME and not osstest's.
I would expect this to be correct if I ran it as osstest.

Should it say "Unknown"?

> > xen-4.5-testing           e3bd3cef e3bd3cef
> 
> You could replace one copy of `e3bd3cef' with `same' or soemthing.

Or "UpToDate" ?

> 
> Ian.

Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is"):
> On Wed, 2015-07-01 at 14:37 +0100, Ian Jackson wrote:
> > These ????-??-?? are quite visually noisy.
> 
> Agreed, I did consider just omitting them. Or would you prefer
> "Unknown"?

It's not so much unknown as n/a.

> >   And the ISO dates are not
> > ideal for reading - how about printing a number of days ago instead ?
> 
> I was a bit lazy and it was easier to get date(1) to give me a date from
> shell. I'll see if I can get the number of days ago out of it by
> extending the Perl.

Ah.  Number of days is easy:

bash:

   now = $(date +%s)
   days = $(( ($then - $now) / 86400 ))

Perl:

   my $now = time;
   my $days = floor(($then - $now) / 86400);

> > > linux-next                2e0a48c9             0  219 ????-??-?? 2014-04-10
> > 
> > I'm not sure what this means.  219 in #Tot would normally be a problem
> > and `1stNew' is from 2014.  But linux-next is not fast-forwarding.
> > And there's allegedly no tip although I think maybe you mean there is
> > no basis.
> 
> Lack of quoting on the call to printf may have confused things due to
> empty parameters.

Ah.

> Quoting things results in:
> 
> linux-next                            2e0a48c9    0  219 ????-??-?? 2014-04-10
> 
> Should basis say "None" in this case IYO?

I think blank is fine but I don't mind None.

> > > osstest                   15d2dd50             0    - ????-??-?? ????-??-??
> > 
> > Is the lack of a Tip here a bug ?
> 
> I think it (ap-fetch-version) is looking in my $HOME and not osstest's.
> I would expect this to be correct if I ran it as osstest.

So it broke ?  Shouldn't the script bomb out ?

> > > xen-4.5-testing           e3bd3cef e3bd3cef
> > 
> > You could replace one copy of `e3bd3cef' with `same' or soemthing.
> 
> Or "UpToDate" ?

For example.

Ian.

Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Campbell]

On Wed, 2015-07-01 at 15:35 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is"):
> > On Wed, 2015-07-01 at 14:37 +0100, Ian Jackson wrote:
> > > These ????-??-?? are quite visually noisy.
> > 
> > Agreed, I did consider just omitting them. Or would you prefer
> > "Unknown"?
> 
> It's not so much unknown as n/a.

Do you prefer the script to say "n/a" rather than nothing?

> > >   And the ISO dates are not
> > > ideal for reading - how about printing a number of days ago instead ?
> > 
> > I was a bit lazy and it was easier to get date(1) to give me a date from
> > shell. I'll see if I can get the number of days ago out of it by
> > extending the Perl.
> 
> Ah.  Number of days is easy:
[...]

Thanks.

> > > > linux-next                2e0a48c9             0  219 ????-??-?? 2014-04-10
> > > 
> > > I'm not sure what this means.  219 in #Tot would normally be a problem
> > > and `1stNew' is from 2014.  But linux-next is not fast-forwarding.
> > > And there's allegedly no tip although I think maybe you mean there is
> > > no basis.
> > 
> > Lack of quoting on the call to printf may have confused things due to
> > empty parameters.
> 
> Ah.
> 
> > Quoting things results in:
> > 
> > linux-next                            2e0a48c9    0  219 ????-??-?? 2014-04-10
> > 
> > Should basis say "None" in this case IYO?
> 
> I think blank is fine but I don't mind None.

I went with blank for no-basis.

> > > > osstest                   15d2dd50             0    - ????-??-?? ????-??-??
> > > 
> > > Is the lack of a Tip here a bug ?
> > 
> > I think it (ap-fetch-version) is looking in my $HOME and not osstest's.
> > I would expect this to be correct if I ran it as osstest.
> 
> So it broke ?  Shouldn't the script bomb out ?

That would be quite annoying since osstest.git is a bit odd/special in
its desire to be in $HOME/testing.git which not everyone will have.

Perhaps if we were to decide this was to be a cron only thing (and not
an end user script) I should a) rename it to cr-* and b) make it fail in
this case instead.

If it is to remain an mg-* then I could make it say "Error!" perhaps?

> > > > xen-4.5-testing           e3bd3cef e3bd3cef
> > > 
> > > You could replace one copy of `e3bd3cef' with `same' or soemthing.
> > 
> > Or "UpToDate" ?
> 
> For example.
> 
> Ian.

Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST v2] mg-all-branch-statuses: Show how up to date each branch is"):
> On Wed, 2015-07-01 at 15:35 +0100, Ian Jackson wrote:
> > It's not so much unknown as n/a.
> 
> Do you prefer the script to say "n/a" rather than nothing?

Midly, I think.  This is all bikeshed stuff.  I don't mind about these
details so long as it's easy to scan down the columns etc., and the
report isn't misleading.

> > > I think it (ap-fetch-version) is looking in my $HOME and not osstest's.
> > > I would expect this to be correct if I ran it as osstest.
> > 
> > So it broke ?  Shouldn't the script bomb out ?
> 
> That would be quite annoying since osstest.git is a bit odd/special in
> its desire to be in $HOME/testing.git which not everyone will have.
> 
> Perhaps if we were to decide this was to be a cron only thing (and not
> an end user script) I should a) rename it to cr-* and b) make it fail in
> this case instead.
> 
> If it is to remain an mg-* then I could make it say "Error!" perhaps?

That would be fine by me.

Ian.