[OSSTEST PATCH 0/4] Use cached git url for trees cloned by xen.git

[OSSTEST PATCH 0/4] Use cached git url for trees cloned by xen.git

[Ian Jackson]

We have tightened up the internal firewall in the Xen Project
Masschusetts test lab.  Now, test boxes do not get access to the
global internet.

Patches 2-4 together arrange that we configure xen.git's .config to
use URLs which go via our git cache.  Without this, builds fail.

Patch 1 provides a better error message for certain internal failures.
(The failure which provoked this patch was a bug in the new firewall,
now fixed.)

I am going to force push these to osstest master, because:

1. I have run an adhoc test of all the build-* jobs, which showed no
   regressions;

2. Without these, things will definitely not work.

I will then re-enable osstest and we can see what other fallout (if
any) there is.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 1/4] Osstest/PDU/eth008: print actual error on failure

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/PDU/eth008.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/PDU/eth008.pm b/Osstest/PDU/eth008.pm
index fc5fa96..12f0bfb 100644
--- a/Osstest/PDU/eth008.pm
+++ b/Osstest/PDU/eth008.pm
@@ -58,7 +58,7 @@ sub pdu_power_state {
 
     my $resp = $ua->get("http://$mo->{PDU}/io.cgi?$op$mo->{Port}=0");
 
-    die "failed" unless $resp->is_success;
+    die "failed ".($resp->status_line) unless $resp->is_success;
     logm($resp->decoded_content);
 }
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 2/4] TestSupport: Break out build_url_vcs

[Ian Jackson]

This encapsulates discovery of the tree url and the applicable VCS,
in a form useable separately (ie, outside build_clone).

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 9e6479a..485a29a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -77,7 +77,9 @@ BEGIN {
                       propname_massage propname_check
          
                       get_stashed open_unique_stashfile compress_stashed
-                      dir_identify_vcs build_clone built_stash built_stash_file
+                      dir_identify_vcs
+                      build_url_vcs build_clone
+                      built_stash built_stash_file
                       built_compress_stashed
                       hg_dir_revision git_dir_revision vcs_dir_revision
                       store_revision store_vcs_revision
@@ -1267,15 +1269,14 @@ sub git_massage_url ($;@) {
     return $url;
 }
 
-sub build_clone ($$$$) {
-    my ($ho, $which, $builddir, $subdir) = @_;
-
-    need_runvars("tree_$which", "revision_$which");
+sub build_url_vcs ($) {
+    my ($which) = @_;
 
     my $tree= $r{"tree_$which"};
-    my $timeout= 4000;
+    return () unless $tree;
 
     my $vcs = $r{"treevcs_$which"};
+    
     if (!defined $vcs) {
 	my $effurl = $tree;
 	$effurl =~ s#\%20[^/]*$##;
@@ -1283,11 +1284,22 @@ sub build_clone ($$$$) {
 	    $vcs= 'hg';
 	} elsif ($effurl =~ m/\.git$/) {
 	    $vcs= 'git';
-	} else {
-	    die "unknown vcs for $which $tree ";
 	}
     }
 
+    return ($tree, $vcs);
+}
+
+sub build_clone ($$$$) {
+    my ($ho, $which, $builddir, $subdir) = @_;
+
+    need_runvars("tree_$which", "revision_$which");
+
+    my ($tree, $vcs) = build_url_vcs($which);
+    die "unknown vcs for $which $tree" unless defined $vcs;
+
+    my $timeout= 4000;
+
     my $rm = "rm -rf $subdir";
 
     if ($vcs eq 'hg') {
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 3/4] ts-xen-build: Factor out config_tree

[Ian Jackson]

Replaces ad-hoc repeated stanzas involving nonempty etc. with a sub
which generates the right output, using build_url_vcs.

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-xen-build | 52 +++++++++++++++++++++-------------------------------
 1 file changed, 21 insertions(+), 31 deletions(-)

diff --git a/ts-xen-build b/ts-xen-build
index 382fe62..5933dd4 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -29,7 +29,22 @@ selectbuildhost(\@ARGV);
 builddirsprops();
 
 my $enable_xsm = ($r{enable_xsm}//'false') =~ m/true/ ? 1 : 0;
-    
+
+sub config_tree ($$$) {
+    my ($which, $configurlvar, $configtagvar) = @_;
+
+    my ($tree,$vcs) = build_url_vcs($which);
+    my $revision = $r{"revision_$which"};
+
+    return
+        (nonempty($tree) ? <<END : '').
+	echo >>.config $configurlvar='$tree'
+END
+        (nonempty($revision) ? <<END : '');
+	echo >>.config $configtagvar='$revision'
+END
+}
+
 sub checkout () {
     prepbuilddirs();
 
@@ -61,36 +76,11 @@ END
 	fi
 	echo >>.config XSM_ENABLE='${build_xsm}'
 END
-               (nonempty($r{tree_qemu}) ? <<END : '').
-	echo >>.config QEMU_REMOTE='$r{tree_qemu}'
-END
-               (nonempty($r{revision_qemu}) ? <<END : '').
-	echo >>.config QEMU_TAG='$r{revision_qemu}'
-END
-               (nonempty($r{tree_qemuu}) ? <<END : '').
-	echo >>.config QEMU_UPSTREAM_URL='$r{tree_qemuu}'
-END
-               (nonempty($r{revision_qemuu}) ? <<END : '').
-	echo >>.config QEMU_UPSTREAM_REVISION='$r{revision_qemuu}'
-END
-               (nonempty($r{tree_seabios}) ? <<END : '').
-	echo >>.config SEABIOS_UPSTREAM_URL='$r{tree_seabios}'
-END
-               (nonempty($r{revision_seabios}) ? <<END : '').
-	echo >>.config SEABIOS_UPSTREAM_TAG='$r{revision_seabios}'
-END
-               (nonempty($r{tree_ovmf}) ? <<END : '').
-	echo >>.config OVMF_UPSTREAM_URL='$r{tree_ovmf}'
-END
-               (nonempty($r{revision_ovmf}) ? <<END : '').
-	echo >>.config OVMF_UPSTREAM_REVISION='$r{revision_ovmf}'
-END
-               (nonempty($r{tree_minios}) ? <<END : '').
-	echo >>.config MINIOS_UPSTREAM_URL='$r{tree_minios}'
-END
-               (nonempty($r{revision_minios}) ? <<END : '').
-	echo >>.config MINIOS_UPSTREAM_REVISION='$r{revision_minios}'
-END
+ (config_tree('qemu',   'QEMU_REMOTE',         'QEMU_TAG')).
+ (config_tree('qemuu',  'QEMU_UPSTREAM_URL',   'QEMU_UPSTREAM_REVISION')).
+ (config_tree('seabios','SEABIOS_UPSTREAM_URL','SEABIOS_UPSTREAM_TAG')).
+ (config_tree('ovmf',   'OVMF_UPSTREAM_URL',   'OVMF_UPSTREAM_REVISION')).
+ (config_tree('minios','MINIOS_UPSTREAM_URL',  'MINIOS_UPSTREAM_REVISION')).
                (nonempty($earlyprintk) ? <<END : '').
 	echo >>.config CONFIG_EARLY_PRINTK=$earlyprintk
 END
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH 4/4] build_clone: Move git_massage_url to build_url_vcs

[Ian Jackson]

The effect is that ts-xen-build (the only other caller of
build_url_vcs) now gets the massaged rather than unmassaged urls.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 485a29a..d0d6ef3 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1287,6 +1287,10 @@ sub build_url_vcs ($) {
 	}
     }
 
+    if (($vcs // '<unknown>') eq 'git') {
+	$tree = git_massage_url($tree);
+    }
+
     return ($tree, $vcs);
 }
 
@@ -1314,11 +1318,9 @@ END
 END
     } elsif ($vcs eq 'git') {
 
-	my $eff_tree = git_massage_url($tree);
-
         target_cmd_build($ho, $timeout, $builddir, <<END.
             $rm
-            git clone '$eff_tree' $subdir
+            git clone '$tree' $subdir
             cd $subdir
 END
                          (length($r{"revision_$which"}) ? <<END : ''));
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel