[PATCH OSSTEST] Stash debug symbols for build results

[PATCH OSSTEST] Stash debug symbols for build results

[Ian Campbell]

Specifically xen-syms (2.5M) and the vmlinux (~90M) file. I'm not sure if 90M
is too big to be stashed, for reference kerndist.tar.gz is ~60M
---
 Osstest/TestSupport.pm |   12 +++++++++++-
 ts-kernel-build        |    1 +
 ts-xen-build           |    1 +
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index f4ec055..95f3541 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -67,7 +67,7 @@ BEGIN {
                       propname_massage
          
                       get_stashed open_unique_stashfile
-                      dir_identify_vcs build_clone built_stash 
+                      dir_identify_vcs build_clone built_stash built_stash_file
                       hg_dir_revision git_dir_revision vcs_dir_revision
                       store_revision store_vcs_revision
 
@@ -1044,6 +1044,16 @@ END
     store_runvar("path_$item", $stashleaf);
 }
 
+sub built_stash_file ($$$$) {
+    my ($ho, $builddir, $item, $fname) = @_;
+    my $build= "build";
+    my $stashleaf= "$build/$item";
+    ensuredir("$stash/$build");
+    target_getfile($ho, 300,
+                   "$builddir/$fname",
+                   "$stash/$stashleaf");
+}
+
 sub vcs_dir_revision ($$$) {
     my ($ho,$builddir,$vcs) = @_;
     no strict qw(refs);
diff --git a/ts-kernel-build b/ts-kernel-build
index 5630dd7..9462278 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -321,3 +321,4 @@ if ($r{tree_linuxfirmware}) {
     fwinstall();
 }
 built_stash($ho, $builddir, 'dist', 'kerndist');
+built_stash_file($ho, $builddir, 'vmlinux', 'linux/vmlinux');
diff --git a/ts-xen-build b/ts-xen-build
index 8a905f5..e5c5725 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -198,6 +198,7 @@ sub stash () {
                     "xen-unstable/dist/${part}install",
                     "${part}dist");
     }
+    built_stash_file($ho, $builddir, "xen-syms", "xen-unstable/xen/xen-syms");
 }
 
 sub checkversions () {
-- 
1.7.10.4

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST] Stash debug symbols for build results"):
> Specifically xen-syms (2.5M) and the vmlinux (~90M) file. I'm not sure if 90M
> is too big to be stashed, for reference kerndist.tar.gz is ~60M

I think doing this would reduce the length of time the other logs can
be kept for by a factor of perhaps two.

Is xen-syms compressed ?

Ian.

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Ian Campbell]

On Mon, 2013-10-28 at 15:46 +0000, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST] Stash debug symbols for build results"):
> > Specifically xen-syms (2.5M) and the vmlinux (~90M) file. I'm not sure if 90M
> > is too big to be stashed, for reference kerndist.tar.gz is ~60M
> 
> I think doing this would reduce the length of time the other logs can
> be kept for by a factor of perhaps two.
> 
> Is xen-syms compressed ?

I don't think so, xen-syms and vmlinux are both raw ELF files.

Compressing with gzip -9 gives 939K and 44M respectively, which I
suspect is still too big. Woking's Debian is too old to have xz which I
would use otherwise (FWIW xz -6 on another system gives a 22M
vmlinux.xz)

The kernel has:
        config DEBUG_INFO_REDUCED
                bool "Reduce debugging information"
                depends on DEBUG_INFO
                help
                  If you say Y here gcc is instructed to generate less debugging
                  information for structure types. This means that tools that
                  need full debugging information (like kgdb or systemtap) won't
                  be happy. But if you merely need debugging information to
                  resolve line numbers there is no loss. Advantage is that
                  build directory object sizes shrink dramatically over a full
                  DEBUG_INFO build and compile times are reduced too.
                  Only works with newer gcc versions.

This seems like a good bet so I will try it.

Ian.

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Ian Campbell]

On Thu, 2013-10-31 at 12:14 +0000, Ian Campbell wrote:

> The kernel has:
>         config DEBUG_INFO_REDUCED
>                 bool "Reduce debugging information"
>                 depends on DEBUG_INFO
>                 help
>                   If you say Y here gcc is instructed to generate less debugging
>                   information for structure types. This means that tools that
>                   need full debugging information (like kgdb or systemtap) won't
>                   be happy. But if you merely need debugging information to
>                   resolve line numbers there is no loss. Advantage is that
>                   build directory object sizes shrink dramatically over a full
>                   DEBUG_INFO build and compile times are reduced too.
>                   Only works with newer gcc versions.
> 
> This seems like a good bet so I will try it.

Result is 30M uncompressed, 13M gzip compressed. Good enough? (with
compression I assume).

Ian.

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH OSSTEST] Stash debug symbols for build results"):
> Result is 30M uncompressed, 13M gzip compressed. Good enough? (with
> compression I assume).

Yes, I think so.

Ian.

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Andrew Cooper]

On 31/10/13 14:13, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH OSSTEST] Stash debug symbols for build results"):
>> Result is 30M uncompressed, 13M gzip compressed. Good enough? (with
>> compression I assume).
> Yes, I think so.
>
> Ian.

If you are concerned with space as opposed to {en/de}compressing time,
why not choose something like bzip2 which is slower but generally more
space efficient?

~Andrew

Re: [PATCH OSSTEST] Stash debug symbols for build results

[Ian Campbell]

On Thu, 2013-10-31 at 14:19 +0000, Andrew Cooper wrote:
> On 31/10/13 14:13, Ian Jackson wrote:
> > Ian Campbell writes ("Re: [Xen-devel] [PATCH OSSTEST] Stash debug symbols for build results"):
> >> Result is 30M uncompressed, 13M gzip compressed. Good enough? (with
> >> compression I assume).
> > Yes, I think so.
> >
> > Ian.
> 
> If you are concerned with space as opposed to {en/de}compressing time,
> why not choose something like bzip2 which is slower but generally more
> space efficient?

bzip2 saves ~0.25M out of a ~13M compressed file. I've already written
the serial and am about to post so I don't think it is worth it ;-)

Ian.