[PATCH v2 0/6] raisin: introduce ovmf and linux

[PATCH v2 0/6] raisin: introduce ovmf and linux

[Stefano Stabellini]

Hi all,

this patch series includes a few clean-ups and introduces a component to
build ovmf and a component to build a linux kernel. The latter is not
enabled by default, and could probably benefit from a bit more work on
the kconfig options to really be useful, but it is a start.

Changes in v2:
- rename _verbose_echo to verbose_echo
- add "raisin: pass --with-system-seabios with seabios was built"
- only enable ovmf in Xen, if we actually have built ovmf


Stefano Stabellini (6):
      raisin: introduce verbose_echo
      raisin: remove duplicate source config in raise
      raisin: rename ARCH to RAISIN_ARCH
      raisin: pass --with-system-seabios with seabios was built
      raisin: introduce ovmf
      raisin: build linux

 README                      |    2 +-
 components/grub             |    6 +--
 components/libvirt          |    2 +-
 components/linux            |  120 +++++++++++++++++++++++++++++++++++++++++++
 components/qemu             |    2 +-
 components/qemu_traditional |    2 +-
 components/seabios          |    6 +--
 components/series           |    2 +
 components/xen              |   16 ++++--
 defconfig                   |    8 ++-
 lib/common-functions.sh     |   50 +++++++-----------
 raise                       |   15 ++----
 scripts/mkdeb               |    8 +--
 scripts/mkrpm               |    2 +-
 14 files changed, 179 insertions(+), 62 deletions(-)
 create mode 100644 components/linux

Cheers,

Stefano

[PATCH v2 1/6] raisin: introduce verbose_echo

[Stefano Stabellini]

A new utility function to make the code more readable and compact:
prints a message if VERBOSE = 1.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

---

Changes in v2:
- rename _verbose_echo to verbose_echo
---
 lib/common-functions.sh |   46 ++++++++++++++++------------------------------
 1 file changed, 16 insertions(+), 30 deletions(-)

diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index dc3a2bb..2cfc5dc 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -1,5 +1,12 @@
 #!/usr/bin/env bash
 
+function verbose_echo() {
+    if [[ $VERBOSE -eq 1 ]]
+    then
+        echo $*
+    fi
+}
+
 # Executed once at the beginning of the script
 function common_init() {
     export BASEDIR=`pwd`
@@ -33,12 +40,9 @@ function common_init() {
     get_arch
     get_components
 
-    if [[ $VERBOSE -eq 1 ]]
-    then
-        echo "Distro: $DISTRO"
-        echo "Arch: $ARCH"
-        echo "Components: $COMPONENTS"
-    fi
+    verbose_echo "Distro: $DISTRO"
+    verbose_echo "Arch: $ARCH"
+    verbose_echo "Components: $COMPONENTS"
 
     for f in $COMPONENTS
     do
@@ -62,10 +66,7 @@ function get_components() {
             if eval [[ ! -z \$"$capital"_REVISION ]]
             then
                 COMPONENTS="$COMPONENTS $component"
-                if [[ $VERBOSE -eq 1 ]]
-                then
-                    echo "Found component $component"
-                fi
+                verbose_echo "Found component $component"
             fi
         done
     fi
@@ -166,10 +167,7 @@ function get_arch() {
 }
 
 function _check-package-deb() {
-    if [[ $VERBOSE -eq 1 ]]
-    then
-        echo "Checking for package ${args[0]}"
-    fi
+    verbose_echo "Checking for package ${args[0]}"
 
     if dpkg -s "$1" 2>/dev/null | grep -q "Status:.*installed"
     then
@@ -184,10 +182,7 @@ function _install-package-deb() {
 }
 
 function _check-package-rpm() {
-    if [[ $VERBOSE -eq 1 ]]
-    then
-        echo "Checking for package $1"
-    fi
+    verbose_echo "Checking for package $1"
 
     if rpm -q "$1" 2>&1 >/dev/null
     then
@@ -273,22 +268,13 @@ function for_each_component () {
         done
         if [[ $found -eq 0 ]]
         then
-            if [[ $VERBOSE -eq 1 ]]
-            then
-                echo "$component" is disabled
-            fi
+            verbose_echo "$component" is disabled
             continue
         fi
 
-        if [[ $VERBOSE -eq 1 ]]
-        then
-            echo calling "$component"_"$1"
-        fi
+        verbose_echo calling "$component"_"$1"
         "$component"_"$1"
-        if [[ $VERBOSE -eq 1 ]]
-        then
-            echo "$component"_"$1" done
-        fi
+        verbose_echo "$component"_"$1" done
     done
 }
 
-- 
1.7.10.4

[PATCH v2 2/6] raisin: remove duplicate source config in raise

[Stefano Stabellini]

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
---
 raise |   15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/raise b/raise
index bce6908..68dbfd8 100755
--- a/raise
+++ b/raise
@@ -17,9 +17,12 @@ _help() {
 }
 
 # Include your defaults
-if [[ -e "./config" ]] ; then
-    . ./config
+if [[ ! -e "./config" ]]
+then
+   echo "No config file found, copying default config"
+   cp defconfig config
 fi
+source ./config
 
 # To use this as a library, set RAISIN_PATH appropriately
 [[ -z "$RAISIN_PATH" ]] && RAISIN_PATH="$PWD/lib"
@@ -29,14 +32,6 @@ source ${RAISIN_PATH}/common-functions.sh
 source ${RAISIN_PATH}/git-checkout.sh
 source ${RAISIN_PATH}/commands.sh
 
-# Include your defaults
-if [[ ! -e "./config" ]] ; then
-   echo "No config file found, copying default config"
-   cp defconfig config
-fi
-
-source ./config
-
 # Set up basic functionality
 common_init
 
-- 
1.7.10.4

[PATCH v2 3/6] raisin: rename ARCH to RAISIN_ARCH

[Stefano Stabellini]

Rename exported environmental variable ARCH to RAISIN_ARCH not to
conflict with any component Makefile variables.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
---
 README                      |    2 +-
 components/grub             |    6 +++---
 components/libvirt          |    2 +-
 components/qemu             |    2 +-
 components/qemu_traditional |    2 +-
 components/seabios          |    6 +++---
 components/xen              |    2 +-
 lib/common-functions.sh     |    6 +++---
 scripts/mkdeb               |    8 ++++----
 scripts/mkrpm               |    2 +-
 10 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/README b/README
index ab8d3e7..b7832da 100644
--- a/README
+++ b/README
@@ -85,7 +85,7 @@ RAISIN_MAKE: make command to run
 SUDO: sudo command to run
 DISTRO: which Linux distribution we are running on, Debian, Fedora, etc
 PKGTYPE: which package format is used by the distro, rpm, deb, etc
-ARCH: which architecture we are running on, x86_64, arm32, etc.
+RAISIN_ARCH: which architecture we are running on, x86_64, arm32, etc.
 BASEDIR: absolute path of the raisin directory
 PREFIX: installation PREFIX
 INST_DIR: absolute path of the installation directory
diff --git a/components/grub b/components/grub
index 7c14a62..fa72b99 100644
--- a/components/grub
+++ b/components/grub
@@ -17,13 +17,13 @@ function grub_check_package() {
     local DEP_CentOS_x86_64="$DEP_Fedora_x86_64"
 
 
-    if [[ $ARCH != "x86_64" && $ARCH != "x86_32" ]]
+    if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
         echo grub is only supported on x86_32 and x86_64
         return
     fi
     echo Checking Grub dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 
@@ -41,7 +41,7 @@ function grub_build() {
       -m ../memdisk.tar -o grub-i386-xen grub-core/*mod
     cp grub-i386-xen "$INST_DIR"/$PREFIX/lib/xen/boot
     ## GRUB64
-    if [[ $ARCH = "x86_64" ]]
+    if [[ $RAISIN_ARCH = "x86_64" ]]
     then
         $RAISIN_MAKE clean
         ./configure --target=amd64 --with-platform=xen
diff --git a/components/libvirt b/components/libvirt
index 5fb1a41..a554643 100644
--- a/components/libvirt
+++ b/components/libvirt
@@ -23,7 +23,7 @@ function libvirt_check_package() {
     local DEP_CentOS_x86_64="$DEP_Fedora_x86_64"
 
     echo Checking Libvirt dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 function libvirt_build() {
diff --git a/components/qemu b/components/qemu
index 00aeeb6..72cfec1 100644
--- a/components/qemu
+++ b/components/qemu
@@ -12,7 +12,7 @@ function qemu_check_package() {
     local DEP_Fedora_x86_64="$DEP_Fedora_common"
 
     echo Checking QEMU dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 function qemu_build() {
diff --git a/components/qemu_traditional b/components/qemu_traditional
index 500cbed..b338007 100644
--- a/components/qemu_traditional
+++ b/components/qemu_traditional
@@ -13,7 +13,7 @@ function qemu_traditional_check_package() {
     local DEP_Fedora_x86_64="$DEP_Fedora_common"
 
     echo Checking QEMU dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 function qemu_traditional_build() {
diff --git a/components/seabios b/components/seabios
index 960a538..ed2c7d2 100644
--- a/components/seabios
+++ b/components/seabios
@@ -12,18 +12,18 @@ function seabios_check_package() {
     local DEP_Fedora_x86_64="$DEP_Fedora_common"
 
 
-    if [[ $ARCH != "x86_64" && $ARCH != "x86_32" ]]
+    if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
         echo seabios is only supported on x86_32 and x86_64
         return
     fi
     echo Checking SeaBIOS dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 
 function seabios_build() {
-    if [[ $ARCH != "x86_64" && $ARCH != "x86_32" ]]
+    if [[ $RAISIN_ARCH != "x86_64" && $RAISIN_ARCH != "x86_32" ]]
     then
         echo seabios is only supported on x86_32 and x86_64
         return
diff --git a/components/xen b/components/xen
index b7258b0..b3426f0 100644
--- a/components/xen
+++ b/components/xen
@@ -20,7 +20,7 @@ function xen_check_package() {
     local DEP_CentOS_x86_64="$DEP_CentOS_x86_32 glibc-devel.i686"
 
     echo Checking Xen dependencies
-    eval check-package \$DEP_"$DISTRO"_"$ARCH"
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
 }
 
 function xen_build() {
diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index 2cfc5dc..3b512d1 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -41,7 +41,7 @@ function common_init() {
     get_components
 
     verbose_echo "Distro: $DISTRO"
-    verbose_echo "Arch: $ARCH"
+    verbose_echo "Arch: $RAISIN_ARCH"
     verbose_echo "Components: $COMPONENTS"
 
     for f in $COMPONENTS
@@ -161,7 +161,7 @@ function get_distro() {
 }
 
 function get_arch() {
-    export ARCH=`uname -m | sed -e s/i.86/x86_32/ -e s/i86pc/x86_32/ -e \
+    export RAISIN_ARCH=`uname -m | sed -e s/i.86/x86_32/ -e s/i86pc/x86_32/ -e \
                 s/amd64/x86_64/ -e s/armv7.*/arm32/ -e s/armv8.*/arm64/ \
                 -e s/aarch64/arm64/`
 }
@@ -296,7 +296,7 @@ function install_package() {
         $SUDO dpkg -i "$1".deb
     elif [[  $DISTRO = "Fedora" ]]
     then
-        $SUDO rpm -i --force "$1"-`git show --oneline | head -1 | cut -d " " -f 1`-0.$ARCH.rpm
+        $SUDO rpm -i --force "$1"-`git show --oneline | head -1 | cut -d " " -f 1`-0.$RAISIN_ARCH.rpm
     else
         echo "Don't know how to install packages on $DISTRO"
     fi
diff --git a/scripts/mkdeb b/scripts/mkdeb
index 0225736..dbd92bd 100755
--- a/scripts/mkdeb
+++ b/scripts/mkdeb
@@ -3,7 +3,7 @@
 # mkdeb: package $INST_DIR output in a .deb 
 #
 # Takes 1 argument: the package name
-# It relies on ARCH and INST_DIR being set correctly
+# It relies on RAISIN_ARCH and INST_DIR being set correctly
 
 set -e
 
@@ -18,12 +18,12 @@ name=$1
 cd "$BASEDIR"
 
 # map the architecture, if necessary
-case "$ARCH" in
+case "$RAISIN_ARCH" in
   x86_32|x86_32p)  arch=i386 ;;
   x86_64)  arch=amd64 ;;
   arm32)   arch=armhf ;;
-  arm64)   arch=$ARCH;;
-  *) echo "Unknown ARCH $ARCH" >&2
+  arm64)   arch=$RAISIN_ARCH;;
+  *) echo "Unknown ARCH $RAISIN_ARCH" >&2
      exit 1
      ;;
 esac
diff --git a/scripts/mkrpm b/scripts/mkrpm
index 8de4508..7965c0c 100755
--- a/scripts/mkrpm
+++ b/scripts/mkrpm
@@ -3,7 +3,7 @@
 # mkrpm: package INST_DIR in an .rpm
 #
 # Takes 1 argument: the package name
-# It relies on ARCH and INST_DIR being set correctly
+# It relies on INST_DIR being set correctly
 
 set -e
 
-- 
1.7.10.4

[PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Stefano Stabellini]

Detect whether we have built seabios and only pass the relative command
line argument to Xen if we actually did.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 components/xen |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/components/xen b/components/xen
index b3426f0..ac9e8fb 100644
--- a/components/xen
+++ b/components/xen
@@ -24,12 +24,17 @@ function xen_check_package() {
 }
 
 function xen_build() {
+    local seabios_opt=""
+
     cd "$BASEDIR"
     git-checkout $XEN_URL $XEN_REVISION xen-dir
     cd xen-dir
+    if [[ -e "$BASEDIR"/seabios-dir/out/bios.bin ]]
+    then
+        seabios_opt="--with-system-seabios="$BASEDIR"/seabios-dir/out/bios.bin"
+    fi
     ./configure --prefix=$PREFIX --with-system-qemu=$PREFIX/lib/xen/bin/qemu-system-i386 \
-        --disable-qemu-traditional --enable-rombios \
-        --with-system-seabios="$BASEDIR"/seabios-dir/out/bios.bin
+        --disable-qemu-traditional --enable-rombios $seabios_opt
     $RAISIN_MAKE
     $RAISIN_MAKE install DESTDIR="$INST_DIR"
     cd "$BASEDIR"
-- 
1.7.10.4

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Ian Campbell]

On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
> Detect whether we have built seabios and only pass the relative command
> line argument to Xen if we actually did.

For this and the following ovmf if we didn't build seabios/ovmf here
then you pass nothing, which will result in xen.git downloading and
doing things itself, is that what is wanted? I think, although I confess
I'm not sure and I haven't checked, you can pass --without-thing to
avoid this and to disable the support (which might hopefully result in
clearer error messages to the user down the line).

Ian.

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Stefano Stabellini]

On Wed, 22 Apr 2015, Ian Campbell wrote:
> On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
> > Detect whether we have built seabios and only pass the relative command
> > line argument to Xen if we actually did.
> 
> For this and the following ovmf if we didn't build seabios/ovmf here
> then you pass nothing, which will result in xen.git downloading and
> doing things itself, is that what is wanted? I think, although I confess
> I'm not sure and I haven't checked, you can pass --without-thing to
> avoid this and to disable the support (which might hopefully result in
> clearer error messages to the user down the line).

This is a good point. I don't know what we want exactly here.

In the case of OVMF if you don't specify anything the default in Xen is
not to build it.
In the case of Seabios if you don't specify anything the default is
clone and build.

Do we want to be absolutely sure to avoid any cloning from the
xen-unstable tree with raisin?
Or do we simply want to fall back to the default behaviour?

I think that both a reasonable answers, I am leaning toward avoiding any
cloning always. This also means disabling stubdoms.  What do you think?

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Ian Campbell]

On Wed, 2015-04-22 at 15:15 +0100, Stefano Stabellini wrote:
> On Wed, 22 Apr 2015, Ian Campbell wrote:
> > On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
> > > Detect whether we have built seabios and only pass the relative command
> > > line argument to Xen if we actually did.
> > 
> > For this and the following ovmf if we didn't build seabios/ovmf here
> > then you pass nothing, which will result in xen.git downloading and
> > doing things itself, is that what is wanted? I think, although I confess
> > I'm not sure and I haven't checked, you can pass --without-thing to
> > avoid this and to disable the support (which might hopefully result in
> > clearer error messages to the user down the line).
> 
> This is a good point. I don't know what we want exactly here.
> 
> In the case of OVMF if you don't specify anything the default in Xen is
> not to build it.
> In the case of Seabios if you don't specify anything the default is
> clone and build.
> 
> Do we want to be absolutely sure to avoid any cloning from the
> xen-unstable tree with raisin?
> Or do we simply want to fall back to the default behaviour?
> 
> I think that both a reasonable answers, I am leaning toward avoiding any
> cloning always.

I think this makes sense. If a users wants ovmf they should ask for it
via raisin, if they don't then they won't get it.

>  This also means disabling stubdoms.  What do you think?

Is the plan to eventually have raisin build stubdoms itself?

Ian.

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Stefano Stabellini]

On Wed, 22 Apr 2015, Ian Campbell wrote:
> On Wed, 2015-04-22 at 15:15 +0100, Stefano Stabellini wrote:
> > On Wed, 22 Apr 2015, Ian Campbell wrote:
> > > On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
> > > > Detect whether we have built seabios and only pass the relative command
> > > > line argument to Xen if we actually did.
> > > 
> > > For this and the following ovmf if we didn't build seabios/ovmf here
> > > then you pass nothing, which will result in xen.git downloading and
> > > doing things itself, is that what is wanted? I think, although I confess
> > > I'm not sure and I haven't checked, you can pass --without-thing to
> > > avoid this and to disable the support (which might hopefully result in
> > > clearer error messages to the user down the line).
> > 
> > This is a good point. I don't know what we want exactly here.
> > 
> > In the case of OVMF if you don't specify anything the default in Xen is
> > not to build it.
> > In the case of Seabios if you don't specify anything the default is
> > clone and build.
> > 
> > Do we want to be absolutely sure to avoid any cloning from the
> > xen-unstable tree with raisin?
> > Or do we simply want to fall back to the default behaviour?
> > 
> > I think that both a reasonable answers, I am leaning toward avoiding any
> > cloning always.
> 
> I think this makes sense. If a users wants ovmf they should ask for it
> via raisin, if they don't then they won't get it.
> 
> >  This also means disabling stubdoms.  What do you think?
> 
> Is the plan to eventually have raisin build stubdoms itself?

Yes, I think it would make sense for raisin to build them.

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[George Dunlap]

On 04/22/2015 03:15 PM, Stefano Stabellini wrote:
> On Wed, 22 Apr 2015, Ian Campbell wrote:
>> On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
>>> Detect whether we have built seabios and only pass the relative command
>>> line argument to Xen if we actually did.
>>
>> For this and the following ovmf if we didn't build seabios/ovmf here
>> then you pass nothing, which will result in xen.git downloading and
>> doing things itself, is that what is wanted? I think, although I confess
>> I'm not sure and I haven't checked, you can pass --without-thing to
>> avoid this and to disable the support (which might hopefully result in
>> clearer error messages to the user down the line).
> 
> This is a good point. I don't know what we want exactly here.
> 
> In the case of OVMF if you don't specify anything the default in Xen is
> not to build it.
> In the case of Seabios if you don't specify anything the default is
> clone and build.
> 
> Do we want to be absolutely sure to avoid any cloning from the
> xen-unstable tree with raisin?
> Or do we simply want to fall back to the default behaviour?
> 
> I think that both a reasonable answers, I am leaning toward avoiding any
> cloning always. This also means disabling stubdoms.  What do you think?

Well I think what people will expect is that if they disable seabios
from ENABLED_COMPONENTS, that it won't be enabled, not that it will
still be built but by Xen instead.

I think it's reasonable to expect users to do some clean-up if they
change their mind about what they want to build.  (I wonder if we could
use a "./raise distclean", which will rm -rf dist, as well as all
components in series but not in COMPONENTS.)

Re stubdoms, I think that we should let the xen component do it until
it's possible to do it out of tree (i.e., no regression in functionality).

 -George

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Stefano Stabellini]

On Wed, 22 Apr 2015, George Dunlap wrote:
> On 04/22/2015 03:15 PM, Stefano Stabellini wrote:
> > On Wed, 22 Apr 2015, Ian Campbell wrote:
> >> On Tue, 2015-04-21 at 17:48 +0100, Stefano Stabellini wrote:
> >>> Detect whether we have built seabios and only pass the relative command
> >>> line argument to Xen if we actually did.
> >>
> >> For this and the following ovmf if we didn't build seabios/ovmf here
> >> then you pass nothing, which will result in xen.git downloading and
> >> doing things itself, is that what is wanted? I think, although I confess
> >> I'm not sure and I haven't checked, you can pass --without-thing to
> >> avoid this and to disable the support (which might hopefully result in
> >> clearer error messages to the user down the line).
> > 
> > This is a good point. I don't know what we want exactly here.
> > 
> > In the case of OVMF if you don't specify anything the default in Xen is
> > not to build it.
> > In the case of Seabios if you don't specify anything the default is
> > clone and build.
> > 
> > Do we want to be absolutely sure to avoid any cloning from the
> > xen-unstable tree with raisin?
> > Or do we simply want to fall back to the default behaviour?
> > 
> > I think that both a reasonable answers, I am leaning toward avoiding any
> > cloning always. This also means disabling stubdoms.  What do you think?
> 
> Well I think what people will expect is that if they disable seabios
> from ENABLED_COMPONENTS, that it won't be enabled, not that it will
> still be built but by Xen instead.
> 
> I think it's reasonable to expect users to do some clean-up if they
> change their mind about what they want to build.  (I wonder if we could
> use a "./raise distclean", which will rm -rf dist, as well as all
> components in series but not in COMPONENTS.)

OK


> Re stubdoms, I think that we should let the xen component do it until
> it's possible to do it out of tree (i.e., no regression in functionality).

On the other hands current stubdoms are not even tested in osstest, so
we might as well wait for the next generation to be ready before having
them in raisin.

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Ian Campbell]

On Wed, 2015-04-22 at 15:49 +0100, Stefano Stabellini wrote:
> On Wed, 22 Apr 2015, George Dunlap wrote:
> > Re stubdoms, I think that we should let the xen component do it until
> > it's possible to do it out of tree (i.e., no regression in functionality).
> 
> On the other hands current stubdoms are not even tested in osstest, so
> we might as well wait for the next generation to be ready before having
> them in raisin.

They are at least build tested.

And George's point about "no regression in functionality" (i.e. for
users, not test systems) is a strong one.

Unless you were suggesting to leave them to xen.git for now?

Ian.

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[George Dunlap]

On Wed, Apr 22, 2015 at 3:49 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
>> Re stubdoms, I think that we should let the xen component do it until
>> it's possible to do it out of tree (i.e., no regression in functionality).
>
> On the other hands current stubdoms are not even tested in osstest, so
> we might as well wait for the next generation to be ready before having
> them in raisin.

I don't really get the logic here.  But it is true that we shipped 4.5
with qemu stubdomains broken and hardly anyone noticed for what, a
month?

 -George

Re: [PATCH v2 4/6] raisin: pass --with-system-seabios with seabios was built

[Stefano Stabellini]

On Wed, 22 Apr 2015, George Dunlap wrote:
> On Wed, Apr 22, 2015 at 3:49 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> >> Re stubdoms, I think that we should let the xen component do it until
> >> it's possible to do it out of tree (i.e., no regression in functionality).
> >
> > On the other hands current stubdoms are not even tested in osstest, so
> > we might as well wait for the next generation to be ready before having
> > them in raisin.
> 
> I don't really get the logic here.  But it is true that we shipped 4.5
> with qemu stubdomains broken and hardly anyone noticed for what, a
> month?

Yes, this is what I was referring to.

The logic is that they are not tested, they are not enabled by default,
and in their current form they get in the way of getting rid of all git
cloning in xen-unstable. I would rather disable them until we get them
properly in raisin.

[PATCH v2 5/6] raisin: introduce ovmf

[Stefano Stabellini]

Add a component to build ovmf and pass the output binary to the xen
build.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

---

Changes in v2:

- only enable ovmf in Xen, if we actually have built ovmf.
---
 components/series |    1 +
 components/xen    |    7 ++++++-
 defconfig         |    6 ++++--
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/components/series b/components/series
index f0f3cfa..fe9092a 100644
--- a/components/series
+++ b/components/series
@@ -1,4 +1,5 @@
 seabios
+ovmf
 xen
 qemu
 qemu_traditional
diff --git a/components/xen b/components/xen
index ac9e8fb..9ff0222 100644
--- a/components/xen
+++ b/components/xen
@@ -25,6 +25,7 @@ function xen_check_package() {
 
 function xen_build() {
     local seabios_opt=""
+    local ovmf_opt=""
 
     cd "$BASEDIR"
     git-checkout $XEN_URL $XEN_REVISION xen-dir
@@ -33,8 +34,12 @@ function xen_build() {
     then
         seabios_opt="--with-system-seabios="$BASEDIR"/seabios-dir/out/bios.bin"
     fi
+    if [[ -e "$BASEDIR"/ovmf-dir/ovmf.bin ]]
+    then
+        ovmf_opt="--enable-ovmf --with-system-ovmf="$BASEDIR"/ovmf-dir/ovmf.bin"
+    fi
     ./configure --prefix=$PREFIX --with-system-qemu=$PREFIX/lib/xen/bin/qemu-system-i386 \
-        --disable-qemu-traditional --enable-rombios $seabios_opt
+        --disable-qemu-traditional --enable-rombios $seabios_opt $ovmf_opt
     $RAISIN_MAKE
     $RAISIN_MAKE install DESTDIR="$INST_DIR"
     cd "$BASEDIR"
diff --git a/defconfig b/defconfig
index d3ef283..7d2a3f7 100644
--- a/defconfig
+++ b/defconfig
@@ -1,12 +1,12 @@
 # Config variables for raisin
 
 # Components
-## All components: seabios xen qemu qemu_traditional grub libvirt
+## All components: seabios ovmf xen qemu qemu_traditional grub libvirt
 ## Core xen functionality: xen
 ## Remove a component from the list below, if you want to disable it
 ## You can manually overwrite this list using the COMPONENTS
 ## environmental variable.
-ENABLED_COMPONENTS="seabios xen qemu qemu_traditional grub libvirt"
+ENABLED_COMPONENTS="seabios ovmf xen qemu qemu_traditional grub libvirt"
 
 # Build config
 ## Make command to run
@@ -27,6 +27,7 @@ QEMU_TRADITIONAL_URL="git://xenbits.xen.org/qemu-xen-unstable.git"
 SEABIOS_URL="git://xenbits.xen.org/seabios.git"
 GRUB_URL="git://git.savannah.gnu.org/grub.git"
 LIBVIRT_URL="git://libvirt.org/libvirt.git"
+OVMF_URL="git://xenbits.xen.org/ovmf.git"
 
 # Software versions.
 XEN_REVISION="master"
@@ -35,3 +36,4 @@ QEMU_TRADITIONAL_REVISION="master"
 SEABIOS_REVISION="master"
 GRUB_REVISION="master"
 LIBVIRT_REVISION="master"
+OVMF_REVISION="master"
-- 
1.7.10.4

[PATCH v2 6/6] raisin: build linux

[Stefano Stabellini]

Add a component, disabled by default, to build a linux kernel with the
Xen kconfig options enabled.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
---
 components/linux  |  120 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 components/series |    1 +
 defconfig         |    4 +-
 3 files changed, 124 insertions(+), 1 deletion(-)
 create mode 100644 components/linux

diff --git a/components/linux b/components/linux
new file mode 100644
index 0000000..f90a894
--- /dev/null
+++ b/components/linux
@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+
+function linux_check_package() {
+    local DEP_Debian_common="build-essential bc openssl"
+    local DEP_Debian_x86_32="$DEP_Debian_common"
+    local DEP_Debian_x86_64="$DEP_Debian_common"
+    local DEP_Debian_arm32="$DEP_Debian_common"
+    local DEP_Debian_arm64="$DEP_Debian_common"
+
+    local DEP_Fedora_common="make gcc bc openssl"
+    local DEP_Fedora_x86_32="$DEP_Fedora_common"
+    local DEP_Fedora_x86_64="$DEP_Fedora_common"
+
+    local DEP_CentOS_common="$DEP_Fedora_common"
+    local DEP_CentOS_x86_32="$DEP_Fedora_x86_32"
+    local DEP_CentOS_x86_64="$DEP_Fedora_x86_64"
+
+    echo Checking Linux dependencies
+    eval check-package \$DEP_"$DISTRO"_"$RAISIN_ARCH"
+}
+
+function _xenify_config() {
+    echo "CONFIG_HYPERVISOR_GUEST=y" >> $1
+    echo "CONFIG_PARAVIRT=y" >> $1
+    echo "CONFIG_PARAVIRT_SPINLOCKS=y" >> $1
+    echo "CONFIG_XEN=y" >> $1
+    echo "CONFIG_XEN_DOM0=y" >> $1
+    echo "CONFIG_XEN_PVHVM=y" >> $1
+    echo "CONFIG_XEN_SAVE_RESTORE=y" >> $1
+    echo "CONFIG_XEN_DEBUG_FS=y" >> $1
+    echo "CONFIG_XEN_PVH=y" >> $1
+    echo "CONFIG_PARAVIRT_CLOCK=y" >> $1
+    echo "CONFIG_BALLOON_COMPACTION=y" >> $1
+    echo "CONFIG_XEN_PCIDEV_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_BLKDEV_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_BLKDEV_BACKEND=y" >> $1
+    echo "CONFIG_XEN_SCSI_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_NETDEV_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_NETDEV_BACKEND=y" >> $1
+    echo "CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_FBDEV_FRONTEND=y" >> $1
+    echo "CONFIG_HVC_XEN=y" >> $1
+    echo "CONFIG_HVC_XEN_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_BALLOON=y" >> $1
+    echo "CONFIG_XEN_SCRUB_PAGES=y" >> $1
+    echo "CONFIG_XEN_DEV_EVTCHN=y" >> $1
+    echo "CONFIG_XEN_BACKEND=y" >> $1
+    echo "CONFIG_XENFS=y" >> $1
+    echo "CONFIG_XEN_COMPAT_XENFS=y" >> $1
+    echo "CONFIG_XEN_SYS_HYPERVISOR=y" >> $1
+    echo "CONFIG_XEN_XENBUS_FRONTEND=y" >> $1
+    echo "CONFIG_XEN_GNTDEV=y" >> $1
+    echo "CONFIG_XEN_GRANT_DEV_ALLOC=y" >> $1
+    echo "CONFIG_SWIOTLB_XEN=y" >> $1
+    echo "CONFIG_XEN_PCIDEV_BACKEND=y" >> $1
+    echo "CONFIG_XEN_PRIVCMD=y" >> $1
+    echo "CONFIG_XEN_HAVE_PVMMU=y" >> $1
+    echo "CONFIG_XEN_ACPI_PROCESSOR=y" >> $1
+    echo "CONFIG_XEN_EFI=y" >> $1
+    echo "CONFIG_XEN_AUTO_XLATE=y" >> $1
+    echo "CONFIG_BRIDGE=y" >> $1
+}
+
+function linux_build() {
+    local vmlinuz
+
+    cd "$BASEDIR"
+    git-checkout $LINUX_URL $LINUX_REVISION linux-dir
+    cd linux-dir
+
+    if [[ ! -e .config ]]
+    then
+        if [[ -e /boot/config-`uname -r` ]]
+        then
+            cp /boot/config-`uname -r` .config
+        else
+            $RAISIN_MAKE defconfig
+        fi
+        _xenify_config .config
+        $RAISIN_MAKE olddefconfig
+    fi
+
+    $RAISIN_MAKE
+    $RAISIN_MAKE modules_install INSTALL_MOD_PATH="$INST_DIR"
+
+    mkdir -p "$INST_DIR"/boot/xen
+    vmlinuz="$INST_DIR"/boot/xen/vmlinuz-$RAISIN_ARCH-$LINUX_REVISION-`date +"%Y%m%d.%H%M%S"`
+
+    if [[ $RAISIN_ARCH = "x86_64" || $RAISIN_ARCH = "x86_32" ]]
+    then
+        cp arch/x86/boot/bzImage "$vmlinuz"
+    elif [[ $RAISIN_ARCH = "arm32" ]]
+    then
+        cp arch/arm/boot/zImage "$vmlinuz"
+    elif [[ $RAISIN_ARCH = "arm64" ]]
+    then
+        cp arch/x86/boot/Image.gz "$vmlinuz"
+    fi
+
+    cd ..
+}
+
+function linux_clean() {
+    cd "$BASEDIR"
+    if [[ -d linux-dir ]]
+    then
+        cd linux-dir
+        $RAISIN_MAKE distclean
+        cd ..
+        rm -rf linux-dir
+    fi
+}
+
+function linux_configure() {
+    :
+}
+
+function linux_unconfigure() {
+    :
+}
diff --git a/components/series b/components/series
index fe9092a..928e78e 100644
--- a/components/series
+++ b/components/series
@@ -5,3 +5,4 @@ qemu
 qemu_traditional
 grub
 libvirt
+linux
diff --git a/defconfig b/defconfig
index 7d2a3f7..b4ed94d 100644
--- a/defconfig
+++ b/defconfig
@@ -1,7 +1,7 @@
 # Config variables for raisin
 
 # Components
-## All components: seabios ovmf xen qemu qemu_traditional grub libvirt
+## All components: seabios ovmf xen qemu qemu_traditional grub libvirt linux
 ## Core xen functionality: xen
 ## Remove a component from the list below, if you want to disable it
 ## You can manually overwrite this list using the COMPONENTS
@@ -28,6 +28,7 @@ SEABIOS_URL="git://xenbits.xen.org/seabios.git"
 GRUB_URL="git://git.savannah.gnu.org/grub.git"
 LIBVIRT_URL="git://libvirt.org/libvirt.git"
 OVMF_URL="git://xenbits.xen.org/ovmf.git"
+LINUX_URL="git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git"
 
 # Software versions.
 XEN_REVISION="master"
@@ -37,3 +38,4 @@ SEABIOS_REVISION="master"
 GRUB_REVISION="master"
 LIBVIRT_REVISION="master"
 OVMF_REVISION="master"
+LINUX_REVISION="master"
-- 
1.7.10.4

Re: [PATCH v2 0/6] raisin: introduce ovmf and linux

[George Dunlap]

On 04/21/2015 05:46 PM, Stefano Stabellini wrote:
> Hi all,
> 
> this patch series includes a few clean-ups and introduces a component to
> build ovmf and a component to build a linux kernel. The latter is not
> enabled by default, and could probably benefit from a bit more work on
> the kconfig options to really be useful, but it is a start.
> 
> Changes in v2:
> - rename _verbose_echo to verbose_echo
> - add "raisin: pass --with-system-seabios with seabios was built"
> - only enable ovmf in Xen, if we actually have built ovmf

Everything looks good:

Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Re: [PATCH v2 0/6] raisin: introduce ovmf and linux

[Stefano Stabellini]

On Tue, 21 Apr 2015, George Dunlap wrote:
> On 04/21/2015 05:46 PM, Stefano Stabellini wrote:
> > Hi all,
> > 
> > this patch series includes a few clean-ups and introduces a component to
> > build ovmf and a component to build a linux kernel. The latter is not
> > enabled by default, and could probably benefit from a bit more work on
> > the kconfig options to really be useful, but it is a start.
> > 
> > Changes in v2:
> > - rename _verbose_echo to verbose_echo
> > - add "raisin: pass --with-system-seabios with seabios was built"
> > - only enable ovmf in Xen, if we actually have built ovmf
> 
> Everything looks good:
> 
> Acked-by: George Dunlap <george.dunlap@eu.citrix.com>

Thanks!