[PATCH 0/2] tests/vm: fix ENOSPC in the openbsd VM image

[PATCH 0/2] tests/vm: fix ENOSPC in the openbsd VM image

[Daniel P. Berrangé]

The openbsd VM builds only have 3.5 GB of space available in /home, out
of the 20 GB disk image. This is insufficient when running iotests in
parallel with all other tests.

Note, after applying this series:

  rm ~/.cache/qemu-vm/images/openbsd.img

is needed for it to take effect if you've previously used the openbsd
image.

Daniel P. Berrangé (2):
  tests/vm: skip X11 in openbsd installation
  tests/vm: custom openbsd partitioning to increase /home space

 tests/vm/openbsd | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

-- 
2.39.2

[PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Daniel P. Berrangé]

As a VM used only for automated testing there is no need to
install the X11 stack.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/vm/openbsd | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index eaeb201e91..6af7afac0d 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
         self.console_wait("Password for root account")
         self.console_send("%s\n" % self._config["root_pass"])
         self.console_wait_send("Start sshd(8)",           "yes\n")
-        self.console_wait_send("X Window System",         "\n")
-        self.console_wait_send("xenodm",                  "\n")
+        self.console_wait_send("X Window System",         "no\n")
         self.console_wait_send("console to com0",         "\n")
         self.console_wait_send("Which speed",             "\n")
 
-- 
2.39.2

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Thomas Huth]

On 22/03/2023 13.36, Daniel P. Berrangé wrote:
> As a VM used only for automated testing there is no need to
> install the X11 stack.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/vm/openbsd | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index eaeb201e91..6af7afac0d 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>           self.console_wait("Password for root account")
>           self.console_send("%s\n" % self._config["root_pass"])
>           self.console_wait_send("Start sshd(8)",           "yes\n")
> -        self.console_wait_send("X Window System",         "\n")
> -        self.console_wait_send("xenodm",                  "\n")
> +        self.console_wait_send("X Window System",         "no\n")
>           self.console_wait_send("console to com0",         "\n")
>           self.console_wait_send("Which speed",             "\n")
>   

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Philippe Mathieu-Daudé]

On 22/3/23 13:36, Daniel P. Berrangé wrote:
> As a VM used only for automated testing there is no need to
> install the X11 stack.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/vm/openbsd | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index eaeb201e91..6af7afac0d 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>           self.console_wait("Password for root account")
>           self.console_send("%s\n" % self._config["root_pass"])
>           self.console_wait_send("Start sshd(8)",           "yes\n")
> -        self.console_wait_send("X Window System",         "\n")
> -        self.console_wait_send("xenodm",                  "\n")
> +        self.console_wait_send("X Window System",         "no\n")

Wasn't this useful to link a X11-ready binary, even if only testing
with -display=none?

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Thomas Huth]

On 22/03/2023 14.38, Philippe Mathieu-Daudé wrote:
> On 22/3/23 13:36, Daniel P. Berrangé wrote:
>> As a VM used only for automated testing there is no need to
>> install the X11 stack.
>>
>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>> ---
>>   tests/vm/openbsd | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
>> index eaeb201e91..6af7afac0d 100755
>> --- a/tests/vm/openbsd
>> +++ b/tests/vm/openbsd
>> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>>           self.console_wait("Password for root account")
>>           self.console_send("%s\n" % self._config["root_pass"])
>>           self.console_wait_send("Start sshd(8)",           "yes\n")
>> -        self.console_wait_send("X Window System",         "\n")
>> -        self.console_wait_send("xenodm",                  "\n")
>> +        self.console_wait_send("X Window System",         "no\n")
> 
> Wasn't this useful to link a X11-ready binary, even if only testing
> with -display=none?

tests/vm/openbsd later installs sdl2 and gtk+3, so I assume that will pull 
in the required libraries if necessary, without all the other non-necessary 
stuff.

  Thomas

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Philippe Mathieu-Daudé]

On 22/3/23 14:40, Thomas Huth wrote:
> On 22/03/2023 14.38, Philippe Mathieu-Daudé wrote:
>> On 22/3/23 13:36, Daniel P. Berrangé wrote:
>>> As a VM used only for automated testing there is no need to
>>> install the X11 stack.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>   tests/vm/openbsd | 3 +--
>>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
>>> index eaeb201e91..6af7afac0d 100755
>>> --- a/tests/vm/openbsd
>>> +++ b/tests/vm/openbsd
>>> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>>>           self.console_wait("Password for root account")
>>>           self.console_send("%s\n" % self._config["root_pass"])
>>>           self.console_wait_send("Start sshd(8)",           "yes\n")
>>> -        self.console_wait_send("X Window System",         "\n")
>>> -        self.console_wait_send("xenodm",                  "\n")
>>> +        self.console_wait_send("X Window System",         "no\n")
>>
>> Wasn't this useful to link a X11-ready binary, even if only testing
>> with -display=none?
> 
> tests/vm/openbsd later installs sdl2 and gtk+3, so I assume that will 
> pull in the required libraries if necessary, without all the other 
> non-necessary stuff.

Good then, thanks for checking!

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Brad Smith]

On 2023-03-22 9:40 a.m., Thomas Huth wrote:
> On 22/03/2023 14.38, Philippe Mathieu-Daudé wrote:
>> On 22/3/23 13:36, Daniel P. Berrangé wrote:
>>> As a VM used only for automated testing there is no need to
>>> install the X11 stack.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>   tests/vm/openbsd | 3 +--
>>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
>>> index eaeb201e91..6af7afac0d 100755
>>> --- a/tests/vm/openbsd
>>> +++ b/tests/vm/openbsd
>>> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>>>           self.console_wait("Password for root account")
>>>           self.console_send("%s\n" % self._config["root_pass"])
>>>           self.console_wait_send("Start sshd(8)", "yes\n")
>>> -        self.console_wait_send("X Window System", "\n")
>>> -        self.console_wait_send("xenodm", "\n")
>>> +        self.console_wait_send("X Window System", "no\n")
>>
>> Wasn't this useful to link a X11-ready binary, even if only testing
>> with -display=none?
>
> tests/vm/openbsd later installs sdl2 and gtk+3, so I assume that will 
> pull in the required libraries if necessary, without all the other 
> non-necessary stuff.


No, it will not. If you want sdl2 / gtk+3 you have to have X enabled.

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Thomas Huth]

On 22/03/2023 16.20, Brad Smith wrote:
> On 2023-03-22 9:40 a.m., Thomas Huth wrote:
>> On 22/03/2023 14.38, Philippe Mathieu-Daudé wrote:
>>> On 22/3/23 13:36, Daniel P. Berrangé wrote:
>>>> As a VM used only for automated testing there is no need to
>>>> install the X11 stack.
>>>>
>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>> ---
>>>>   tests/vm/openbsd | 3 +--
>>>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>>>
>>>> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
>>>> index eaeb201e91..6af7afac0d 100755
>>>> --- a/tests/vm/openbsd
>>>> +++ b/tests/vm/openbsd
>>>> @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
>>>>           self.console_wait("Password for root account")
>>>>           self.console_send("%s\n" % self._config["root_pass"])
>>>>           self.console_wait_send("Start sshd(8)", "yes\n")
>>>> -        self.console_wait_send("X Window System", "\n")
>>>> -        self.console_wait_send("xenodm", "\n")
>>>> +        self.console_wait_send("X Window System", "no\n")
>>>
>>> Wasn't this useful to link a X11-ready binary, even if only testing
>>> with -display=none?
>>
>> tests/vm/openbsd later installs sdl2 and gtk+3, so I assume that will pull 
>> in the required libraries if necessary, without all the other 
>> non-necessary stuff.
> 
> No, it will not. If you want sdl2 / gtk+3 you have to have X enabled.

I meant for compiling, not for running QEMU... and we're only running in 
non-graphical mode here later, so I hope that should be fine.

  Thomas

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Daniel P. Berrangé]

On Wed, Mar 22, 2023 at 11:20:08AM -0400, Brad Smith wrote:
> On 2023-03-22 9:40 a.m., Thomas Huth wrote:
> > On 22/03/2023 14.38, Philippe Mathieu-Daudé wrote:
> > > On 22/3/23 13:36, Daniel P. Berrangé wrote:
> > > > As a VM used only for automated testing there is no need to
> > > > install the X11 stack.
> > > > 
> > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > > ---
> > > >   tests/vm/openbsd | 3 +--
> > > >   1 file changed, 1 insertion(+), 2 deletions(-)
> > > > 
> > > > diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> > > > index eaeb201e91..6af7afac0d 100755
> > > > --- a/tests/vm/openbsd
> > > > +++ b/tests/vm/openbsd
> > > > @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
> > > >           self.console_wait("Password for root account")
> > > >           self.console_send("%s\n" % self._config["root_pass"])
> > > >           self.console_wait_send("Start sshd(8)", "yes\n")
> > > > -        self.console_wait_send("X Window System", "\n")
> > > > -        self.console_wait_send("xenodm", "\n")
> > > > +        self.console_wait_send("X Window System", "no\n")
> > > 
> > > Wasn't this useful to link a X11-ready binary, even if only testing
> > > with -display=none?
> > 
> > tests/vm/openbsd later installs sdl2 and gtk+3, so I assume that will
> > pull in the required libraries if necessary, without all the other
> > non-necessary stuff.
> 
> 
> No, it will not. If you want sdl2 / gtk+3 you have to have X enabled.

That does not appear to be required for QEMU build / test scenarios,
as 'make vm-build-openbsd' succeeded and included GTK/SDL2 support
AFAICT.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/2] tests/vm: skip X11 in openbsd installation

[Daniel P. Berrangé]

On Wed, Mar 22, 2023 at 02:38:13PM +0100, Philippe Mathieu-Daudé wrote:
> On 22/3/23 13:36, Daniel P. Berrangé wrote:
> > As a VM used only for automated testing there is no need to
> > install the X11 stack.
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >   tests/vm/openbsd | 3 +--
> >   1 file changed, 1 insertion(+), 2 deletions(-)
> > 
> > diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> > index eaeb201e91..6af7afac0d 100755
> > --- a/tests/vm/openbsd
> > +++ b/tests/vm/openbsd
> > @@ -106,8 +106,7 @@ class OpenBSDVM(basevm.BaseVM):
> >           self.console_wait("Password for root account")
> >           self.console_send("%s\n" % self._config["root_pass"])
> >           self.console_wait_send("Start sshd(8)",           "yes\n")
> > -        self.console_wait_send("X Window System",         "\n")
> > -        self.console_wait_send("xenodm",                  "\n")
> > +        self.console_wait_send("X Window System",         "no\n")
> 
> Wasn't this useful to link a X11-ready binary, even if only testing
> with -display=none?

I've just confirmed /usr/X11R6/include/X11/  still exists in the image
even after this change, as does /usr/X11R6/lib/

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH 2/2] tests/vm: custom openbsd partitioning to increase /home space

[Daniel P. Berrangé]

The openbsd image is 20GB in size, but the automatic partitioning
done by the installer leaves /home with a mere ~3.5 GB of space,
wasting free space across many other partitions that are not
used by our build process:

openbsd$ df
Filesystem  512-blocks      Used     Avail Capacity  Mounted on
/dev/sd0a      1229692    213592    954616    18%    /
/dev/sd0k      7672220        40   7288572     0%    /home
/dev/sd0d      1736604        24   1649752     0%    /tmp
/dev/sd0f      4847676   2505124   2100172    54%    /usr
/dev/sd0g      1326684    555656    704696    44%    /usr/X11R6
/dev/sd0h      4845436   1445932   3157236    31%    /usr/local
/dev/sd0j     10898972         4  10354020     0%    /usr/obj
/dev/sd0i      3343644         4   3176460     0%    /usr/src
/dev/sd0e      2601212     19840   2451312     1%    /var

This change tells the installer todo custom partitioning with
4 GB on /, 256 MB swap, and the remaining ~15GB for /home

openbsd$ df
Filesystem  512-blocks      Used     Avail Capacity  Mounted on
/dev/sd0a      7932412   4740204   2795588    63%    /
/dev/sd0d     32164636        40  30556368     0%    /home

This will avoid ENOSPC failures when tests that need to create
big files (disk images) run in parallel.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/vm/openbsd | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 6af7afac0d..6b4fc29793 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -123,7 +123,32 @@ class OpenBSDVM(basevm.BaseVM):
         self.console_wait_send("timezone",                "UTC\n")
         self.console_wait_send("root disk",               "\n")
         self.console_wait_send("(W)hole disk",            "\n")
-        self.console_wait_send("(A)uto layout",           "\n")
+        self.console_wait_send("(A)uto layout",           "c\n")
+
+        # 4000 MB / as /dev/sd0a, at start of disk
+        self.console_wait_send("sd0>", "a a\n")
+        self.console_wait_send("offset:", "\n")
+        self.console_wait_send("size:", "4000M\n")
+        self.console_wait_send("FS type", "4.2BSD\n")
+        self.console_wait_send("mount point:", "/\n")
+
+        # 256 MB swap as /dev/sd0b
+        self.console_wait_send("sd0*>", "a b\n")
+        self.console_wait_send("offset:", "\n")
+        self.console_wait_send("size:", "256M\n")
+        self.console_wait_send("FS type", "swap\n")
+
+        # All remaining space for /home as /dev/sd0d
+        # NB, 'c' isn't allowed to be used.
+        self.console_wait_send("sd0*>", "a d\n")
+        self.console_wait_send("offset:", "\n")
+        self.console_wait_send("size:", "\n")
+        self.console_wait_send("FS type", "4.2BSD\n")
+        self.console_wait_send("mount point:", "/home\n")
+
+        self.console_wait_send("sd0*>", "q\n")
+        self.console_wait_send("Write new label?:", "y\n")
+
         self.console_wait_send("Location of sets",        "cd0\n")
         self.console_wait_send("Pathname to the sets",    "\n")
         self.console_wait_send("Set name(s)",             "\n")
-- 
2.39.2

Re: [PATCH 2/2] tests/vm: custom openbsd partitioning to increase /home space

[Thomas Huth]

On 22/03/2023 13.36, Daniel P. Berrangé wrote:
> The openbsd image is 20GB in size, but the automatic partitioning
> done by the installer leaves /home with a mere ~3.5 GB of space,
> wasting free space across many other partitions that are not
> used by our build process:
> 
> openbsd$ df
> Filesystem  512-blocks      Used     Avail Capacity  Mounted on
> /dev/sd0a      1229692    213592    954616    18%    /
> /dev/sd0k      7672220        40   7288572     0%    /home
> /dev/sd0d      1736604        24   1649752     0%    /tmp
> /dev/sd0f      4847676   2505124   2100172    54%    /usr
> /dev/sd0g      1326684    555656    704696    44%    /usr/X11R6
> /dev/sd0h      4845436   1445932   3157236    31%    /usr/local
> /dev/sd0j     10898972         4  10354020     0%    /usr/obj
> /dev/sd0i      3343644         4   3176460     0%    /usr/src
> /dev/sd0e      2601212     19840   2451312     1%    /var
> 
> This change tells the installer todo custom partitioning with

s/todo/to do/

> 4 GB on /, 256 MB swap, and the remaining ~15GB for /home
> 
> openbsd$ df
> Filesystem  512-blocks      Used     Avail Capacity  Mounted on
> /dev/sd0a      7932412   4740204   2795588    63%    /
> /dev/sd0d     32164636        40  30556368     0%    /home
> 
> This will avoid ENOSPC failures when tests that need to create
> big files (disk images) run in parallel.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/vm/openbsd | 27 ++++++++++++++++++++++++++-
>   1 file changed, 26 insertions(+), 1 deletion(-)

Acked-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 2/2] tests/vm: custom openbsd partitioning to increase /home space

[Philippe Mathieu-Daudé]

On 22/3/23 13:36, Daniel P. Berrangé wrote:
> The openbsd image is 20GB in size, but the automatic partitioning
> done by the installer leaves /home with a mere ~3.5 GB of space,
> wasting free space across many other partitions that are not
> used by our build process:
> 
> openbsd$ df
> Filesystem  512-blocks      Used     Avail Capacity  Mounted on
> /dev/sd0a      1229692    213592    954616    18%    /
> /dev/sd0k      7672220        40   7288572     0%    /home
> /dev/sd0d      1736604        24   1649752     0%    /tmp
> /dev/sd0f      4847676   2505124   2100172    54%    /usr
> /dev/sd0g      1326684    555656    704696    44%    /usr/X11R6
> /dev/sd0h      4845436   1445932   3157236    31%    /usr/local
> /dev/sd0j     10898972         4  10354020     0%    /usr/obj
> /dev/sd0i      3343644         4   3176460     0%    /usr/src
> /dev/sd0e      2601212     19840   2451312     1%    /var
> 
> This change tells the installer todo custom partitioning with
> 4 GB on /, 256 MB swap, and the remaining ~15GB for /home
> 
> openbsd$ df
> Filesystem  512-blocks      Used     Avail Capacity  Mounted on
> /dev/sd0a      7932412   4740204   2795588    63%    /
> /dev/sd0d     32164636        40  30556368     0%    /home
> 
> This will avoid ENOSPC failures when tests that need to create
> big files (disk images) run in parallel.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/vm/openbsd | 27 ++++++++++++++++++++++++++-
>   1 file changed, 26 insertions(+), 1 deletion(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 0/2] tests/vm: fix ENOSPC in the openbsd VM image

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> The openbsd VM builds only have 3.5 GB of space available in /home, out
> of the 20 GB disk image. This is insufficient when running iotests in
> parallel with all other tests.
>
> Note, after applying this series:
>
>   rm ~/.cache/qemu-vm/images/openbsd.img
>
> is needed for it to take effect if you've previously used the openbsd
> image.
>
> Daniel P. Berrangé (2):
>   tests/vm: skip X11 in openbsd installation
>   tests/vm: custom openbsd partitioning to increase /home space
>
>  tests/vm/openbsd | 30 +++++++++++++++++++++++++++---
>  1 file changed, 27 insertions(+), 3 deletions(-)

Queued to pr/170323-for-8.0-1, thanks.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro