QEMU's Haiku CI image

QEMU's Haiku CI image

[Thomas Huth]

  Hi,

while researching the different "sed" options on our supported build 
platform today, I started "make vm-build-haiku.x86_64" in my QEMU build 
directory for the first time since many months again. And I had to discover 
that this is completely out of date. The image does not contain any version 
of Python 3 yet which we require for compilation since more than a year now 
already, and the Haiku version in there seems to be too old to do a "pkgman 
install -y python3" ... so this has been completely been bitrotting since 
more than a year now. Is anybody still interested in keeping the Haiku 
support in QEMU? If so, please help to get the VM image updated. Thanks!

  Thomas

Re: QEMU's Haiku CI image

[Alexander von Gluck IV]

February 16, 2022 6:31 AM, "Thomas Huth" <thuth@redhat.com> wrote:
>
> while researching the different "sed" options on our supported build platform today, I started
> "make vm-build-haiku.x86_64" in my QEMU build directory for the first time since many months again.
> And I had to discover that this is completely out of date. The image does not contain any version
> of Python 3 yet which we require for compilation since more than a year now already, and the Haiku
> version in there seems to be too old to do a "pkgman install -y python3" ... so this has been
> completely been bitrotting since more than a year now. Is anybody still interested in keeping the
> Haiku support in QEMU? If so, please help to get the VM image updated. Thanks!

I submitted
https://patchwork.kernel.org/project/qemu-devel/patch/20220216154208.2985103-1-kallisti5@unixzen.com/
to fix this issue.  The build runs as expected after that patchset.

Likely cause is us no longer packing a "python" binary, deferring to "python2" vs "python3"

I'm still the most likely maintainer.  Are there still plans to automate the tests for Haiku to
prevent this from happening again in the future?

Manually running qemu vm tests is a drop in huge bucket of tasks, so it's pretty likely to get
forgotten until someone runs into an issue. :-)

 -- Alex

Re: QEMU's Haiku CI image

[Thomas Huth]

On 16/02/2022 16.52, Alexander von Gluck IV wrote:
> February 16, 2022 6:31 AM, "Thomas Huth" <thuth@redhat.com> wrote:
>>
>> while researching the different "sed" options on our supported build platform today, I started
>> "make vm-build-haiku.x86_64" in my QEMU build directory for the first time since many months again.
>> And I had to discover that this is completely out of date. The image does not contain any version
>> of Python 3 yet which we require for compilation since more than a year now already, and the Haiku
>> version in there seems to be too old to do a "pkgman install -y python3" ... so this has been
>> completely been bitrotting since more than a year now. Is anybody still interested in keeping the
>> Haiku support in QEMU? If so, please help to get the VM image updated. Thanks!
> 
> I submitted
> https://patchwork.kernel.org/project/qemu-devel/patch/20220216154208.2985103-1-kallisti5@unixzen.com/
> to fix this issue.  The build runs as expected after that patchset.
> 
> Likely cause is us no longer packing a "python" binary, deferring to "python2" vs "python3"
> 
> I'm still the most likely maintainer.  Are there still plans to automate the tests for Haiku to
> prevent this from happening again in the future?

AFAIK we still don't have a machine where we could properly run VM-based 
tests in the CI, do we? Peter? Cleber?

I recently added NetBSD and OpenBSD via KVM on Cirrus to the gitlab-CI :

  https://gitlab.com/qemu-project/qemu/-/commit/f11b0a4892ae22b872

But the jobs are marked as "manual" since the setup is rather fragile. We 
could still at least try to add a job like this for Haiku, too, so that it's 
at least semi-integrated into our CI setup?

  Thomas

Re: QEMU's Haiku CI image

[Philippe Mathieu-Daudé via]

On 16/2/22 17:32, Thomas Huth wrote:
> On 16/02/2022 16.52, Alexander von Gluck IV wrote:
>> February 16, 2022 6:31 AM, "Thomas Huth" <thuth@redhat.com> wrote:
>>>
>>> while researching the different "sed" options on our supported build 
>>> platform today, I started
>>> "make vm-build-haiku.x86_64" in my QEMU build directory for the first 
>>> time since many months again.
>>> And I had to discover that this is completely out of date. The image 
>>> does not contain any version
>>> of Python 3 yet which we require for compilation since more than a 
>>> year now already, and the Haiku
>>> version in there seems to be too old to do a "pkgman install -y 
>>> python3" ... so this has been
>>> completely been bitrotting since more than a year now. Is anybody 
>>> still interested in keeping the
>>> Haiku support in QEMU? If so, please help to get the VM image 
>>> updated. Thanks!
>>
>> I submitted
>> https://patchwork.kernel.org/project/qemu-devel/patch/20220216154208.2985103-1-kallisti5@unixzen.com/ 
>>
>> to fix this issue.  The build runs as expected after that patchset.
>>
>> Likely cause is us no longer packing a "python" binary, deferring to 
>> "python2" vs "python3"
>>
>> I'm still the most likely maintainer.  Are there still plans to 
>> automate the tests for Haiku to
>> prevent this from happening again in the future?
> 
> AFAIK we still don't have a machine where we could properly run VM-based 
> tests in the CI, do we? Peter? Cleber?

We still have unused fosshost.org resources. What we don't have is a
sysadmin willing to install the VM and maintain it over time.

Re: QEMU's Haiku CI image

[Thomas Huth]

On 16/02/2022 18.16, Philippe Mathieu-Daudé wrote:
> On 16/2/22 17:32, Thomas Huth wrote:
>> On 16/02/2022 16.52, Alexander von Gluck IV wrote:
>>> February 16, 2022 6:31 AM, "Thomas Huth" <thuth@redhat.com> wrote:
>>>>
>>>> while researching the different "sed" options on our supported build 
>>>> platform today, I started
>>>> "make vm-build-haiku.x86_64" in my QEMU build directory for the first 
>>>> time since many months again.
>>>> And I had to discover that this is completely out of date. The image 
>>>> does not contain any version
>>>> of Python 3 yet which we require for compilation since more than a year 
>>>> now already, and the Haiku
>>>> version in there seems to be too old to do a "pkgman install -y python3" 
>>>> ... so this has been
>>>> completely been bitrotting since more than a year now. Is anybody still 
>>>> interested in keeping the
>>>> Haiku support in QEMU? If so, please help to get the VM image updated. 
>>>> Thanks!
>>>
>>> I submitted
>>> https://patchwork.kernel.org/project/qemu-devel/patch/20220216154208.2985103-1-kallisti5@unixzen.com/ 
>>>
>>> to fix this issue.  The build runs as expected after that patchset.
>>>
>>> Likely cause is us no longer packing a "python" binary, deferring to 
>>> "python2" vs "python3"
>>>
>>> I'm still the most likely maintainer.  Are there still plans to automate 
>>> the tests for Haiku to
>>> prevent this from happening again in the future?
>>
>> AFAIK we still don't have a machine where we could properly run VM-based 
>> tests in the CI, do we? Peter? Cleber?
> 
> We still have unused fosshost.org resources. What we don't have is a
> sysadmin willing to install the VM and maintain it over time.

I thought that one does not have KVM enabled?

  Thomas

Re: QEMU's Haiku CI image

[Daniel P. Berrangé]

On Wed, Feb 16, 2022 at 06:16:10PM +0100, Philippe Mathieu-Daudé via wrote:
> On 16/2/22 17:32, Thomas Huth wrote:
> > On 16/02/2022 16.52, Alexander von Gluck IV wrote:
> > > February 16, 2022 6:31 AM, "Thomas Huth" <thuth@redhat.com> wrote:
> > > > 
> > > > while researching the different "sed" options on our supported
> > > > build platform today, I started
> > > > "make vm-build-haiku.x86_64" in my QEMU build directory for the
> > > > first time since many months again.
> > > > And I had to discover that this is completely out of date. The
> > > > image does not contain any version
> > > > of Python 3 yet which we require for compilation since more than
> > > > a year now already, and the Haiku
> > > > version in there seems to be too old to do a "pkgman install -y
> > > > python3" ... so this has been
> > > > completely been bitrotting since more than a year now. Is
> > > > anybody still interested in keeping the
> > > > Haiku support in QEMU? If so, please help to get the VM image
> > > > updated. Thanks!
> > > 
> > > I submitted
> > > https://patchwork.kernel.org/project/qemu-devel/patch/20220216154208.2985103-1-kallisti5@unixzen.com/
> > > 
> > > to fix this issue.  The build runs as expected after that patchset.
> > > 
> > > Likely cause is us no longer packing a "python" binary, deferring to
> > > "python2" vs "python3"
> > > 
> > > I'm still the most likely maintainer.  Are there still plans to
> > > automate the tests for Haiku to
> > > prevent this from happening again in the future?
> > 
> > AFAIK we still don't have a machine where we could properly run VM-based
> > tests in the CI, do we? Peter? Cleber?
> 
> We still have unused fosshost.org resources. What we don't have is a
> sysadmin willing to install the VM and maintain it over time.

I feel like there must be scope for sharing some of this burden with
libvirt since we've got essentially the same problem & requirements.

For libvirt we're using lcitool for building our VMs, and have worked
on a GitLab custom executor for launching throw-away VMs per CI job
from a read-only base template.

  https://gitlab.com/eskultety/libvirt-gitlab-executor

With this there's an admin burden setting up the gitlab executor on
the host and preparing the VM templates, but after that the admin
burden should be lightweight, as most of the risk of breakage is in
the VMs getting messed up, but they get thrown away after 1 job.

With QEMU's recent use of lcitool we're better ready  to share some
of this work than in the past. The main issue is that for non-Linux,
we don't have full automation for building the VM templates. We need
someone to prepare the image by getting it able to run and expose
SSH, whereupon we can provision the build-deps. We also only have
package mappings for FreeBSD, not Haiku, NetBSD or OpenBSD, but that's
fairly straightforward to address.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: QEMU's Haiku CI image

[Thomas Huth]

On 16/02/2022 20.21, Daniel P. Berrangé wrote:
> [...] The main issue is that for non-Linux,
> we don't have full automation for building the VM templates. We need
> someone to prepare the image by getting it able to run and expose
> SSH, whereupon we can provision the build-deps.

That's easy: In QEMU build folder, type:

  make vm-build-netbsd
  make vm-build-openbsd
  make vm-build-haiku.x86_64

... and then you can find the images in the ~/.cache/qemu-vm/images/ folder.

  Thomas