Re: qemu bsd-user plans

[Warner Losh <imp@bsdimp.com>]

[-- Attachment #1: Type: text/plain, Size: 4312 bytes --]

On Sat, Jan 9, 2021 at 10:03 AM Kyle Evans <kevans@freebsd.org> wrote:

> On Fri, 8 Jan 2021 at 19:43, Warner Losh <imp@bsdimp.com> wrote:
> >
> > The FreeBSD project has rewritten bsd-user. We've been working on this
> for quite some time (the earliest commits date from 2013). Maybe a dozen
> people have worked on this over time, and there's 3 or 4 active developers
> focused on FreeBSD changes at the moment.
> >
> > For a while, we'd merge in upstream changes from qemu. This worked great
> for us, but left us with a big backlog that was hard to upstream. Each of
> the updates took some time, so we got a little behind.
> >
> > So, a few years ago, I spent several weeks converting the tangled merge
> mess into a set of linear patches and started moving that forward. This was
> around the time 4.0 was released. I only managed to get the rebase forward
> to 3.1 release at the time before I hit problems related to poor testing
> environment making it hard to verify newer versions were still working.
> Plus, we found a few bugs that took a while to resolve for a number of
> reasons. Now that they are resolved, we're able to use qemu-bsd-user to
> build ~30k packages for arm, and ~20k for different types of mips in
> FreeBSD "ports" system. We now have great confidence that it's working well
> again.
> >
> > Now that those bugs are resolved, I started trying to forward-port the
> two-year-old base and immediately found myself hitting a number of
> problems. A big problem was that I was re-doing a lot of work that was due
> to innoculous changes upstream that I wouldn't have to do if the bsd-user
> changes were upstream. These changes get in the way of dealing with the
> more substantial structural changes in qemu that have happened.
> >
> > There had been talk of doing a remove and replace update of bsd-user.
> This talk was before I managed to rebase things as far forward as 3.1 even.
> This appealed to me because we've accumulated about 150 patches to date,
> many quite large, and curating them into a set of maybe 400 or 500 changes
> to match the size and scope of most patches I've seen posted to qemu-devel
> seemed overwhelming.
> >
> > However, it's been another year since that plan was hatched, and it's
> become clear to me that plan won't end in success. The closest I've been
> able to get is 3.1 when 4.1 was current (about 6 months behind). It's time
> for a new plan.
> >
>
> As one of the developers on the FreeBSD side, I agree with this. In
> addition to the reasons cited, reintroducing it would really take a
> lot more time and effort and I'm not convinced it would ever be
> completed because, IMO, going that route should really entail
> redesigning it from the ground-up based on an abstraction of
> linux-user. I find that right now I'm playing a lot of catch-up
> because we seem to be largely copied from linux-user without later
> improvement; there is a really healthy amount of platform-independent
> stuff that really should end up ultimately shared in a qemu-user or
> something to that effect to mitigate duplication of effort.
>

Yes. Once we have things upstream the normal 'refactoring' work can be done
there, rather than in a side repo where it will quickly go stale. Also,
once it's in upstream, then the NetBSD and OpenBSD folks can come back to
the table with a much smaller delta to get things working again than they
have today.


> > So, my new plan is to rebase what changes I can to the tip of master and
> submit those for review. I'll work with the developers on the FreeBSD side
> to ensure they are included in reviews in addition to the normal qemu-devel
> list. This will allow us to pare down the deltas between our code and
> upstream to allow us to make progress. The changes will be held to the
> standard 'makes things better'. Given how broken bsd-user is today in qemu
> upstream, at first that will a very easy standard to make.
> >
> > The first patch I'll submit will be changing MAINTAINERS to point to me,
> since I'm acting as the point person in this effort. I'll then re-submit
> some other changes that I've submitted in the past, but CC the FreeBSD
> folks that are currently active (they were only CC'd to former developers
> who lack the time to review).
> >
>
> Thanks for taking this on!
>

You bet.

Warner

[-- Attachment #2: Type: text/html, Size: 5078 bytes --]