[Qemu-devel] RFC: This project needs a stable branch

[Qemu-devel] RFC: This project needs a stable branch

[Julian Seward]

I am a great fan of QEMU, and have used it more or less continuously
for the past 2+ years.  Over that time I've installed and operated
various Linux and Windows guests with varying degrees of success.

The recently released 0.9.0 seems a big step forward in the
stability/usability department, which is excellent.  But there are
still residual worries -- for example, qcow2 images corrupted for no
obvious reason -- which, whilst a boring problem, is important for
folks like me who want to run VMs 24x7 with the hope of complete
reliability.

Pretty much all mature projects which have achieved widespread usage
have one or more stable branches along with the main development
branch (trunk).  Think GCC, the kernel, KDE, ... the list is endless.

Maintaining a stable branch is extra hassle and overhead, but it is
the standard way to operate, for reasons which are obvious: the
majority of users care more about stability, reliability and usability
than they do about the latest new features, and delivering stability
from a branch used for bleeding-edge development work is pretty much
impossible.  That is not, of course, a criticism of the bleeding edge
developers, since it is they who ultimately drive the project along.

I am writing to propose that a stable branch be made from the 0.9.0
release point.  The aim would be to maximise stability for (IMO) the
subset of functionality that has the largest potential user base:
i386-softmmu + Accelerator and x86_64-softmmu + Accelerator, but
excluding -kernel-kqemu due to limitations described in
http://qemu.org/kqemu-doc.html#SEC7.

Subsequent releases of the branch would contain no functionality
enhancements, but just bug fixes, with the eventual aim of achieving
'it just works' status for any x86/x86_64 guest I try to install/run.
I know that's a tall order, and that 0.9.0 may not be able to supply
that for all guests.  But it is an important goal to strive for.

My impression is that (at least as I perceive it) the lack of emphasis
on maximising stability on a stable branch, and the lack of a bug
tracker, is artificially restricting QEMU's user base, and therefore
indirectly its long term prospects.  This is a shame, because QEMU is
a very remarkable and useful project, which should be used (and
usable) by everybody and anybody.

J

Re: [Qemu-devel] RFC: This project needs a stable branch

[Anthony Liguori]

I'm not necessarily sure I agree that a stable branch is the best thing 
to have (verses aiming for never introducing regressions).

I do agree that a bug tracker would be terribly useful for tracking 
regressions.  Bug trackers quickly get out of hand though unless someone 
spends a lot of time keeping them tidy.

Any thoughts on the subject?

Regards,

Anthony Liguori

Julian Seward wrote:
> I am a great fan of QEMU, and have used it more or less continuously
> for the past 2+ years.  Over that time I've installed and operated
> various Linux and Windows guests with varying degrees of success.
>
> The recently released 0.9.0 seems a big step forward in the
> stability/usability department, which is excellent.  But there are
> still residual worries -- for example, qcow2 images corrupted for no
> obvious reason -- which, whilst a boring problem, is important for
> folks like me who want to run VMs 24x7 with the hope of complete
> reliability.
>
> Pretty much all mature projects which have achieved widespread usage
> have one or more stable branches along with the main development
> branch (trunk).  Think GCC, the kernel, KDE, ... the list is endless.
>
> Maintaining a stable branch is extra hassle and overhead, but it is
> the standard way to operate, for reasons which are obvious: the
> majority of users care more about stability, reliability and usability
> than they do about the latest new features, and delivering stability
> from a branch used for bleeding-edge development work is pretty much
> impossible.  That is not, of course, a criticism of the bleeding edge
> developers, since it is they who ultimately drive the project along.
>
> I am writing to propose that a stable branch be made from the 0.9.0
> release point.  The aim would be to maximise stability for (IMO) the
> subset of functionality that has the largest potential user base:
> i386-softmmu + Accelerator and x86_64-softmmu + Accelerator, but
> excluding -kernel-kqemu due to limitations described in
> http://qemu.org/kqemu-doc.html#SEC7.
>
> Subsequent releases of the branch would contain no functionality
> enhancements, but just bug fixes, with the eventual aim of achieving
> 'it just works' status for any x86/x86_64 guest I try to install/run.
> I know that's a tall order, and that 0.9.0 may not be able to supply
> that for all guests.  But it is an important goal to strive for.
>
> My impression is that (at least as I perceive it) the lack of emphasis
> on maximising stability on a stable branch, and the lack of a bug
> tracker, is artificially restricting QEMU's user base, and therefore
> indirectly its long term prospects.  This is a shame, because QEMU is
> a very remarkable and useful project, which should be used (and
> usable) by everybody and anybody.
>
> J
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
>
>   

Re: [Qemu-devel] RFC: This project needs a stable branch

[Julian Seward]

On Thursday 15 March 2007 13:48, Anthony Liguori wrote:
> I'm not necessarily sure I agree that a stable branch is the best thing
> to have (verses aiming for never introducing regressions).

Aiming for no regressions is a worthy aim, but I believe unachieveable
in a project of any size.  For sure it's impossible if there is ever a
need to make large-scale infrastructural changes, which inevitably is
occasionally the case if the project is to live a long time.

For example, if the dyngen/gcc-based backend is replaced by a
self-contained handwritten one, I would be amazed if there were not
a few obscure regressions whilst the new backend is brought up to the 
same level of stability as the current one.  At least, that is what
I know from my own code generator hacking.

J

Re: [Qemu-devel] RFC: This project needs a stable branch

[Paul Brook]

> Subsequent releases of the branch would contain no functionality
> enhancements, but just bug fixes, with the eventual aim of achieving
> 'it just works' status for any x86/x86_64 guest I try to install/run.
> I know that's a tall order, and that 0.9.0 may not be able to supply
> that for all guests.  But it is an important goal to strive for.

While I agree stability is a desirable goal, and there is obviously users want 
a stable product, I'm not sure a qemu is mature enough to make a stable 
branch worthwhile.  Especially considering the very limited technical 
resources we have available.

> I am writing to propose that a stable branch be made from the 0.9.0
> release point.  The aim would be to maximise stability for (IMO) the
> subset of functionality that has the largest potential user base:
> i386-softmmu + Accelerator and x86_64-softmmu + Accelerator, but
> excluding -kernel-kqemu due to limitations described in
> http://qemu.org/kqemu-doc.html#SEC7.

Whereas I think the single easiest way to increase the user base would be to 
merge the kvm patches. Definitely not something suitable for a stable branch.

> My impression is that (at least as I perceive it) the lack of emphasis
> on maximising stability on a stable branch, and the lack of a bug
> tracker, is artificially restricting QEMU's user base, and therefore
> indirectly its long term prospects.  This is a shame, because QEMU is
> a very remarkable and useful project, which should be used (and
> usable) by everybody and anybody.

A bug tracker doesn't help unless you've got someone who triages, monitors and 
eventually fixes those bugs. There is a bug tracker on savannah, but noone 
has the time or motivation to use it.

Are you volunteering to maintain this stable branch, and look after the bug 
tracker?

Paul

Re: [Qemu-devel] RFC: This project needs a stable branch

[Avi Kivity]

Paul Brook wrote:
> Whereas I think the single easiest way to increase the user base would be to 
> merge the kvm patches.
>
>   

This is good to hear.  I should really have submitted patches to 
qemu-devel long ago, but have run out of cycles.  In addition, I am a 
little concerned about the kvm userspace interface being in flux and 
causing breakage to users.  I guess it's not too bad on the development 
branch.

Another concern that's occurring to me is that we are now adding 
additional functionality such as ballooning and paravirtualized 
networking, which are not traditional emulator functionality.  What are 
your views regarding that?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] RFC: This project needs a stable branch

[Julian Seward]

On Thursday 15 March 2007 14:53, Paul Brook wrote:
> > Subsequent releases of the branch would contain no functionality
> > enhancements, but just bug fixes, with the eventual aim of achieving
> > 'it just works' status for any x86/x86_64 guest I try to install/run.
> > I know that's a tall order, and that 0.9.0 may not be able to supply
> > that for all guests.  But it is an important goal to strive for.
>
> While I agree stability is a desirable goal, and there is obviously users
> want a stable product, I'm not sure a qemu is mature enough to make a
> stable branch worthwhile.  Especially considering the very limited
> technical resources we have available.

Limited effort is always a problem, granted.

So here's a broader question, which I'm surprised nobody has asked
before (afaik).  Think forward to a hypothetical QEMU 1.0 release.
What criteria are required for such a release?

J

Re: [Qemu-devel] RFC: This project needs a stable branch

[Rob Landley]

On Tuesday 20 March 2007 6:19 pm, Julian Seward wrote:
> Limited effort is always a problem, granted.
> 
> So here's a broader question, which I'm surprised nobody has asked
> before (afaik).  Think forward to a hypothetical QEMU 1.0 release.
> What criteria are required for such a release?

*cough* *cough* QOPS *cough* *cough*

Rob
-- 
Vista: Windows Millenium Second Edition

Re: [Qemu-devel] RFC: This project needs a stable branch

[Johannes Schindelin]

Hi,

On Thu, 22 Mar 2007, Rob Landley wrote:

> On Tuesday 20 March 2007 6:19 pm, Julian Seward wrote:
> > Limited effort is always a problem, granted.
> > 
> > So here's a broader question, which I'm surprised nobody has asked
> > before (afaik).  Think forward to a hypothetical QEMU 1.0 release.
> > What criteria are required for such a release?
> 
> *cough* *cough* QOPS *cough* *cough*

*cough* *cough* patches? *cough* *cough*

Ciao,
Dscho

Re: [Qemu-devel] RFC: This project needs a stable branch

[Rob Landley]

On Thursday 22 March 2007 7:00 pm, Johannes Schindelin wrote:
> Hi,
> 
> On Thu, 22 Mar 2007, Rob Landley wrote:
> 
> > On Tuesday 20 March 2007 6:19 pm, Julian Seward wrote:
> > > Limited effort is always a problem, granted.
> > > 
> > > So here's a broader question, which I'm surprised nobody has asked
> > > before (afaik).  Think forward to a hypothetical QEMU 1.0 release.
> > > What criteria are required for such a release?
> > 
> > *cough* *cough* QOPS *cough* *cough*
> 
> *cough* *cough* patches? *cough* *cough*

Do you mean you're asking me to break up Paul Brook's QOPS tree at 
https://nowt.dyndns.org and submit it to mainline?  I can do this thing, if 
you really think it would help.  Seems a bit roundabout to submit Paul 
Brook's work to Paul Brook, though, when he has CVS commit access and I 
don't.

Rob
-- 
Vista: Windows Millenium Second Edition

Re: [Qemu-devel] RFC: This project needs a stable branch

[Paul Brook]

> Do you mean you're asking me to break up Paul Brook's QOPS tree at
> https://nowt.dyndns.org and submit it to mainline?  I can do this thing, if
> you really think it would help.

If you implement all the missing bits in the process it'll help ;-)

Paul

Re: [Qemu-devel] RFC: This project needs a stable branch

[Thiemo Seufer]

Rob Landley wrote:
> On Thursday 22 March 2007 7:00 pm, Johannes Schindelin wrote:
> > Hi,
> > 
> > On Thu, 22 Mar 2007, Rob Landley wrote:
> > 
> > > On Tuesday 20 March 2007 6:19 pm, Julian Seward wrote:
> > > > Limited effort is always a problem, granted.
> > > > 
> > > > So here's a broader question, which I'm surprised nobody has asked
> > > > before (afaik).  Think forward to a hypothetical QEMU 1.0 release.
> > > > What criteria are required for such a release?
> > > 
> > > *cough* *cough* QOPS *cough* *cough*
> > 
> > *cough* *cough* patches? *cough* *cough*
> 
> Do you mean you're asking me to break up Paul Brook's QOPS tree at 
> https://nowt.dyndns.org and submit it to mainline?  I can do this thing, if 
> you really think it would help.  Seems a bit roundabout to submit Paul 
> Brook's work to Paul Brook, though, when he has CVS commit access and I 
> don't.

The patches need some updates AFAIK, and you might have more time than
Paul has.


Thiemo

Re: [Qemu-devel] RFC: This project needs a stable branch

[Rob Landley]

On Thursday 22 March 2007 7:29 pm, Thiemo Seufer wrote:
> > Do you mean you're asking me to break up Paul Brook's QOPS tree at 
> > https://nowt.dyndns.org and submit it to mainline?  I can do this thing, 
if 
> > you really think it would help.  Seems a bit roundabout to submit Paul 
> > Brook's work to Paul Brook, though, when he has CVS commit access and I 
> > don't.
> 
> The patches need some updates AFAIK, and you might have more time than
> Paul has.

Time yes, expertise no.  But I'm downloading the most recent version of the 
patch as we speak.  Brace yourself for some really for stupid questions...

(But not before this weekend at the earliest.)

> Thiemo

Rob
-- 
Vista: Windows Millenium Second Edition

Re: [Qemu-devel] RFC: This project needs a stable branch

[Julian Seward]

On Thursday 22 March 2007 23:27, Paul Brook wrote:
> > Do you mean you're asking me to break up Paul Brook's QOPS tree at
> > https://nowt.dyndns.org and submit it to mainline?  I can do this thing,
> > if you really think it would help.
>
> If you implement all the missing bits in the process it'll help ;-)

What bits would they be then?

FWIW, I snarfed the patch last Sunday and tested it on amd64 host / 
x86 guest, and successfully booted a couple of linux distros.  So it's
not obviously broken, at least for my mundane host/guest choice.
It also seemed marginally slower on a big compile in the guest - 
395.4 host cpu seconds for mainline vs 422.9 with qops.  Is this 
expected?

J

Re: [Qemu-devel] RFC: This project needs a stable branch

[Paul Brook]

On Thursday 22 March 2007 23:57, Julian Seward wrote:
> On Thursday 22 March 2007 23:27, Paul Brook wrote:
> > > Do you mean you're asking me to break up Paul Brook's QOPS tree at
> > > https://nowt.dyndns.org and submit it to mainline?  I can do this
> > > thing, if you really think it would help.
> >
> > If you implement all the missing bits in the process it'll help ;-)
>
> What bits would they be then?

The m68k target is the only one that uses qops for all its code generation. 
Arm is about half-and-half, x86 has the easy bits converted, and the other 
targets still use dyngen pretty much exclusively.

amd64 host support is fairly good, x86 hosts mostly work, and ppc has 
bitrotted a bit.

> FWIW, I snarfed the patch last Sunday and tested it on amd64 host /
> x86 guest, and successfully booted a couple of linux distros.  So it's
> not obviously broken, at least for my mundane host/guest choice.
> It also seemed marginally slower on a big compile in the guest -
> 395.4 host cpu seconds for mainline vs 422.9 with qops.  Is this
> expected?

Yes. The generated code should run at similar speed, but doing the translation 
takes a bit longer. I've been seeing 10-20% slowdown for booting linux.

Paul

Re: [Qemu-devel] RFC: This project needs a stable branch

[Johannes Schindelin]

Hi,

On Thu, 22 Mar 2007, Julian Seward wrote:

> On Thursday 22 March 2007 23:27, Paul Brook wrote:
> > > Do you mean you're asking me to break up Paul Brook's QOPS tree at
> > > https://nowt.dyndns.org and submit it to mainline?  I can do this thing,
> > > if you really think it would help.
> >
> > If you implement all the missing bits in the process it'll help ;-)
> 
> What bits would they be then?

AFAICT almost _all_ ops for the i86 target are missing.

> FWIW, I snarfed the patch last Sunday and tested it on amd64 host / x86 
> guest, and successfully booted a couple of linux distros.  So it's not 
> obviously broken, at least for my mundane host/guest choice. It also 
> seemed marginally slower on a big compile in the guest - 395.4 host cpu 
> seconds for mainline vs 422.9 with qops.

I saw something similar (qops performing worse than non-qops), which I 
could only explain by the non-optimizing nature of not relying on GCC's 
optimizer.

Ciao,
Dscho