Re: [Hackathon Minutes] Xen 4.4 Planning

[Alex Bligh <alex@alex.org.uk>]

--On 13 June 2013 15:00:46 +0100 Lars Kurth <lars.kurth@xen.org> wrote:

> We should aim to reduce the release cycle to 4 months (or maybe 6 months
> as an intermediate step) from the current 9 months. A 4 months relase
> cycle should help accelerate development and lead to fewer patches being
> queued up. The implications are that we would have to operate a 2-3 weeks
> merge window.

Disclaimer: I don't claim to be a xen developer, despite writing a few
patches. But we are a heavy libxl API user. This perspective may or may
not be useful from a 'consumer of your development' perspective. Delete
or ignore if not.

The thing we like the least about Xen is (was?), stuff breaking between
major releases. If you have to drop 90% of your new features in order
not to break stuff, please do just that. Xen 3.x -> Xen 4.1 (we mostly
skipped 4.0) was a huge change, requiring a lot of rewriting. Xen 4.1
to Xen 4.2 was far, far worse; had we not needed it to support
qemu-upstream, we'd have probably stuck with 4.1. We talk to 4 hypervisors,
and have never had this difficulty with any other hypervisor. You can
imagine our joy when we found we could compile against 4.3 (we haven't
tried running yet) without a single line code changed. Please, please,
keep it like this. If we can also run unchanged against 4.3, this will
be even better.

The thing we like the second least about Xen is how long it seems to take
to get what we count as serious bugs fixed, even in stable releases. We
like it even less if we have to find them and fix them. By 'serious'
I mean basic functionality not working, crashes dom0, etc. I don't know
if this says something bad about us, but we've not found any such bug
ever in kvm (in well over 5 years). On the positive side, we've found
xen-devel pretty friendly and receptive. The perception is, however, that
new development takes priority over stable releases. I recognise that I
have a bias here, so this may not be fair.

The result of this is two-fold. Firstly, we've never (yet) been able to
run a production version of xen which is a standard xen release. We've
always had to maintain our own patches even on 'stable' releases.
Frankly, this is a pain. Secondly, there is a perception that moving
versions of Xen is going to be a huge PITA; that perception does not
exist for other hypervisors. I'm really really hoping Xen 4.3 dispels
this, and signs are good so far (we've done a lot of testing at xl
level, not so much at libxl level).

What does this mean for the development process?

1. I think more testing would be useful, particularly against API driven
   stuff. I find the current test stuff a bit confusing. What I wan't
   to know is whether commit X works or not - if things fail randomly,
   they aren't useful tests, particularly if it's difficult to
   distinguish them from other failures. Spoken as someone who's broken
   things :-(

2. My concern about early branching of -next (at -rc1 for instance) is
   that developing new stuff is far more interesting than fixing bugs.
   We liked the way stuff we raised with 4.3 (e.g. tsc issues) got looked
   at, and would hope that continues.

3. I would quite like a slightly shorter development cycle IFF it
   doesn't impact on stability. EG I thought we were probably bending the
   rules to get live migrate on qemu-upstream backported into 4.2,
   and had 4.3 been available sooner, we wouldn't have pushed. At
   a guess, 6 months would be about right, 4 months would be too short.

4. Following xen has been 10 times easier since you've moved to git.
   I agree with George's statement that you aren't using it enough -
   particularly branches.

5. At risk of repetition, we don't really care, so long as you don't break
   stuff.

-- 
Alex Bligh