[Qemu-devel] New kvm-related qemu patch queue

[Qemu-devel] New kvm-related qemu patch queue

[Avi Kivity]

In order to improve qemu.git kvm integration quality wrt performance, 
features, and reliability Marcelo and I will begin to maintain a patch 
queue based on qemu.git containing kvm-related patches.  We will review 
and apply patches to this queue, test them using the same test suite 
that is used for qemu-kvm.git, and regularly submit them for inclusion 
in qemu.git, mimicking the relationship between kvm.git and Linus' 
linux-2.6.git.

One of the problems of qemu.git kvm support is that it is a clean 
reimplementation, and thus some of the nuances that were carefully 
ironed out in qemu-kvm.git are lost.  To that end, we would like to 
change the process of adding features as follows:

  - first, the feature in qemu-kvm.git master is morphed to a form 
suitable for merging into qemu.git
  - when that has been accomplished, the feature is broken into patches 
and merged into the patch queue

This process ensures that important details are not lost: the morphing 
step attempts to preserve all the nuances, and it is much easier to spot 
a behaviour change in a review than to spot a missing behaviour in a new 
feature.  This was successfully used to merge i386 and x86_64 arch 
support in Linux.

Some qemu-kvm.git features (primarily support for very old kernels) can 
be dropped, simplifying the convergence process.

Over time, qemu-kvm.git master will converge with qemu.git master and 
all that will remain is the patch queue.

Features in qemu-kvm.git not present in qemu.git include:
- smp support
- in-kernel irqchip
- in-kernel pit
- tpr patching
- device assignment
- kvm paravirtualization
- boot=on (extboot, or a real option rom)
- test suite
- ia64
- signalfd support

Many others are implemented differently in the two trees.

The patch queue will appear as a branch 'uq/master' (for 'upstream 
queue); if necessary branches for the stable series will also be created.

In order to get a kvm feature into qemu.git, please observe the 
following process:

- post a patch series against qemu-kvm.git/master that implements the 
feature, or changes an existing feature to use qemu.git infrastructure
- once that's merged, post a patch series against qemu-kvm.git/uq/master 
that brings the same code into qemu.git.

The new branch will be subject to the same automatic testing that 
qemu-kvm.git is.

Please post patches to both qemu-devel and kvm mailing lists.

This won't work for all features, so we'll have to feel our way and make 
decisions on a case by case basis.


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

[Qemu-devel] Re: New kvm-related qemu patch queue

[Gleb Natapov]

On Sun, Jan 10, 2010 at 02:02:27PM +0200, Avi Kivity wrote:
> In order to improve qemu.git kvm integration quality wrt
> performance, features, and reliability Marcelo and I will begin to
> maintain a patch queue based on qemu.git containing kvm-related
> patches.  We will review and apply patches to this queue, test them
> using the same test suite that is used for qemu-kvm.git, and
> regularly submit them for inclusion in qemu.git, mimicking the
> relationship between kvm.git and Linus' linux-2.6.git.
> 
> One of the problems of qemu.git kvm support is that it is a clean
> reimplementation, and thus some of the nuances that were carefully
> ironed out in qemu-kvm.git are lost.  To that end, we would like to
> change the process of adding features as follows:
> 
>  - first, the feature in qemu-kvm.git master is morphed to a form
> suitable for merging into qemu.git
>  - when that has been accomplished, the feature is broken into
> patches and merged into the patch queue
> 
If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to
whom and who is merged where? I am confused. We have much of duplicated
code between qemu-kvm.git and qemu.git it is often almost, but not exactly the
same. Is it really productive to set rules who should be morphed and how
at this point?

> This process ensures that important details are not lost: the
> morphing step attempts to preserve all the nuances, and it is much
> easier to spot a behaviour change in a review than to spot a missing
> behaviour in a new feature.  This was successfully used to merge
> i386 and x86_64 arch support in Linux.
> 
> Some qemu-kvm.git features (primarily support for very old kernels)
> can be dropped, simplifying the convergence process.
> 
> Over time, qemu-kvm.git master will converge with qemu.git master
> and all that will remain is the patch queue.
> 
> Features in qemu-kvm.git not present in qemu.git include:
> - smp support
> - in-kernel irqchip
> - in-kernel pit
> - tpr patching
> - device assignment
> - kvm paravirtualization
> - boot=on (extboot, or a real option rom)
> - test suite
> - ia64
> - signalfd support
> 
> Many others are implemented differently in the two trees.
> 
> The patch queue will appear as a branch 'uq/master' (for 'upstream
> queue); if necessary branches for the stable series will also be
> created.
> 
> In order to get a kvm feature into qemu.git, please observe the
> following process:
> 
> - post a patch series against qemu-kvm.git/master that implements
> the feature, or changes an existing feature to use qemu.git
> infrastructure
It was other way around till last week. Why change?

> - once that's merged, post a patch series against
> qemu-kvm.git/uq/master that brings the same code into qemu.git.
> 
> The new branch will be subject to the same automatic testing that
> qemu-kvm.git is.
> 
> Please post patches to both qemu-devel and kvm mailing lists.
> 
> This won't work for all features, so we'll have to feel our way and
> make decisions on a case by case basis.
> 
> 

--
			Gleb.

[Qemu-devel] Re: New kvm-related qemu patch queue

[Avi Kivity]

On 01/10/2010 04:28 PM, Gleb Natapov wrote:
> On Sun, Jan 10, 2010 at 02:02:27PM +0200, Avi Kivity wrote:
>    
>> In order to improve qemu.git kvm integration quality wrt
>> performance, features, and reliability Marcelo and I will begin to
>> maintain a patch queue based on qemu.git containing kvm-related
>> patches.  We will review and apply patches to this queue, test them
>> using the same test suite that is used for qemu-kvm.git, and
>> regularly submit them for inclusion in qemu.git, mimicking the
>> relationship between kvm.git and Linus' linux-2.6.git.
>>
>> One of the problems of qemu.git kvm support is that it is a clean
>> reimplementation, and thus some of the nuances that were carefully
>> ironed out in qemu-kvm.git are lost.  To that end, we would like to
>> change the process of adding features as follows:
>>
>>   - first, the feature in qemu-kvm.git master is morphed to a form
>> suitable for merging into qemu.git
>>   - when that has been accomplished, the feature is broken into
>> patches and merged into the patch queue
>>
>>      
> If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to
> whom and who is merged where? I am confused. We have much of duplicated
> code between qemu-kvm.git and qemu.git it is often almost, but not exactly the
> same. Is it really productive to set rules who should be morphed and how
> at this point?
>    

If the feature is already in both, then morph qemu-kvm.git into what is 
already in qemu.git.  Hopefully anything missing in qemu.git will be 
discovered while making the changes.

>> In order to get a kvm feature into qemu.git, please observe the
>> following process:
>>
>> - post a patch series against qemu-kvm.git/master that implements
>> the feature, or changes an existing feature to use qemu.git
>> infrastructure
>>      
> It was other way around till last week. Why change?
>    

There were a lot of regressions.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: New kvm-related qemu patch queue

[Gleb Natapov]

On Sun, Jan 10, 2010 at 04:30:44PM +0200, Avi Kivity wrote:
> On 01/10/2010 04:28 PM, Gleb Natapov wrote:
> >On Sun, Jan 10, 2010 at 02:02:27PM +0200, Avi Kivity wrote:
> >>In order to improve qemu.git kvm integration quality wrt
> >>performance, features, and reliability Marcelo and I will begin to
> >>maintain a patch queue based on qemu.git containing kvm-related
> >>patches.  We will review and apply patches to this queue, test them
> >>using the same test suite that is used for qemu-kvm.git, and
> >>regularly submit them for inclusion in qemu.git, mimicking the
> >>relationship between kvm.git and Linus' linux-2.6.git.
> >>
> >>One of the problems of qemu.git kvm support is that it is a clean
> >>reimplementation, and thus some of the nuances that were carefully
> >>ironed out in qemu-kvm.git are lost.  To that end, we would like to
> >>change the process of adding features as follows:
> >>
> >>  - first, the feature in qemu-kvm.git master is morphed to a form
> >>suitable for merging into qemu.git
> >>  - when that has been accomplished, the feature is broken into
> >>patches and merged into the patch queue
> >>
> >If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to
> >whom and who is merged where? I am confused. We have much of duplicated
> >code between qemu-kvm.git and qemu.git it is often almost, but not exactly the
> >same. Is it really productive to set rules who should be morphed and how
> >at this point?
> 
> If the feature is already in both, then morph qemu-kvm.git into what
> is already in qemu.git.  Hopefully anything missing in qemu.git will
> be discovered while making the changes.
> 
What about bugs that are present only in qemu.git? Like this:
http://patchwork.ozlabs.org/patch/42298/. Should it go through
qemu-kvm.git/uq/master?

What about this one: http://patchwork.ozlabs.org/patch/42447/ should it
be postponed untill qemu.git and qemu-kvm.git converge on using the same
cpuid infrastructure, or add similar functionality to qemu-kvm to,
or add new cpu flags to qemu-kvm only and when cpuid code converge
qemu.git will have it too?

> >>In order to get a kvm feature into qemu.git, please observe the
> >>following process:
> >>
> >>- post a patch series against qemu-kvm.git/master that implements
> >>the feature, or changes an existing feature to use qemu.git
> >>infrastructure
> >It was other way around till last week. Why change?
> 
> There were a lot of regressions.
> 
Yeah :(

--
			Gleb.

[Qemu-devel] Re: New kvm-related qemu patch queue

[Avi Kivity]

On 01/10/2010 04:49 PM, Gleb Natapov wrote:
>
>> If the feature is already in both, then morph qemu-kvm.git into what
>> is already in qemu.git.  Hopefully anything missing in qemu.git will
>> be discovered while making the changes.
>>
>>      
> What about bugs that are present only in qemu.git? Like this:
> http://patchwork.ozlabs.org/patch/42298/. Should it go through
> qemu-kvm.git/uq/master?
>    

Yes.  So there is a central place for kvm patches, and so they see autotest.


> What about this one: http://patchwork.ozlabs.org/patch/42447/ should it
> be postponed untill qemu.git and qemu-kvm.git converge on using the same
> cpuid infrastructure, or add similar functionality to qemu-kvm to,
> or add new cpu flags to qemu-kvm only and when cpuid code converge
> qemu.git will have it too?
>    

Best to make qemu-kvm.git and qemu.git converge first.  Duplicating the 
patch extends the problem.  Of course if something is urgent we can 
bypass the process.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: New kvm-related qemu patch queue

[Gleb Natapov]

On Sun, Jan 10, 2010 at 04:52:48PM +0200, Avi Kivity wrote:
> On 01/10/2010 04:49 PM, Gleb Natapov wrote:
> >
> >>If the feature is already in both, then morph qemu-kvm.git into what
> >>is already in qemu.git.  Hopefully anything missing in qemu.git will
> >>be discovered while making the changes.
> >>
> >What about bugs that are present only in qemu.git? Like this:
> >http://patchwork.ozlabs.org/patch/42298/. Should it go through
> >qemu-kvm.git/uq/master?
> 
> Yes.  So there is a central place for kvm patches, and so they see autotest.
> 
So I assume you'll take it then.

> 
> >What about this one: http://patchwork.ozlabs.org/patch/42447/ should it
> >be postponed untill qemu.git and qemu-kvm.git converge on using the same
> >cpuid infrastructure, or add similar functionality to qemu-kvm to,
> >or add new cpu flags to qemu-kvm only and when cpuid code converge
> >qemu.git will have it too?
> 
> Best to make qemu-kvm.git and qemu.git converge first.  Duplicating
> the patch extends the problem.  Of course if something is urgent we
> can bypass the process.
> 
Actually, looking into it, this patch is needed for qemu-kvm.git and qemu.git
to converge. For qemu-kvm to use kvm -cpu flags the patch below should
be applied on top of the previous patch.

Signed-off-by: Gleb Natapov <gleb@redhat.com>
diff --git a/qemu-kvm-x86.c b/qemu-kvm-x86.c
index 82e362c..b3c371c 100644
--- a/qemu-kvm-x86.c
+++ b/qemu-kvm-x86.c
@@ -1308,7 +1308,7 @@ int kvm_arch_init_vcpu(CPUState *cenv)
     pv_ent = &cpuid_ent[cpuid_nent++];
     memset(pv_ent, 0, sizeof(*pv_ent));
     pv_ent->function = KVM_CPUID_FEATURES;
-    pv_ent->eax = get_para_features(kvm_context);
+    pv_ent->eax = cenv->cpuid_kvm_features & get_para_features(kvm_context);
 #endif
 
     kvm_trim_features(&cenv->cpuid_features,
--
			Gleb.

[Qemu-devel] Re: New kvm-related qemu patch queue

[Anthony Liguori]

On 01/10/2010 06:02 AM, Avi Kivity wrote:
> In order to improve qemu.git kvm integration quality wrt performance, 
> features, and reliability Marcelo and I will begin to maintain a patch 
> queue based on qemu.git containing kvm-related patches.  We will 
> review and apply patches to this queue, test them using the same test 
> suite that is used for qemu-kvm.git, and regularly submit them for 
> inclusion in qemu.git, mimicking the relationship between kvm.git and 
> Linus' linux-2.6.git.

Thanks for setting this up Avi!

I just want to stress that everyone continue CC'ing qemu-devel on all 
KVM patches.  Even if the patch is qemu-kvm specific for the moment, I 
think it's important for qemu-devel to be exposed to the refactoring work.

It might be good to prefix qemu-kvm.git patches in some manner to make 
it clear which repository they belong to.

Regards,

Anthony Liguori

[Qemu-devel] Re: New kvm-related qemu patch queue

[Avi Kivity]

On 01/11/2010 05:30 PM, Anthony Liguori wrote:
> On 01/10/2010 06:02 AM, Avi Kivity wrote:
>> In order to improve qemu.git kvm integration quality wrt performance, 
>> features, and reliability Marcelo and I will begin to maintain a 
>> patch queue based on qemu.git containing kvm-related patches.  We 
>> will review and apply patches to this queue, test them using the same 
>> test suite that is used for qemu-kvm.git, and regularly submit them 
>> for inclusion in qemu.git, mimicking the relationship between kvm.git 
>> and Linus' linux-2.6.git.
>
> Thanks for setting this up Avi!
>
> I just want to stress that everyone continue CC'ing qemu-devel on all 
> KVM patches.  Even if the patch is qemu-kvm specific for the moment, I 
> think it's important for qemu-devel to be exposed to the refactoring 
> work.
>
> It might be good to prefix qemu-kvm.git patches in some manner to make 
> it clear which repository they belong to.

[patch kvm] - qemu-kvm.git uq/master
[patch qemu-kvm] - qemu-kvm.git master
[patch kvm stable-0.12] - qemu-kvm uq/stable-0.12

etc.

-- 
error compiling committee.c: too many arguments to function