Split kvm source tarballs

Split kvm source tarballs

[Avi Kivity]

As part of the preparations for a formal maintenance branch, we're now 
providing userspace-only source tarballs and kernel module-only source 
tarballs.  The combined kernel/userspace package is still provided.

Userspace package names will take the form 'qemu-kvm-devel-xx'.
Kernel module package names will take the form 'kvm-kmod-devel-xx'.
The combined package is named, as usual, kvm-xx'.

When maintenance branch packages become available, -devel will be 
dropped, of course.  Only the split packages will be available for the 
maintenance branch.

The userspace package can run against modules from the matching kernel 
module package, or against modules from a released Linux kernel (version 
2.6.25 or higher).  In almost all cases, you can mix and match userspace 
and kernel -devel packages.

Please test the release candidates in 
http://userweb.kernel.org/~avi/kvm-85rc1/ for any packaging errors.  
Suggestions of any kind are also welcome.

-- 
error compiling committee.c: too many arguments to function

Re: Split kvm source tarballs

[Anthony Liguori]

Avi Kivity wrote:
> As part of the preparations for a formal maintenance branch, we're now 
> providing userspace-only source tarballs and kernel module-only source 
> tarballs.  The combined kernel/userspace package is still provided.
>
> Userspace package names will take the form 'qemu-kvm-devel-xx'.
> Kernel module package names will take the form 'kvm-kmod-devel-xx'.
> The combined package is named, as usual, kvm-xx'.
>
> When maintenance branch packages become available, -devel will be 
> dropped, of course.  Only the split packages will be available for the 
> maintenance branch.
>
> The userspace package can run against modules from the matching kernel 
> module package, or against modules from a released Linux kernel 
> (version 2.6.25 or higher).  In almost all cases, you can mix and 
> match userspace and kernel -devel packages.
>
> Please test the release candidates in 
> http://userweb.kernel.org/~avi/kvm-85rc1/ for any packaging errors.  
> Suggestions of any kind are also welcome.

Hrm, I notice that for qemu-kvm, you basically just pulled out the 
module source.

What do you think about going a step further and building a proper qemu 
repository?

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Avi Kivity]

Anthony Liguori wrote:
>
> Hrm, I notice that for qemu-kvm, you basically just pulled out the 
> module source.
>

That's step 1.

> What do you think about going a step further and building a proper 
> qemu repository?
>

Step 2.  I have scripts that do that and will merge with Eduardo's 
script that also rewrites the qemu commit hashes to correspond to 
qemu.git on git.kernel.org.

-- 
error compiling committee.c: too many arguments to function

Re: Split kvm source tarballs

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>
>> Hrm, I notice that for qemu-kvm, you basically just pulled out the 
>> module source.
>>
>
> That's step 1.
>
>> What do you think about going a step further and building a proper 
>> qemu repository?
>>
>
> Step 2.  I have scripts that do that and will merge with Eduardo's 
> script that also rewrites the qemu commit hashes to correspond to 
> qemu.git on git.kernel.org.

That's what I figured.  FWIW, the split tarballs work just fine for me.

It may be worth waiting to do step 2 until the IO thread is merged.  I 
think once that happens, we could probably do a sprint to get rid of 
libkvm in kvm-userspace.  That would certainly simplify things.

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Christoph Hellwig]

On Wed, Mar 25, 2009 at 08:44:58AM -0500, Anthony Liguori wrote:
> That's what I figured.  FWIW, the split tarballs work just fine for me.
>
> It may be worth waiting to do step 2 until the IO thread is merged.  I  
> think once that happens, we could probably do a sprint to get rid of  
> libkvm in kvm-userspace.  That would certainly simplify things.

Yeah.  And having the both common and split repos just confuses the
heck out of any user of the repository.  I think the right way to split
it to wait for libkvm going away and just have a qemu-kvm repository
and an entirely separate kernel module repository.  It's not like there
is anything common but the few exported ABI headers, and we can either
keep them in both (would mean qemu-kvm can always build against a
defined set of headers) or make qemu-kvm require a kernel source like
the current kvm support in upstream qemu.

Re: Split kvm source tarballs

[Avi Kivity]

Christoph Hellwig wrote:
> On Wed, Mar 25, 2009 at 08:44:58AM -0500, Anthony Liguori wrote:
>   
>> That's what I figured.  FWIW, the split tarballs work just fine for me.
>>
>> It may be worth waiting to do step 2 until the IO thread is merged.  I  
>> think once that happens, we could probably do a sprint to get rid of  
>> libkvm in kvm-userspace.  That would certainly simplify things.
>>     
>
> Yeah.  And having the both common and split repos just confuses the
> heck out of any user of the repository.  I think the right way to split
> it to wait for libkvm going away and just have a qemu-kvm repository
> and an entirely separate kernel module repository.  It's not like there
> is anything common but the few exported ABI headers, and we can either
> keep them in both (would mean qemu-kvm can always build against a
> defined set of headers) or make qemu-kvm require a kernel source like
> the current kvm support in upstream qemu.
>   

While I strongly dislike duplicating code under source control, I'm 
beginning to lean in this direction, since the situation is beginning to 
confuse me too.

So how about this:

- keep copies of the headers in the qemu repository. 'make sync' becomes 
a maintainer tool rather than a developer tool
- move qemu to the root of the repository, and reparent libkvm/ user/ 
and friends under it.  this will give us easier merging.
- move the external module kit into kvm.git

We end up with a standalone kvm-userspace.git which is easily mergable 
to and from qemu.git, and kvm.git which can build either a Linux kernel 
(and is easily mergable to and from Linus' tree and others) or an 
external module.

No git submodules or inter-repository dependencies.  What's not to like?

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Split kvm source tarballs

[Christoph Hellwig]

On Wed, Mar 25, 2009 at 08:02:48PM +0200, Avi Kivity wrote:
> So how about this:
>
> - keep copies of the headers in the qemu repository. 'make sync' becomes  
> a maintainer tool rather than a developer tool

Yeah.  That similar how we maintain the headers and some shared source
file for XFS and libxfs in xfsprogs.

> - move qemu to the root of the repository, and reparent libkvm/ user/  
> and friends under it.  this will give us easier merging.

Yeah.  While you're at it user/ might be renamed to something more
descriptive.

> - move the external module kit into kvm.git

As in your kvm development kernel tree?  Not sure it's a good idea
to suck this into a full kernel tree.  Probably worth making it a small
repository of it's own.

Re: Split kvm source tarballs

[Anthony Liguori]

Hi Avi,

I spent some time today putting together an approximation of the KVM 
stable release based on QEMU 0.10.x.  In principle, it's not too bad at 
all because it's just a matter of creating a branch in kvm-userspace 
that's based a kvm commit sometime after the QEMU 0.10.x release but 
before the next qemu-cvs merge you did that happened post QEMU 0.10.x.  
Basically, it was a merge of 72ee81f and ddecccc.

The problem for me is pulling in the QEMU stable fixes.  You don't have 
a branch that tracks the QEMU stable tree and I can pull in the QEMU 
stable git tree without rewriting history since the directory layout is 
different.

But if you created a qemu-svn-stable branch that followed the QEMU 
stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, 
then it would be pretty easy to create and maintain a kvm_stable_0_10 
branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this?  I suspect it's just a matter of creating 
the branch based off of the qemu-cvs tree at ddecccc and then doing a 
git-svn fetch.

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Christoph Hellwig]

On Wed, Mar 25, 2009 at 04:34:31PM -0500, Anthony Liguori wrote:
> But if you created a qemu-svn-stable branch that followed the QEMU  
> stable tree in kvm-userspace, like the qemu-cvs branch follows trunk,  
> then it would be pretty easy to create and maintain a kvm_stable_0_10  
> branch of whatever you'd like to call it in kvm-userspace.
>
> Any chance you could do this?  I suspect it's just a matter of creating  
> the branch based off of the qemu-cvs tree at ddecccc and then doing a  
> git-svn fetch.

Slightly offtopic, but I always wondered why qemu is hosted in svn.  For
all project having semi-forks of qemu that they try to keep in sync
or even merge back a distributed scm would work so much better.

Re: Split kvm source tarballs

[Avi Kivity]

Christoph Hellwig wrote:
>> - move qemu to the root of the repository, and reparent libkvm/ user/  
>> and friends under it.  this will give us easier merging.
>>     
>
> Yeah.  While you're at it user/ might be renamed to something more
> descriptive.
>   

The subdirectory test/ under it can be ported to run under qemu and 
contributed into qemu's test suite.

>> - move the external module kit into kvm.git
>>     
>
> As in your kvm development kernel tree?  Not sure it's a good idea
> to suck this into a full kernel tree.  Probably worth making it a small
> repository of it's own.
>   

If the repo contains only the kit (external-module.h and the hack 
scripts) we'll be left with dual repositories with their confusion and 
unbisectability.  If the repo contains both the kit and the code, I'll 
need to commit every kvm.git change into that repository, which I'm sure 
to botch now and then.

Do you see any specific problem with dropping kvm-userspace.git/kernel/* 
under kvm.git/scripts/kvm/?

-- 
error compiling committee.c: too many arguments to function

Re: Split kvm source tarballs

[Avi Kivity]

Anthony Liguori wrote:
> Hi Avi,
>
> I spent some time today putting together an approximation of the KVM 
> stable release based on QEMU 0.10.x.  In principle, it's not too bad 
> at all because it's just a matter of creating a branch in 
> kvm-userspace that's based a kvm commit sometime after the QEMU 0.10.x 
> release but before the next qemu-cvs merge you did that happened post 
> QEMU 0.10.x.  Basically, it was a merge of 72ee81f and ddecccc.
>
> The problem for me is pulling in the QEMU stable fixes.  You don't 
> have a branch that tracks the QEMU stable tree and I can pull in the 
> QEMU stable git tree without rewriting history since the directory 
> layout is different.
>
> But if you created a qemu-svn-stable branch that followed the QEMU 
> stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, 
> then it would be pretty easy to create and maintain a kvm_stable_0_10 
> branch of whatever you'd like to call it in kvm-userspace.
>
> Any chance you could do this?  I suspect it's just a matter of 
> creating the branch based off of the qemu-cvs tree at ddecccc and then 
> doing a git-svn fetch.
>

I pushed this under the old naming convention, as kvm-userspace.git 
maint/2.6.30.  I'll rename it once I convert the repo.


-- 
error compiling committee.c: too many arguments to function

Re: Split kvm source tarballs

[Anthony Liguori]

Christoph Hellwig wrote:
> On Wed, Mar 25, 2009 at 04:34:31PM -0500, Anthony Liguori wrote:
>   
>> But if you created a qemu-svn-stable branch that followed the QEMU  
>> stable tree in kvm-userspace, like the qemu-cvs branch follows trunk,  
>> then it would be pretty easy to create and maintain a kvm_stable_0_10  
>> branch of whatever you'd like to call it in kvm-userspace.
>>
>> Any chance you could do this?  I suspect it's just a matter of creating  
>> the branch based off of the qemu-cvs tree at ddecccc and then doing a  
>> git-svn fetch.
>>     
>
> Slightly offtopic, but I always wondered why qemu is hosted in svn.  For
> all project having semi-forks of qemu that they try to keep in sync
> or even merge back a distributed scm would work so much better.
>   

I'm going to switch it to git fwiw.

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>> Hi Avi,
>>
>> I spent some time today putting together an approximation of the KVM 
>> stable release based on QEMU 0.10.x.  In principle, it's not too bad 
>> at all because it's just a matter of creating a branch in 
>> kvm-userspace that's based a kvm commit sometime after the QEMU 
>> 0.10.x release but before the next qemu-cvs merge you did that 
>> happened post QEMU 0.10.x.  Basically, it was a merge of 72ee81f and 
>> ddecccc.
>>
>> The problem for me is pulling in the QEMU stable fixes.  You don't 
>> have a branch that tracks the QEMU stable tree and I can pull in the 
>> QEMU stable git tree without rewriting history since the directory 
>> layout is different.
>>
>> But if you created a qemu-svn-stable branch that followed the QEMU 
>> stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, 
>> then it would be pretty easy to create and maintain a kvm_stable_0_10 
>> branch of whatever you'd like to call it in kvm-userspace.
>>
>> Any chance you could do this?  I suspect it's just a matter of 
>> creating the branch based off of the qemu-cvs tree at ddecccc and 
>> then doing a git-svn fetch.
>>
>
> I pushed this under the old naming convention, as kvm-userspace.git 
> maint/2.6.30.  I'll rename it once I convert the repo.
>
>
Sweet, thanks!

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Christoph Hellwig]

On Thu, Mar 26, 2009 at 11:09:07AM +0200, Avi Kivity wrote:
> If the repo contains only the kit (external-module.h and the hack  
> scripts) we'll be left with dual repositories with their confusion and  
> unbisectability.  If the repo contains both the kit and the code, I'll  
> need to commit every kvm.git change into that repository, which I'm sure  
> to botch now and then.
>
> Do you see any specific problem with dropping kvm-userspace.git/kernel/*  
> under kvm.git/scripts/kvm/?

Adding more stuff to a kernel tree that doesn't apply to building that
tree seems rather odd.  But if it makes your life easier go for it, it's
defintively much better than the current setup.

Re: Split kvm source tarballs

[Christoph Hellwig]

On Thu, Mar 26, 2009 at 01:19:46PM -0500, Anthony Liguori wrote:
>> Slightly offtopic, but I always wondered why qemu is hosted in svn.  For
>> all project having semi-forks of qemu that they try to keep in sync
>> or even merge back a distributed scm would work so much better.
>>   
>
> I'm going to switch it to git fwiw.

If it's soon enough maybe the kvm repository reorganization should wait
for it so that the new repository can be cloned from upstream qemu?

Re: Split kvm source tarballs

[Anthony Liguori]

Christoph Hellwig wrote:
> On Thu, Mar 26, 2009 at 01:19:46PM -0500, Anthony Liguori wrote:
>   
>>> Slightly offtopic, but I always wondered why qemu is hosted in svn.  For
>>> all project having semi-forks of qemu that they try to keep in sync
>>> or even merge back a distributed scm would work so much better.
>>>   
>>>       
>> I'm going to switch it to git fwiw.
>>     
>
> If it's soon enough maybe the kvm repository reorganization should wait
> for it so that the new repository can be cloned from upstream qemu?
>   

It'll be compatible with the kernel.org git tree so there's no need to wait.

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Christoph Hellwig]

Any idea when the split will happen?

Re: Split kvm source tarballs

[Anthony Liguori]

Christoph Hellwig wrote:
> Any idea when the split will happen?
>   
AFAIK, shortly after kvm-85 is released.

Regards,

Anthony Liguori

Re: Split kvm source tarballs

[Avi Kivity]

Christoph Hellwig wrote:
> Any idea when the split will happen?
>   

I'm working on it, but hit a snag when rewriting upstream qemu commits 
to match the official qemu.git:
- my repo and qemu.git used different -k options ("$keyword $" 
squashing), so I can't match tree objects
- some non-UTF-8 characters got corrupted, so I can't match log messages 
(some are dups anyway)
- I can't match the git-svn-id line since I ripped it off in 
kvm-userspace.git
- I can't match the ordinal position of the commit in the changelog 
since some bogus commits made their way into kvm-userspace.git

I expect I'll work around these issues, but it isn't like I have my 
script ready to roll.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.