stand-alone kvmtool

stand-alone kvmtool

[Andre Przywara]

Hi,

as I found it increasingly inconvenient to use kvmtool[1] as part of a
Linux repository, I decided to give it a go and make it a stand-alone
project. So I filtered all the respective commits, adjusted the paths in
there (while keeping authorship and commit date, of course) and then
added the missing bits to let it compile without a kernel tree nearby.
The result is now available on:

git://linux-arm.org/kvmtool.git
http://linux-arm.org/kvmtool.git

You can simply check it out, type make and use "./lkvm run" for a quick
test. So far I briefly tested x86-64, arm and arm64, the later two were
also cross-compiled. For sure there are rough edges in there (for
instance copying a few non-uapi header files into), but I deem it worthy
enough to get some public comments.
For me that also fixed some nasty warnings about libfdt, which now are
gone due it using your system library version of it.
I also managed to get rid of the libc-i386-dev dependency when compiling
for x86-64, but that still needs to be cleaned up and thus is not in the
current HEAD.
I haven't got around to compile-test the other supported architectures,
but supporting them should be as easy as copying over the uapi kvm.h
header file (see the respective ARM commit). Contributions (and tests!)
are welcome.

Please give it a go and tell me what you think. I don't want to fork the
project, so I am happy if someone "official" picks it up.

Cheers,
Andre.

[1] https://github.com/penberg/linux-kvm/tree/master/tools/kvm

stand-alone kvmtool

[Claudio Fontana]

Hello Andre,

On 13.02.2015 11:39, Andre Przywara wrote:
> Hi,
> 
> as I found it increasingly inconvenient to use kvmtool[1] as part of a
> Linux repository, I decided to give it a go and make it a stand-alone
> project. So I filtered all the respective commits, adjusted the paths in
> there (while keeping authorship and commit date, of course) and then
> added the missing bits to let it compile without a kernel tree nearby.
> The result is now available on:
> 
> git://linux-arm.org/kvmtool.git
> http://linux-arm.org/kvmtool.git

It builds fine on x86_64, but when I tried to crosscompile from x86_64 to AArch64,
I get in trouble because of libfdt: I have the aarch64 libs (static and shared), but how do I instruct the build system to get it from the right place?

Thanks,

Claudio


> 
> You can simply check it out, type make and use "./lkvm run" for a quick
> test. So far I briefly tested x86-64, arm and arm64, the later two were
> also cross-compiled. For sure there are rough edges in there (for
> instance copying a few non-uapi header files into), but I deem it worthy
> enough to get some public comments.
> For me that also fixed some nasty warnings about libfdt, which now are
> gone due it using your system library version of it.
> I also managed to get rid of the libc-i386-dev dependency when compiling
> for x86-64, but that still needs to be cleaned up and thus is not in the
> current HEAD.
> I haven't got around to compile-test the other supported architectures,
> but supporting them should be as easy as copying over the uapi kvm.h
> header file (see the respective ARM commit). Contributions (and tests!)
> are welcome.
> 
> Please give it a go and tell me what you think. I don't want to fork the
> project, so I am happy if someone "official" picks it up.
> 
> Cheers,
> Andre.
> 
> [1] https://github.com/penberg/linux-kvm/tree/master/tools/kvm
> _______________________________________________
> kvmarm mailing list
> kvmarm at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> 


-- 
Claudio Fontana
Server Virtualization Architect
Huawei Technologies Duesseldorf GmbH
Riesstra?e 25 - 80992 M?nchen

office: +49 89 158834 4135
mobile: +49 15253060158

stand-alone kvmtool

[Andre Przywara]

Ciao Claudio,

On 13/02/15 14:30, Claudio Fontana wrote:
> Hello Andre,
> 
> On 13.02.2015 11:39, Andre Przywara wrote:
>> Hi,
>>
>> as I found it increasingly inconvenient to use kvmtool[1] as part of a
>> Linux repository, I decided to give it a go and make it a stand-alone
>> project. So I filtered all the respective commits, adjusted the paths in
>> there (while keeping authorship and commit date, of course) and then
>> added the missing bits to let it compile without a kernel tree nearby.
>> The result is now available on:
>>
>> git://linux-arm.org/kvmtool.git
>> http://linux-arm.org/kvmtool.git
> 
> It builds fine on x86_64, but when I tried to crosscompile from x86_64 to AArch64,
> I get in trouble because of libfdt: I have the aarch64 libs (static and shared), but how do I instruct the build system to get it from the right place?

You have to install them into your cross-compiler's SYSROOT.
Get the location of that by executing
$ ${CROSS_COMPILE}gcc -print-sysroot.
If it's just for libfdt, it's probably the easiest to copy them
manually, the header files into $SYSROOT/usr/include, the libraries into
$SYSROOT/usr/lib/aarch64-linux-gnu
That fixed it for me ;-)

For a more robust approach you would use your distribution's packaging
system to install the aarch64 package into $SYSROOT.

Cheers,
Andre.

> 
>>
>> You can simply check it out, type make and use "./lkvm run" for a quick
>> test. So far I briefly tested x86-64, arm and arm64, the later two were
>> also cross-compiled. For sure there are rough edges in there (for
>> instance copying a few non-uapi header files into), but I deem it worthy
>> enough to get some public comments.
>> For me that also fixed some nasty warnings about libfdt, which now are
>> gone due it using your system library version of it.
>> I also managed to get rid of the libc-i386-dev dependency when compiling
>> for x86-64, but that still needs to be cleaned up and thus is not in the
>> current HEAD.
>> I haven't got around to compile-test the other supported architectures,
>> but supporting them should be as easy as copying over the uapi kvm.h
>> header file (see the respective ARM commit). Contributions (and tests!)
>> are welcome.
>>
>> Please give it a go and tell me what you think. I don't want to fork the
>> project, so I am happy if someone "official" picks it up.
>>
>> Cheers,
>> Andre.
>>
>> [1] https://github.com/penberg/linux-kvm/tree/master/tools/kvm
>> _______________________________________________
>> kvmarm mailing list
>> kvmarm at lists.cs.columbia.edu
>> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>>
> 
> 

stand-alone kvmtool

[Claudio Fontana]

On 13.02.2015 15:40, Andre Przywara wrote:
> Ciao Claudio,
> 
> On 13/02/15 14:30, Claudio Fontana wrote:
>> Hello Andre,
>>
>> On 13.02.2015 11:39, Andre Przywara wrote:
>>> Hi,
>>>
>>> as I found it increasingly inconvenient to use kvmtool[1] as part of a
>>> Linux repository, I decided to give it a go and make it a stand-alone
>>> project. So I filtered all the respective commits, adjusted the paths in
>>> there (while keeping authorship and commit date, of course) and then
>>> added the missing bits to let it compile without a kernel tree nearby.
>>> The result is now available on:
>>>
>>> git://linux-arm.org/kvmtool.git
>>> http://linux-arm.org/kvmtool.git
>>
>> It builds fine on x86_64, but when I tried to crosscompile from x86_64 to AArch64,
>> I get in trouble because of libfdt: I have the aarch64 libs (static and shared), but how do I instruct the build system to get it from the right place?
> 
> You have to install them into your cross-compiler's SYSROOT.
> Get the location of that by executing
> $ ${CROSS_COMPILE}gcc -print-sysroot.
> If it's just for libfdt, it's probably the easiest to copy them
> manually, the header files into $SYSROOT/usr/include, the libraries into
> $SYSROOT/usr/lib/aarch64-linux-gnu
> That fixed it for me ;-)

Thanks!

> 
> For a more robust approach you would use your distribution's packaging
> system to install the aarch64 package into $SYSROOT.
> 
> Cheers,
> Andre.

I still prefer to be forced to understand things so I actually prefer the manual route.

Ciao,

Claudio

>>
>>>
>>> You can simply check it out, type make and use "./lkvm run" for a quick
>>> test. So far I briefly tested x86-64, arm and arm64, the later two were
>>> also cross-compiled. For sure there are rough edges in there (for
>>> instance copying a few non-uapi header files into), but I deem it worthy
>>> enough to get some public comments.
>>> For me that also fixed some nasty warnings about libfdt, which now are
>>> gone due it using your system library version of it.
>>> I also managed to get rid of the libc-i386-dev dependency when compiling
>>> for x86-64, but that still needs to be cleaned up and thus is not in the
>>> current HEAD.
>>> I haven't got around to compile-test the other supported architectures,
>>> but supporting them should be as easy as copying over the uapi kvm.h
>>> header file (see the respective ARM commit). Contributions (and tests!)
>>> are welcome.
>>>
>>> Please give it a go and tell me what you think. I don't want to fork the
>>> project, so I am happy if someone "official" picks it up.
>>>
>>> Cheers,
>>> Andre.
>>>
>>> [1] https://github.com/penberg/linux-kvm/tree/master/tools/kvm
>>> _______________________________________________
>>> kvmarm mailing list
>>> kvmarm at lists.cs.columbia.edu
>>> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>>>
>>
>>

stand-alone kvmtool

[Will Deacon]

Hi Andre,

Thanks for doing this. Since it looks unlikely that kvmtool will ever be
merged back into the kernel tree, it makes sense to cut the dependency
in my opinion.

On Fri, Feb 13, 2015 at 10:39:33AM +0000, Andre Przywara wrote:
> as I found it increasingly inconvenient to use kvmtool[1] as part of a
> Linux repository, I decided to give it a go and make it a stand-alone
> project. So I filtered all the respective commits, adjusted the paths in
> there (while keeping authorship and commit date, of course) and then
> added the missing bits to let it compile without a kernel tree nearby.
> The result is now available on:
> 
> git://linux-arm.org/kvmtool.git
> http://linux-arm.org/kvmtool.git
> 
> You can simply check it out, type make and use "./lkvm run" for a quick
> test. So far I briefly tested x86-64, arm and arm64, the later two were
> also cross-compiled. For sure there are rough edges in there (for
> instance copying a few non-uapi header files into), but I deem it worthy
> enough to get some public comments.
> For me that also fixed some nasty warnings about libfdt, which now are
> gone due it using your system library version of it.
> I also managed to get rid of the libc-i386-dev dependency when compiling
> for x86-64, but that still needs to be cleaned up and thus is not in the
> current HEAD.
> I haven't got around to compile-test the other supported architectures,
> but supporting them should be as easy as copying over the uapi kvm.h
> header file (see the respective ARM commit). Contributions (and tests!)
> are welcome.
> 
> Please give it a go and tell me what you think. I don't want to fork the
> project, so I am happy if someone "official" picks it up.

In which case, it's probably best to post the patches for review rather
than just point me at your git repo!

Will

stand-alone kvmtool

[Andre Przywara]

Hi Will,

On 18/02/15 15:50, Will Deacon wrote:
> Hi Andre,
> 
> Thanks for doing this. Since it looks unlikely that kvmtool will ever be
> merged back into the kernel tree, it makes sense to cut the dependency
> in my opinion.
> 
> On Fri, Feb 13, 2015 at 10:39:33AM +0000, Andre Przywara wrote:
>> as I found it increasingly inconvenient to use kvmtool[1] as part of a
>> Linux repository, I decided to give it a go and make it a stand-alone
>> project. So I filtered all the respective commits, adjusted the paths in
>> there (while keeping authorship and commit date, of course) and then
>> added the missing bits to let it compile without a kernel tree nearby.
>> The result is now available on:
>>
>> git://linux-arm.org/kvmtool.git
>> http://linux-arm.org/kvmtool.git
>>
>> You can simply check it out, type make and use "./lkvm run" for a quick
>> test. So far I briefly tested x86-64, arm and arm64, the later two were
>> also cross-compiled. For sure there are rough edges in there (for
>> instance copying a few non-uapi header files into), but I deem it worthy
>> enough to get some public comments.
>> For me that also fixed some nasty warnings about libfdt, which now are
>> gone due it using your system library version of it.
>> I also managed to get rid of the libc-i386-dev dependency when compiling
>> for x86-64, but that still needs to be cleaned up and thus is not in the
>> current HEAD.
>> I haven't got around to compile-test the other supported architectures,
>> but supporting them should be as easy as copying over the uapi kvm.h
>> header file (see the respective ARM commit). Contributions (and tests!)
>> are welcome.
>>
>> Please give it a go and tell me what you think. I don't want to fork the
>> project, so I am happy if someone "official" picks it up.
> 
> In which case, it's probably best to post the patches for review rather
> than just point me at your git repo!

Makes some sense, although part of the exercise was to get rid of the
huge, now unneeded Linux kernel code base.
So this approach required a fresh repository, and due to the different
paths there is no out-of-the-box patch compatibility between the two.
Also I wanted to provide an easy way for people to give it a test.

So what I could do is to send the top-most patches against Pekka's
github repository, which would eliminate the references to the kernel
directory (at the cost of duplicating some files).
Once this is settled, acked and applied, one could try to create a new
repository with the tools/kvm directory being the new root.

Let me know if that makes more sense and I will rework the patches to
apply against the current upstream kvmtool.

Cheers,
Andre.

P.S. Although both approaches still provide the kvmtool patch history,
they do not compile before the dependency cut patches. If that is an
issue, one could think about injecting those new patches back into the
repository time line. Admittedly that sounds scary, but would solve the
problem.

stand-alone kvmtool

[Alex Bennée]

Andre Przywara <andre.przywara@arm.com> writes:

> Hi Will,
>
> On 18/02/15 15:50, Will Deacon wrote:
>> Hi Andre,
>> 
>> Thanks for doing this. Since it looks unlikely that kvmtool will ever be
>> merged back into the kernel tree, it makes sense to cut the dependency
>> in my opinion.
>> 
<snip>
>
> P.S. Although both approaches still provide the kvmtool patch history,
> they do not compile before the dependency cut patches. If that is an
> issue, one could think about injecting those new patches back into the
> repository time line. Admittedly that sounds scary, but would solve the
> problem.

If you can have it all it would be nice to preserve buildability all
through your history for bisecting (and the moon on a stick please ;-)

Is the dependency on the kernel sources something that has been stable
over the projects history or something that's been declining/increasing
over time?

-- 
Alex Benn?e

stand-alone kvmtool

[Pekka Enberg]

On 2/26/15 1:02 PM, Alex Benn?e wrote:
> If you can have it all it would be nice to preserve buildability all
> through your history for bisecting (and the moon on a stick please ;-)
>
> Is the dependency on the kernel sources something that has been stable
> over the projects history or something that's been declining/increasing
> over time?

AFAICT, it's been mostly stable after the initial sweep to use 
kernel-provided data structures such as lists and rbtrees.

- Pekka

stand-alone kvmtool

[Pekka Enberg]

Hi,

On 2/18/15 5:50 PM, Will Deacon wrote:
> Thanks for doing this. Since it looks unlikely that kvmtool will ever be
> merged back into the kernel tree, it makes sense to cut the dependency
> in my opinion.

I am certainly OK with a standalone repository which preserves the 
history. Will, would you like to take over the proposed new repository 
and put it somewhere on git.kernel.org, perhaps?

- Pekka

stand-alone kvmtool

[Will Deacon]

On Mon, Feb 23, 2015 at 05:23:58PM +0000, Pekka Enberg wrote:
> Hi,

Hi Pekka,

Sorry for the delay, I've been away from email for a few days.

> On 2/18/15 5:50 PM, Will Deacon wrote:
> > Thanks for doing this. Since it looks unlikely that kvmtool will ever be
> > merged back into the kernel tree, it makes sense to cut the dependency
> > in my opinion.
> 
> I am certainly OK with a standalone repository which preserves the 
> history. Will, would you like to take over the proposed new repository 
> and put it somewhere on git.kernel.org, perhaps?

Sure thing. I'll sync-up with Andre and reply back here when we've got
something sensible.

One extra point that I don't think has been mentioned in this thread so
far is that separating kvmtool from the kernel sources is likely a
prerequisite for distribution packaging. Once we've got something sorted
out, I'll poke some friendly local debian developers and see if they can't
package it up for us.

Will

stand-alone kvmtool

[Will Deacon]

On Mon, Feb 23, 2015 at 05:23:58PM +0000, Pekka Enberg wrote:
> On 2/18/15 5:50 PM, Will Deacon wrote:
> > Thanks for doing this. Since it looks unlikely that kvmtool will ever be
> > merged back into the kernel tree, it makes sense to cut the dependency
> > in my opinion.
> 
> I am certainly OK with a standalone repository which preserves the 
> history. Will, would you like to take over the proposed new repository 
> and put it somewhere on git.kernel.org, perhaps?

Finally got around to this with Andre's help, so I'll post a separate
mail advertising it.

Will

stand-alone kvmtool

[Sasha Levin]

On 02/13/2015 05:39 AM, Andre Przywara wrote:
> Hi,
> 
> as I found it increasingly inconvenient to use kvmtool[1] as part of a
> Linux repository, I decided to give it a go and make it a stand-alone
> project. So I filtered all the respective commits, adjusted the paths in
> there (while keeping authorship and commit date, of course) and then
> added the missing bits to let it compile without a kernel tree nearby.
> The result is now available on:
> 
> git://linux-arm.org/kvmtool.git
> http://linux-arm.org/kvmtool.git

Hi Andre,

What inconvenience is caused by having it sit inside the kernel tree
beyond an increased requirement in disk space?

Moving it out will make us lose all the new features and bug fixes we
gain from using the kernel code directly rather than copying it once
in a while.

With your suggestion we'll end up needing something that copies stuff
from the kernel into that standalone tree, just like what qemu does.


Thanks,
Sasha

stand-alone kvmtool

[Andre Przywara]

Hi Sasha,

thanks for taking a look!

On 19/02/15 10:56, Sasha Levin wrote:
> On 02/13/2015 05:39 AM, Andre Przywara wrote:
>> Hi,
>>
>> as I found it increasingly inconvenient to use kvmtool[1] as part of a
>> Linux repository, I decided to give it a go and make it a stand-alone
>> project. So I filtered all the respective commits, adjusted the paths in
>> there (while keeping authorship and commit date, of course) and then
>> added the missing bits to let it compile without a kernel tree nearby.
>> The result is now available on:
>>
>> git://linux-arm.org/kvmtool.git
>> http://linux-arm.org/kvmtool.git
> 
> Hi Andre,
> 
> What inconvenience is caused by having it sit inside the kernel tree
> beyond an increased requirement in disk space?

Reduced disk space is admittedly one of the benefits of this exercise.
Also it makes cloning a lot easier and would allow easier packaging.

Many of the issues we face here come from the fact that kvmtool lives in
_a_ kernel repository, but it's not upstream. So we loose the benefit of
joined kernel-userland development. In fact we have to do regular merges
of mostly unrelated upstream kernel code into the branch to get it
compiled with a new feature.
Also having a pure userland tool in the kernel repository sounds just
wrong to me, especially as KVM has a nice API with compatibility
features. There is a clear interface between the KVM kernel and the
controlling userland, so they should not need to share code beyond the
API defining header files. Having a shared code base lures people into
breaking the interface.

> Moving it out will make us lose all the new features and bug fixes we
> gain from using the kernel code directly rather than copying it once
> in a while.

Which code are you exactly thinking of?
>From the code I copied I don't see that rbtree or the Linux list
implementations for instance justify a common code base. If in dire
need, one could setup alerts on the few code files copied to spot
upstream bug fixes.
I see there is a slight drawback in this regard, but I think the
benefits outweigh it.

> With your suggestion we'll end up needing something that copies stuff
> from the kernel into that standalone tree, just like what qemu does.

While I see that copying is not the best solution, QEMU lives very well
with it, doesn't it? With KVM's feature compatibility API and the
kernel's "don't break userland" policy there should be no real problem.
Also with the current situation we just replace "copy uapi header files"
with "merge in upstream kernel code base", which is also manually
triggered and much more ugly IMHO.


I agree that the whole argumentation would be much different if kvmtool
would be upstream, but it is not and as Will pointed out will probably
never be. So to make it's usage easier for the users and distribution
package maintainers, I'd like to see it live on in a separate
(kernel.org) repository.
I could imagine that the easier accessibility would make it more
appealing to potential users (and packagers!)

Cheers,
Andre.

stand-alone kvmtool

[Russell King - ARM Linux]

On Thu, Feb 19, 2015 at 05:56:45AM -0500, Sasha Levin wrote:
> What inconvenience is caused by having it sit inside the kernel tree
> beyond an increased requirement in disk space?

I've come across this problem with the perf tools - luckily, the perf
tools allow the source to be exported from the kernel tree, but it is
far from a good solution.

The problem is when you're primarily cross-building the kernel on a
system where you don't have the target libraries (because, eg, you're
running in a build environment for multiple different target systems.)
Having to build userspace tools in that scenario is a _major_ pita.

Yes, of course it's possible to pull the 1GB of kernel GIT respository
down onto the target just to build some silly userspace tool, but when
your rootfs lives on an 8GB SD card or a USB memory stick (as is the
case with the ARM Juno 64-bit platform), and when the userspace tool
somehow depends on the kernel source tree being configured, it really
starts getting painful.

TBH, I don't much care provided there is a way to export a source
tarball for the tool from the kernel (like perf does) which can then
be transferred to the target and built there.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

stand-alone kvmtool

[Christoffer Dall]

Hi,

On Thu, Feb 19, 2015 at 05:56:45AM -0500, Sasha Levin wrote:

[...]

> 
> What inconvenience is caused by having it sit inside the kernel tree
> beyond an increased requirement in disk space?
> 
FWIW: I would prefer seeing this outside the kernel tree; I think it is
slightly confusing to keep it as part of a non-upstream kernel repo and
it is simpler to view git change logs etc. in gitweb for a stand-alone
repo.

-Christoffer