Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

On 03/08/2012 04:43 PM, Anthony Liguori wrote:
> On 03/08/2012 01:34 PM, Lucas Meneghel Rodrigues wrote:
>> On 03/08/2012 03:57 PM, Anthony Liguori wrote:
>>> On 03/08/2012 09:19 AM, Lucas Meneghel Rodrigues wrote:
>>>> Before I forget, I'd like to ask you about this:
>>>>
>>>> On 03/08/2012 10:36 AM, Anthony Liguori wrote:
>>>>> I'm really not a fan of buildroot. Note that in order to ship
>>>>> binaries,
>>>>> full source needs to be provided in order to comply with the GPL. The
>>>>> FSF at least states that referring to another website for source
>>>>> that's
>>>>> not under your control doesn't satisfy the requirements of the GPL.
>>>>
>>>> About using buildroot, what is up with it, since it is mature and
>>>> works well?
>>>> You mentioned than providing all the sources is harder than it looks
>>>> like, and I
>>>> surely think this might be the case.
>>>
>>> buildroot is a full blown distribution. But instead of distributing
>>> binaries, it only distributes source code. Think of it like Gentoo--.
>>>
>>> It relies on third party links to fetch said source code which means
>>> that it's not unusual
>>
>> By this definition, qemu test fetch source code from 3rd party
>> repositories just
>> as much, after all you have to fetch your linux and busybox code from
>> somewhere,
>> I assume.
>
> It uses git submodules with repositories hosted on git.qemu.org.

The linux, uclibc, gcc and and busybox repositories are nowhere to be 
seen there. Also, from .gitmodules for qemu-jeos:

[submodule "busybox"]
         path = busybox
         url = git://busybox.net/busybox.git
[submodule "linux"]
         path = linux
         url = 
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[submodule "uClibc"]
         path = uClibc
         url = git://uclibc.org/uClibc.git
[submodule "binutils"]
         path = binutils
         url = git://sources.redhat.com/git/binutils.git
[submodule "gcc"]
         path = gcc
         url = git://gcc.gnu.org/git/gcc.git

So still pretty much a lot of code outside the qemu.org realm.

>>
>> at all for buildroot to straight out fail. Because
>>> it needs a GCC that can build shared libraries, it also requires
>>> rebuilding GCC which increases the build time even for a minimalistic
>>> configuration.
>>
>> But is this additional build time a problem? Is the person expected to be
>> messing around with re-creating the minimal guest images all the time?
>
> I think it's common enough to want to update the guest kernel that yes,
> this is something that many people need to be able to do in order for
> the test cases to be effective. If it takes 24hr to rebuild the images,
> that's problematic.

I'd say more like 2 hours on commodity hardware to do a full rebuild. 
Incremental rebuilds are much quicker (under 20 minutes), since gcc and 
all userspace was already built, so no need to do a full rebuild each 
time you want to update the kernel.

>>> Having a JeOS it not meant to replace having proper guests (like Fedora
>>> or Ubuntu). As I've said in other notes, the point of having a JeOS is
>>> to essentially use Linux as a libOS when writing unit tests.
>>
>> That is why we would have a reference binary jeos image that can be
>> downloaded
>> (although not a replacement, having a small guest makes setup time
>> smaller, as
>> well as smaller time to run the tests). If people need to tweak
>> kernel, etc for
>> some reason, I'm assuming that person would have their own clone of
>> the repo and
>> rebuild the images with the modification, and use that for the tests,
>> I guess
>> just the way you would do with qemu-jeos.
>
> Yes, but this is a common thing. This is something that needs to be
> designed for upfront for the infrastructure to be useful.
>
>>>> But in all my naiveness, if the problem is to ship the exact source
>>>> with the
>>>> images have been built, couldn't I just ask buildroot to fetch all the1:30
>>>> tarball
>>>> sources (there's a function to perform source download only) and add
>>>> them to the
>>>> appropriate git branch?
>>>
>>> Okay, let's say I'm a developer and I want to use a new Linux kernel to
>>> test the new virtio-pci feature I added with buildroot. Where do I
>>> begin? What patch do I submit?
>>
>> You change the configuration file to build your linux from git rather
>> than the
>> tarball, change the linux config and type 'make'. At the end, you'll
>> have a
>> patch that can be sent and kept in another autotest-buildroot branch,
>> if it's
>> proven to be useful *for other people*.
>
> Except you also need to update those tarballs too that you're storing in
> git, remember.
>
> Herein lies the problem. You forgot and it's your proposal :-)

Ok, fair enough :) But still, qemu-jeos points out to external 
repositories, just as much as buildroot. It seems to me that the whole 
point about FSF requiring the source to be under your control is no 
longer valid here.