Re: [Qemu-devel] Yet another git submodule rant

["Daniel P. Berrange" <berrange@redhat.com>]

On Wed, Nov 08, 2017 at 01:43:15PM +0100, Michal Suchánek wrote:
> On Wed, 8 Nov 2017 12:34:17 +0000
> "Daniel P. Berrange" <berrange@redhat.com> wrote:
> 
> > On Wed, Nov 08, 2017 at 01:05:27PM +0100, Michal Suchánek wrote:
> > > On Wed, 8 Nov 2017 10:25:29 +0000
> > > "Daniel P. Berrange" <berrange@redhat.com> wrote:
> > >   
> > > > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote:  
> > > > > 
> > > > > That automatic git submodule stuff now broke my workflow again.
> > > > > I usually keep the git repository on my laptop and then simply
> > > > > rsync the sources (without .git directories) to my target
> > > > > machine to compile it there. Used to work great for years. Now
> > > > > it's broken, the build process complains:    
> > > > 
> > > > Excluding the .git dir will be a problem - that needs to exist,
> > > > unless you are building from tar.xz (where we will bundled all
> > > > the submodule sources together in the tar.xz).
> > > > 
> > > > If you really want to exclude the .git directories, then instead
> > > > of doing an rsync, then use the scripts/archive-source.sh tool to
> > > > create an archive of all the sources (which bundles required
> > > > submodules) and unpack that on your target machine. Then all
> > > > submodule handling on the target will be skipped.  
> > > 
> > > This probably does not work because it archives committed sources
> > > and not actual sources.  
> > 
> > Actually that's not quite correct - if it sees you have non-comitted
> > changes, and will use 'git stash create' to capture them in the
> > archive it creates.  So it would only miss brand new files which have
> > not yet been 'git add'ed
> 
> So there really isn't a good solution.
> 
> > 
> > > > > I've got the feeling that all this submodule crap is constantly
> > > > > causing pain ... do we really need this? Can't we find another
> > > > > solution instead? Or at least stop modifying files
> > > > > automatically in the $SRC_PATH ?    
> > > > 
> > > > We *have* to modify SRC_PATH, as that's where git submodule
> > > > checkouts live. If we didn't do that, then people would end up
> > > > having failed build due to submodules not existing, or even
> > > > worse, having a silently incorrect build due to using the wrong
> > > > checked out version.  
> > > 
> > > And that's another reason why submodules are bad thing.
> > > 
> > > So again, can this be done without submodules?
> > > 
> > > Let somebody else do the testing and polishing of this pre-alpha git
> > > feature.  
> > 
> > Honestly I think the submodule stuff is working pretty well - a
> > handful of people have mentioned problems and we've addressed most of
> > them or illustrated alternative approaches to deal with it. For most
> > developers it has been pretty much transparent and just works, or we
> > would have seen alot more noise.
> 
> > On 11/08/2017 06:57 AM, Thomas Huth wrote:
> 
> > 
> > Also yesterday on IRC:
> > 
> > <RaV3N> [...] I downloaded the qemu source from git and tried to
> > compile it. I am getting this:
> > 
> > ./configure --static && make && sudo make install
> >  CC      ui/input-keymap.o
> > ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c:
> > No such file or directory
> > 
> > <cota> [...] you might want to do "git submodule init && git submodule
> > update"
> > 
> 
>  .. you call that transparent.
> 
> Yeah, right.
> 
> Please don't pretend that something is working when it clearly isn't.
> 
> Repositories using submodules are far from first class citizens as far
> as support from git is concerned.
> 
> Therefore projects that don't have git development at least as
> secondary focus should not use them until that changes.

That's really overstating the problem. There are some caveats with use of
submodules, but they are hardly showstoppers that prevent their usage
entirely. There are plenty of projects using GIT with submodules for
*years* (libvirt has used them for 8 years).

Some of the problems pointed out have been clear bugs in the way we've
integrated into QEMU's build system, which I've been addressing. Others
have been due to usage scenarios I didn't anticipate, which I've attempted
to provide solutions for, either by improving the build system, or suggesting
alternative approaches. We'll continue to try to improve this, as we would
for any other major change to the buildsystem we've been through, which
caused problems for some workflows. I none the less believe this is working
for the vast majority of developers of QEMU, given the size of our community
vs the number of problems reported.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|