[Qemu-devel] qemu compile error

[Qemu-devel] qemu compile error

[Chen, Farrah]

Hi,

When I build qemu with commit from aef45d51d1204f3335fb99de6658e0c5612c2b67 to the latest one, the following error occurred:

tsocks git clone git://git.qemu.org/qemu.git
cd qemu

./configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc --disable-gtk --enable-libusb --enable-numa --disable-sdl
make
  GEN     config-host.h
  GIT     dtc
  GEN     trace/generated-tcg-tracers.h
  GEN     trace/generated-helpers-wrappers.h
  GEN     trace/generated-helpers.h
  GEN     module_block.h
  GEN     trace/generated-helpers.c
make: *** [git-submodule-update] Error 1


Would you mind telling me how to avoid this error?


Thanks,
Fan Chen

Re: [Qemu-devel] qemu compile error

[Daniel P. Berrange]

On Fri, Oct 20, 2017 at 12:13:05PM +0000, Chen, Farrah wrote:
> Hi,
> 
> When I build qemu with commit from aef45d51d1204f3335fb99de6658e0c5612c2b67 to the latest one, the following error occurred:
> 
> tsocks git clone git://git.qemu.org/qemu.git

The use of 'tsocks' is probably the problem you have

> cd qemu
> 
> ./configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc --disable-gtk --enable-libusb --enable-numa --disable-sdl
> make
>   GEN     config-host.h
>   GIT     dtc

When we checkout the submodule we just call "git" and expect it to
work - we've no knowledge of tsocks.

>   GEN     trace/generated-tcg-tracers.h
>   GEN     trace/generated-helpers-wrappers.h
>   GEN     trace/generated-helpers.h
>   GEN     module_block.h
>   GEN     trace/generated-helpers.c
> make: *** [git-submodule-update] Error 1
> 
> 
> Would you mind telling me how to avoid this error?

You need to make sure the 'git' command in the $PATH can checkout from
qemu.org directly. Since you seem to require tsocks, then the way to
achieve this is to create a wrapper script and make sure it appears
first in $PATH.

eg something like

   $ cat > $HOME/.local/bin/git <<EOF
   #!/bin/sh

   exec tsocks /usr/bin/git "$@"
   EOF
   chmod +x $HOME/.local/bin/git
   export PATH=$HOME/.local/bin:$PATH

so now just typing 'git' transparently runs tsocks.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] qemu compile error

[Peter Maydell]

On 20 October 2017 at 13:13, Chen, Farrah <farrah.chen@intel.com> wrote:
> Hi,
>
> When I build qemu with commit from aef45d51d1204f3335fb99de6658e0c5612c2b67 to the latest one, the following error occurred:
>
> tsocks git clone git://git.qemu.org/qemu.git
> cd qemu
>
> ./configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc --disable-gtk --enable-libusb --enable-numa --disable-sdl
> make
>   GEN     config-host.h
>   GIT     dtc
>   GEN     trace/generated-tcg-tracers.h
>   GEN     trace/generated-helpers-wrappers.h
>   GEN     trace/generated-helpers.h
>   GEN     module_block.h
>   GEN     trace/generated-helpers.c
> make: *** [git-submodule-update] Error 1

Dan -- I thought we were going to not have the git-submodule.sh
script throw away the errors from git, but the version in
master still seems to do that...

Fan: this happens because we tried to run a 'git submodule'
command and it failed. If you edit scripts/git-submodule.sh
to remove the "1>/dev/null 2>&1" part of the 'git submodule
update' command line then rerun make it should print the
error messages from git which will tell you what the
problem is.

(The fact you needed to say 'tsocks git clone...' on your
original command suggests to me that the problem is that
plain 'git submodule update' can't access the git repo.)

thanks
-- PMM

Re: [Qemu-devel] qemu compile error

[Daniel P. Berrange]

On Fri, Oct 20, 2017 at 01:41:59PM +0100, Peter Maydell wrote:
> On 20 October 2017 at 13:13, Chen, Farrah <farrah.chen@intel.com> wrote:
> > Hi,
> >
> > When I build qemu with commit from aef45d51d1204f3335fb99de6658e0c5612c2b67 to the latest one, the following error occurred:
> >
> > tsocks git clone git://git.qemu.org/qemu.git
> > cd qemu
> >
> > ./configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc --disable-gtk --enable-libusb --enable-numa --disable-sdl
> > make
> >   GEN     config-host.h
> >   GIT     dtc
> >   GEN     trace/generated-tcg-tracers.h
> >   GEN     trace/generated-helpers-wrappers.h
> >   GEN     trace/generated-helpers.h
> >   GEN     module_block.h
> >   GEN     trace/generated-helpers.c
> > make: *** [git-submodule-update] Error 1
> 
> Dan -- I thought we were going to not have the git-submodule.sh
> script throw away the errors from git, but the version in
> master still seems to do that...

Urgh, yes, we did fix that, but it seems it got lost in one of the
rebases somewhere along the way. I'll send a patch to fix it.

> (The fact you needed to say 'tsocks git clone...' on your
> original command suggests to me that the problem is that
> plain 'git submodule update' can't access the git repo.)

Yep, creating the wrapper script called 'git' that runs
'exec tsocks /usr/bin/git "$@"'  is the way to deal with this for
QEMU, and indeed any project using git.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] qemu compile error

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]

On 10/20/2017 07:41 AM, Peter Maydell wrote:
> Fan: this happens because we tried to run a 'git submodule'
> command and it failed. If you edit scripts/git-submodule.sh
> to remove the "1>/dev/null 2>&1" part of the 'git submodule
> update' command line then rerun make it should print the
> error messages from git which will tell you what the
> problem is.
> 
> (The fact you needed to say 'tsocks git clone...' on your
> original command suggests to me that the problem is that
> plain 'git submodule update' can't access the git repo.)

It is also possible to use 'git config' to set up rewrite rules; if you
can't use git to access a git:// repo, but CAN use it to access an
http[s]:// repo, then setting up:

[url "http://server-of-http-clone/..."]
	insteadof = git://git.qemu-project.org/

I haven't personally tried it for git (it's been a while since I've been
a machine where I had to use rewriting rules to get past inability to
access git://), but have used the trick in other projects in the past.
I'm also fairly certain we have at least a github clone of all qemu
projects involved in qemu submodules, so that you can rewrite all of
your URLs to point to the github http:// service.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] qemu compile error

[Chen, Farrah]

Thank you very much!
I removed "1>/dev/null 2>&1" in scripts/git-submodule.sh and found I can't git clone git://git.qemu-project.org/dtc.git without tsocks.
I used Daniel's approach, created the wrapper script called 'git' that runs 'exec tsocks /usr/bin/git "$@"', it succeeded.
Considering we still need to use bare 'git'  to access our internal repo, so, for the time being, I use tsocks to git clone git://git.qemu-project.org/dtc.git and git://git.qemu.org/keycodemapdb.git first, then modified url in .gitmodules before compiling. It works.


Thanks,
Fan

 
-----Original Message-----
From: Peter Maydell [mailto:peter.maydell@linaro.org] 
Sent: Friday, October 20, 2017 8:42 PM
To: Chen, Farrah <farrah.chen@intel.com>
Cc: qemu-devel@nongnu.org; Hu, Robert <robert.hu@intel.com>; Daniel P. Berrange <berrange@redhat.com>
Subject: Re: [Qemu-devel] qemu compile error

On 20 October 2017 at 13:13, Chen, Farrah <farrah.chen@intel.com> wrote:
> Hi,
>
> When I build qemu with commit from aef45d51d1204f3335fb99de6658e0c5612c2b67 to the latest one, the following error occurred:
>
> tsocks git clone git://git.qemu.org/qemu.git cd qemu
>
> ./configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc 
> --disable-gtk --enable-libusb --enable-numa --disable-sdl make
>   GEN     config-host.h
>   GIT     dtc
>   GEN     trace/generated-tcg-tracers.h
>   GEN     trace/generated-helpers-wrappers.h
>   GEN     trace/generated-helpers.h
>   GEN     module_block.h
>   GEN     trace/generated-helpers.c
> make: *** [git-submodule-update] Error 1

Dan -- I thought we were going to not have the git-submodule.sh script throw away the errors from git, but the version in master still seems to do that...

Fan: this happens because we tried to run a 'git submodule'
command and it failed. If you edit scripts/git-submodule.sh to remove the "1>/dev/null 2>&1" part of the 'git submodule update' command line then rerun make it should print the error messages from git which will tell you what the problem is.

(The fact you needed to say 'tsocks git clone...' on your original command suggests to me that the problem is that plain 'git submodule update' can't access the git repo.)

thanks
-- PMM

Re: [Qemu-devel] qemu compile error

[Daniel P. Berrange]

On Mon, Oct 23, 2017 at 01:43:30PM +0000, Chen, Farrah wrote:
> Thank you very much!
> I removed "1>/dev/null 2>&1" in scripts/git-submodule.sh and found I can't
> git clone git://git.qemu-project.org/dtc.git without tsocks.
> I used Daniel's approach, created the wrapper script called 'git' that runs
> 'exec tsocks /usr/bin/git "$@"', it succeeded.
> Considering we still need to use bare 'git'  to access our internal repo,
> so, for the time being, I use tsocks to git clone
> git://git.qemu-project.org/dtc.git and git://git.qemu.org/keycodemapdb.git
> first, then modified url in .gitmodules before compiling. It works.

To avoid need to create the wrapper script, I copied you on a patch that
would let you run

  ./configure --with-git="tsocks git"

As a further alternative, I think it might also be possible to just run

  tsocks ./scripts/git-submodule.sh dtc ui/keycodemapdb

immediately after configure, but before running make.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] qemu compile error

[Chen, Farrah]

Yes, I saw that patch, thank you !


Thanks,
Fan



-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com] 
Sent: Monday, October 23, 2017 9:46 PM
To: Chen, Farrah <farrah.chen@intel.com>
Cc: Peter Maydell <peter.maydell@linaro.org>; qemu-devel@nongnu.org; Hu, Robert <robert.hu@intel.com>
Subject: Re: [Qemu-devel] qemu compile error

On Mon, Oct 23, 2017 at 01:43:30PM +0000, Chen, Farrah wrote:
> Thank you very much!
> I removed "1>/dev/null 2>&1" in scripts/git-submodule.sh and found I 
> can't git clone git://git.qemu-project.org/dtc.git without tsocks.
> I used Daniel's approach, created the wrapper script called 'git' that 
> runs 'exec tsocks /usr/bin/git "$@"', it succeeded.
> Considering we still need to use bare 'git'  to access our internal 
> repo, so, for the time being, I use tsocks to git clone 
> git://git.qemu-project.org/dtc.git and 
> git://git.qemu.org/keycodemapdb.git
> first, then modified url in .gitmodules before compiling. It works.

To avoid need to create the wrapper script, I copied you on a patch that would let you run

  ./configure --with-git="tsocks git"

As a further alternative, I think it might also be possible to just run

  tsocks ./scripts/git-submodule.sh dtc ui/keycodemapdb

immediately after configure, but before running make.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|