Unable to fork off sideband demultiplexer

Unable to fork off sideband demultiplexer

[Randy Brandenburg]

I have searched high and low to find a resolution to this error I am seeing 
when attempting to push or pull from a repository (either remote or on the same 
machine).  The error reported is as follows: 

# git push
error: cannot create thread: Unknown error 
fatal: fetch-pack: Unable to fork off sideband demultiplexer 
Broken pipe 

# git pull
error: cannot create thread: Unknown error 
fatal: receive-pack: Unable to fork off sideband demultiplexer 
Broken pipe 

This is a Solaris 9 installation (git-1.7.5-sol9-sparc-local.gz obtained from 
Sunfreeware.com) including all of the required package dependencies listed. 
I followed the same install on a Solaris 10 box and have no issues. Not sure if 
this is a permissions issue or something not configured 
correctly.
 
Any help is greatly appreciated. 

Re: Unable to fork off sideband demultiplexer

[Junio C Hamano]

Randy Brandenburg <randy.brandenburg@woh.rr.com> writes:

> # git push
> error: cannot create thread: Unknown error 

Perhaps threading library does not really work on your platform? Does
rebuilding git with "make NO_PTHREADS=YesPlease" help?

Re: Unable to fork off sideband demultiplexer

[Jeff King]

On Wed, Jun 01, 2011 at 10:15:54AM -0700, Junio C Hamano wrote:

> Randy Brandenburg <randy.brandenburg@woh.rr.com> writes:
> 
> > # git push
> > error: cannot create thread: Unknown error 
> 
> Perhaps threading library does not really work on your platform? Does
> rebuilding git with "make NO_PTHREADS=YesPlease" help?

I suspect it is more subtle than that. We've had several people recently
reporting the same issue, and all are using pre-built binaries on
Solaris 9. Given the weird errno value ("unknown error"), my guess is
that the packages are built on Solaris 10, and there is some ABI
incompatibility between the two platforms.

I'd be curious if building on Solaris 9, even without NO_PTHREADS set,
fixes the issue.

-Peff

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Jeff King <peff <at> peff.net> writes:
> 
> I suspect it is more subtle than that. We've had several people recently
> reporting the same issue, and all are using pre-built binaries on
> Solaris 9. Given the weird errno value ("unknown error"), my guess is
> that the packages are built on Solaris 10, and there is some ABI
> incompatibility between the two platforms.
> 
> I'd be curious if building on Solaris 9, even without NO_PTHREADS set,
> fixes the issue.
> 
> -Peff
> 

Thank you for the insight. I will attempt to rebuild on the target platform and 
see what happens. 

I will post the results when I have an answer on whther that helped or not.

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Randy Brandenburg <randy.brandenburg <at> woh.rr.com> writes:

> 
> Jeff King <peff <at> peff.net> writes:
> > 
> > I suspect it is more subtle than that. We've had several people recently
> > reporting the same issue, and all are using pre-built binaries on
> > Solaris 9. Given the weird errno value ("unknown error"), my guess is
> > that the packages are built on Solaris 10, and there is some ABI
> > incompatibility between the two platforms.
> > 
> > I'd be curious if building on Solaris 9, even without NO_PTHREADS set,
> > fixes the issue.
> > 
> > -Peff
> > 
> 
> Thank you for the insight. I will attempt to rebuild on the target platform 
and 
> see what happens. 
> 
> I will post the results when I have an answer on whther that helped or not.
> 
> 

I rebuilt git-1.7.5 from source on the Solaris 9 platform with gcc-3.4.6, GNU 
make 3.82, and GNU install (from coreutils 8.11). Following these steps

----------------------------------------------------------------------------
# ./configure CC=gcc

Edit the Makefile - set the path to "ar", "gcc" and GNU "install".
CC = /usr/local/bin/gcc
AR = /usr/ccs/bin/ar
INSTALL = /usr/local/bin/install
Look for ifeq ($(uname_S),SunOS) and set INSTALL to /usr/lcoal/bin/install

# gmake NO_TCLTK=1 NO_CURL=1 NO_PTHREADS=YesPlease install prefix=/opt/sfw
# git --version
git version 1.7.5
----------------------------------------------------------------------------
Preliminary testing indicates a working system -- will investigate thourough 
today and tomorrow.

Thanks for the tips!

- Randy

Re: Unable to fork off sideband demultiplexer

[Jeff King]

On Thu, Jun 02, 2011 at 03:29:04PM +0000, Randy Brandenburg wrote:

> I rebuilt git-1.7.5 from source on the Solaris 9 platform with gcc-3.4.6, GNU 
> make 3.82, and GNU install (from coreutils 8.11). Following these steps
> 
> ----------------------------------------------------------------------------
> # ./configure CC=gcc

Hmm, I would have thought configure would set up most of what you needed
below, but it's possible our autoconf is not very complete (unlike many
projects, it's not necessary to use it to build).

> Edit the Makefile - set the path to "ar", "gcc" and GNU "install".
> CC = /usr/local/bin/gcc
> AR = /usr/ccs/bin/ar
> INSTALL = /usr/local/bin/install
> Look for ifeq ($(uname_S),SunOS) and set INSTALL to /usr/lcoal/bin/install

Rather than edit the Makefile, you can put these in "config.mak" which
is sourced automatically by the Makefile (after the defaults are set up,
so you can override them).

> # gmake NO_TCLTK=1 NO_CURL=1 NO_PTHREADS=YesPlease install prefix=/opt/sfw
> # git --version
> git version 1.7.5
> ----------------------------------------------------------------------------
> Preliminary testing indicates a working system -- will investigate thourough 
> today and tomorrow.

Great, I'm glad it's working. With NO_PTHREADS set, you will be
fork()ing off the sideband process instead of using a thread. So you
avoid pthreads altogether and it works, but you can't do multi-threaded
delta compression.

If you build without NO_PTHREADS set on a Solaris 9 box, does it work on
that same box? That would confirm or deny my ABI compatibility theory
(and hopefully point us in the right direction for the sunfreeware
people to build a binary that works on Solaris 9 and 10).

-Peff

Re: Unable to fork off sideband demultiplexer

[Ben Walton]

Excerpts from Randy Brandenburg's message of Wed Jun 01 10:16:32 -0400 2011:

Hi Randy,

> This is a Solaris 9 installation (git-1.7.5-sol9-sparc-local.gz
> obtained from Sunfreeware.com) including all of the required package
> dependencies listed.  I followed the same install on a Solaris 10
> box and have no issues. Not sure if this is a permissions issue or
> something not configured correctly.

Have you considered the OpenCSW git[1] packages?  I'm a little stale
at 1.7.3 right now but I have 1.7.5 ready to go...

It's built with thread support and should work on both 9 and 10.

Thanks
-Ben

[1] http://opencsw.org/packages/git
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Ben Walton <bwalton <at> artsci.utoronto.ca> writes:

Hey Ben,

> 
> Have you considered the OpenCSW git[1] packages?  I'm a little stale
> at 1.7.3 right now but I have 1.7.5 ready to go...
> 

I did download the 1.7.3.2 version using pkg-get as the next option if I 
couldnt get the 1.7.5 to build and install. I would be all for a working 
package install if available, since we will have to install Git on a developer 
machine each time one is upgraded from a standard install of Solaris 9. I would 
like to add a package to our install instead of actually building from source 
each time.

Please keep me posted when the package is available for download!

Thanks,
Randy Brandenburg

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Jeff King <peff <at> peff.net> writes:

> 
> If you build without NO_PTHREADS set on a Solaris 9 box, does it work on
> that same box? That would confirm or deny my ABI compatibility theory
> (and hopefully point us in the right direction for the sunfreeware
> people to build a binary that works on Solaris 9 and 10).
> 
> -Peff
> 

I will attempt building without NO_PTHREADS set today along with the other 
suggestions about modifying the config.mak instead of the actual Makefile. Stay 
tuned.

- Randy

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Jeff King <peff <at> peff.net> writes:

> Great, I'm glad it's working. With NO_PTHREADS set, you will be
> fork()ing off the sideband process instead of using a thread. So you
> avoid pthreads altogether and it works, but you can't do multi-threaded
> delta compression.
> 
> If you build without NO_PTHREADS set on a Solaris 9 box, does it work on
> that same box? That would confirm or deny my ABI compatibility theory
> (and hopefully point us in the right direction for the sunfreeware
> people to build a binary that works on Solaris 9 and 10).
> 

Building without NO_PTHREADS set results in the original "broken" behaviour. I 
am staying with the working version in the interest of time. May investigate 
more later. 

Thanks for all the insight and tips.

- Randy

Re: Unable to fork off sideband demultiplexer

[Jeff King]

On Fri, Jun 03, 2011 at 01:30:00PM +0000, Randy Brandenburg wrote:

> > If you build without NO_PTHREADS set on a Solaris 9 box, does it
> > work on that same box? That would confirm or deny my ABI
> > compatibility theory (and hopefully point us in the right direction
> > for the sunfreeware people to build a binary that works on Solaris 9
> > and 10).
> > 
> 
> Building without NO_PTHREADS set results in the original "broken"
> behaviour. I am staying with the working version in the interest of
> time. May investigate more later.

Thanks, that's a good data point. I know you are probably out of time
for fooling with such things, but if you get a chance, could you try
building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the
problem is as simple as that.

-Peff

Re: Unable to fork off sideband demultiplexer

[Randy Brandenburg]

Jeff King <peff <at> peff.net> writes:

> 
> Thanks, that's a good data point. I know you are probably out of time
> for fooling with such things, but if you get a chance, could you try
> building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the
> problem is as simple as that.
> 
> -Peff
> 

OK, rebuilt with NO_PTHREADS=0 and PTHREAD_CFLAGS=-pthreads and it seems to be 
working. Just tested quickly for the basics (push and pull). I will be able to 
test more in depth next week, but I would assume that all will be okay.

Hope this helps.

-Randy

Re: Unable to fork off sideband demultiplexer

[Jeff King]

On Fri, Jun 03, 2011 at 06:52:32PM +0000, Randy Brandenburg wrote:

> OK, rebuilt with NO_PTHREADS=0 and PTHREAD_CFLAGS=-pthreads and it seems to be 
> working. Just tested quickly for the basics (push and pull). I will be able to 
> test more in depth next week, but I would assume that all will be
> okay.

I think that will still build without pthreads at all, as we check
"ifdef NO_PTHREADS" in the Makefile. You have to use "NO_PTHREADS=".

-Peff

Re: Unable to fork off sideband demultiplexer

[Brandon Casey]

On 06/03/2011 09:49 AM, Jeff King wrote:
> On Fri, Jun 03, 2011 at 01:30:00PM +0000, Randy Brandenburg wrote:
> 
>>> If you build without NO_PTHREADS set on a Solaris 9 box, does it
>>> work on that same box? That would confirm or deny my ABI
>>> compatibility theory (and hopefully point us in the right direction
>>> for the sunfreeware people to build a binary that works on Solaris 9
>>> and 10).
>>>
>>
>> Building without NO_PTHREADS set results in the original "broken"
>> behaviour. I am staying with the working version in the interest of
>> time. May investigate more later.
> 
> Thanks, that's a good data point. I know you are probably out of time
> for fooling with such things, but if you get a chance, could you try
> building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the
> problem is as simple as that.

I generally build on 5.7 using Sun WSPro compilers and then test that
binary on 5.7 and 5.9.  The latest that I have compiled is 1.7.5.3 and
I haven't seen this thread error.

Also note, I only do the above for testing purposes (and fun :). Solaris
is not a primary development platform, so it does not get a lot of
active testing in practice.  The 5.9 system doesn't even have a compiler
installed, so I can't try to duplicate this problem.

-Brandon

Re: Unable to fork off sideband demultiplexer

[Ben Walton]

Excerpts from Randy Brandenburg's message of Fri Jun 03 08:29:21 -0400 2011:

Hi Randy,

> I did download the 1.7.3.2 version using pkg-get as the next option

Try pkgutil.  It's much more capable. :)

> Please keep me posted when the package is available for download!

I'll push these for release shortly, but in the meantime, you can get
1.7.5.4 packages here:

http://buildfarm.opencsw.org/experimental.html#git

Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302