test suite fails if sh != bash || tar != GNU tar

test suite fails if sh != bash || tar != GNU tar

[David Frech]

I've built git on several BSD (FreeBSD and DragonFlyBSD) systems, and
while it builds fine on both platforms, the test suite *fails* on
both.

I tracked most of the problems down to dependencies on bash and GNU
tar. I'm not sure yet what the specific differences are between BSD's
/bin/sh and bash (the two are separate in the BSD world), but it's
obvious what the problem is with tar: the git-archive test parses
datestamps printed out by tar, and the two tar's print datestamps
differently.

If I  set SHELL_PATH=/usr/local/bin/bash and TAR=gtar (the name of GNU
tar on BSD systems) I am able to get thru all the tests on FreeBSD
(this is with a checkout of the v1.5.2.3 tag).

On DragonFly there is another problem that I have not yet tracked
down. git-mailinfo fails on 0004 - it fails to extract *anything* from
the message.

Anyone running Linux or Mac OSX won't see these problems, because on
both platforms /bin/sh == bash and tar == GNU tar.

Cheers,

- David
-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: test suite fails if sh != bash || tar != GNU tar

[Johannes Schindelin]

Hi,

On Tue, 10 Jul 2007, David Frech wrote:

> Anyone running Linux or Mac OSX won't see these problems, because on 
> both platforms /bin/sh == bash and tar == GNU tar.

Actually, Ubuntu seems to ship with dash as the default shell, and we had 
quite some fallout from that change.

As for GNU tar, I think that this is easily fixable if you have access to 
non-GNU tar.  We do not use tar _ever_, except for the test suite, to 
verify that things are working as expected.

If you could put in some time to make the tests work, that would be 
awesome.

Ciao,
Dscho

Re: test suite fails if sh != bash || tar != GNU tar

[Linus Torvalds]

On Wed, 11 Jul 2007, Johannes Schindelin wrote:
> 
> Actually, Ubuntu seems to ship with dash as the default shell, and we had 
> quite some fallout from that change.

Well, part of it was really dash bugs, no? The whole "'shift' on an empty 
argument list" seemed to be a total and outright dash bug, and wasn't 
about relying on bash "features".

But we've had tons of issues with different spacing for "wc -l" etc, so 
yeah, the silly details matter even when there aren't outright bugs in 
tools.

> As for GNU tar, I think that this is easily fixable if you have access to 
> non-GNU tar.  We do not use tar _ever_, except for the test suite, to 
> verify that things are working as expected.
> 
> If you could put in some time to make the tests work, that would be 
> awesome.

Yeah, it would be a good thing to get rid of any bashisms, but to _keep_ 
it working somebody would then need to test every once in a while that it 
still works ;)

		Linus

Re: test suite fails if sh != bash || tar != GNU tar

[David Frech]

On 7/10/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Wed, 11 Jul 2007, Johannes Schindelin wrote:
> > As for GNU tar, I think that this is easily fixable if you have access to
> > non-GNU tar.  We do not use tar _ever_, except for the test suite, to
> > verify that things are working as expected.
> >
> > If you could put in some time to make the tests work, that would be
> > awesome.
>
> Yeah, it would be a good thing to get rid of any bashisms, but to _keep_
> it working somebody would then need to test every once in a while that it
> still works ;)

I'll see what I can do. As I'm planning on running git on both FreeBSD
and DragonFly for the forseeable future, and plan to track git's
evolution (running stable releases if not more bleeding-edge code), I
can run the test suite every time I build a new git.

Cheers,

- David

>
>                 Linus
>

-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: test suite fails if sh != bash || tar != GNU tar

[Johannes Schindelin]

Hi,

On Tue, 10 Jul 2007, David Frech wrote:

> On 7/10/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> > On Wed, 11 Jul 2007, Johannes Schindelin wrote:
> > > As for GNU tar, I think that this is easily fixable if you have 
> > > access to non-GNU tar.  We do not use tar _ever_, except for the 
> > > test suite, to verify that things are working as expected.
> > >
> > > If you could put in some time to make the tests work, that would be 
> > > awesome.
> > 
> > Yeah, it would be a good thing to get rid of any bashisms, but to 
> > _keep_ it working somebody would then need to test every once in a 
> > while that it still works ;)
> 
> I'll see what I can do. As I'm planning on running git on both FreeBSD 
> and DragonFly for the forseeable future, and plan to track git's 
> evolution (running stable releases if not more bleeding-edge code), I 
> can run the test suite every time I build a new git.

If you want to, I can help you setting up a nightly cron job to fetch what 
is the current "next", run the tests, and report failures by email.

Ciao,
Dscho

Re: test suite fails if sh != bash || tar != GNU tar

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I'll see what I can do. As I'm planning on running git on both FreeBSD 
>> and DragonFly for the forseeable future, and plan to track git's 
>> evolution (running stable releases if not more bleeding-edge code), I 
>> can run the test suite every time I build a new git.
>
> If you want to, I can help you setting up a nightly cron job to fetch what 
> is the current "next", run the tests, and report failures by email.

Wow.  Nightly builds of 'next' on various platforms would
actually be quite useful, especially from non Linux and non bash
world.

Re: test suite fails if sh != bash || tar != GNU tar

[David Frech]

On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> >> I'll see what I can do. As I'm planning on running git on both FreeBSD
> >> and DragonFly for the forseeable future, and plan to track git's
> >> evolution (running stable releases if not more bleeding-edge code), I
> >> can run the test suite every time I build a new git.
> >
> > If you want to, I can help you setting up a nightly cron job to fetch what
> > is the current "next", run the tests, and report failures by email.
>
> Wow.  Nightly builds of 'next' on various platforms would
> actually be quite useful, especially from non Linux and non bash
> world.

I found and fixed the shell issues. Once I've got a "fix" for tar I'll
send a patch. I think the BSD sh has a bug wrt to negating the return
code from a pipeline.

I'd be happy to do a nightly build on my DragonFly box, and that
should catch anything that also doesn't work for FreeBSD. The failure
modes were exactly the same - though the DFly box has an additional
iconv-related problem (with git-mailinfo) that I still haven't tracked
down...

Is there a canned script to get me started?

One issue is that my server is on dynamic IP, and my lame ISP (the
local telco) doesn't give me a proper SMTP relay - they want us to
send our mail via HTTP to MSN! Completely lame.

So sending mail can be an issue, if the receiver blocks mail from dynamic IPs.

- David

-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: test suite fails if sh != bash || tar != GNU tar

[Junio C Hamano]

"David Frech" <david@nimblemachines.com> writes:

> One issue is that my server is on dynamic IP, and my lame ISP (the
> local telco) doesn't give me a proper SMTP relay - they want us to
> send our mail via HTTP to MSN! Completely lame.
>
> So sending mail can be an issue, if the receiver blocks mail from dynamic IPs.

I think I heard gmail has incoming SMTP for its subscribers, and
I would not be surprised if other free e-mail providers have the
same.  Perhaps you can use one of them for this purpose?

Re: test suite fails if sh != bash || tar != GNU tar

[David Frech]

On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> "David Frech" <david@nimblemachines.com> writes:
>
> > One issue is that my server is on dynamic IP, and my lame ISP (the
> > local telco) doesn't give me a proper SMTP relay - they want us to
> > send our mail via HTTP to MSN! Completely lame.
> >
> > So sending mail can be an issue, if the receiver blocks mail from dynamic IPs.
>
> I think I heard gmail has incoming SMTP for its subscribers, and
> I would not be surprised if other free e-mail providers have the
> same.  Perhaps you can use one of them for this purpose?

Yes, I have managed to relay thru gmail. It was a bit hard to set up -
build Postfix with SASL, sniff the traffic to figure out who signed
gmail's SSL cert (Thawte), download CA cert - but I know how to do it.
;-)

The downside is, the gmail relay rewrites the headers in annoying
ways, so I'm not using it in general. If necessary I could relay just
the "build breakage" thru them.

Actually my incoming nimblemachines.com mail gets sent from my dynamic
IP thru normal (non-AUTH, non-SSL) SMTP to gmail, and they accept it
fine. So if I can send the build results to a gmail account, there are
no problems. Hmm - and that account could forward to the git list...

- David

-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: test suite fails if sh != bash || tar != GNU tar

[Johannes Schindelin]

Hi,

On Wed, 11 Jul 2007, David Frech wrote:

> On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > 
> > >> I'll see what I can do. As I'm planning on running git on both 
> > >> FreeBSD and DragonFly for the forseeable future, and plan to track 
> > >> git's evolution (running stable releases if not more bleeding-edge 
> > >> code), I can run the test suite every time I build a new git.
> > >
> > > If you want to, I can help you setting up a nightly cron job to 
> > > fetch what is the current "next", run the tests, and report failures 
> > > by email.
> > 
> > Wow.  Nightly builds of 'next' on various platforms would actually be 
> > quite useful, especially from non Linux and non bash world.
> 
> I found and fixed the shell issues. Once I've got a "fix" for tar I'll 
> send a patch. I think the BSD sh has a bug wrt to negating the return 
> code from a pipeline.

Cool!  Please be sure to give Documentation/SubmittingPatches a quick 
glance before sending, to avoid hassles for the reviewers.

> I'd be happy to do a nightly build on my DragonFly box, and that should 
> catch anything that also doesn't work for FreeBSD. The failure modes 
> were exactly the same - though the DFly box has an additional 
> iconv-related problem (with git-mailinfo) that I still haven't tracked 
> down...
> 
> Is there a canned script to get me started?

Well, I would have started from scratch, as in

5 0 * * *       (cd /xx/git && sh test-it.sh)

And in test-it.sh there could be something like

	(git pull origin next && make test > test-it.out 2>&1 ) || 
	some-script-that-sends-the-email.sh

> One issue is that my server is on dynamic IP, and my lame ISP (the local 
> telco) doesn't give me a proper SMTP relay - they want us to send our 
> mail via HTTP to MSN! Completely lame.

It is lame.

> So sending mail can be an issue, if the receiver blocks mail from 
> dynamic IPs.

But maybe you can just upload the status somewhere public?  Or ssh into a 
machine which allows you to send an email, with public key authentication?

Ciao,
Dscho