git default behavior seems odd from a Unix command line point of view

git default behavior seems odd from a Unix command line point of view

[Andrew Schein]

Hi all -

I am recently working within git after some experience with mercurial.
 I am observing what I believe to be an odd default behavior from the
perspective of UNIX command line tools.  I thought I'd share in case
this hasn't occurred to git maintainers or in case somebody has
developed good workaround practices.

ais@ace:bio[1]$ pwd
/home/ais/repo/nps/projects/bio
ais@ace:bio$ git status
# On branch master
nothing to commit (working directory clean)
ais@ace:bio[1]$ git commit -a
# On branch master
nothing to commit (working directory clean)
ais@ace:bio[1]$


The [1] in my prompt indicates the exit code of the git commands. What
I find odd is that even with the -q option, you get this verbose
output.  Also, you get  a non-zero exit status (which I would expect
only on a failure such as presence of an unresolved conflict).  My git
usage is to have a number of small repositories and use a shell script
to loop over them and perform a sync with a centralized server.
Having all this wordy output on a "no sync necessary" scenario seems
counter the desired properties of output only when work is taking
place or when an error occurs.

Have others developed git practices to sync a bunch or repositories
without all this verbose output on a "no change" scenario?

Andy

-- 
Andrew I. Schein
www.andrewschein.com

Re: git default behavior seems odd from a Unix command line point of view

[Junio C Hamano]

Andrew Schein <andrew@andrewschein.com> writes:

> ais@ace:bio$ git status
> # On branch master
> nothing to commit (working directory clean)
> ais@ace:bio[1]$ git commit -a

This is merely a historical accident that you cannot change.  "git status"
was originally a dual-purpose implementation detail of "git commit"
command to:

 (1) report if there is anything you can commit with its exit code; and

 (2) if there is something to be committed, show what will and what won't
     be committed.

For the purpose of (1), the calling command "commit" was told by the user
to make a commit, so the situation where there is nothing to commit was
signalled as an error.  The code conceptually looked like:

	(git-commit.sh)

	do the preparation
	if git-status
        then
        	# message "there is nothing to commit" already given
		revert the preparation
                exit 1
	fi
        really make a commit

These days, "git commit" has been rewritten and the logic to see if there
is anything to commit is built-in, but people still use "git status"
primarily for purpose (2), the listing of what's committed and what's
left.

Oh, and your example is wrong.  If you are going to commit with -a, and if
you want to see if such a "git commit" has actually something to commit,
then you should be giving -a to status as well, like:

	git status -a && git commit -a

Also, unless you ask explicitly, "git commit" will not make an empty
commit, so you can always omit the "git status -a &&" part and have "git
commit" itself do the checking.

Re: git default behavior seems odd from a Unix command line point of view

[Andrew Schein]

Hi Junio -

Thanks for your reply giving historical context.  The command line
examples I gave were intended to give examples of the output rather
than my own usage pattern.

Here is my actual usage pattern... from a file called "sync.sh"
(would love feedback on whether this is the gitly? way to solve my use
case):

# environment set up occurs before loop, I pull before "pushing" in an
attempt to prevent
# conflicts from being left on the shared repository.
for dir in ./* ; do
    if [ ! -d $dir ]      ; then continue ; fi #not a directory
    if [ ! -e $dir/.git ] ; then continue ; fi #not a git repo
    dir=`basename $dir`
    echo "syncing: $dir"
    set +e # commit returns an error if there is nothing to commit.
    (cd ./$dir ; git commit -a)
    set -e
    (cd ./$dir ; git pull $UP "$REPO/$dir" master)        #pull
    ssh $HOST "mkdir -p $DEST_CACHE/$LOC/$dir"  # these three lines
handle "push"
    rsync -rl --delete ./$dir/.git -e ssh "$DEST:$DEST_CACHE/$LOC/$dir/.git"
    ssh $DEST "(cd $LOC/$dir ; /tools/bin/git pull
$DEST_CACHE/$LOC/$dir/.git master)"

done

Re: git default behavior seems odd from a Unix command line point of view

[Jeff King]

On Tue, May 12, 2009 at 12:24:56PM -0400, Andrew Schein wrote:

> # environment set up occurs before loop, I pull before "pushing" in an
> attempt to prevent
> # conflicts from being left on the shared repository.
> for dir in ./* ; do
>     if [ ! -d $dir ]      ; then continue ; fi #not a directory
>     if [ ! -e $dir/.git ] ; then continue ; fi #not a git repo
>     dir=`basename $dir`
>     echo "syncing: $dir"
>     set +e # commit returns an error if there is nothing to commit.
>     (cd ./$dir ; git commit -a)
>     set -e

One trick to avoid playing with "set -e" is just:

  thing_whose_error_you_want_ignore || true

But that aside, it might be nice _not_ to ignore the result from "commit
-a", since it could also be warning you that it failed to correctly
commit. You probably want to do:

  git diff --quiet HEAD || git commit -a

That is "either there is nothing to commit, or we succeed in committing
everything". That of course has a race condition between the two
commands, though (i.e., if you modify files in between). If you want
something truly atomic, I think we would need "git commit" to
distinguish error codes between "I had nothing to commit" and "an error
occurred" (returning "1" right now could mean either).

>     (cd ./$dir ; git pull $UP "$REPO/$dir" master)        #pull
>     ssh $HOST "mkdir -p $DEST_CACHE/$LOC/$dir"  # these three lines
> handle "push"
>     rsync -rl --delete ./$dir/.git -e ssh "$DEST:$DEST_CACHE/$LOC/$dir/.git"
>     ssh $DEST "(cd $LOC/$dir ; /tools/bin/git pull
> $DEST_CACHE/$LOC/$dir/.git master)"

Pushing directly should be a lot more efficient in the case of repacking
(git will realize that the remote already has certain objects, but if
the filenames change, rsync will not).

-Peff

Re: git default behavior seems odd from a Unix command line point of view

[Daniel Barkalow]

On Tue, 12 May 2009, Andrew Schein wrote:

> Hi Junio -
> 
> Thanks for your reply giving historical context.  The command line
> examples I gave were intended to give examples of the output rather
> than my own usage pattern.
> 
> Here is my actual usage pattern... from a file called "sync.sh"
> (would love feedback on whether this is the gitly? way to solve my use
> case):

What *is* your use case? What you're doing seems nuts to me (like, you're 
going to send out files with this script that someone is in the middle of 
editting), but I don't know what you're trying to do.

> # environment set up occurs before loop, I pull before "pushing" in an
> attempt to prevent
> # conflicts from being left on the shared repository.
> for dir in ./* ; do
>     if [ ! -d $dir ]      ; then continue ; fi #not a directory
>     if [ ! -e $dir/.git ] ; then continue ; fi #not a git repo
>     dir=`basename $dir`
>     echo "syncing: $dir"
>     set +e # commit returns an error if there is nothing to commit.
>     (cd ./$dir ; git commit -a)
>     set -e
>     (cd ./$dir ; git pull $UP "$REPO/$dir" master)        #pull
>     ssh $HOST "mkdir -p $DEST_CACHE/$LOC/$dir"  # these three lines
> handle "push"
>     rsync -rl --delete ./$dir/.git -e ssh "$DEST:$DEST_CACHE/$LOC/$dir/.git"
>     ssh $DEST "(cd $LOC/$dir ; /tools/bin/git pull
> $DEST_CACHE/$LOC/$dir/.git master)"
> 
> done

What are you trying to avoid by not using "git push"? Why are you 
committing whatever changes happen to be in working directories? Are you 
intending to be able to handle non-trivial merges?

	-Daniel
*This .sig left intentionally blank*

Re: git default behavior seems odd from a Unix command line point of view

[Andrew Schein]

> What *is* your use case? What you're doing seems nuts to me (like, you're
> going to send out files with this script that someone is in the middle of
> editting), but I don't know what you're trying to do.

I am new to git... so my first instinct is to try to reproduce a work
flow that I know works with mercurial setup.  It is possible that the
concepts don't translate correctly.  Here goes...

I have a bunch of separate project-related repositories.  There are
very few users of the system.  Most of the time I am the only user.  I
want a system for syncing my local repositories to a single shared
repository.  For example some days I work on my laptop, and some days
from my desktop.  A third "shared/public" repository "on campus"
serves as an always available repository that anyone I collaborate
with can pull from.  Also it is backed up, and for this reason I
designate it the "shared" version.  So the purpose of the sync.sh
script is to synchronize the personal laptop/desktop repository to the
on-campus version.

Something I have learned from using mercurial in industry is that when
somebody messes up a "public repo" with conflicts they frequently
don't clean up the mess.  This can be a sign that they have not
learned the lessons of cleanliness rather than ill intent.  Otherwise
(and similarly) this messiness can be caused from not noticing that
they have left a mess.

The motivation of having a sync script that is run on each user's
local repository is to decrease the likelihood of a mess.  This is
achieved by first pulling from the common repository and resolving
conflicts _before_ "pushing" (note quotations) their changes to the
common repository.  There is a possibility of a race condition that
leaves a conflict on the shared repository, however the risk is
diminished.

Finally, I use "push" in quotes because actually the script uses only
uses the pull command.  This prevents proliferation of branches on the
shared repository.

Is there a better way to achieve this in git than the sync.sh script I
sent around?

Thanks,

Andy

-- 
Andrew I. Schein
www.andrewschein.com

Re: git default behavior seems odd from a Unix command line point of view

[Daniel Barkalow]

On Tue, 12 May 2009, Andrew Schein wrote:

> > What *is* your use case? What you're doing seems nuts to me (like, you're
> > going to send out files with this script that someone is in the middle of
> > editting), but I don't know what you're trying to do.
> 
> I am new to git... so my first instinct is to try to reproduce a work
> flow that I know works with mercurial setup.  It is possible that the
> concepts don't translate correctly.  Here goes...
> 
> I have a bunch of separate project-related repositories.  There are
> very few users of the system.  Most of the time I am the only user.  I
> want a system for syncing my local repositories to a single shared
> repository.  For example some days I work on my laptop, and some days
> from my desktop.  A third "shared/public" repository "on campus"
> serves as an always available repository that anyone I collaborate
> with can pull from.  Also it is backed up, and for this reason I
> designate it the "shared" version.  So the purpose of the sync.sh
> script is to synchronize the personal laptop/desktop repository to the
> on-campus version.

In general, you should probably not have the script run "commit", but run 
it by hand whenever you've finished making changes (and probably done at 
least a quick syntax test, if possible). Otherwise, you'll often get the 
situation where you get distracted halfway through modifying a line, save 
something by habit, and then find this state getting sent to the system 
that other people pull from, and they'll find that they can't build your 
latest code.

> Something I have learned from using mercurial in industry is that when
> somebody messes up a "public repo" with conflicts they frequently
> don't clean up the mess.  This can be a sign that they have not
> learned the lessons of cleanliness rather than ill intent.  Otherwise
> (and similarly) this messiness can be caused from not noticing that
> they have left a mess.

In the normal process of pushing changes to a shared repository, there is 
no possibility of leaving conflicts, because there's no way to get git to 
attempt a merge on push; the push simply fails, and you then pull, resolve 
any conflicts, commit the merge, and try again at pushing.

> The motivation of having a sync script that is run on each user's
> local repository is to decrease the likelihood of a mess.  This is
> achieved by first pulling from the common repository and resolving
> conflicts _before_ "pushing" (note quotations) their changes to the
> common repository.  There is a possibility of a race condition that
> leaves a conflict on the shared repository, however the risk is
> diminished.
> 
> Finally, I use "push" in quotes because actually the script uses only
> uses the pull command.  This prevents proliferation of branches on the
> shared repository.

It's better to configure the push refspec to do what you want, which seems 
to be "refs/heads/master:refs/heads/master", and use the fact that git 
push actually does what you're trying to do in a race-free and efficient 
way.

Also, the "refs/heads/*:refs/heads/*" refspec is only different in the 
case where you use branches other than "master", and seems like it would 
be useful for your use case; if you actually want to send everything you 
do to the shared location, you want to have all of the branches you've 
made. (That is, you get one branch on the shared repository per distinct 
name of a branch that you use on a local repository, not one "master" for 
each local repository's "master")

	-Daniel
*This .sig left intentionally blank*

Re: git default behavior seems odd from a Unix command line point of view

[Jeff King]

On Tue, May 12, 2009 at 11:18:25AM -0400, Andrew Schein wrote:

> The [1] in my prompt indicates the exit code of the git commands. What
> I find odd is that even with the -q option, you get this verbose
> output.  Also, you get  a non-zero exit status (which I would expect
> only on a failure such as presence of an unresolved conflict).  My git
> usage is to have a number of small repositories and use a shell script
> to loop over them and perform a sync with a centralized server.
> Having all this wordy output on a "no sync necessary" scenario seems
> counter the desired properties of output only when work is taking
> place or when an error occurs.
> 
> Have others developed git practices to sync a bunch or repositories
> without all this verbose output on a "no change" scenario?

Yes, I have such a script. I check:

  git ls-files -m -o -d --exclude-standard --directory --no-empty-directory

If it produces any output, then there is something to commit (either a
change in a tracked file, or an untracked file that might need to be
added).

I also do a fetch and check to see if we have any commits that need to
be merged:

  git rev-list master..origin

or any commits that we need to push:

  git rev-list origin..master

(actually, it is a bit more complicated, since "master" and "origin" are
just convention; I actually parse the config to find the branch pairs).

-Peff