"git add $ignored_file" fail

"git add $ignored_file" fail

[Santi]

Hi *,

      When you try to add ignored files with the git-add command it
fails because the call to:

git-ls-files -z \
        --exclude-from="$GIT_DIR/info/exclude" \
        --others --exclude-per-directory=.gitignore

      does not output this file because it is ignored. I know I can do it with:

git-update-index --add $ignored_file

I understand the behaviour of git-ls-files but I think it is no the
expected for git-add, at least for me.

    Thanks

    Santi

Re: "git add $ignored_file" fail

[Linus Torvalds]

On Wed, 17 May 2006, Santi wrote:
> 
>      When you try to add ignored files with the git-add command it
> fails because the call to:
> 
> git-ls-files -z \
>        --exclude-from="$GIT_DIR/info/exclude" \
>        --others --exclude-per-directory=.gitignore
> 
>      does not output this file because it is ignored. I know I can do it with:
> 
> git-update-index --add $ignored_file
> 
> I understand the behaviour of git-ls-files but I think it is no the
> expected for git-add, at least for me.

Well, the thing is, git-add doesn't really take a "file name", it takes a 
filename _pattern_.

Clearly we can't add everything that matches the pattern, because one 
common case is to add a whole subdirectory, and thus clearly the 
.gitignore file must override the pattern.

So it's consistent that it overrides it also for a single filename case, 
no?

		Linus

Re: "git add $ignored_file" fail

[Jakub Narebski]

Linus Torvalds wrote:

> Well, the thing is, git-add doesn't really take a "file name", it takes a 
> filename _pattern_.
> 
> Clearly we can't add everything that matches the pattern, because one 
> common case is to add a whole subdirectory, and thus clearly the 
> .gitignore file must override the pattern.
> 
> So it's consistent that it overrides it also for a single filename case, 
> no?

Well, if shell expansion cannot find a file matching pattern, it uses
pattern as file name literaly.

It would be nice to have easy (git core porcelain level) way to add files
which match ignore pattern.

-- 
Jakub Narebski
Warsaw, Poland

Re: "git add $ignored_file" fail

[Santi]

2006/5/17, Linus Torvalds <torvalds@osdl.org>:
>
>
> On Wed, 17 May 2006, Santi wrote:
> >
> >      When you try to add ignored files with the git-add command it
> > fails because the call to:
> >
> > git-ls-files -z \
> >        --exclude-from="$GIT_DIR/info/exclude" \
> >        --others --exclude-per-directory=.gitignore
> >
> >      does not output this file because it is ignored. I know I can do it with:
> >
> > git-update-index --add $ignored_file
> >
> > I understand the behaviour of git-ls-files but I think it is no the
> > expected for git-add, at least for me.
>
> Well, the thing is, git-add doesn't really take a "file name", it takes a
> filename _pattern_.
>
> Clearly we can't add everything that matches the pattern, because one
> common case is to add a whole subdirectory, and thus clearly the
> .gitignore file must override the pattern.
>
> So it's consistent that it overrides it also for a single filename case,
> no?
>

It's consistent from an implementation point of view, but not from the
(my?) user point of view. This is why I say I understand it for
git-ls-files. For the case of git-add even the usage and the man page
talk about <file>...

Clearly for the case of a whole subdirectory, or even ".",  the
.gitignore file must override the pattern, but not for the case of a
pattern that is a single existing file.

Santi

Re: "git add $ignored_file" fail

[Johannes Schindelin]

On Wed, 17 May 2006, Santi wrote:

> 2006/5/17, Linus Torvalds <torvalds@osdl.org>:
> > 
> > 
> > On Wed, 17 May 2006, Santi wrote:
> > >
> > >      When you try to add ignored files with the git-add command it
> > > fails [...]
> > >
> > [...]
> >
> > So it's consistent that it overrides it also for a single filename case,
> > no?
> > 
> 
> It's consistent from an implementation point of view, but not from the
> (my?) user point of view.

So, you told git to ignore the file. And then you say "please add it". I 
find it highly consistent that git does not do anything, because *you* 
decided it should ignore it in the first place.

Ciao,
Dscho

Re: "git add $ignored_file" fail

[Santi]

2006/5/17, Alex Riesen <raa.lkml@gmail.com>:
> On 5/17/06, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > It's consistent from an implementation point of view, but not from the
> > > (my?) user point of view.
> >
> > So, you told git to ignore the file. And then you say "please add it". I
> > find it highly consistent that git does not do anything, because *you*
> > decided it should ignore it in the first place.
> >
>
> Well, he didn't say to ignore exactly this file. And Santi didn't know
> git-add uses git-ls-files here. So it actually is unexpected for a novice.
> It was unexpected for me too, until I looked into git-add.sh
>

Actually I'm not a novice, but that is the point.

In the other way, now I find the value of being able to say:

$ git add t*

and be sure that it does not add an ignored file. Unfortunately
git-add cannot distinguish between both. So what I propose is to
document it explicitly, something like:

diff --git a/Documentation/git-add.txt /Documentation/git-add.txt
index 5e31129..42f1e33 100644
--- a/Documentation/git-add.txt
+++ b/Documentation/git-add.txt
@@ -7,18 +7,20 @@ git-add - Add files to the index file

 SYNOPSIS
 --------
-'git-add' [-n] [-v] [--] <file>...
+'git-add' [-n] [-v] [--] <filespec>...

 DESCRIPTION
 -----------
 A simple wrapper for git-update-index to add files to the index,
 for people used to do "cvs add".

+It only adds non-ignored files, to add ignored files use
+"git-update-index --add".

 OPTIONS
 -------
-<file>...::
-       Files to add to the index.
+<filespec>...::
+       Files to add to the index (see git-ls-files).

 -n::
         Don't actually add the file(s), just show if they exist.
diff --git a/git-add.sh b/git-add.sh
index d6a4bc7..394793f 100755
--- a/git-add.sh
+++ b/git-add.sh
@@ -1,6 +1,6 @@
 #!/bin/sh

-USAGE='[-n] [-v] <file>...'
+USAGE='[-n] [-v] <filespec>...'
 SUBDIRECTORY_OK='Yes'
 . git-sh-setup

Re: "git add $ignored_file" fail

[Jakub Narebski]

Santi wrote:

> In the other way, now I find the value of being able to say:
> 
> $ git add t*
> 
> and be sure that it does not add an ignored file. Unfortunately
> git-add cannot distinguish between both.

Well, it could. If 'git add <filespec>' would result in NO files 
added, take <filespec> as literate <file> (filename), regardless
of ignores.

-- 
Jakub Narebski
Warsaw, Poland

Re: "git add $ignored_file" fail

[Johannes Schindelin]

Hi,

On Wed, 17 May 2006, Santi wrote:

> In the other way, now I find the value of being able to say:
> 
> $ git add t*
> 
> and be sure that it does not add an ignored file. Unfortunately
> git-add cannot distinguish between both. So what I propose is to
> document it explicitly, something like:
> 
> [...]

How about a "--force" AKA "-f" flag that overrides the ignore file?

I would implement it if I had time, but I am gone for three days in about 
1 minute :-(

Ciao,
Dscho

Re: "git add $ignored_file" fail

[Linus Torvalds]

On Wed, 17 May 2006, Santi wrote:
> 
> In the other way, now I find the value of being able to say:
> 
> $ git add t*

Exactly. The shell will expand the path, so the fact that the path exists 
on the command line does not mean that "git add" should just trust it.

> and be sure that it does not add an ignored file. Unfortunately
> git-add cannot distinguish between both. So what I propose is to
> document it explicitly, something like:

Documenting this more clearly (and the workaround of forcing it with 
"git-update-index --add" by hand) is certainly a good idea.

			Linus

Re: "git add $ignored_file" fail

[Pavel Roskin]

On Wed, 2006-05-17 at 15:46 +0200, Jakub Narebski wrote:
> Santi wrote:
> 
> > In the other way, now I find the value of being able to say:
> > 
> > $ git add t*
> > 
> > and be sure that it does not add an ignored file. Unfortunately
> > git-add cannot distinguish between both.
> 
> Well, it could. If 'git add <filespec>' would result in NO files 
> added, take <filespec> as literate <file> (filename), regardless
> of ignores.

Can we apply the ignore rules to the directories but not the files?

This way, "git-add *" would add all files (rarely a good idea), whereas 
"git-add ." would respect the ignore rules.

Kludgy as it is, this approach would generally produce more expected
results than others.  If you let the shell expand the pattern, expect
all junk to be added.  If you let git expand the pattern, expect it to
adhere to the ignore rules.

-- 
Regards,
Pavel Roskin

Re: "git add $ignored_file" fail

[Sean]

On Wed, 17 May 2006 15:23:06 -0400
Pavel Roskin <proski@gnu.org> wrote:

> Can we apply the ignore rules to the directories but not the files?
> 
> This way, "git-add *" would add all files (rarely a good idea), whereas 
> "git-add ." would respect the ignore rules.
> 
> Kludgy as it is, this approach would generally produce more expected
> results than others.  If you let the shell expand the pattern, expect
> all junk to be added.  If you let git expand the pattern, expect it to
> adhere to the ignore rules. 

Shouldn't git just always respect the ignore rules?  Forcing someone to
remove a file from the .gitignore or employ the other work around
mentioned earlier doesn't seem too bad.  How often are people adding
files that are explicitly ignored?

Sean

Re: "git add $ignored_file" fail

[Jakub Narebski]

Sean wrote:


> Shouldn't git just always respect the ignore rules?  Forcing someone to
> remove a file from the .gitignore [...]

Or adding exclude rule (!filename) for specific file to .gitignore.

-- 
Jakub Narebski
Warsaw, Poland

Re: "git add $ignored_file" fail

[Pavel Roskin]

On Wed, 2006-05-17 at 15:39 -0400, Sean wrote:
> On Wed, 17 May 2006 15:23:06 -0400
> Pavel Roskin <proski@gnu.org> wrote:
> 
> Shouldn't git just always respect the ignore rules?  Forcing someone to
> remove a file from the .gitignore or employ the other work around
> mentioned earlier doesn't seem too bad.  How often are people adding
> files that are explicitly ignored?

That's a good idea!  And the implementation should be easy - if the file
is present, but git-ls-file doesn't show it, tell the user to
adjust .gitignore or to use some flag like --force.

Libification of git-ls-files would allow even more precise messages.

-- 
Regards,
Pavel Roskin

Re: "git add $ignored_file" fail

[Linus Torvalds]

On Wed, 17 May 2006, Pavel Roskin wrote:

> On Wed, 2006-05-17 at 15:39 -0400, Sean wrote:
> > On Wed, 17 May 2006 15:23:06 -0400
> > Pavel Roskin <proski@gnu.org> wrote:
> > 
> > Shouldn't git just always respect the ignore rules?  Forcing someone to
> > remove a file from the .gitignore or employ the other work around
> > mentioned earlier doesn't seem too bad.  How often are people adding
> > files that are explicitly ignored?
> 
> That's a good idea!  And the implementation should be easy - if the file
> is present, but git-ls-file doesn't show it, tell the user to
> adjust .gitignore or to use some flag like --force.

Umm. That's exactly the semantics for "git add" right now. We _always_ 
respect the ignore rules.

That was what people were complaining about.

Although I think Santi realized why we do it, and isn't even complaining 
any more. 

So we're all good again.

		Linus

Re: "git add $ignored_file" fail

[Jakub Narebski]

Linus Torvalds wrote:

> On Wed, 17 May 2006, Pavel Roskin wrote:

>> [...] implementation should be easy - if the file
>> is present, but git-ls-file doesn't show it, tell the user to
>> adjust .gitignore or to use some flag like --force.
> 
> Umm. That's exactly the semantics for "git add" right now. We _always_ 
> respect the ignore rules.
> 
> That was what people were complaining about.
> 
> Although I think Santi realized why we do it, and isn't even complaining 
> any more. 
> 
> So we're all good again.

The changes in docummentation are nice and dandy, but it would be even nicer
if "git add" told us about "git update-index --add" when it adds no files,
as it is certainly the case when something is wrong (perhaps user
expectations, but still...).

-- 
Jakub Narebski
Warsaw, Poland

Re: "git add $ignored_file" fail

[Linus Torvalds]

On Wed, 17 May 2006, Jakub Narebski wrote:
> 
> The changes in docummentation are nice and dandy, but it would be even nicer
> if "git add" told us about "git update-index --add" when it adds no files,
> as it is certainly the case when something is wrong (perhaps user
> expectations, but still...).

Well, with the new-and-improved builtin "git add", you could probably do 
something like the appended (on top of my most recent patch).

It says

	No added files - did you perhaps mean to do a 'git update-index'?

whenever it finds that "git add" has ignored a file and not actually added 
anything. That, btw, can happen either because it refused to see the file 
in the first place (ie it was ignored), or because all the files listed 
were already added.

In both cases the warning may or may not be sensible.

Anyway, I dunno. I don't have any strong opinions on this.

		Linus

---
diff --git a/builtin-add.c b/builtin-add.c
index 82e8f44..8641137 100644
--- a/builtin-add.c
+++ b/builtin-add.c
@@ -12,6 +12,8 @@ #include "dir.h"
 static const char builtin_add_usage[] =
 "git-add [-n] [-v] <filepattern>...";
 
+static int ignored;
+
 static int common_prefix(const char **pathspec)
 {
 	const char *path, *slash, *next;
@@ -123,8 +125,10 @@ static void prune_directory(struct dir_s
 
 		/* Existing file? We must have ignored it */
 		match = pathspec[i];
-		if (!lstat(match, &st))
+		if (!lstat(match, &st)) {
+			ignored = 1;
 			continue;
+		}
 		die("pathspec '%s' did not match any files", match);
 	}
 }
@@ -257,6 +261,9 @@ int cmd_add(int argc, const char **argv,
 	for (i = 0; i < dir.nr; i++)
 		add_file_to_index(dir.entries[i]->name, verbose);
 
+	if (ignored && !active_cache_changed)
+		fprintf(stderr, "No added files - did you perhaps mean to do a 'git update-index'?\n");
+
 	if (active_cache_changed) {
 		if (write_cache(newfd, active_cache, active_nr) ||
 		    commit_index_file(&cache_file))

Re: "git add $ignored_file" fail

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Well, with the new-and-improved builtin "git add", you could probably do 
> something like the appended (on top of my most recent patch).
>
> It says
>
> 	No added files - did you perhaps mean to do a 'git update-index'?
>
> whenever it finds that "git add" has ignored a file and not actually added 
> anything. That, btw, can happen either because it refused to see the file 
> in the first place (ie it was ignored), or because all the files listed 
> were already added.
>
> In both cases the warning may or may not be sensible.
>
> Anyway, I dunno. I don't have any strong opinions on this.

If you give a pattern that would match two files but one of them
were hidden by .gitignore, this approach would not help you
much.

Even if we wanted to say something like "if the user explicitly
tells us to add etc/mtab~ by spelling it out, we should ignore
*~ entry in .gitignore", the shell expansion bites us because it
is done before we get to what the user give us.  We cannot
distinguish that with the user typing etc/?tab* for example.

If somebody (Jakub, perhaps?) cares strong enough, we could show
by default "matched the pathspec but ignored by .gitignore"
paths with fprintf(stderr, "ignoring '%s'\n"), and have an
option -q to squelch it.

I do not have strong feeling on this, so I'll see if somebody
comes up with a better implementation.

Re: "git add $ignored_file" fail

[Linus Torvalds]

On Wed, 17 May 2006, Junio C Hamano wrote:
> 
> If you give a pattern that would match two files but one of them
> were hidden by .gitignore, this approach would not help you
> much.

Correct. On the other hand, what could you do?

I think that the common case for that is literally something like

	git add dirname/

or

	git add file*

and it turns out that there are object files under the directory, or that 
there's a file.c, file.h _and_ a already-compiled file.o file.

Under both of those circumstances (one pattern that matches multiple files 
but ignores others - or several different patterns where _some_ of the 
patterns are ignored), we actually do what I think is the only sane thing. 

Namely to just silently add everything that makes sense to add.

> Even if we wanted to say something like "if the user explicitly
> tells us to add etc/mtab~ by spelling it out, we should ignore
> *~ entry in .gitignore", the shell expansion bites us because it
> is done before we get to what the user give us.  We cannot
> distinguish that with the user typing etc/?tab* for example.

Right.

The only case that we cando anything at all about is really the case where 
we didn't add anything at all, and then we might reasonably ask "do you 
know what the heck you're doing".

That's kind of what my last patch did. It's a total special case, but it's 
the _only_ special case that I can see that is at all relevant (ie in all 
other cases it would just be annoying as _hell_ if we started talking 
about how we're ignoring object files. Of _course_ we're ignoring them, 
and that's why they are listed in .gitignore).

So I'd love to have the built-in "git add", but quite frankly, if you drop 
that last patch as "too ugly to live", I certainly won't complain. I sent 
it out more as a "we -could- do this" thing rather than anything more 
serious.

			Linus