[PATCH] Ignore file filter

[PATCH] Ignore file filter

[David Greaves]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Hi Petr

This is an inline filter that introduces the concept of .git/ignore
It is intended to be used within the cogito scripts like the other cg-X*
files.


Signed-off-by: David Greaves <david@dgreaves.com>

-- 


[-- Attachment #2: cg-Xignore --]
[-- Type: text/plain, Size: 2346 bytes --]

#!/usr/bin/env bash
#
# Takes a list of files on stdin and only passes valid ones agording to .git/ignore
# Copyright (c) David Greaves, 2005
#
# This filter implements cogito ignore rules and should typically be used in find pipelines
#
# Synopsis
# cg-Xignore [-debug] [-f] [-h] [-d] < file-list >useful-file-list
#
# Options
# -debug::
#     produce helpful debug output
#
# -q::
#     don't say what paths are ignored
#
# -f::
#     passes files
#
# -d::
#     passes directories
#
# -h::
#     passes symbolic links
#
# The default is to pass all file types that are not ignored.
#
# Note that the .git/ignore file contains multiple expressions, 1 per line
# Lines beginning with a '#' are ignored (allowing comments)
# These are 'bash regular expressions' not glob patterns
# This allows ignore rules to take the directory into account
# Suggested contents:
#   # bash regexps (not globs)
#   ^\.[^/]
#   /\.
#   /$
#   .*\.o$


# This doesn't allow the -h which is the [ arg for symlinks...
#. ${COGITO_LIB}cg-Xlib
_git=${GIT_DIR:-.git}

IGNORE_FILE="$_git/ignore"

if [ "$1" = "-0" ]; then
    # doesn't work :(
    zerosep=$'-d "\0"'
    shift
fi

# Defaults
pass_files=0
pass_dirs=0
pass_links=0
pass_all=1
while [ $# -gt 0 ]; do
    case $1 in
	"-f")
	    pass_all=0
	    pass_files=1
	    ;;
	"-d")
	    pass_all=0
	    pass_dirs=1
	    ;;
	"-h")
	    pass_all=0
	    pass_links=1
	    ;;
	"-q")
	    quiet=1
	    ;;
	"-debug")
	    debug=1
	    ;;
	esac
    shift
done

# save stderr
exec 5>&2
if [ $quiet ]; then
    # turn off noise
    exec 2>&-
fi

if [ $debug ]; then
    exec 4>&5
else
    exec 4>/dev/null
fi


# Strip out the common leading ./ allowing "find ."
sed 's:^./::' | \
while read $zerosep file; do
    echo "consider file: $file" >&4
    ignore=0
    if [ -f $IGNORE_FILE ]; then
	exec 3<$IGNORE_FILE
	while read -r -u3 patt ; do
	    if [[ $patt =~ "^\w*#" ]]; then
		continue
	    fi
	    echo "consider pattern: $patt" >&4
	    if [[ $file =~ $patt ]]; then
		ignore=1
		echo "Ignoring $file because of $patt" >&2
		break
	    fi
	done
    fi
    echo "passing file: $file" >&4
    
    if [ $ignore != "1" \
	-a \( $pass_all -eq 1 \
	   -o \( $pass_files -eq 1 -a -f $file \) \
	   -o \( $pass_dirs  -eq 1 -a -d $file \) \
	   -o \( $pass_links -eq 1 -a -h $file \) \
	   \) \
	]; then
	echo $file
    fi
done

Re: [PATCH] Ignore file filter

[David Greaves]

Junio C Hamano wrote:

>Just a half theoretical question.  How well does this perform
>when your filenames have:
>
>    - ' '  (ASCII 0x20)
>    - '\t' (ASCII 0x09)
>    - '\n' (ASCII 0x0a)
>    - '`'  (ASCII 0x60) backtick
>    - '$'  (ASCII 0x24) dollar sign
>
>in them?  Or is it the case that the rest of the Cogito is not
>careful enough and it does not matter to be careful only in this
>script?
>  
>
That's what the:
zerosep=$'-d "\0"'
is for.
It looks like the shell 'read' doesn't honour it though - I couldn't
make it work.

However, thanks, after I fixed the quotes I missed in:
-o \( $pass_files -eq 1 -a -f "$file" \) \
-o \( $pass_dirs -eq 1 -a -d "$file" \) \
-o \( $pass_links -eq 1 -a -h "$file" \) \

it handles all cases above except \n (of course)

Frankly I think we're beyond shell programming and we should be using
perl (IMHO as the 'best' and most portable text handler)
It also has a *fantastic* test harness available.

David

Re: [PATCH] Ignore file filter

[Petr Baudis]

Dear diary, on Thu, May 12, 2005 at 11:30:32PM CEST, I got a letter
where David Greaves <david@dgreaves.com> told me that...
> # This doesn't allow the -h which is the [ arg for symlinks...

But so is -L. And I'd just use -l...

> #. ${COGITO_LIB}cg-Xlib
> _git=${GIT_DIR:-.git}

...but it makes no sense anyway I think to reinclude this stuff from a
cg-Xfile you are including from other scripts anyway.

> 	    if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
We're already bash-only, but further reducing that to bash3 really won't
work. I *might* get convinced to add some bash2+-only feature, but only
if you'll be really good at explaining that it makes sense.

Besides, I'd prefer just the shell globs in the ignore file, as it is
done in the rest of the world, and in all the real-world scenarios I've
seen, the globs were powerful enough.

Also, how does this interact with git-ls-files --exclude and
.git/exclude? We would have two ignoring mechanisms...

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH] Ignore file filter

[David Greaves]

Petr Baudis wrote:

>Dear diary, on Thu, May 12, 2005 at 11:30:32PM CEST, I got a letter
>where David Greaves <david@dgreaves.com> told me that...
>  
>
>># This doesn't allow the -h which is the [ arg for symlinks...
>>    
>>
>
>But so is -L. And I'd just use -l...
>  
>
OK

>>#. ${COGITO_LIB}cg-Xlib
>>_git=${GIT_DIR:-.git}
>>    
>>
>
>...but it makes no sense anyway I think to reinclude this stuff from a
>cg-Xfile you are including from other scripts anyway.
>  
>
cg-Xignore isn't included - only called.
it's also just a library program.
Also I don't think cg-Xlib should be doing arg handling.
As an include it should provide an arg handling function that the
scripts call.

>>	    if [[ $file =~ $patt ]]; then
>>    
>>
>
>I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
>We're already bash-only, but further reducing that to bash3 really won't
>work. I *might* get convinced to add some bash2+-only feature, but only
>if you'll be really good at explaining that it makes sense.
>  
>
Ah
OK
I don't know how to do that.
I was actually aiming for glob matching when I came upon this in the
manpage.
I just thought it was bash and didn't think to check what version it was
introduced with.

>Besides, I'd prefer just the shell globs in the ignore file, as it is
>done in the rest of the world, and in all the real-world scenarios I've
>seen, the globs were powerful enough.
>
>Also, how does this interact with git-ls-files --exclude and
>.git/exclude? We would have two ignoring mechanisms...
>
>  
>
because one was cogito's and one was git's.
Cogitos was supposed to have a more powerful, pattern based abroach.

David

-- 

Re: [PATCH] Ignore file filter

[Junio C Hamano]

>>>>> "DG" == David Greaves <david@dgreaves.com> writes:

>>> if [[ $file =~ $patt ]]; then
>> 
>> I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
DG> OK
DG> I don't know how to do that.

Is that regexp or shell glob?  If regexp, expr is your friend,
like this:

    if expr "$file" : "$patt" >/dev/null; then

For a glob:

    patt='?.sh'
    file=1.sh

    case "$file" in
    $patt)
            echo Yeah ;;
    *)
            echo No ;;
    esac

Re: [PATCH] Ignore file filter

[Petr Baudis]

Dear diary, on Sat, May 14, 2005 at 10:28:24AM CEST, I got a letter
where David Greaves <david@dgreaves.com> told me that...
> >>#. ${COGITO_LIB}cg-Xlib
> >>_git=${GIT_DIR:-.git}
> >>    
> >>
> >
> >...but it makes no sense anyway I think to reinclude this stuff from a
> >cg-Xfile you are including from other scripts anyway.
> >  
> >
> cg-Xignore isn't included - only called.

Oh yes, I'm stupid.

> it's also just a library program.
> Also I don't think cg-Xlib should be doing arg handling.
> As an include it should provide an arg handling function that the
> scripts call.

I'd prefer the few and scattered users which don't want arg handling to
explicitly set some magic variable before calling cg-Xlib rather than
adding the arg parser function call everywhere else.

> >>	    if [[ $file =~ $patt ]]; then
> >>    
> >>
> >
> >I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
> >We're already bash-only, but further reducing that to bash3 really won't
> >work. I *might* get convinced to add some bash2+-only feature, but only
> >if you'll be really good at explaining that it makes sense.
> >  
> >
> Ah
> OK
> I don't know how to do that.
> I was actually aiming for glob matching when I came upon this in the
> manpage.

Ok, so what's the outcome? Are you going to stop at this point, or will
you change the scripts so that they use the glob list?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH] Ignore file filter

[Petr Baudis]

Dear diary, on Sat, May 14, 2005 at 11:01:49AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "DG" == David Greaves <david@dgreaves.com> writes:
> 
> >>> if [[ $file =~ $patt ]]; then
> >> 
> >> I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
> DG> OK
> DG> I don't know how to do that.
> 
> Is that regexp or shell glob?  If regexp, expr is your friend,
> like this:
> 
>     if expr "$file" : "$patt" >/dev/null; then

Oh, this looks nice. I didn't know expr can do that. :-)

Still, I'd prefer the old-fashioned globs as primary matching mechanism.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH] Ignore file filter

[David Greaves]

Petr Baudis wrote:

>Dear diary, on Sat, May 14, 2005 at 10:28:24AM CEST, I got a letter
>where David Greaves <david@dgreaves.com> told me that...
>  
>
>>Also I don't think cg-Xlib should be doing arg handling.
>>As an include it should provide an arg handling function that the
>>scripts call.
>>    
>>
>
>I'd prefer the few and scattered users which don't want arg handling to
>explicitly set some magic variable before calling cg-Xlib rather than
>adding the arg parser function call everywhere else.
>  
>
OK

>>>>	    if [[ $file =~ $patt ]]; then
>>>>        
>>>>
>>>I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
>>>We're already bash-only, but further reducing that to bash3 really won't
>>>work. I *might* get convinced to add some bash2+-only feature, but only
>>>if you'll be really good at explaining that it makes sense.
>>>      
>>>
>>OK
>>I don't know how to do that.
>>I was actually aiming for glob matching when I came upon this in the
>>manpage.
>>    
>>
>Ok, so what's the outcome? Are you going to stop at this point, or will
>you change the scripts so that they use the glob list?
>  
>
Well, Junio solved that for me - I'll gather the comments and resubmit.

David

-- 

Re: [PATCH] Ignore file filter

[David Greaves]

Petr Baudis wrote:

>Dear diary, on Sat, May 14, 2005 at 11:01:49AM CEST, I got a letter
>where Junio C Hamano <junkio@cox.net> told me that...
>  
>
>>>>>>>"DG" == David Greaves <david@dgreaves.com> writes:
>>>>>>>              
>>>>>>>
>>>>>if [[ $file =~ $patt ]]; then
>>>>>          
>>>>>
>>>>I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.
>>>>        
>>>>
>>DG> OK
>>DG> I don't know how to do that.
>>
>>Is that regexp or shell glob?  If regexp, expr is your friend,
>>like this:
>>
>>    if expr "$file" : "$patt" >/dev/null; then
>>    
>>
>
>Oh, this looks nice. I didn't know expr can do that. :-)
>
>Still, I'd prefer the old-fashioned globs as primary matching mechanism.
>  
>
OK
I was wondering about supporting _both_ globs and re's
right now my ignore file has a # to precede comment lines
maybe re: precedes regexp lines and unadorned lines are globs.

However the re's provided by regex(7) are too weedy to be worth
bothering with.
If however, there is a serious plan to go to perl, it may be worth
providing for this now in the ignore syntax.

Additionally this causes problems with sharing the same exclude file as
used by git.
However...
I really think git's exclude file capability and cogito's are different.
Cogito is aiming to provide full-blown SCM capabilities - git isn't

I am also concerned that a centralised ignore file is not flexible enough.
Certainly limiting if we support globs only.
It may be that you want different rules in different trees - someone on
lkml mentioned that excludes vary in different parts of the source.
Eg .s files may be generally ignored - but not in the asm parts of the tree.

Also... you haven't mentioned perl for a while - can you give us an update?
I personally think we're making life needlessly unpleasant by sticking
with shell.


David

-- 

[RFD] Ignore rules

[Petr Baudis]

Here, I would like more people to speak up, plaese, especially the
authors of other layers over git than Cogito, since I think it'd be
great if we could agree on common ignore rules format and we could just
call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.


Dear diary, on Sat, May 14, 2005 at 05:13:08PM CEST, I got a letter
where David Greaves <david@dgreaves.com> told me that...
> I was wondering about supporting _both_ globs and re's
> right now my ignore file has a # to precede comment lines

I assume \# will override that?

> maybe re: precedes regexp lines and unadorned lines are globs.

Or maybe even /, which is the common regexp prefix anyway...

To mention it in this mail too, I think leading '!' should do the
"ignore exclude" - that is, it would override any possible previous
ignore decisions about the file. E.g. '!*' would throw away all
previously applied ignore rules.

> However the re's provided by regex(7) are too weedy to be worth
> bothering with.
> If however, there is a serious plan to go to perl, it may be worth
> providing for this now in the ignore syntax.
> 
> Also... you haven't mentioned perl for a while - can you give us an update?
> I personally think we're making life needlessly unpleasant by sticking
> with shell.

If there is still a serious plan, it is much more long-term now, since
shell turns out to keep doing fine and everything we need, and that all
reasonably fast.

That said, I think it's fine to use Perl regexps. I think they rule. :-)
But what do others think? Should we stick with POSIX regexps (I assume
at least extended instead of basic), or go with Perl regexps?

> Additionally this causes problems with sharing the same exclude file as
> used by git.
> However...
> I really think git's exclude file capability and cogito's are different.
> Cogito is aiming to provide full-blown SCM capabilities - git isn't

If we get to agree on some common format, I'm thinking whether it
wouldn't be actually good to extend the --exclude option to support it.
How much of an issue would that be? What do others think?

> I am also concerned that a centralised ignore file is not flexible enough.
> Certainly limiting if we support globs only.
> It may be that you want different rules in different trees - someone on
> lkml mentioned that excludes vary in different parts of the source.
> Eg .s files may be generally ignored - but not in the asm parts of the tree.

I imagine it as (ignore rules applied in this order):

<default ignore list>:

Some builtin ignore list catching files like *.o, *.a and such.
Remember that you can throw it away with !* if you don't like it.

/.git/ignore:

Per-repository ignore list, not version tracked etc; really a local thing.

/.gitignore
/**/.gitignore:

(Applied in the order from the project root to the current directory.)
Version tracked ignore list, which concerns the current directory, BUT
may match pathnames instead of just filenames (but no ..). That is, you
could do something like

	echo '*.o' >.gitignore

to ignore all the object files in the current directory, and

	echo '**.o' >.gitignore

to also ignore the object files in all the subdirectories.


Opinions?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [RFD] Ignore rules

[David Greaves]

Petr Baudis wrote:

>Here, I would like more people to speak up, plaese, especially the
>authors of other layers over git than Cogito, since I think it'd be
>great if we could agree on common ignore rules format and we could just
>call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.
>
>
>Dear diary, on Sat, May 14, 2005 at 05:13:08PM CEST, I got a letter
>where David Greaves <david@dgreaves.com> told me that...
>  
>
>>I was wondering about supporting _both_ globs and re's
>>right now my ignore file has a # to precede comment lines
>>    
>>
>
>I assume \# will override that?
>  
>
doesn't match /^\w#/ so yes

>>maybe re: precedes regexp lines and unadorned lines are globs.
>>    
>>
>Or maybe even /, which is the common regexp prefix anyway...
>  
>
I thought about it but thought it would look strange since we won't want
a closing /
OTOH, no globs will start with a / since they have to be relative so
maybe a good choice after all.

>To mention it in this mail too, I think leading '!' should do the
>"ignore exclude" - that is, it would override any possible previous
>ignore decisions about the file. E.g. '!*' would throw away all
>previously applied ignore rules.
>  
>
Well, CVS has a lone ! meaning just that.
I wonder if we should simply say that most patterns are 'ignore' patterns.
'!' patterns are 'accept' patterns.
The last seen pattern wins

so given:
*.o
!fre*.o
freddy.o

frog.o is ignored
fred.o is seen
freddy.o is ignored

This means !* (or is that !**) has the same effect as a lone ! in CVS.

>>However the re's provided by regex(7) are too weedy to be worth
>>bothering with.
>>If however, there is a serious plan to go to perl, it may be worth
>>providing for this now in the ignore syntax.
>>
>>Also... you haven't mentioned perl for a while - can you give us an update?
>>I personally think we're making life needlessly unpleasant by sticking
>>with shell.
>>    
>>
>
>If there is still a serious plan, it is much more long-term now, since
>shell turns out to keep doing fine and everything we need, and that all
>reasonably fast.
>
>That said, I think it's fine to use Perl regexps. I think they rule. :-)
>But what do others think? Should we stick with POSIX regexps (I assume
>at least extended instead of basic), or go with Perl regexps?
>  
>
I vote perl re's
You have the power if needed but most of the time it'll be basic regexps.
I think if we have a central 'path' matching .git/ignore then regexps
are needed.
Then if we have them, we may as well allow them.
OTOH: A plethora of .gitignore files with the !negation mechanism may
make globs adequate.

>>Additionally this causes problems with sharing the same exclude file as
>>used by git.
>>However...
>>I really think git's exclude file capability and cogito's are different.
>>Cogito is aiming to provide full-blown SCM capabilities - git isn't
>>    
>>
>
>If we get to agree on some common format, I'm thinking whether it
>wouldn't be actually good to extend the --exclude option to support it.
>How much of an issue would that be? What do others think?
>  
>
I'm not convinced it needs to be extended.
It's trivial to take git-ls-files and filter it's results.
The rest of git will need filtering.

>>I am also concerned that a centralised ignore file is not flexible enough.
>>Certainly limiting if we support globs only.
>>It may be that you want different rules in different trees - someone on
>>lkml mentioned that excludes vary in different parts of the source.
>>Eg .s files may be generally ignored - but not in the asm parts of the tree.
>>    
>>
>
>I imagine it as (ignore rules applied in this order):
>
><default ignore list>:
>
>Some builtin ignore list catching files like *.o, *.a and such.
>  
>
Why builtin?
Why not have a default setup for .git/ignore
The way I'd do it is in cg-init :
[ -f cogito.ignore ] && mv cogito.ignore .git/ignore
[ ! -f .git/ignore ] && cat <<STD_IGNORE > .git/ignore

>Remember that you can throw it away with !* if you don't like it.
>  
>
but it means reading the man pages to find it out whilst you're editing
.git/gitignore
I could never remember what CVS ignored...

>/.git/ignore:
>
>Per-repository ignore list, not version tracked etc; really a local thing.
>  
>
~/.gitignore ?? CVS has it - personal prefs for your editor's strange
numbered backups etc

>/.gitignore
>/**/.gitignore:
>
>(Applied in the order from the project root to the current directory.)
>  
>
>Version tracked ignore list, which concerns the current directory, BUT
>may match pathnames instead of just filenames (but no ..). That is, you
>could do something like
>
>	echo '*.o' >.gitignore
>
>to ignore all the object files in the current directory, and
>
>	echo '**.o' >.gitignore
>
>to also ignore the object files in all the subdirectories.
>  
>
well, is the matching against files or paths?
And do .gitignores affect their sub-tree or just their dir?
And if they're vc'ed then git needs to stop ignoring .files

>
>Opinions?
>  
>

I think the global ignores contain both:
* regexps against the path relative to the project root
* regexps against the filename
* shell globs against the filename
I think the .gitignore's operate on a per-tree basis (so their directory
and lower)
I think .gitignore matching should be against filenames (not paths)

# comments
/regexp
!/inverted regexp
shellglob
!inverted shellglob




Lesson from the CVS manual:
Specifying `-I !' to `cvs import' will import everything, which is
generally what you want to do if you are importing files from a
pristine distribution or any other source which is known to not contain
any extraneous files. However, looking at the rules above you will see
there is a fly in the ointment; if the distribution contains any
`.cvsignore' files, then the patterns from those files will be
processed even if `-I !' is specified. The only workaround is to
remove the `.cvsignore' files in order to do the import. Because this
is awkward, in the future `-I !' might be modified to override
`.cvsignore' files in each directory.





-- 

Re: [RFD] Ignore rules

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

PB> Here, I would like more people to speak up, plaese, especially the
PB> authors of other layers over git than Cogito, since I think it'd be
PB> great if we could agree on common ignore rules format and we could just
PB> call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.

Purely off the top of my head, without even regurgitating what
you wrote in the message I am responding to (sorry, I am about
to leave for the day).

 * Two lists, one propagated with merges and another repository
   private one.

 * GIT_DIR/info/ignore is the one propagated with merge but it
   is not a list itself.  It records the name of a file that is
   GIT tracked.  GIT_DIR/ignore is the repository private one.

 * Repository private one and then merge propagated one are read
   in order, one line at a time.  The first hit determines
   whether it is taken or ignored.

 * Each entry in the list is not a shell glob but a regexp.  The
   ignore list rarely changes, so more expressiveness with a bit
   higher learning curve and a bit more typing would not hurt
   much.  The entry is implicitly anchored at the left and
   matched against the path relative to what is recorded in
   GIT_INDEX_FILE (so 'Makefile' matches "Makefile" in
   linux-2.6.git/ but not "fs/Makefile").  The regexp can
   optionally prefixed with a '!' to mean negation.  A line that
   starts with a '#' is a comment.

Re: [RFD] Ignore rules

[Jon Seymour]

Is there value in:

a. pushing the ignore logic into the core git tools such as git-ls-files

b. including the current ignore .* rule as a default ignore rule that
can be overridden by a .gitignore file


jon.
-- 
homepage: http://www.zeta.org.au/~jon/
blog: http://orwelliantremors.blogspot.com/

Re: [RFD] Ignore rules

[Junio C Hamano]

>>>>> "JS" == Jon Seymour <jon.seymour@gmail.com> writes:

JS> Is there value in:

JS> a. pushing the ignore logic into the core git tools such as git-ls-files

When there is an agreed upon Porcelain layer ignore logic,
git-ls-files --others --exclude-from=... should be changed so
that it uses the same file format and same semantics, and
probably use the default ignore file without being explicitly
told.  Otherwise things would get quite confusing, so yes, I see
value in there.

I do not however think this should apply to things like
"git-update-cache --add"; because even though user may say
"git-update-cache --add *" and wish it to ignore things on the
ignore list, that is not how shell parameter expansion works.

I would like to see a git-path-helper command that can act as
filter between "find -print0" and "xargs -0" like this:

    find * -print0 |
    git-path-helper -z --ignore-file=.git/info/ignore |
    xargs -r -0 git-update-cache --add --

I envision that we should not even need --ignore-file parameter,
once we have "an agreed upon Porcelain layer ignore logic".  It
should read the "agreed upon" location, somewhere under the
$GIT_DIR and perform the "agreed upon" filtering logic.

I further envision that this would work from anywhere in the
work tree, not only from the directory that corresponds to the
top of the tree structure GIT_INDEX_FILE describes.  For
example, in linux-2.6 git tree, you _ought_ to be able to say
something like this:

    cd fs
    find ext? ../include/linux -type f -print0 |
    git-path-helper -z |
    xargs -r -0 git-update-cache --add --

The git-path-helper command could internally run getpwd() and
find out the top directory (in this case, the parent directory
of our current working directory "fs"), then canonicalize the
incoming filenames (either relative to getpwd() or the full
filesystem path) to paths relative to the top directory, apply
the ignore filter and send the surviving paths downstream.
Anything downstream driven with xargs -0 would get relative
paths suitable for core GIT consumption.

JS> b. including the current ignore .* rule as a default ignore rule that
JS> can be overridden by a .gitignore file

Knowing the number of places that assume .* are irrelevant, I
would not be looking forward to doing that myself, but that
behaviour would be ideal.

Re: [RFD] Ignore rules

[Junio C Hamano]

>>>>> "JCH" == Junio C Hamano <junkio@cox.net> writes:

JCH> ...  For example, in linux-2.6 git tree, you _ought_ to be
JCH> able to say something like this:

JCH>     cd fs
JCH>     find ext? ../include/linux -type f -print0 |
JCH>     git-path-helper -z |
JCH>     xargs -r -0 git-update-cache --add --

The above example has an obvious thinko/typo.  It should have
been like this:

     cd fs
     find ext? ../include/linux -type f -print0 |
     git-path-helper -z | {
         cd .. && xargs -r -0 git-update-cache --add --
     }

[RFD] git-run-with-user-path

[Junio C Hamano]

Petr, I have not written the implementation yet, but do you
think Cogito would find something like this useful?  I've seen
some patch to add running from subdirectory support to cg-add in
the mailing list and I assume Cogito is in the process of being
enhanced in this area.

JIT has been supporting running from the subdirectory from the
day one, but has been using a bit different mechanism.  If we
can agree on a core-ish helper (I consider this just as non-core
as git-diff-helper is), I would want to switch to use something
like this one.

This _deliberately_ conflates the user-path canonicalization
with the ignore-list processing discussed recently on the list.
When you think about it, they are very closely related and
handling them at the same place makes more sense than having
them separately.  It is a way to munge paths given by users and
"find -print0 | xargs -0" into a form appropriate for core GIT
consumption. Similar reasoning with which you accepted the use
of cg-Xignore by David Greaves in cg-commit.

Linus, and everybody else, suggestions and comments?

------------

git-run-with-user-path(1)
=========================
v0.1, May 2005

NAME
----
git-run-with-user-path - Run command from the top after canonicalizing paths.


SYNOPSIS
--------
'git-run-with-user-path' [options] <command> <argument>... '--' <path>...

DESCRIPTION
-----------
This command takes a <command>, zero or more <argument> and zero
or more <path> arguments.  <path> arguments name objects on the
filesystem, <command> is typically a core GIT command, and
<argument> are the initial arguments to the <command>.

It first finds the project top directory (the directory that
corresponds to the top of the tree structure GIT_INDEX_FILE
describes), and canonicalizes the given <path> to be relative to
the project top.  It then chdir(2)'s to the project top
directory and runs the given <command>, with <argument> and
these canonicalized <path> arguments.

This is useful for the Porcelain layer to run core GIT commands
from subdirectories.  For example, if linux-2.6.git tree is
checked out in /usr/src/linux, you can do:

    $ cd /usr/src/linux/fs
    $ ... work in fs directory making changes ...
    $ git-run-with-user-path git-diff-tree -r HEAD -- ext? ../include/linux
    $ find ext? ! -type d -print0 |
      xargs -0 git-run-with-user-path git-update-cache --add -- --

The above is roughly equivalent to:

    $ cd /usr/src/linux
    $ git-diff-tree -r HEAD fs/ext? include/linux
    $ find fs/ext? include/linux ! -type d -print0 |
      xargs git-update-cache --add --

OPTIONS
-------
--no-ignore::

	By default, the path arguments are filtered with the
	same ignore rules Porcelain layers use.  With
	--no-ignore flag, there is no such filtering done.

Author
------
Written by Junio C Hamano <junkio@cox.net>

Documentation
--------------
Documentation by Junio C Hamano.

GIT
---
Part of the link:git.html[git] suite

Re: [RFD] Ignore rules

[Matthias Urlichs]

Hi, Jon Seymour wrote:

> a. pushing the ignore logic into the core git tools such as git-ls-files
> 
> b. including the current ignore .* rule as a default ignore rule that
> can be overridden by a .gitignore file

I'd say YES to both.

My preferred ignore file logic would be:

- stop at first match (that's more efficient)
- !pattern prevents exclusion of matching files
- bash-style shell globs, except that ...
  - a pattern that starts with / is a regexp
  - * doesn't cross directory boundaries, but ** does
- I don't need a per-repository (i.e. non-checked-in/propagated)
  ignore file.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de

Re: [RFD] Ignore rules

[David Greaves]

Matthias Urlichs wrote:

>Hi, Jon Seymour wrote:
>
>  
>
>>a. pushing the ignore logic into the core git tools such as git-ls-files
>>
>>b. including the current ignore .* rule as a default ignore rule that
>>can be overridden by a .gitignore file
>>    
>>
>
>I'd say YES to both.
>
>My preferred ignore file logic would be:
>
>- stop at first match (that's more efficient)
>  
>
more efficient true - but then surely 98% of the time you have to check
_all_ patterns since files aren't generally ignored.
And the ability to override earlier matches makes life much easier.
So I say no shortcuts, last pattern to match decides ignore/accept status

>- !pattern prevents exclusion of matching files
>- bash-style shell globs, except that ...
>  - a pattern that starts with / is a regexp
>  - * doesn't cross directory boundaries, but ** does
>  
>
>- I don't need a per-repository (i.e. non-checked-in/propagated)
>  ignore file.
>
I agree.
But for the sake of checking a couple of files it makes sense to define
a complete set of locations.

David


--