Interpretation of '/' changed sparse-checkout

Interpretation of '/' changed sparse-checkout

[Caleb Marchent]

I have a system where I checkout by default everything except one directory. 
Then the rules in the top-level sources can selectivly reintroduce 
subdirectories as required for the build target.
I have deployed, using git 1.7.2.5, a set of git trees which use the following 
sparse-checkout rules to achieve this.

/
!exclude/
exclude/reinclude

I have however discovered that if the reincluded directory is the same as the 
first part of any other path then that path will also be checked out.

/
!exlcude
exclude/wanted

will checkout both:

exclude/wanted
exclude/wanted_not

This appears to have bee fixed in git 1.7.10.2 but from that version of git 
onwards using '/' to request everything and then exclude a specific directory 
from it no longer works.

While I have been able to contrive the same effect with recent versions of git 
using the following updated sparse-checkout rules

/*
!exclude/
exclude/reinclude

Even with this work-around there are two problems:

1) If I have a path: exclude/reinclude/exclude, this will now be exluded where I 
would expect it to be included as it was with 1.7.2.5

2) I have a large number of deployed trees across a number of machines that have 
the existing format which will break when git operations are performed after a 
upgrade to the latest git.

I believe that the interpretation of '/' should be 'everything in the repo', 
from which later rules can exclude. I think this is a bug in recent versions of 
git. I have been working on some ideas for a fix to the git code based along 
these lines. Can anyone confirm if my analysis is correct or provide an 
alternate solution?

Also I would like to know if anyone knows the solution to point 1) above as this 
also blocks upgrading to the latest version of git.

Re: Interpretation of '/' changed sparse-checkout

[Caleb Marchent]

Caleb Marchent <cmarchent <at> aminocom.com> writes:

Corrected cut-paste errors on first submission, seems I missed a bunch of 
terminating '/' when hand-copying the text in from the sparse-checkout files :-( 
Sorry.

I have a system where I checkout by default everything except one directory. 
Then the rules in the top-level sources can selectivly reintroduce 
subdirectories as required for the build target.
I have deployed, using git 1.7.2.5, a set of git trees which use the following 
sparse-checkout rules to achieve this.

/
!exclude/
exclude/reinclude/


I have however discovered that if the reincluded directory is the same as the 
first part of any other path then that path will also be checked out.
 
/
!exlcude/
exclude/wanted/

 
will checkout both:

exclude/wanted/
exclude/wanted_not/


This appears to have bee fixed in git 1.7.10.2 but from that version of git 
onwards using '/' to request everything and then exclude a specific directory 
from it no longer works.
 
While I have been able to contrive the same effect with recent versions of git 
using the following updated sparse-checkout rules
 
/*
!exclude/
exclude/reinclude/

 
Even with this work-around there are two problems:

1) If I have a path: exclude/reinclude/exclude, this will now be exluded where 
I would expect it to be included as it was with 1.7.2.5
 
2) I have a large number of deployed trees across a number of machines that 
have the existing format which will break when git operations are performed 
after a upgrade to the latest git.
 
I believe that the interpretation of '/' should be 'everything in the repo', 
from which later rules can exclude. I think this is a bug in recent versions 
of git. I have been working on some ideas for a fix to the git code based along 
these lines. Can anyone confirm if my analysis is correct or provide an 
alternate solution?
 
Also I would like to know if anyone knows the solution to point 1) above as 
this also blocks upgrading to the latest version of git.
 
 

Re: Interpretation of '/' changed sparse-checkout

[Nguyen Thai Ngoc Duy]

Hi,

On Wed, May 23, 2012 at 02:09:24PM +0000, Caleb Marchent wrote:
> Caleb Marchent <cmarchent <at> aminocom.com> writes:
> 
> Corrected cut-paste errors on first submission, seems I missed a bunch of 
> terminating '/' when hand-copying the text in from the sparse-checkout files :-( 
> Sorry.
> 
> I have a system where I checkout by default everything except one directory. 
> Then the rules in the top-level sources can selectivly reintroduce 
> subdirectories as required for the build target.
> I have deployed, using git 1.7.2.5, a set of git trees which use the following 
> sparse-checkout rules to achieve this.
> 
> /
> !exclude/
> exclude/reinclude/
> 
> 
> I have however discovered that if the reincluded directory is the same as the 
> first part of any other path then that path will also be checked out.
>  
> /
> !exlcude/
> exclude/wanted/
> 
>  
> will checkout both:
> 
> exclude/wanted/
> exclude/wanted_not/
> 
> 
> This appears to have bee fixed in git 1.7.10.2 but from that version of git 
> onwards using '/' to request everything and then exclude a specific directory 
> from it no longer works.

As a user I agree that "/" should mean "everything under the sky" and
your use case should work (i.e. it is broken in 1.7.10.2). If you can
build git, you can try the patch below. I'll submit a patch for
inclusion to git later (perhaps with optimization for this special case).

-- 8< --
diff --git a/unpack-trees.c b/unpack-trees.c
index bcee99c..188c196 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -838,7 +838,8 @@ static int clear_ce_flags_dir(struct cache_entry **cache, int nr,
 	int dtype = DT_DIR;
 	int ret = excluded_from_list(prefix, prefix_len, basename, &dtype, el);
 
-	prefix[prefix_len++] = '/';
+	if (prefix_len)
+		prefix[prefix_len++] = '/';
 
 	/* If undecided, use matching result of parent dir in defval */
 	if (ret < 0)
@@ -953,10 +954,12 @@ static int clear_ce_flags(struct cache_entry **cache, int nr,
 			    struct exclude_list *el)
 {
 	char prefix[PATH_MAX];
-	return clear_ce_flags_1(cache, nr,
-				prefix, 0,
-				select_mask, clear_mask,
-				el, 0);
+	*prefix = '\0';
+	return clear_ce_flags_dir(cache, nr,
+				  prefix, 0,
+				  "",
+				  select_mask, clear_mask,
+				  el, 0);
 }
 
 /*

-- 8< --

>  
> While I have been able to contrive the same effect with recent versions of git 
> using the following updated sparse-checkout rules
>  
> /*
> !exclude/
> exclude/reinclude/
> 
>  
> Even with this work-around there are two problems:
> 
> 1) If I have a path: exclude/reinclude/exclude, this will now be exluded where 
> I would expect it to be included as it was with 1.7.2.5
>  
> 2) I have a large number of deployed trees across a number of machines that 
> have the existing format which will break when git operations are performed 
> after a upgrade to the latest git.
>  
> I believe that the interpretation of '/' should be 'everything in the repo', 
> from which later rules can exclude. I think this is a bug in recent versions 
> of git. I have been working on some ideas for a fix to the git code based along 
> these lines. Can anyone confirm if my analysis is correct or provide an 
> alternate solution?
>  
> Also I would like to know if anyone knows the solution to point 1) above as 
> this also blocks upgrading to the latest version of git.

-- 
Duy

Re: Interpretation of '/' changed sparse-checkout

[Caleb Marchent]

Nguyen Thai Ngoc Duy <pclouds <at> gmail.com> writes:

> 
> As a user I agree that "/" should mean "everything under the sky" and
> your use case should work (i.e. it is broken in 1.7.10.2). If you can
> build git, you can try the patch below. I'll submit a patch for
> inclusion to git later (perhaps with optimization for this special case).
> 

Many Thanks Duy!, that patch works so please do submit the patch.


That just leaves the other issue I noted in my original posting - if I 
exclude a path from the top-level with a 1.7.2.5 format sparse-checkout
 file, git 1.7.10.2 will exclude a sub-directory with the same name:

/
!exclude/

Unfortunately (for me), I think this is correct behaviour of 1.7.10.2 
because in 1.7.10.2 you can now do: 

/
!/exclude/
/exlucde/reinclude/

which makes sense and correctly excludes "/exclude", but not 
"/exclude/reinclude/exclude". The flexibility to have:

/
!CVS/

or something like that is required; so I don't believe should be changed?