[PATCH] git-svnimport.perl: fix for 'arg list too long...'

[PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Sasha Khapyorsky]

Hello,

This fixes 'arg list too long' problem when passed to git-ls-files.

Sasha.


This fixes 'arg list too long..' problem with git-ls-files.

Note that second arg list separation loop (with 'git-update-index') is
needed since git-ls-files arguments can be directories.

Signed-off-by: Sasha Khapyorsky <sashak@voltaire.com>

diff --git a/git-svnimport.perl b/git-svnimport.perl
index 6e3a44a..b6799d8 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -546,23 +546,30 @@ sub commit {
 			}
 		}
 
-		if(@old) {
-			open my $F, "-|", "git-ls-files", "-z", @old or die $!;
-			@old = ();
+		while(@old) {
+			my @o1;
+			if(@old > 55) {
+				@o1 = splice(@old,0,50);
+			} else {
+				@o1 = @old;
+				@old = ();
+			}
+			open my $F, "-|", "git-ls-files", "-z", @o1 or die $!;
+			@o1 = ();
 			local $/ = "\0";
 			while(<$F>) {
 				chomp;
-				push(@old,$_);
+				push(@o1,$_);
 			}
 			close($F);
 
-			while(@old) {
+			while(@o1) {
 				my @o2;
-				if(@old > 55) {
-					@o2 = splice(@old,0,50);
+				if(@o1 > 55) {
+					@o2 = splice(@o1,0,50);
 				} else {
-					@o2 = @old;
-					@old = ();
+					@o2 = @o1;
+					@o1 = ();
 				}
 				system("git-update-index","--force-remove","--",@o2);
 				die "Cannot remove files: $?\n" if $?;

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Junio C Hamano]

Sasha Khapyorsky <sashak@voltaire.com> writes:

> This fixes 'arg list too long..' problem with git-ls-files.

I wonder if feeding the list into a pipe to 'update-index --stdin' 
would be an option.

BTW, please do not do this:

        Mail-Followup-To: Junio C Hamano <junkio@cox.net>,
                Matthias Urlichs <smurf@smurf.noris.de>, git@vger.kernel.org

It is simply rude.

 * I and/or Smurf may be uninterested in hearing the whole
   discussion thread started by you.  We might well be, but that
   is not something you shoudl decide upfront in your message.

 * When a recipient of your message tries to "[R]eply" to *you*,
   that Mail-Followup-To: would cause the MUA to address that
   message to me and/or smurf by default, not to you.  I had to
   edit the resulting To: header by hand to address this
   response to you.  Why force extra work on others?

I know why you did it --- you subscribe to git list and
otherwise you would get two copies.

That is _not_ a good excuse.  That is how mailing lists work;
either you filter the duplicates on your end, or you learn to
live with it.  Please don't force extra work on others.

I think the only header mucking of this kind that is acceptable
is to remove yourself from CC: line when the mailing list you
subscribe to is already on the CC: line.

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Sasha Khapyorsky]

On 12:27 Wed 01 Feb     , Junio C Hamano wrote:
> Sasha Khapyorsky <sashak@voltaire.com> writes:
> 
> > This fixes 'arg list too long..' problem with git-ls-files.
> 
> I wonder if feeding the list into a pipe to 'update-index --stdin' 
> would be an option.

This can be option for git-update-index, but my problem was with
git-ls-files, git-update-index was ok (arg list processing were splitted
already).

> 
> BTW, please do not do this:
> 
>         Mail-Followup-To: Junio C Hamano <junkio@cox.net>,
>                 Matthias Urlichs <smurf@smurf.noris.de>, git@vger.kernel.org
> 
> It is simply rude.

OK, will not. Thanks for pointing.

Sasha.

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Junio C Hamano]

Sasha Khapyorsky <sashak@voltaire.com> writes:

> On 12:27 Wed 01 Feb     , Junio C Hamano wrote:
>> Sasha Khapyorsky <sashak@voltaire.com> writes:
>> 
>> > This fixes 'arg list too long..' problem with git-ls-files.
>> 
>> I wonder if feeding the list into a pipe to 'update-index --stdin' 
>> would be an option.
>
> This can be option for git-update-index, but my problem was with
> git-ls-files, git-update-index was ok (arg list processing were splitted
> already).

I was commenting on the "while (@o1)" loop that splices at 50
when the list has more than 55 items to feed update-index.  You
could accumulate output from multiple invocations of ls-files
and feed everything to a single "update-index --stdin" after the
"while (@old)" loop is done.  Batch of 50 vs a single batch may
not matter though [*1*].

But you are right; that git-ls-files can get too many arguments
unless you split like that.

A casual skimming over the rest of the code tells me that this
fixes the last instance of such a command invocation with too
many arguments.  Good catch.

Smurf, I do not have problems with Sasha's patch.  Are you OK
if I apply it?


-- >8 --

*1* I do not think this makes much of a difference but here is what
I mean.

diff --git a/git-svnimport.perl b/git-svnimport.perl
index b6799d8..d76a595 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -546,7 +546,8 @@ sub commit {
 			}
 		}
 
-		while(@old) {
+		my @u = ();
+		while (@old) {
 			my @o1;
 			if(@old > 55) {
 				@o1 = splice(@old,0,50);
@@ -555,26 +556,19 @@ sub commit {
 				@old = ();
 			}
 			open my $F, "-|", "git-ls-files", "-z", @o1 or die $!;
-			@o1 = ();
 			local $/ = "\0";
 			while(<$F>) {
 				chomp;
-				push(@o1,$_);
+				push(@u,$_);
 			}
 			close($F);
-
-			while(@o1) {
-				my @o2;
-				if(@o1 > 55) {
-					@o2 = splice(@o1,0,50);
-				} else {
-					@o2 = @o1;
-					@o1 = ();
-				}
-				system("git-update-index","--force-remove","--",@o2);
-				die "Cannot remove files: $?\n" if $?;
-			}
 		}
+		open my $F, "|-",
+			qw(git-update-index --force-remove -z --stdin)
+				or die $!;
+		print $F "$_\0" for @u;
+		close $F or die $!;
+
 		while(@new) {
 			my @n2;
 			if(@new > 12) {

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Matthias Urlichs]

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

Hi,

Junio C Hamano:
> Smurf, I do not have problems with Sasha's patch.  Are you OK
> if I apply it?
> 
Go ahead.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Circumstances can force a generalized incompetent to become competent, at
least in a specialized field.
					-- Frank R. Freemon

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

[Sasha Khapyorsky]

On 14:50 Wed 01 Feb     , Junio C Hamano wrote:
> 
> *1* I do not think this makes much of a difference but here is what
> I mean.
> 
> diff --git a/git-svnimport.perl b/git-svnimport.perl

Tested. Works good. I think your version is cleaner. Cosmetic comment btw:

> +		open my $F, "|-",
> +			qw(git-update-index --force-remove -z --stdin)
> +				or die $!;
> +		print $F "$_\0" for @u;
> +		close $F or die $!;

It is not critical, but this block may be under if(@u) {..} to prevent
"empty" git-update-index invocations.

Sasha.