Re: Importing Mozilla CVS into git

["Jon Smirl" <jonsmirl@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]

With the attached patch you can parse the entire Mozilla tree. The
tree has over 100,000 files in it and about 300 branches.

I ran this overnight and it failed with out of memory on a 1GB machine
with 1GB swap. If failed in the branch processing, the parse phase
succeeded.

How much memory does something like this need?

If you want to quickly check out the branch processing problems use
rsync to pull down just a copy of repository files.

I am getting 1000s of warnings like these and I haven't figured out why yet.

Warning: ../mozilla/mozilla/build/mac/build_scripts/Attic/MozillaCheckoutList.txt,v:
unnamed branch from master
Warning: ../mozilla/mozilla/build/unix/run-mozilla.sh,v: unnamed
branch from master
Warning: ../mozilla/mozilla/Makefile.in,v: unnamed branch from master of 99855
Warning: ../mozilla/mozilla/Makefile.in,v: unnamed branch from master
Warning: ../mozilla/mozilla/allmakefiles.sh,v: unnamed branch from master99855
Warning: ../mozilla/mozilla/allmakefiles.sh,v: unnamed branch from master
Warning: ../mozilla/mozilla/cmd/macfe/MailNews/AddressBook/Attic/UAddressBookUtilities.cp,v:
unnamed branch from master
Warning: ../mozilla/mozilla/cmd/macfe/MailNews/AddressBook/Attic/UAddressBookUtilities.h,v:
unnamed branch from master
Warning: ../mozilla/mozilla/cmd/macfe/central/Attic/msv2dsk.cp,v:
unnamed branch from master

-- 
Jon Smirl
jonsmirl@gmail.com

[-- Attachment #2: patch --]
[-- Type: application/octet-stream, Size: 1618 bytes --]

diff --git a/gram.y b/gram.y
index 502991e..4566628 100644
--- a/gram.y
+++ b/gram.y
@@ -18,6 +18,7 @@
  */
 
 #include "cvs.h"
+#define YYDEBUG 1
     
 void yyerror (char *msg);
 
@@ -72,6 +73,7 @@ header		: HEAD NUMBER SEMI
 		| symbollist
 		  { this_file->symbols = $1; }
 		| LOCKS SEMI lock_type SEMI
+		| LOCKS NAME COLON NUMBER SEMI lock_type SEMI
 		| COMMENT DATA SEMI
 		| EXPAND DATA SEMI
 		;
@@ -91,6 +93,21 @@ symbol		: NAME COLON NUMBER
 			$$->name = $1;
 			$$->number = $3;
 		  }
+		| NUMBER COLON NUMBER
+		  {
+		  	char buffer[200];
+		  	int i, j;
+		  	
+		  	j = 0;
+		  	for (i = 0; i < $1.c; i++) {
+		  		if (i != 0)
+			  		j += sprintf(&buffer[j], ".");
+		  		j += sprintf(&buffer[j], "%d", $1.n[i]);
+		  	}
+			$$ = calloc (1, sizeof (cvs_symbol));
+			$$->name = atom(buffer);
+			$$->number = $3;
+		  }
 		;
 revisions	: revision revisions
 		  { $1->next = $2; $$ = $1; }
diff --git a/lex.l b/lex.l
index f25b8a5..956eed7 100644
--- a/lex.l
+++ b/lex.l
@@ -52,7 +52,7 @@ #define YY_INPUT(buf,result,max_size) { 
 					BEGIN(INITIAL);
 					return TEXT_DATA;
 				}
-<CONTENT>[-a-zA-Z_0-9+][-a-zA-Z_0-9+/]* {
+<CONTENT>[-a-zA-Z_+][-a-zA-Z_0-9+/.%]* {
 					yylval.s = atom (yytext);
 					return NAME;
 				}
diff --git a/parsecvs.c b/parsecvs.c
index df083c6..e9b8056 100644
--- a/parsecvs.c
+++ b/parsecvs.c
@@ -27,7 +27,9 @@ #endif
 
 cvs_file	*this_file;
 
-rev_execution_mode rev_mode = ExecuteGit;
+//rev_execution_mode rev_mode = ExecuteGit;
+//rev_execution_mode rev_mode = ExecuteGraph;
+rev_execution_mode rev_mode = ExecuteSplits;
 
 int elide = 0;
 int difffiles = 1;