Re: Importing Mozilla CVS into git

[Keith Packard <keithp@keithp.com>]

[-- Attachment #1: Type: text/plain, Size: 2212 bytes --]

On Thu, 2006-06-01 at 20:55 -0400, Jon Smirl wrote:
> With the attached patch you can parse the entire Mozilla tree. The
> tree has over 100,000 files in it and about 300 branches.

that's good news.

> I ran this overnight and it failed with out of memory on a 1GB machine
> with 1GB swap. If failed in the branch processing, the parse phase
> succeeded.

yeah, parsecvs has some internal storage inefficiencies which need to be
addressed. In particular, every commit has a pointer to the related
revision of every file in the commit. Much like git used to store every
filename in the commit object and was changed to share common directory
contents, parsecvs should be fixed to do the same.

> How much memory does something like this need?

It's basically 4 * nrev * nfile bytes on a 32-bit machine, multiply by 2
for a 64-bit machine.

> If you want to quickly check out the branch processing problems use
> rsync to pull down just a copy of repository files.
> 
> I am getting 1000s of warnings like these and I haven't figured out why yet.
> 
> Warning: ../mozilla/mozilla/build/mac/build_scripts/Attic/MozillaCheckoutList.txt,v:
> unnamed branch from master
> Warning: ../mozilla/mozilla/build/unix/run-mozilla.sh,v: unnamed
> branch from master
> Warning: ../mozilla/mozilla/Makefile.in,v: unnamed branch from master of 99855
> Warning: ../mozilla/mozilla/Makefile.in,v: unnamed branch from master
> Warning: ../mozilla/mozilla/allmakefiles.sh,v: unnamed branch from master99855
> Warning: ../mozilla/mozilla/allmakefiles.sh,v: unnamed branch from master
> Warning: ../mozilla/mozilla/cmd/macfe/MailNews/AddressBook/Attic/UAddressBookUtilities.cp,v:
> unnamed branch from master
> Warning: ../mozilla/mozilla/cmd/macfe/MailNews/AddressBook/Attic/UAddressBookUtilities.h,v:
> unnamed branch from master
> Warning: ../mozilla/mozilla/cmd/macfe/central/Attic/msv2dsk.cp,v:
> unnamed branch from master

yeah, these happen when vendor branches go awry. I'll pull the
repository and take a look. X.org had similar 'issues' as the current
CVS repo was built by merging mesa, XFree86 and X.org together in a
rather haphazard fashion.

-- 
keith.packard@intel.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]