using git for file management while writing a thesis...

using git for file management while writing a thesis...

[Jordan Miller]

Hello,

Thank you to the developers for Git!

I am using git 1.5.3.1 on OS X 10.5.2 for file versioning for LaTeX  
files while writing my thesis. I have looked at other possible ways to  
manage text files while writing a thesis, but I think git is the best  
one available.

I am a bit confused on how best to create backups of my commits on  
external hard drives and USB keys.

After I had created my git repository on my local hard drive, I  
connected an external hard drive and a USB key and peformed a "git  
clone" from local to external. Everything was very quick.

Now, every day I stage files and perform a commit on my local drive to  
my local repository. Then, I connect my external hard drives and  
perform a "git pull" from each of their repositories on the repository  
on my local drive to update these external hard drive repositories and  
also checkout the latest files to these external repositories.

The problem I am having is as follows:
Everything works beautifully and incredibly speedily on my external  
3.5" hard drive connected via Firewire.
On my USB keys, a huge number of changes are seen and git takes a very  
long time assessing ("deltifying") what are the changes needed (more  
than 10 times the number of files are deltified!). Shouldn't git just  
realize that it only needs to make the changes that were made in the  
last commit, or am I using the "pull" command incorrectly?

Why would it take so long for a "pull" command to happen on the USB2  
key while it is extremely fast on an external drive? I have also tried  
wiping the USB key and issuing a new "git clone" command, and that  
happens extremely fast. But, shouldn't it be even faster to pull only  
the latest commit onto the USB key?

Finally, I have also tried changing the disk formatting of the USB key  
to try to diagnose the problem. However, the problem is the same  
whether the format of the USB key is HFS+ Journaled or MS-DOS FAT32.

So, my question is what am I doing wrong with "git pull" and is there  
a better way to use git for the task at hand? Unfortunately, I have  
not yet been able to find a solution anywhere on the  
interwebnetblagosphere.


thanks in advance,
Jordan

Re: using git for file management while writing a thesis...

[David Tweed]

On Thu, Apr 3, 2008 at 9:58 PM, Jordan Miller <jmil@rice.edu> wrote:
>  I am using git 1.5.3.1 on OS X 10.5.2 for file versioning for LaTeX files
[snip]
>  Everything works beautifully and incredibly speedily on my external 3.5"
> hard drive connected via Firewire.
>  On my USB keys, a huge number of changes are seen and git takes a very long
> time assessing ("deltifying") what are the changes needed (more than 10
> times the number of files are deltified!). Shouldn't git just realize that
> it only needs to make the changes that were made in the last commit, or am
>  Finally, I have also tried changing the disk formatting of the USB key to
> try to diagnose the problem. However, the problem is the same whether the
> format of the USB key is HFS+ Journaled or MS-DOS FAT32.
>
>  So, my question is what am I doing wrong with "git pull" and is there a
> better way to use git for the task at hand? Unfortunately, I have not yet
> been able to find a solution anywhere on the interwebnetblagosphere.

Not really directly relevant, but since no-one has replied: I daily
copy several revisions onto (git push) and back from (git pull) to USB
key (MS-DOS) on x86-Linux and it never takes more than a couple of
seconds. (Repo is around 17MB packed, maybe 10-400 objects updated per
push.) I know nothing about OS X, but the discrepancy between firewire
and usb suggests some performance issue in usb handling. I don't know
off the top of my head if packs on the receiveing end of a push are
mmap()'d (to find branch heads?), but OS X is said to have poor mmap
performance: maybe it interacts with usb driver to be even worse?

Anyway, only suggestion I've got is if you've got easy access to a
Linux machine with git available, try pushing from that and see if the
speed differs.

There have been reports of people using git when writing books, and I
use git to track papers I'm writing (amongst other things), so your
usage pattern is entirely normal.

HTH,
-- 
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"while having code so boring anyone can maintain it, use Python." --
attempted insult seen on slashdot