How to go to git from svn without checkout

How to go to git from svn without checkout

[Mercedes6s]

Hello.  I've been falling more and more in love with git and I want to move
our biggest project because it is my biggest pain over to git to make our
lives easier.  The problem is our project is about 50 gigs (A lot of media
files involved).  Also, we got developers all over the world that are
running on slow connections.  Some took a week to get the project up and
running in the first place.  Luckily, they are only isolated instances. 
What I'm wondering is can I have all of them turn their projects into git
repositories locally and have all of them sync with a master repository once
they are done.  The actual project is only about 500 megs and the changes
are probably only a few K since they keep their projects up to date, so
that's more acceptable, but I don't want them to bring down all those media
files again and I doubt they will be willing to do it.  Is this possible,
and how would I do it?  

thanks
-- 
View this message in context: http://www.nabble.com/How-to-go-to-git-from-svn-without-checkout-tp22640020p22640020.html
Sent from the git mailing list archive at Nabble.com.

Re: How to go to git from svn without checkout

[Jeff King]

On Sat, Mar 21, 2009 at 01:25:38PM -0700, Mercedes6s wrote:

> Hello.  I've been falling more and more in love with git and I want to move
> our biggest project because it is my biggest pain over to git to make our
> lives easier.  The problem is our project is about 50 gigs (A lot of media
> files involved).  Also, we got developers all over the world that are
> running on slow connections.  Some took a week to get the project up and
> running in the first place.  Luckily, they are only isolated instances. 
> What I'm wondering is can I have all of them turn their projects into git
> repositories locally and have all of them sync with a master repository once
> they are done.  The actual project is only about 500 megs and the changes
> are probably only a few K since they keep their projects up to date, so
> that's more acceptable, but I don't want them to bring down all those media
> files again and I doubt they will be willing to do it.  Is this possible,
> and how would I do it?

I'm not sure you can. "git svn clone" will give a stable commit ID in
git; that is, two runs of the same import will yield interoperable git
repositories. However, I'm not sure if that is actually useful to you;
the people with svn checkouts don't _have_ the history. So the clone
operation will have to talk to the server.

However, you should really consider whether you want those 50G in the
git repository at all. Git is pretty good at not looking at parts of the
repository that aren't necessary to an operation, but whole-repository
operations like packing and cloning are going to be absurdly slow.

Are those files actually changing? Would it be feasible to put the
"main" part of the project in git and just include something like
symlinks in the repository pointing to your media? Then each local
developer could clone the git project and just move their existing 50G
of media files into place.

-Peff

RE: How to go to git from svn without checkout

[Samman, Bassel]

Yes, unfortunately they do change occasionally and they grow with time.  It's a product catalogue, so new products are added all the time, just not daily.  That's part of why I want to move to Git, SVN has been giving me lots of trouble when adding files at deep levels and I like how Git does adding files and how each repository is really it's own repository.  Also, I'm not really concerned about history as the main purpose of the repository is really to make the syncing job easier.  The code is very minimal and I do the majority of the maintenance on the code, so I can easily jump back to svn if I need history.  The main reason we even need to share the project is that the sales guys need to go without internet at times and they need to have the website locally on their machines so they can easily demo the latest products to their clients.  Business requirements by sales guys can be crazy at times, but what can we do.  Also, this is a far fetch, since all the projects are installed under the same directories and are really exact clones of each other in every way, can I copy the .git directory and hope to God that it will magically work.  I really doubt it, but I figure it's worth asking.

-----Original Message-----
From: Jeff King [mailto:peff@peff.net] 
Sent: Saturday, March 21, 2009 10:20 PM
To: Samman, Bassel
Cc: git@vger.kernel.org
Subject: Re: How to go to git from svn without checkout

On Sat, Mar 21, 2009 at 01:25:38PM -0700, Mercedes6s wrote:

> Hello.  I've been falling more and more in love with git and I want to move
> our biggest project because it is my biggest pain over to git to make our
> lives easier.  The problem is our project is about 50 gigs (A lot of media
> files involved).  Also, we got developers all over the world that are
> running on slow connections.  Some took a week to get the project up and
> running in the first place.  Luckily, they are only isolated instances. 
> What I'm wondering is can I have all of them turn their projects into git
> repositories locally and have all of them sync with a master repository once
> they are done.  The actual project is only about 500 megs and the changes
> are probably only a few K since they keep their projects up to date, so
> that's more acceptable, but I don't want them to bring down all those media
> files again and I doubt they will be willing to do it.  Is this possible,
> and how would I do it?

I'm not sure you can. "git svn clone" will give a stable commit ID in
git; that is, two runs of the same import will yield interoperable git
repositories. However, I'm not sure if that is actually useful to you;
the people with svn checkouts don't _have_ the history. So the clone
operation will have to talk to the server.

However, you should really consider whether you want those 50G in the
git repository at all. Git is pretty good at not looking at parts of the
repository that aren't necessary to an operation, but whole-repository
operations like packing and cloning are going to be absurdly slow.

Are those files actually changing? Would it be feasible to put the
"main" part of the project in git and just include something like
symlinks in the repository pointing to your media? Then each local
developer could clone the git project and just move their existing 50G
of media files into place.

-Peff

Re: How to go to git from svn without checkout

[Jeff King]

On Mon, Mar 23, 2009 at 10:21:41AM -0500, Samman, Bassel wrote:

> Yes, unfortunately they do change occasionally and they grow with
> time.  It's a product catalogue, so new products are added all the
> time, just not daily.  That's part of why I want to move to Git, SVN

Well, I would first try just importing _one_ repo into git and seeing
how it behaves. I suspect you may run into assumptions that a 50G repo
breaks.

In theory, I think you could pack all of the large files once into a
single pack, mark it with .keep, and then not have it negatively impact
further repacks.

> has been giving me lots of trouble when adding files at deep levels
> and I like how Git does adding files and how each repository is really
> it's own repository.  Also, I'm not really concerned about history as
> the main purpose of the repository is really to make the syncing job
> easier.  The code is very minimal and I do the majority of the

Well, if you're not too concerned about the history, then I would
suggest setting up something like this:

  - all of the big files get a unique name; new versions of the same
    file have names unique from their previous version (you can do this
    either with human-readable names, or just using the sha-1 of the
    file contents)

  - make all of the big files available through rsync, http, ftp, or
    whatever

  - the git (or svn) repo contains a list of all desired files, and a
    script that checks what you have versus what is desired (deleting
    any that aren't wanted, and pulling any that are missing)

Your repository of big files will grow, but each file within it is
immutable. The git repository points to the files by name, but doesn't
contain them. So updating from a client is just:

  git pull && ./update-script

but you can still go back to a snapshot in history if you want to:

  git checkout HEAD@{3.weeks.ago} && ./update-script

Make sense?

> can be crazy at times, but what can we do.  Also, this is a far fetch,
> since all the projects are installed under the same directories and
> are really exact clones of each other in every way, can I copy the
> .git directory and hope to God that it will magically work.  I really
> doubt it, but I figure it's worth asking.

I'm not sure what you mean by that. You can copy the .git directory to a
new working tree, yes. It will be a new repo, and you can see how what
it thinks should be checked out differs from what is in the new working
tree by running "git diff".

-Peff

How to go to git from svn without checkout

[Mercedes6s]

Hello.  I've been falling more and more in love with git and I want to move
our biggest project biggest it is my biggest pain over to git to make our
lives easier.  The problem is our project is about 50 gigs (A lot of media
files involved).  Also, we got developers all over the world that are
running on slow connections.  Some took a week to get the project up and
running in the first place.  Luckily, they are only isolated instances. 
What I'm wondering is can I have all of them turn their projects into git
repositories locally and have all of them sync with a master repository once
they are done.  The actual project is only about 500 megs and the changes
are probably only a few K since they keep their projects up to date, so
that's more acceptable, but I don't want them to bring down all those media
files again and I doubt they will be willing to do it.  Is this possible,
and how would I do it?  

thanks
-- 
View this message in context: http://www.nabble.com/How-to-go-to-git-from-svn-without-checkout-tp22638146p22638146.html
Sent from the git mailing list archive at Nabble.com.