About single user setup for lightweights

About single user setup for lightweights

[Harry Putnam]

Hold your fire on this one if possible.  I'm not just a lazy slug who
can't think and study for himself.

I'm a little confused about the different way of using rcs that git,
mercurial bazaar and probably several others offer.

I've not used anything but cvs.  I use it at least every couple of
days but really only a limited set of commands, and no deep knowledge
even of that style.

My usage is basically to keep up with rc files for the several OSs' I
tinker with on my home lan and a fair bit of scripting and
experimenting with shell, perl, c, etc. that I've built up over the
years and find reason to change a bit every so often.

I keep a central cvs repo and on each host I do a check out of the
entire thing from the base up.  Mostly to have copies of various style
of rc files the  OSs need but also to keep the scripts I've written
over the years and learned to rely on, available and in sync.

To me, keeping up with cvs is always a PITA.  I've never hit on a
handy and efficient way to do it. Even for a just my light usage.

And I do mean lignt.  For example, even after yrs of using cvs my
$CVROOT is still tiny:

  du -sh /usr/local/cvsroot
  12M	/usr/local/cvsroot

So, I'm wondering if one of the newer systems would be less of a PITA.

How would a workflow actually go:
I'd create and populate a repo, then what?.  Create clones on each
machine I guess and if I found a need to change or add files, I'd then
push back to the original repo?  Its sounding a whole lot like cvs so far.

So, am I likely to see some improvement in the chore of keeping up an
rcs system with git, mercurial or bazaar?

Anther thing I'm really curious about concerns binary rcs.  I'm thinking
of photo editing and things like flash where I might be changing a
project over time and want access to past versions.

I'm told cvs is not good for that... consequently I've never tried
it.  Am I likely to find that one of git, mercurial or bazaar is far
better for that?

Re: About single user setup for lightweights

[Avery Pennarun]

On Thu, Mar 18, 2010 at 9:53 PM, Harry Putnam <reader@newsguy.com> wrote:
> I keep a central cvs repo and on each host I do a check out of the
> entire thing from the base up.  Mostly to have copies of various style
> of rc files the  OSs need but also to keep the scripts I've written
> over the years and learned to rely on, available and in sync.
>
> To me, keeping up with cvs is always a PITA.  I've never hit on a
> handy and efficient way to do it. Even for a just my light usage.
[...]
> How would a workflow actually go:
> I'd create and populate a repo, then what?.  Create clones on each
> machine I guess and if I found a need to change or add files, I'd then
> push back to the original repo?  Its sounding a whole lot like cvs so far.

Yes.  Or you could skip the central repo and pull directly from one
machine's working tree to another.  If that has any value to you, then
it's the only likely reason a DVCS would do you any good for this
trivial case.

The real question is: what makes your current setup a PITA?  If you
can't answer that concisely, then you don't know what to look for in a
supposedly better solution.

> Anther thing I'm really curious about concerns binary rcs.  I'm thinking
> of photo editing and things like flash where I might be changing a
> project over time and want access to past versions.
>
> I'm told cvs is not good for that... consequently I've never tried
> it.  Am I likely to find that one of git, mercurial or bazaar is far
> better for that?

git sucks at handling large binary files (>50 megs or so) unless you
have boatloads of RAM.  If your binary files are moderately sized (a
few megs) then it'll probably be reasonably efficient.  I don't know
about hg and bzr for memory usage.

It's better to store uncompressed binary files (eg *.tar) instead of
compressed ones (*.tar.gz) in order to allow useful delta compression.
 That means raw images instead of png/gif/jpg.  And probably completed
flash files are compressed.  The best thing to do is actually try it
and see if your repository size and memory usage is reasonable.

Have fun,

Avery

Re: About single user setup for lightweights

[Martin Geisler]

[-- Attachment #1: Type: text/plain, Size: 863 bytes --]

Avery Pennarun <apenwarr@gmail.com> writes:

> git sucks at handling large binary files (>50 megs or so) unless you
> have boatloads of RAM. If your binary files are moderately sized (a
> few megs) then it'll probably be reasonably efficient. I don't know
> about hg and bzr for memory usage.

Mercurial also uses lots of RAM, way more than I had hoped. I did some
tests with this recently:

  http://markmail.org/message/uxqtmmnkyimxse5b

They show a factor 3-6 blowup when working with a 256 MB file.

We don't really recommend storing such large files in Mercurial. Instead
we recommend storing the files outside of the tree, e.g., on a server
with a huge disk. The bfiles extension can do this:

  http://mercurial.selenic.com/wiki/BfilesExtension

-- 
Martin Geisler

Fast and powerful revision control: http://mercurial.selenic.com/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: About single user setup for lightweights

[Avery Pennarun]

On Fri, Mar 19, 2010 at 5:53 AM, Martin Geisler <mg@lazybytes.net> wrote:
> Avery Pennarun <apenwarr@gmail.com> writes:
>> git sucks at handling large binary files (>50 megs or so) unless you
>> have boatloads of RAM. If your binary files are moderately sized (a
>> few megs) then it'll probably be reasonably efficient. I don't know
>> about hg and bzr for memory usage.
>
> Mercurial also uses lots of RAM, way more than I had hoped. I did some
> tests with this recently:
>
>  http://markmail.org/message/uxqtmmnkyimxse5b
>
> They show a factor 3-6 blowup when working with a 256 MB file.
>
> We don't really recommend storing such large files in Mercurial. Instead
> we recommend storing the files outside of the tree, e.g., on a server
> with a huge disk. The bfiles extension can do this:
>
>  http://mercurial.selenic.com/wiki/BfilesExtension

You might find my "bup" program entertaining:

  http://github.com/apenwarr/bup/

It happens to use the git file format, but the hashsplitting algorithm
would work with any repo and the code is written mostly in python.
Because it breaks larges files into chunks, it tends to avoid the
memory growth problems (at the cost of somewhat worse compression and
deltas).  At least you can then store them in your repository.

bup is intended for use as a full-system backup tool, but it would be
interesting to take the same techniques and use them to solve the
general case of large files in git/hg.

Have fun,

Avery

Re: About single user setup for lightweights

[Ben Finney]

Harry Putnam <reader@newsguy.com> writes:

> Hold your fire on this one if possible. I'm not just a lazy slug who
> can't think and study for himself.

Hopefully you're in for a smooth ride; I'd be disappointed if any of
these communities are hostile to an honest and specific enquiry like
yours.

Welcome, and thanks for learning about modern VCS options!

> I'm a little confused about the different way of using rcs that git,
> mercurial bazaar and probably several others offer.

I got a bit confised by that sentence. You would do well to use “VCS”
(Version Control System) to refer to the class of program. “RCS” refers
to a particular VCS, one that is still in current use for some purposes.

> I've not used anything but cvs.  I use it at least every couple of
> days but really only a limited set of commands, and no deep knowledge
> even of that style.

Good news; you'll have little un-learning to do :-)

> My usage is basically to keep up with rc files for the several OSs' I
> tinker with
[…]
> To me, keeping up with cvs is always a PITA.  I've never hit on a
> handy and efficient way to do it. Even for a just my light usage.

I don't know what “keep up with CVS” means. Can you explain what part of
your workflow is a PITA?

> How would a workflow actually go:
> I'd create and populate a repo, then what?.

You could choose which files should be common between different
machines, add those to the repository and choose not to track the rest
in the VCS. Other ways of doing it are also feasible.

> Create clones on each machine I guess and if I found a need to change
> or add files, I'd then push back to the original repo? Its sounding a
> whole lot like cvs so far.

> So, am I likely to see some improvement in the chore of keeping up an
> [VCS] system with git, mercurial or bazaar?

Pending an explanation of what “keeping up” means, I think one of the
big benefits you'll get is that the modern VCS designs:

* treat the whole working tree as the thing to be represented in each
  revision; and

* have significantly better merging capability compared with CVS.

> Anther thing I'm really curious about concerns binary rcs. I'm
> thinking of photo editing and things like flash where I might be
> changing a project over time and want access to past versions.

You can do this with any VCS, but binary files don't have a good generic
way to represent differences efficiently, whereas text files do. So
tracking binary files will work, but will be rather inefficient in terms
of memory usage and repository data.

-- 
 \         “If you can do no good, at least do no harm.” —_Slapstick_, |
  `\                                                     Kurt Vonnegut |
_o__)                                                                  |
Ben Finney

Re: About single user setup for lightweights

[Sitaram Chamarty]

On Fri, Mar 19, 2010 at 7:23 AM, Harry Putnam <reader@newsguy.com> wrote:

> I keep a central cvs repo and on each host I do a check out of the
> entire thing from the base up.  Mostly to have copies of various style
> of rc files the  OSs need but also to keep the scripts I've written
> over the years and learned to rely on, available and in sync.

about the most important thing, you ought to notice is that you don't
need to stay connected to your "server" to commit.  I do this too, and
having a DVCS (in my case, git) helps a lot.

I didn't, on a quick read, see that mentioned in any of the replies so far.