[GSoC update extra!] git-remote-svn: Week 9

[GSoC update extra!] git-remote-svn: Week 9

[Ramkumar Ramachandra]

Hi,

I figured that I haven't posted a recent update about the RA
svnclient, so here it is. The good news is that using a combination of
trial-and-error and twisted logic, I seem to have to fix the memory
leaks and pool lifetime issues. I also managed to ouput the
text-content-length header by writing the dump to a temporary file and
stat'ing it. Unfortunately, the headers are still malformed, and this
requires some more work to get right. The client runs successfully and
manages to dump 5000 revisions of the ASF repository:

11.34s user 3.50s system 21% cpu 1:07.69 total

That's about 75 revisions per second, which isn't too bad I hope. This
is over the network, and my internet connection isn't all that fast.

The bad news is that the client seems to signal SIGABRT when it
finishes its work for some strange reason. Fortunately, I suspect that
this is a trivial issue where I'm leaking some error or forgetting to
clean up something. Hopefully, I'll be able to resolve this and fix
the dump headers within the next few days. After that, I'll scrub the
history clean and roll a series.

-- Ram

Re: [GSoC update extra!] git-remote-svn: Week 9

[Tay Ray Chuan]

Hi,

On Sat, Jul 3, 2010 at 5:57 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:
> [snip]
> The client runs successfully and
> manages to dump 5000 revisions of the ASF repository:
>
> 11.34s user 3.50s system 21% cpu 1:07.69 total
>
> That's about 75 revisions per second, which isn't too bad I hope. This
> is over the network, and my internet connection isn't all that fast.

I wonder if you could mirror the test SVN repository locally with
svnsync so that we can a) spare their bandwidth b) minimize the effect
of network bottlenecks on stats - I'm sure you could get more
impressive stats that way.

Even better, having the SVN repo and the "exported" git repo on
separate hard disks to minimize I/O as a bottleneck.

(Great work, by the way.)

-- 
Cheers,
Ray Chuan

Re: [GSoC update extra!] git-remote-svn: Week 9

[Ramkumar Ramachandra]

Hi Tay,

Tay Ray Chuan writes:
> Hi,
> 
> On Sat, Jul 3, 2010 at 5:57 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:
> > [snip]
> > The client runs successfully and
> > manages to dump 5000 revisions of the ASF repository:
> >
> > 11.34s user 3.50s system 21% cpu 1:07.69 total
> >
> > That's about 75 revisions per second, which isn't too bad I hope. This
> > is over the network, and my internet connection isn't all that fast.
> 
> I wonder if you could mirror the test SVN repository locally with
> svnsync so that we can a) spare their bandwidth b) minimize the effect
> of network bottlenecks on stats - I'm sure you could get more
> impressive stats that way.

Excellent idea. Unfortunately, I don't have enough disk space to
mirror the whole ASF repository on my laptop- I'll try it when I get
back to my desktop (in 15~20 days).

In my prelimiary benchmark, I also found that I beat `svnsync` over
the same network connection :)

> Even better, having the SVN repo and the "exported" git repo on
> separate hard disks to minimize I/O as a bottleneck.

Unfortunately, a lot of middleware components are missing and we
cannot export it to a Git repository just yet- see my previous email
about this [1]. Great idea though- will try it in future.

> (Great work, by the way.)

Thanks, and thanks for the feedback! :)

[1]: http://article.gmane.org/gmane.comp.version-control.git/150035

-- Ram