Another question about importing SVN with fast-import

Another question about importing SVN with fast-import

[Julian Phillips]

First off, I would just like to say fast-import rocks.  It's well named 
too ...

Now the question.  Shawn recently added C and R operations - almost as 
soon as they were asked for too.  However, how do you copy a file from a 
particular revision?  I have just hit a point where someone deleted a 
directory, and then copied one of the files from that directory back from 
an old revision (as two separate commits).  Since I'm not tracking any 
branch contents in my front-end, and the copy operation only works from 
the current branch head I seem to be stuck ... or have I missed something?

-- 
Julian

  ---
What I want to find out is -- do parrots know much about Astro-Turf?

Re: Another question about importing SVN with fast-import

[Shawn O. Pearce]

Julian Phillips <julian@quantumfyre.co.uk> wrote:
> First off, I would just like to say fast-import rocks.  It's well named 
> too ...

I'm glad you like it.  ;-)
 
> Now the question.  Shawn recently added C and R operations - almost as 
> soon as they were asked for too.

Sometimes I'm responsive...

> However, how do you copy a file from a 
> particular revision?

and sometimes not so much.  You can't do that right now.  I've wanted
to open up the data subcommand to allow another form that lets you
specify data from a branch and file path (thus selecting a blob
from another revision) but I haven't gotten around to it.  I also
don't have time to do it during the earlier part of this week.
Maybe I'll get to it later near the end of the week.

It shouldn't be too difficult now with the tree_content_get()
function that I recently defined for the C/R commands.  The SHA-1
comes back in the tree_entry leaf, but that codepath is only valid
for the tip of a branch that fast-import knows about in memory.
If it doesn't then you probably need to fallback into raw tree
parsing.  Ugh.

The way prior frontends have handled this is they assigned marks
to every blob, and then had a translation table within the frontend
of revision->mark, so that anytime it needed a given revision of a
file it knew what mark to send to fast-import.  This does require
that the frontend maintain basically everything...

> I have just hit a point where someone deleted a 
> directory, and then copied one of the files from that directory back from 
> an old revision (as two separate commits).  Since I'm not tracking any 
> branch contents in my front-end, and the copy operation only works from 
> the current branch head I seem to be stuck ... or have I missed something?

Yea, I got nothing.  I can't think of any method to make that work
aside from using marks.  Or teaching fast-import how to do tree
lookup, like I described above.

-- 
Shawn.

Re: Another question about importing SVN with fast-import

[Julian Phillips]

On Mon, 16 Jul 2007, Shawn O. Pearce wrote:

> Julian Phillips <julian@quantumfyre.co.uk> wrote:
>> However, how do you copy a file from a
>> particular revision?
>
> and sometimes not so much.  You can't do that right now.  I've wanted
> to open up the data subcommand to allow another form that lets you
> specify data from a branch and file path (thus selecting a blob
> from another revision) but I haven't gotten around to it.  I also
> don't have time to do it during the earlier part of this week.
> Maybe I'll get to it later near the end of the week.

I was wondering if this was still a vague todo item, or if it was being 
worked on?  It really is the biggest thing blocking my import at the 
moment and I'm willing to have a bash at it myself, but I didn't want to 
start messing around inside the fast-import code if someone else was 
already there ...

>
> It shouldn't be too difficult now with the tree_content_get()
> function that I recently defined for the C/R commands.  The SHA-1
> comes back in the tree_entry leaf, but that codepath is only valid
> for the tip of a branch that fast-import knows about in memory.
> If it doesn't then you probably need to fallback into raw tree
> parsing.  Ugh.
>
> The way prior frontends have handled this is they assigned marks
> to every blob, and then had a translation table within the frontend
> of revision->mark, so that anytime it needed a given revision of a
> file it knew what mark to send to fast-import.  This does require
> that the frontend maintain basically everything...

Not sure how that would apply to trees ... and I would rather minimise the 
amount of state retained in the frontend.

-- 
Julian

  ---
Disco is to music what Etch-A-Sketch is to art.

Re: Another question about importing SVN with fast-import

[Shawn O. Pearce]

Julian Phillips <julian@quantumfyre.co.uk> wrote:
> On Mon, 16 Jul 2007, Shawn O. Pearce wrote:
> 
> >Julian Phillips <julian@quantumfyre.co.uk> wrote:
> >>However, how do you copy a file from a
> >>particular revision?
> >
> >and sometimes not so much.  You can't do that right now.  I've wanted
> >to open up the data subcommand to allow another form that lets you
> >specify data from a branch and file path (thus selecting a blob
> >from another revision) but I haven't gotten around to it.  I also
> >don't have time to do it during the earlier part of this week.
> >Maybe I'll get to it later near the end of the week.
> 
> I was wondering if this was still a vague todo item, or if it was being 
> worked on?  It really is the biggest thing blocking my import at the 
> moment and I'm willing to have a bash at it myself, but I didn't want to 
> start messing around inside the fast-import code if someone else was 
> already there ...

No, its not vague.  I'm still working on it.  Actually its on
my to-do list for tonight, and if I don't finish it tonight then
Thursday.
 
-- 
Shawn.

Re: Another question about importing SVN with fast-import

[David Frech]

On 7/16/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
> Now the question.  Shawn recently added C and R operations - almost as
> soon as they were asked for too.  However, how do you copy a file from a
> particular revision?  I have just hit a point where someone deleted a
> directory, and then copied one of the files from that directory back from
> an old revision (as two separate commits).  Since I'm not tracking any
> branch contents in my front-end, and the copy operation only works from
> the current branch head I seem to be stuck ... or have I missed something?

I have to second Julian's question.

The only things I have left to implement (in my svn dump to
fast-import translator) before I can call my code "complete"  are
branch copies (which will generate "from" lines in commits so
fast-import can properly initialize the new branch) and a generic
"copy <path> from <rev>" that Julian also needs.

I mark my commits and keep track of the mapping from svn revs to git
commits, so when the svn dump asks for a <path> from a specific <svn
rev> I can correlate that to a mark (ie, a git commit). But I can't
tell fast-import to *do* anything with it.

This is kind of a request and a comment/question. The request is:
there is no way to do *reasonably* in the front end what fast-import
can do somewhat reasonably: namely, copy a <path> (file or directory!)
from an arbitrary previously committed revision/mark to the current
branch.

The comment/question is: how different is this, really, from being
able to  specify a "from" line in a commit? In both cases I'm asking
fast-import to reach into its memory (or the repo) and pull out a
tree, and to add (some or all of it) to my current branch. Isn't the
kind of generic C command that Julian and I are asking for the same
thing, only instead of taking the whole tree (from the specified
commit) it takes a single file or directory?

I hope I haven't missed the point entirely.

Lastly, do we really need "R"? With this generic copy - and I think
there should be *only* a generic version, not a "streamlined local
copy" version and a "reach into history arbitrarily" version - we can,
as an earlier poster pointed out, do R by doing a C and then a D. This
is, in fact, how svn dump files represent file and directory renames.

It would be nice to keep the fast-import command set small and orthogonal.

My few cents.

Cheers,

- David

-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: Another question about importing SVN with fast-import

[Shawn O. Pearce]

David Frech <david@nimblemachines.com> wrote:
> On 7/16/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
> >Now the question.  Shawn recently added C and R operations - almost as
> >soon as they were asked for too.  However, how do you copy a file from a
> >particular revision?
> 
> I have to second Julian's question.

I'm getting to it.  Maybe this weekend.  Right now I have git-gui
work to finish, and that work is more important to me this week than
fast-import is.  Not that fast-import isn't important to me either,
its just the way things are this week.
 
> This is kind of a request and a comment/question. The request is:
> there is no way to do *reasonably* in the front end what fast-import
> can do somewhat reasonably: namely, copy a <path> (file or directory!)
> from an arbitrary previously committed revision/mark to the current
> branch.

I agree.
 
> The comment/question is: how different is this, really, from being
> able to  specify a "from" line in a commit? In both cases I'm asking
> fast-import to reach into its memory (or the repo) and pull out a
> tree, and to add (some or all of it) to my current branch. Isn't the
> kind of generic C command that Julian and I are asking for the same
> thing, only instead of taking the whole tree (from the specified
> commit) it takes a single file or directory?

It is possible.  I'm just not sure what the syntax for it should be.
Suggestions?  I really want to stay backwards compatible with the
current "C" command, so:

	'C' SP commit SP path SP path

is out because its ambiguous with the current meaning where the
second (destination) path can contain SP without being quoted by
the frontend.

> Lastly, do we really need "R"? With this generic copy - and I think
> there should be *only* a generic version, not a "streamlined local
> copy" version and a "reach into history arbitrarily" version - we can,
> as an earlier poster pointed out, do R by doing a C and then a D. This
> is, in fact, how svn dump files represent file and directory renames.

The code for "R" is so short that I just don't see a need to remove it.
Its also already out in the wild, as it has been in Junio's master for
a little while now.
 
-- 
Shawn.

Re: Another question about importing SVN with fast-import

[Julian Phillips]

On Fri, 20 Jul 2007, Shawn O. Pearce wrote:

> David Frech <david@nimblemachines.com> wrote:
>> On 7/16/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
>>> Now the question.  Shawn recently added C and R operations - almost as
>>> soon as they were asked for too.  However, how do you copy a file from a
>>> particular revision?
>>
>> I have to second Julian's question.
>
> I'm getting to it.  Maybe this weekend.  Right now I have git-gui
> work to finish, and that work is more important to me this week than
> fast-import is.  Not that fast-import isn't important to me either,
> its just the way things are this week.

No complaints from me - git-gui is one of the reasons I want to be able to 
use fast-import ;)

>> The comment/question is: how different is this, really, from being
>> able to  specify a "from" line in a commit? In both cases I'm asking
>> fast-import to reach into its memory (or the repo) and pull out a
>> tree, and to add (some or all of it) to my current branch. Isn't the
>> kind of generic C command that Julian and I are asking for the same
>> thing, only instead of taking the whole tree (from the specified
>> commit) it takes a single file or directory?
>
> It is possible.  I'm just not sure what the syntax for it should be.
> Suggestions?  I really want to stay backwards compatible with the
> current "C" command, so:
>
> 	'C' SP commit SP path SP path
>
> is out because its ambiguous with the current meaning where the
> second (destination) path can contain SP without being quoted by
> the frontend.

You could always make it part of the 'M' command?

'M' sp mode sp 'copy' sp path_str lf (ref_str | hexsha1 | sha1exp_str | 
idnum) SP path_str;

Or just make it a new command, O (for other) or E (for existing) maybe? :S

-- 
Julian

  ---
It is easier to change the specification to fit the program than vice versa.

Re: Another question about importing SVN with fast-import

[David Frech]

On 7/19/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
> On Fri, 20 Jul 2007, Shawn O. Pearce wrote:
>
> > It is possible.  I'm just not sure what the syntax for it should be.
> > Suggestions?  I really want to stay backwards compatible with the
> > current "C" command, so:
> >
> >       'C' SP commit SP path SP path
> >
> > is out because its ambiguous with the current meaning where the
> > second (destination) path can contain SP without being quoted by
> > the frontend.
>
> You could always make it part of the 'M' command?
>
> 'M' sp mode sp 'copy' sp path_str lf (ref_str | hexsha1 | sha1exp_str |
> idnum) SP path_str;
>
> Or just make it a new command, O (for other) or E (for existing) maybe? :S

Since we'll be referring to past commits via marks (with start with
":") how about this:

'C' SP srcpath:mark SP dstpath

In the case of making it a new command I can't think of any really
compelling one letter names. ;-)

- David

>
> --
> Julian
>
>   ---
> It is easier to change the specification to fit the program than vice versa.
>

-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: Another question about importing SVN with fast-import

[Julian Phillips]

On Thu, 19 Jul 2007, David Frech wrote:

> On 7/19/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
>>  On Fri, 20 Jul 2007, Shawn O. Pearce wrote:
>> 
>> >  It is possible.  I'm just not sure what the syntax for it should be.
>> >  Suggestions?  I really want to stay backwards compatible with the
>> >  current "C" command, so:
>> > 
>> >        'C' SP commit SP path SP path
>> > 
>> >  is out because its ambiguous with the current meaning where the
>> >  second (destination) path can contain SP without being quoted by
>> >  the frontend.
>>
>>  You could always make it part of the 'M' command?
>>
>>  'M' sp mode sp 'copy' sp path_str lf (ref_str | hexsha1 | sha1exp_str |
>>  idnum) SP path_str;
>>
>>  Or just make it a new command, O (for other) or E (for existing) maybe? :S
>
> Since we'll be referring to past commits via marks (with start with
> ":") how about this:
>
> 'C' SP srcpath:mark SP dstpath

Why only via marks?  That may be what you happen to want, but no other 
command restricts you to _only_ using marks ...

>
> In the case of making it a new command I can't think of any really
> compelling one letter names. ;-)
>
> - David
>
>>
>>  --
>>  Julian
>>
>>    ---
>>  It is easier to change the specification to fit the program than vice
>>  versa.
>> 
>
>

-- 
Julian

  ---
What say you, Mr. American Ambassador?

Fuck Canada!

Re: Another question about importing SVN with fast-import

[David Frech]

On 7/20/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
> On Thu, 19 Jul 2007, David Frech wrote:
> > Since we'll be referring to past commits via marks (with start with
> > ":") how about this:
> >
> > 'C' SP srcpath:mark SP dstpath
>
> Why only via marks?  That may be what you happen to want, but no other
> command restricts you to _only_ using marks ...

I was using the royal "we". ;-)

Yes, you're right of course. My bad.

- David

> --
> Julian
>
>   ---
> What say you, Mr. American Ambassador?
>
> Fuck Canada!
>


-- 
If I have not seen farther, it is because I have stood in the
footsteps of giants.

Re: Another question about importing SVN with fast-import

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Fri, Jul 20, 2007 at 01:11:42 -0400, Shawn O. Pearce wrote:
> It is possible.  I'm just not sure what the syntax for it should be.
> Suggestions?  I really want to stay backwards compatible with the
> current "C" command, so:
> 
> 	'C' SP commit SP path SP path
> 
> is out because its ambiguous with the current meaning where the
> second (destination) path can contain SP without being quoted by
> the frontend.

I'd suggest one of two variants:
 1) 'M' SP <mode> SP <dataref> SP <path> LF
    where <dataref> would be extended to understand the
    {tag-id|commit-id|tree-id}:path notation git-ref-parse understands plus
    mark:path where mark points to commit.
 2) 'C' SP <dataref> SP <path> LF
    where again <dataref> can be {tag-id|commit-id|tree-id|mark}:path -- or
    just path which implies current head.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]