Updating shared ref from remote helper, or fetch hook

Updating shared ref from remote helper, or fetch hook

[Jed Brown]

I'm working on an hg remote helper that uses git notes for the sha1
revision, so that git users can more easily refer to specific commits
when communicating with hg users.  Since there may be multiple
concurrent fast-import streams, I write the notes to a private ref
(refs/notes/hg-REMOTE), to be merged eventually using

  git notes --ref hg merge hg-REMOTE*

There will never be conflicts because each hg commit translates to a
unique git commit, thus even if multiple concurrent remotes process the
same commit, the corresponding note will match.

Unfortunately, I couldn't find a safe way to get this run after a fetch
or clone.  Of course I can ask the user to arrange to have this command
run, but it would be a better interface to have it run automatically
since it is a natural responsibility of the remote helper.  Am I missing
a way to do this or a viable alternative approach?

Re: Updating shared ref from remote helper, or fetch hook

[Max Horn]

Hi Jed, all,


On 28.01.2013, at 06:19, Jed Brown wrote:

> I'm working on an hg remote helper that uses git notes for the sha1
> revision, so that git users can more easily refer to specific commits
> when communicating with hg users.

For the record, I am also working on that very same thing; it is yet another git-remote-hg alternative, based on Felipe's code but with many improvements. You can find it here: <https://github.com/buchuki/gitifyhg>.

Anyway, back to Jed's (and also my) question:

>  Since there may be multiple
> concurrent fast-import streams, I write the notes to a private ref
> (refs/notes/hg-REMOTE), to be merged eventually using
> 
>  git notes --ref hg merge hg-REMOTE*
> 
> There will never be conflicts because each hg commit translates to a
> unique git commit, thus even if multiple concurrent remotes process the
> same commit, the corresponding note will match.
> 
> Unfortunately, I couldn't find a safe way to get this run after a fetch
> or clone.  Of course I can ask the user to arrange to have this command
> run, but it would be a better interface to have it run automatically
> since it is a natural responsibility of the remote helper.  Am I missing
> a way to do this or a viable alternative approach?

One idea we (well, Jed :-) had when brain storming about this was that perhaps one could (or even should?) (ab)use the "checkpoint" feature for that.

Basically, when the remote-helper is almost done with everything, issue a "checkpoint" command, to flush out everything we just imported to the HD. 

Then once this completed, we can perform the notes merge. The main remaining problem with that is: How would we know when the "checkpoint" actually completed? Any ideas?

Perhaps a way to do that would be to make use of the new "bidi-import" remote helper capability -- if I understand it right, then this effectively connects the fast-import stdout to the stdin of the remote helper. Thus, if one were to follow the "checkpoint" by a "progress" command, then by waiting for that progress command's output to appear back on stdin, the remote helper could determine whether the import succeeded, and perform finalization actions (like merging notes, as in our case).


Does that sound viable? Crazy? Anybody got better a idea?


Cheers,
Max

Re: Updating shared ref from remote helper, or fetch hook

[Jed Brown]

Jed Brown <jed@59A2.org> writes:

> I'm working on an hg remote helper that uses git notes for the sha1
> revision, so that git users can more easily refer to specific commits
> when communicating with hg users.  Since there may be multiple
> concurrent fast-import streams, I write the notes to a private ref
> (refs/notes/hg-REMOTE), to be merged eventually using
>
>   git notes --ref hg merge hg-REMOTE*

A related issue is that when a remote helper replies to an 'import' with
_only_ a commit in refs/notes/, git (fetch or pull) produces an error
message like

  error: refs/notes/hg-84b3865b750a567acb16929c21e14c4a45a5639b does not point to a valid object

but successfully updates the ref (which is indeed valid) and returns
0. I have not been able to determine what exactly git thinks is
invalid. As long as there is at least one non-notes commit in the
stream, no such error message is produced.

Is this behavior intended?