Re: git-p4

Re: git-p4

[Alejandro Riveira Fernández]

El Fri, 10 Sep 2010 15:54:16 -0400
Kent Borg <kentborg@borg.org> escribió:

 [ CCing git mailing list. Looks like a better place to ask this question]

> I have a git-p4 question: I work in a Perforce shop and am doing Linux
> kernel work, I need to share that work with colleagues who see the world
> as a Perforce place.  The kernel I have came from Linus' tree and has a
> lot of history.  When I try to do my first a "git p4 submit" it chokes
> as it looks back in the entire git history until it fails looking for
> the ancestor of the first commit (linux-2.6.12-rc2!), I think it is
> looking for the last time it did a git-p4 submit so it knows how far
> back to go--but it has never done a submit in this new relationship
> between p4 and git.  There is plenty of git history that is not
> reflected in p4, and I don't want it in p4, I just want new work in p4.
> 
> I fear that git-p4 is for git people to contribute to bits natively
> p4-homed code, not this case where the code is natively git-homed code
> and it is the p4 people who will be contributing bits. 
> 
> My attempt at a work around was this:
> 
>  - create a director on the p4 side, and from the p4 side submit the
> files that match my latest git submit.
> 
>  - sync with git-p4
> 
>  - try to submit a file with git-p4...and that fails as it runs all the
> way back through the history.  (Thank goodness it didn't succeed in
> submitting kernel activity since 2005!)
> 
> 
> I was hoping I could merge the p4/master with master to force a stopping
> point in the git-p4 submit, but that doesn't work, it is searching
> through git history to find the stopping point.  What might I put in the
> git history to force the stop (or maybe make a change to git-p4 to help
> my plight along)?
> 
> 
> Thanks,
> 
> -kb, the Kent who has always hated Perforce, alas.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: git-p4

[Tor Arvid Lund]

2010/9/10 Alejandro Riveira Fernández <ariveira@gmail.com>:
> El Fri, 10 Sep 2010 15:54:16 -0400
> Kent Borg <kentborg@borg.org> escribió:
>
>  [ CCing git mailing list. Looks like a better place to ask this question]
>
>> I have a git-p4 question: I work in a Perforce shop and am doing Linux
>> kernel work, I need to share that work with colleagues who see the world
>> as a Perforce place.  The kernel I have came from Linus' tree and has a
>> lot of history.  When I try to do my first a "git p4 submit" it chokes
>> as it looks back in the entire git history until it fails looking for
>> the ancestor of the first commit (linux-2.6.12-rc2!), I think it is
>> looking for the last time it did a git-p4 submit so it knows how far
>> back to go--but it has never done a submit in this new relationship
>> between p4 and git.  There is plenty of git history that is not
>> reflected in p4, and I don't want it in p4, I just want new work in p4.

Yes, you are right in that git-p4 looks for the last point in history
where p4 and git were "in sync".

>> I fear that git-p4 is for git people to contribute to bits natively
>> p4-homed code, not this case where the code is natively git-homed code
>> and it is the p4 people who will be contributing bits.

This is also correct, I'd say.

>> My attempt at a work around was this:
>>
>>  - create a director on the p4 side, and from the p4 side submit the
>> files that match my latest git submit.
>>
>>  - sync with git-p4

This is how I would do it too...

>>  - try to submit a file with git-p4...and that fails as it runs all the
>> way back through the history.  (Thank goodness it didn't succeed in
>> submitting kernel activity since 2005!)

Well, when you did the "sync with git-p4", did it create a branch in
refs/remotes/p4/master or something like that? I think that's
generally how it solves the issue of knowing what to submit to p4; On
the git side you have a number of commits that you _don't_ want to
submit, and the most recent of these commits should have a log message
that ends with:

[git-p4: depot-paths = "//Path/To/Your/Project/": change = 30049]

.. where the 'change' number is the number of the perforce changelist
you synced using git-p4.

So - work that you want to submit to p4 should be rebased on top of
such a commit. Then it should work to do git-p4 submit.

-Tor Arvid-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
> [git-p4: depot-paths = "//Path/To/Your/Project/": change = 30049]
> .. where the 'change' number is the number of the perforce changelist
> you synced using git-p4.
>
> So - work that you want to submit to p4 should be rebased on top of
> such a commit. Then it should work to do git-p4 submit.
>   

That doesn't work.  I am suspecting there is a tangle in my branches.

Firing up a Python session and manually running the git-p4
findUpstreamBranchPoint() I get back something reminiscent of the commit
I made at your suggestion:

  ['remotes/p4/master', {'change': '160991', 'depot-paths':
['//depot/imgeng/sw/inferno/kernel/linux-2.6.33-quatro']}]

I thought it looked reasonable, but when I try to run the git p4 summit,
at line 811 in git-p4 it is doing:

  git rev-list --no-merges remotes/p4/master..master

And then reversing the results and trying to apply them all, starting
with 1da177e (which is Linux-2.16.12-rc), and again it fails.  (Again,
thank goodness it failed at that.)  How is git rev-list supposed to get
a limited list back?

Anyone spot my silliness?  Next I'm going to see if I can create a
normal (and working) case for comparison.  (As a newbie git is one
thing, trying to understand git-p4 at the same time makes it that much
harder.)


Thanks a bunch,

-kb, the Kent who Monday morning (UTC-0400) would love to casually say
"Oh, yes, git-p4 is working great, both directions, no problem.".

Re: git-p4

[Tor Arvid Lund]

2010/9/12 Kent Borg <kentborg@borg.org>:
> Firing up a Python session and manually running the git-p4
> findUpstreamBranchPoint() I get back something reminiscent of the commit
> I made at your suggestion:
>
>  ['remotes/p4/master', {'change': '160991', 'depot-paths':
> ['//depot/imgeng/sw/inferno/kernel/linux-2.6.33-quatro']}]
>
> I thought it looked reasonable, but when I try to run the git p4 summit,
> at line 811 in git-p4 it is doing:
>
>  git rev-list --no-merges remotes/p4/master..master

Ok, so if you call this on the cmd line, it should output sha1's on
all commits that will be submitted (in reverse order) to p4. If it
doesn't, this may well be a good place to dig for a solution.

Say that you have only one commit to submit to p4. We'll call it X.

The parent of X should then be the commit that remotes/p4/master is
pointing to. If it isn't, you can try to:

1) git checkout -b my_p4_branch p4/master
2) git cherry-pick X # we'll call this cherry-picked commit Y
3) git rev-list --no-merges remotes/p4/master..my_p4_branch
  * if it lists Y and only Y, try
4) git p4 submit

-Tor Arvid-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
> Ok, so if you call this on the cmd line, it should output sha1's on
> all commits that will be submitted (in reverse order) to p4. If it
> doesn't, this may well be a good place to dig for a solution.

It does output the commits it wants to submit.  In my broken case far
too many.

I created a simple working case to see what working looks like, and the
exact same command outputs just the one commit.  Looking more carefully
at my gitg pictures in the good and bad cases, I realize that I in the
broken case I probably don't have the graph I need.


Good case:

  (master)
   |
  (p4/HEAD)  (p4/master)
   |
  initial test import from existing depot


Bad case:

  (master)   trivial change I am trying to submit
   |
  (tmp)  commit with the "[git-p4: depot-paths = "//depot/[...blah
blah...]": change  = 160991]" string at the end of the description
   |
  Merge remote branch 'p4/master'
   | \
   |  (p4/HEAD)  (p4/master)  a commit that is real work
   |   |
   |   more real work
   |   |
   |   more real work
   |   |
   Merge remote branch 'p4/master'
   |  \|
   some commit in my thrashing about
   |   |
   |   more real work
   |   |
   .   .
   .   .
   .   .
   |   |
   |   initial import of no-history linux sources that I put in manually
via Perforce
   |
   .
   .
   .
   |
   Linux-2.6.12-rc2
 

I think my understanding of merges and rebases needs more depth...and I
think I have mangled branches.


I tried a checkout of master and a "git rebase remotes/p4/master" and
that produced thousands of conflicts.  Was that due to my initial Linux
sources put in on the Perforce side?


How do I untangle myself here?


I think I am about to be saved, thanks so much in advance for that!


-kb

Re: git-p4

[Tor Arvid Lund]

2010/9/12 Kent Borg <kentborg@borg.org>:
> Tor Arvid Lund wrote:
>> Ok, so if you call this on the cmd line, it should output sha1's on
>> all commits that will be submitted (in reverse order) to p4. If it
>> doesn't, this may well be a good place to dig for a solution.
>
> It does output the commits it wants to submit.  In my broken case far
> too many.
>
> I created a simple working case to see what working looks like, and the
> exact same command outputs just the one commit.  Looking more carefully
> at my gitg pictures in the good and bad cases, I realize that I in the
> broken case I probably don't have the graph I need.
>
>
> Good case:
>
>  (master)
>   |
>  (p4/HEAD)  (p4/master)
>   |
>  initial test import from existing depot
>
>
> Bad case:
>
>  (master)   trivial change I am trying to submit
>   |
>  (tmp)  commit with the "[git-p4: depot-paths = "//depot/[...blah
> blah...]": change  = 160991]" string at the end of the description
>   |
>  Merge remote branch 'p4/master'
>   | \
>   |  (p4/HEAD)  (p4/master)  a commit that is real work
<snip>

Hmm.. Let's define a "commit that contains that [git-p4:
depot-paths=...] stuff" as a git/perforce commit. Then I would say
that p4/master should always point to the most recent git/perforce
commit. In your "Bad case", it seems that it doesn't do that, because
the most recent git/perforce commit is (tmp).

When messing about with git-p4 I have sometimes messed up that
p4/master ref myself. I can normally fix it by setting it manually: In
git, all existing branches are simply files with a sha1 in them (or
symbolic refs that point to other refs, like "ref:
refs/heads/master"). You should thus be able to put the sha1 of your
(tmp) commit into refs/remotes/p4/master.

You could try that and see if git rev-list gives the expected result.

> I think my understanding of merges and rebases needs more depth...and I
> think I have mangled branches.
>
>
> I tried a checkout of master and a "git rebase remotes/p4/master" and
> that produced thousands of conflicts.  Was that due to my initial Linux
> sources put in on the Perforce side?

Well, when you do that, git will look back until it finds a common
ancestor commit of master and p4/master. I am guessing, that a common
ancestor for those branches does not exist in your case... If my guess
is correct, git will go back to the beginning of master (to linux
2.6.12-rc2) and try to apply each of those commits on top of
p4/master. Naturally, that will cause a world of pain and conflicts ;)

-Tor Arvid-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
> When messing about with git-p4 I have sometimes messed up that
> p4/master ref myself. I can normally fix it by setting it manually

Um, I maybe just did that before I saw your e-mail.

I edited:

  .git/info/refs
  .git/ORIG_HEAD
  .git/packed/refs

...and put in the sha1 of the "[git-p4:"-commit.

When I now do:

  $ git rev-list --no-merges remotes/p4/master..master

I get just the one commit that I hoped to get!

...did I just fix it?

I don't have time to try the "git p4 submit" now, I am going to be late
in meeting my wife.  Tomorrow morning...will it work?


Thanks so very much for your help,

-kb

Re: git-p4

[Tor Arvid Lund]

2010/9/12 Kent Borg <kentborg@borg.org>:
> Tor Arvid Lund wrote:
>> When messing about with git-p4 I have sometimes messed up that
>> p4/master ref myself. I can normally fix it by setting it manually
>
> Um, I maybe just did that before I saw your e-mail.
>
> I edited:
>
>  .git/info/refs
>  .git/ORIG_HEAD
>  .git/packed/refs
>
> ...and put in the sha1 of the "[git-p4:"-commit.
>
> When I now do:
>
>  $ git rev-list --no-merges remotes/p4/master..master
>
> I get just the one commit that I hoped to get!
>
> ...did I just fix it?

My guess: Yes! :)

> I don't have time to try the "git p4 submit" now, I am going to be late
> in meeting my wife.  Tomorrow morning...will it work?

I'm optimistic :)

-TA-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
>> I don't have time to try the "git p4 submit" now, I am going to be late
>> in meeting my wife.  Tomorrow morning...will it work?
>>     
>
> I'm optimistic :)

Um, not so great.

Two problems:

 - I was on the wrong branch this morning when I did the
   "git p4 submit" (stupid me),

 - I realize that even if I had done that right, part of my
   history is a big lump of Linux kernel files with no history
   but a sync out of Perforce.

So it seems what I need to do is get on a clean branch that has good
history and cherry-pick all the bits I want off the branch with the bad
ancestry onto the clean branch.  (And get a good "[git-p4: depot-paths:
..." in place.)

Then...just point p4/HEAD and p4/master and master to the newest commit
on that new clean branch.  Voila!

Possibly one of my last stupid questions on this: How do I do that? 
"p4/master" looks like a remote, but "git remote" doesn't seem to know
about it.  In gitg "master" is green, just like a branch I might create,
so I figure I can delete and re-create it where I want, but "p4/master"
and "p4/HEAD" is blue.  What are these blue things?  How do I move them
to a different commit?

Thanks,

-kb

Re: git-p4

[Tor Arvid Lund]

On Mon, Sep 13, 2010 at 4:23 PM, Kent Borg <kentborg@borg.org> wrote:
> Tor Arvid Lund wrote:
>>> I don't have time to try the "git p4 submit" now, I am going to be late
>>> in meeting my wife.  Tomorrow morning...will it work?
>>>
>>
>> I'm optimistic :)
>
> Um, not so great.
>
> Two problems:
>
>  - I was on the wrong branch this morning when I did the
>   "git p4 submit" (stupid me),
>
>  - I realize that even if I had done that right, part of my
>   history is a big lump of Linux kernel files with no history
>   but a sync out of Perforce.

I don't know if I understand what you mean here... If "git diff
p4/master.." gives output equivalent to what you would want to submit
to p4, then that seems correct to me.

> So it seems what I need to do is get on a clean branch that has good
> history and cherry-pick all the bits I want off the branch with the bad
> ancestry onto the clean branch.  (And get a good "[git-p4: depot-paths:
> ..." in place.)
>
> Then...just point p4/HEAD and p4/master and master to the newest commit
> on that new clean branch.  Voila!
>
> Possibly one of my last stupid questions on this: How do I do that?
> "p4/master" looks like a remote, but "git remote" doesn't seem to know
> about it.  In gitg "master" is green, just like a branch I might create,
> so I figure I can delete and re-create it where I want, but "p4/master"
> and "p4/HEAD" is blue.  What are these blue things?  How do I move them
> to a different commit?

Well, I don't know gitg, but I can guess that it shows branches as
blue if they are files created inside refs/remotes/*. This is the case
for p4/master and p4/HEAD. They are placed there because normally,
users should not update these refs, nor should they directly do
checkouts of these branches (but your case is not exactly that
normal).

git remote, shows remotes as defined in your .git/config file (the
[remote "someremotename"] sections). git-p4 does not need nor create
such entries.

-Tor Arvid-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
> git remote, shows remotes as defined in your .git/config file (the
> [remote "someremotename"] sections). git-p4 does not need nor create
> such entries.

I am confused trying to understand what kind of beast the p4/master is.

I think my fundamental problem is that I need p4/master to point at a
branch with a real git history, not a sync from Perforce (which has no
git history).

I tried pointing p4/master at a clean branch in my repository with "git
update-ref refs/remotes/p4/master ...", but every time I tried a "git p4
sync --verbose" it sprang back to pointing at the old commit (the one
without a full git history). 

So I figured I have brokenness I might be able to leave behind if I
tried making a new "git p4 clone", doing a "git remote add ..." to the
branch I like in my old git repository, and trying it again...but I get
the same result. 

After my update-ref I grep in .git for the commit hash that keeps
grabbing my p4/master...and that hash is only in
logs/refs/remotes/p4/master.    I do a "git p4 sync" and the hash
reappears in refs/remotes/p4/master.  Where is it coming from?

How do I change where p4/master points?  (Put another way, how can I
have a complete Linux history in git but only new work in Perforce?)


Thanks,

-kb, the Kent who is getting a lot smarter about git in the last few
days, but for whom git-p4 is not yet in control.

Re: git-p4

[Tor Arvid Lund]

On Mon, Sep 13, 2010 at 6:28 PM, Kent Borg <kentborg@borg.org> wrote:
> Tor Arvid Lund wrote:
>> git remote, shows remotes as defined in your .git/config file (the
>> [remote "someremotename"] sections). git-p4 does not need nor create
>> such entries.
>
> I am confused trying to understand what kind of beast the p4/master is.
>
> I think my fundamental problem is that I need p4/master to point at a
> branch with a real git history, not a sync from Perforce (which has no
> git history).

Well, then I think you are a bit confused ;)

The p4/master branch is git's view of your p4 history. So p4/master
points to the most recent git/perforce commit. An important side point
here, is that if you have another remote (which you do in your case)
that is a pure git remote that knows nothing about p4, then the
p4/master branch and the origin/master branch will be disjoint.

> How do I change where p4/master points?  (Put another way, how can I
> have a complete Linux history in git but only new work in Perforce?)

Ehm, thinking about this some more, I wonder if this whole endeavor
might be continually painful...

Say that you clone Linus' kernel tree with git, and that commit X is
the master where you will start your work.

You then import your tree at commit X into p4, and sync it back using
git-p4. That git-p4 sync will give you a commit X' - Such that
checking out X or X' will yield the same checkout tree, but they will
have differing commit IDs... Do you have a strategy if you:

1) Commit something, Y', on top of X' in git (and syncs with p4).
2) Someone else commits Z on top of X, which you get when you do git fetch.

What do you do in p4? You can't be sure that it'll work to apply Z on
top of Y', nor the other way around...

It kind of seems like you want to have two different repositories, and
you want these to be equal (no one is the "one true repository"), and
this presents problems in the scenario I described above, right?

I think maybe draw a few branch trees for git and p4/git-p4 and your
own work on a good old piece of paper... It often helps to clear the
head and think freshly (is that a word?).

-Tor Arvid-

Re: git-p4

[Kent Borg]

Tor Arvid Lund wrote:
> Well, then I think you are a bit confused ;)
>   

That I know is true, but I am making progress.

 - I have "git p4 rebase"-d changes from p4 world out to
   git.  More than once even.

 - I have "git p4 submit"-ted changes from git back into
   p4 world.  Again, more than once.

 - I can pull and push from/to this git repository to my primary git
repository.

> The p4/master branch is git's view of your p4 history. So p4/master
> points to the most recent git/perforce commit. 

Yes.

> An important side point
> here, is that if you have another remote (which you do in your case)
> that is a pure git remote that knows nothing about p4, then the
> p4/master branch and the origin/master branch will be disjoint.
>   

That, I think I fixed!  The first commit on the p4/master branch used to
be a sync from p4, but after surgery on branch references (correct
term?) my gateway git-p4 repository's p4/master branch now has history
all the way back to the beginning of time in the git universe (back to
good ol' Linux 2.6.12-rc2).  The recent commits have git-p4 comments
that mark the matching p4 changesets.  I am not sure exactly how I did
it, but it seems that doing a "git-p4 rebase" instead of "git-p4 sync"
made my surgery work.

One odd thing that had me worried was seeing the git side of the gateway
repository show a single history back and then show a short split
history and then a single history, flopping as I ran transactions
through it.  I am not sure what was going on, but I think git-p4 is
doing an amend of the last commit to put its notes in the message, and
if I have anything newer hanging from that commit this is a very bad
thing.  I am still worried but less so as long as I behave myself about
not expecting it to make amendments to anything but the newest commits.

Part of the consideration is to simply be very aware of those "[git-p4:
..." notes and decide where this should propagate to and design the
workflow accordingly.  (lkml probably won't want to see p4 notations...)

But anyway, I seem to have git-p4 working in both directions, with a
complete beginning-of-time history on the git side.


Tor Arvid: I owe you a beer (or whatever you drink when someone offers
you a beer), how often do you visit Boston?


Thanks for everyone's patience with a newbie,

-kb

Re: git-p4

[Tor Arvid Lund]

On Thu, Sep 16, 2010 at 2:14 PM, Kent Borg <kentborg@borg.org> wrote:
> One odd thing that had me worried was seeing the git side of the gateway
> repository show a single history back and then show a short split
> history and then a single history, flopping as I ran transactions
> through it.  I am not sure what was going on, but I think git-p4 is
> doing an amend of the last commit to put its notes in the message, and
> if I have anything newer hanging from that commit this is a very bad
> thing.  I am still worried but less so as long as I behave myself about
> not expecting it to make amendments to anything but the newest commits.

This is true. git-p4 does rebase (which usually rewrites history) the
active branch as the last step when you do git-p4 submit. So, as you
say, it is important to be aware of this.

If HEAD points to X, and you do git-p4 submit, then if you have
another branch YY on top of X, you may want to checkout YY and do git
rebase X' (where X' is what git-p4 produces after it amends its
[git-p4: ...] stuff).

> Part of the consideration is to simply be very aware of those "[git-p4:
> ..." notes and decide where this should propagate to and design the
> workflow accordingly.  (lkml probably won't want to see p4 notations...)
>
> But anyway, I seem to have git-p4 working in both directions, with a
> complete beginning-of-time history on the git side.

Good stuff! Congrats :)

> Tor Arvid: I owe you a beer (or whatever you drink when someone offers
> you a beer), how often do you visit Boston?

Appreciated :) Well, its been ~9 years, so maybe I should go again
soon :) Otherwise, say hello whenever you're in Oslo ;)

-TA-

git-p4

[dhruva]

Hi,
 I am eager to have a full blown git-p4 with ability to _seamlessly_ submit to perforce. The one way pulling of p4 changesets to git works fine except for the un-expanding of RCS keywords like "$Id: ......$" to "$Id$". When submiting back to p4, the file would not have recorded any diff in the keyword which would have changed in p4 due to modifications. Applying the patch during the submission process fails. Also, the 'git-p4 submit --continue' fails saying that '--continue' is an invalid argument.
 I can work on fixing these issues but need help from someone with knowledge of this script 'git-p4'. I have basic expereince in git and am a newbie in p4. I am willing to invest my time in this area. I have written a p4 to hg converter in perl which I have been using. I feel git-p4 is more complete than the script I have written especially in handling file mode.

-dhruva


      Unlimited freedom, unlimited storage. Get it now, on http://help.yahoo..com/l/in/yahoo/mail/yahoomail/tools/tools-08.html/

git-p4

[dhruva]

Hi,
 I am eager to have a full blown git-p4 with ability to _seamlessly_ submit to perforce. The one way pulling of p4 changesets to git works fine except for the un-expanding of RCS keywords like "$Id: ......$" to "$Id$". When submiting back to p4, the file would not have recorded any diff in the keyword which would have changed in p4 due to modifications. Applying the patch during the submission process fails. Also, the 'git-p4 submit --continue' fails saying that '--continue' is an invalid argument.
 I can work on fixing these issues but need help from someone with knowledge of this script 'git-p4'. I have basic expereince in git and am a newbie in p4. I am willing to invest my time in this area. I have written a p4 to hg converter in perl which I have been using. I feel git-p4 is more complete than the script I have written especially in handling file mode.

-dhruva


      Bollywood news, movie reviews, film trailers and more! Go to http://in..movies.yahoo.com/