Git History Rewriting in a public repository - capability to remove one or more commits from a public repository

Git History Rewriting in a public repository - capability to remove one or more commits from a public repository

[Saravanan Shanmugham (sarvi)]

Hi Git Leads,
      I am looking for git capability/way to be able to remove commits
from a public repository.

Background:
We are looking for a multi-stage commit process where commits get pushed
into a public ³testing-stage² repository.
Where we do testing of commits before they are pushed to another public
³mainline² repository.

When there are failures seen in the public ³testing-stage² repository.
We would like to implement some process to go identify the bad patch and
completely eject it from that public ³testing-stage² repository, as if it
was not connected.

The plan is to use the Git History Rewriting capability described here
https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History
So I can pull a pull workspace from the public ³testing-stage² repository
use the above mechanism to eject one or more commits from it.

Now I would like to be able push it back to public ³testing-stage²
repository. 
And allow other people to be able to sync their workspaces to this public
³testing-stage² repository, correctly.

This as I understand is not supported?/recommended? in GIT.

Mercurial addresses this with the capability to mark commits with a phase
such as ³Draft² or ³Experimental² and having a workflow around them.
Described here
https://www.mercurial-scm.org/wiki/Phases

http://www.gerg.ca/evolve/user-guide.html#evolve-user-guide



Question:
What are the issues?
What needs to be done in terms of development, to support this
functionality and make it work properly in GIT?
Is there additional development that needs to be done to git core to allow
this development process?


Thanks,
Sarvi
-----
Occam's Razor Rules

Re: Git History Rewriting in a public repository - capability to remove one or more commits from a public repository

[Stefan Beller]

On Tue, Feb 23, 2016 at 1:30 PM, Saravanan Shanmugham (sarvi)
<sarvi@cisco.com> wrote:
>
>
> Hi Git Leads,
>       I am looking for git capability/way to be able to remove commits
> from a public repository.
>
> Background:
> We are looking for a multi-stage commit process where commits get pushed
> into a public ³testing-stage² repository.
> Where we do testing of commits before they are pushed to another public
> ³mainline² repository.

instead of different repositories you could have different branches,
but as you like...

>
> When there are failures seen in the public ³testing-stage² repository.
> We would like to implement some process to go identify the bad patch and
> completely eject it from that public ³testing-stage² repository, as if it
> was not connected.

So git revert is not an option, but you really want to purge the commit
as if it never existed?

That is not possible without rewriting history (which is considered bad
behavior for public repositories)

You can use a cherry-picking workflow where you cherry-pick the good
commits, once you are sure they are good indeed.

>
> The plan is to use the Git History Rewriting capability described here
> https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History
> So I can pull a pull workspace from the public ³testing-stage² repository
> use the above mechanism to eject one or more commits from it.
>
> Now I would like to be able push it back to public ³testing-stage²
> repository.

when using the cherry-pick workflow above (cherry picking from testing
to public),
you can still merge back public to testing

> And allow other people to be able to sync their workspaces to this public
> ³testing-stage² repository, correctly.

    git fetch testing && git reset --hard testing-stage test-branch

would do that client side.

>
> This as I understand is not supported?/recommended? in GIT.

It is supported as Git is a toolkit and you can align your workflow using
different tools from the box.

But it's not what anyone with prior knowledge of "How to work with Git"
expects things to be.

>
> Mercurial addresses this with the capability to mark commits with a phase
> such as ³Draft² or ³Experimental² and having a workflow around them.
> Described here
> https://www.mercurial-scm.org/wiki/Phases
>
> http://www.gerg.ca/evolve/user-guide.html#evolve-user-guide
>
>
>
> Question:
> What are the issues?

Rewriting history is considered bad practice, and that is probably the reason
why there are no good tools AFAIK to verify rewritten history easily.

When you consider published history permanent, you can use gpg or local tags
to know what "is already good" and only inspect new incoming deltas
for correctness.

> What needs to be done in terms of development, to support this
> functionality and make it work properly in GIT?
> Is there additional development that needs to be done to git core to allow
> this development process?

I think the workflow you described can be done using current tools.
You would just need to
polish things or create aliases for things as this seems to be an
unusual workflow?

Thanks,
Stefan

>
>
> Thanks,
> Sarvi
> -----
> Occam's Razor Rules
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git History Rewriting in a public repository - capability to remove one or more commits from a public repository

[Ilya Terentyev]

Hi Saravanan,

Changes that rewrite history, including (but not limited to) deleted 
commits,
can be pushed with the --force or --force-with-lease options, like this:

     $ git push --force remote branch

--force pushes your changes unconditionally, which may overwrite changes
that someone else pushed between the moment you cloned the repo and pushed
your own. --force-with-lease will check for others' pushes, so you can 
use it
in a dry run (without actually changing anything in the remote repository)
like this:

     $ git push -n --force-with-lease remote branch

If someone else (like another developer with access to "testing-stage")
pushes anything before your attempt to push, you will receive a message 
like:

     $ git push -n --force-with-lease remote branch
     ! [rejected]        branch -> branch (stale info)
     error: failed to push some refs to remote

Generally speaking, your idea is, probably, better implemented with patches
or pull requests:

  1) Your developers rewrite their local history as they wish
  2) They generate patches from their commits (with git format-patch,
     for instance)
  3) Send those patches to "testing-stage"
  4) Apply them to staging area (without committing)
  5) Run required checks
  6) If checks don't pass, discard those changes
  7) If checks pass, commit those patches
  8) Push committed changes to "mainline"

But in any case, you should better consider using feature branches for that.

Best regards,
Ilya T.


On 02/24/2016 12:30 AM, Saravanan Shanmugham (sarvi) wrote:
> Hi Git Leads,
>        I am looking for git capability/way to be able to remove commits
> from a public repository.
>
> Background:
> We are looking for a multi-stage commit process where commits get pushed
> into a public ³testing-stage² repository.
> Where we do testing of commits before they are pushed to another public
> ³mainline² repository.
>
> When there are failures seen in the public ³testing-stage² repository.
> We would like to implement some process to go identify the bad patch and
> completely eject it from that public ³testing-stage² repository, as if it
> was not connected.
>
> The plan is to use the Git History Rewriting capability described here
> https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History
> So I can pull a pull workspace from the public ³testing-stage² repository
> use the above mechanism to eject one or more commits from it.
>
> Now I would like to be able push it back to public ³testing-stage²
> repository.
> And allow other people to be able to sync their workspaces to this public
> ³testing-stage² repository, correctly.
>
> This as I understand is not supported?/recommended? in GIT.
>
> Mercurial addresses this with the capability to mark commits with a phase
> such as ³Draft² or ³Experimental² and having a workflow around them.
> Described here
> https://www.mercurial-scm.org/wiki/Phases
>
> http://www.gerg.ca/evolve/user-guide.html#evolve-user-guide
>
>
>
> Question:
> What are the issues?
> What needs to be done in terms of development, to support this
> functionality and make it work properly in GIT?
> Is there additional development that needs to be done to git core to allow
> this development process?
>
>
> Thanks,
> Sarvi
> -----
> Occam's Razor Rules
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message tomajordomo@vger.kernel.org
> More majordomo info athttp://vger.kernel.org/majordomo-info.html

Re: Git History Rewriting in a public repository - capability to remove one or more commits from a public repository

[Saravanan Shanmugham (sarvi)]

Thanks Ilya and Stefan for your detailed responses.

In general I am trying to avoid having to work with patch sets/queues.
This may be acceptable as a replacement for the “testing-stage” repo.

But we don’t want to have the developer deal with patches or patch queues
in his workflow.

1. The developer will pulling from the “mainline” repository to  do
development.
2. They will be committing and pushing to the “testing-stage” repository
or branch as you suggest.
3. To do (2) above they will need to sync to “testing-stage” repository or
branch, which might pull in 20-30 commits that are already in that branch
or repository.
4. He might have to resolve some conflicts before he commits it into that
“testing-stage” repository or branch.
5. During the window of (4) above, Auto bisection testing process
happening on the “testing-stage” repository or branch might have kicked
out 3 commits from that repository
6. The kicking out of the 3 commits in (5) above, would require the
developer sync to the “testing-stage” repository or branch again before
pushing it.

We want the developer workflow at (3) and (6) above, to not have to deal
with patch queues and stuff.
For him to be able to do the “sync" to the “testing-stage” repository
branch in (3) and (6) above, allow him to merge the the commits he has
already made in his local repository with the rewritten history of commits
coming from the “testing-stage” repository or branch.


What I am hearing you suggest as the way forward is the following
1. Removing bad patches in “testin-stage” repository or branch
   1. Use may be branch instead of a separate repository for
“testing-stage”.
   2. Pull a workspace repo for removing the bad commit and related
commits in that workspace by rewriting history in it.
   3. Do git push -n --force-with-lease remote branch. If that fails
because additional commits have come bring in the additional changes and
re-attempt the push.
   4. When 3 completes successfully the history would have been rewritten
into the “testing-stage”

2. Developer has made commits to history space.
3. Developer syncs to “testing-stage” repo or branch, the first time, in
preparation for pushing his changes to that branch or repo. All should go
well. Since his repo has not seen history from that branch or repo till
that time.
3. Takes time resolve issues.(While the history has changed in the
“testing-stage” repo or branch)

4. He syncs again, which is where we will have a problem. “git pull” needs
to recognize that history has changed and fail. Will it do that?

5. Drawing from Stefan Beller’s response, it looks the developer has to
either manually or through git extension
    1. Recognize that the history has changed, during a git pull failure.
    2. Use "git cherrypick” get his own commits in his workspace have them
recommited over the new history that is being pulled from “testing-stage”
repo or branch. Do something along the lines of
http://think-like-a-git.net/sections/rebase-from-the-ground-up/using-git-ch
erry-pick-to-simulate-git-rebase.html to achieve that.

Do I have the workflow right?

Thanks,
Sarvi
-----
Occam's Razor Rules





On 2/23/16, 2:15 PM, "Ilya Terentyev" <bacondropped@gmail.com> wrote:

>Hi Saravanan,
>
>Changes that rewrite history, including (but not limited to) deleted
>commits,
>can be pushed with the --force or --force-with-lease options, like this:
>
>     $ git push --force remote branch
>
>--force pushes your changes unconditionally, which may overwrite changes
>that someone else pushed between the moment you cloned the repo and pushed
>your own. --force-with-lease will check for others' pushes, so you can
>use it
>in a dry run (without actually changing anything in the remote repository)
>like this:
>
>     $ git push -n --force-with-lease remote branch
>
>If someone else (like another developer with access to "testing-stage")
>pushes anything before your attempt to push, you will receive a message
>like:
>
>     $ git push -n --force-with-lease remote branch
>     ! [rejected]        branch -> branch (stale info)
>     error: failed to push some refs to remote
>
>Generally speaking, your idea is, probably, better implemented with
>patches
>or pull requests:
>
>  1) Your developers rewrite their local history as they wish
>  2) They generate patches from their commits (with git format-patch,
>     for instance)
>  3) Send those patches to "testing-stage"
>  4) Apply them to staging area (without committing)
>  5) Run required checks
>  6) If checks don't pass, discard those changes
>  7) If checks pass, commit those patches
>  8) Push committed changes to "mainline"
>
>But in any case, you should better consider using feature branches for
>that.
>
>Best regards,
>Ilya T.
>
>
>On 02/24/2016 12:30 AM, Saravanan Shanmugham (sarvi) wrote:
>> Hi Git Leads,
>>        I am looking for git capability/way to be able to remove commits
>> from a public repository.
>>
>> Background:
>> We are looking for a multi-stage commit process where commits get pushed
>> into a public ³testing-stage² repository.
>> Where we do testing of commits before they are pushed to another public
>> ³mainline² repository.
>>
>> When there are failures seen in the public ³testing-stage² repository.
>> We would like to implement some process to go identify the bad patch and
>> completely eject it from that public ³testing-stage² repository, as if
>>it
>> was not connected.
>>
>> The plan is to use the Git History Rewriting capability described here
>> https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History
>> So I can pull a pull workspace from the public ³testing-stage²
>>repository
>> use the above mechanism to eject one or more commits from it.
>>
>> Now I would like to be able push it back to public ³testing-stage²
>> repository.
>> And allow other people to be able to sync their workspaces to this
>>public
>> ³testing-stage² repository, correctly.
>>
>> This as I understand is not supported?/recommended? in GIT.
>>
>> Mercurial addresses this with the capability to mark commits with a
>>phase
>> such as ³Draft² or ³Experimental² and having a workflow around them.
>> Described here
>> https://www.mercurial-scm.org/wiki/Phases
>>
>> http://www.gerg.ca/evolve/user-guide.html#evolve-user-guide
>>
>>
>>
>> Question:
>> What are the issues?
>> What needs to be done in terms of development, to support this
>> functionality and make it work properly in GIT?
>> Is there additional development that needs to be done to git core to
>>allow
>> this development process?
>>
>>
>> Thanks,
>> Sarvi
>> -----
>> Occam's Razor Rules
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>