[PATCH v2 0/5] Check replacement object type and minor updates

[PATCH v2 0/5] Check replacement object type and minor updates

[Christian Couder]

The only changes compared to the previous version are those
suggested by Johannes:

- the error message is now:

	"Objects must be of the same type.\n"
	"'%s' points to a replaced object of type '%s'\n"
	"while '%s' points to a replacement object of type '%s'."

- the test uses "." instead of "'\''"

Only patchs 1/5 and 3/5 were changed.

Christian Couder (5):
  replace: forbid replacing an object with one of a different type
  Documentation/replace: state that objects must be of the same type
  t6050-replace: test that objects are of the same type
  t6050-replace: add test to clean up all the replace refs
  Documentation/replace: add Creating Replacement Objects section

 Documentation/git-replace.txt | 23 ++++++++++++++++++++---
 builtin/replace.c             | 10 ++++++++++
 t/t6050-replace.sh            | 19 +++++++++++++++++++
 3 files changed, 49 insertions(+), 3 deletions(-)

-- 
1.8.4.rc1.26.gdd51ee9

[PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Christian Couder]

Users replacing an object with one of a different type were not
prevented to do so, even if it was obvious, and stated in the doc,
that bad things would result from doing that.

To avoid mistakes, it is better to just forbid that though.

There is no case where one object can be replaced with one of a
different type while keeping the history valid, because:

* Annotated tags contain the type of the tagged object.

* The tree/parent lines in commits must be a tree and commits, resp.

* The object types referred to by trees are specified in the 'mode'
  field:
    100644 and 100755    blob
    160000               commit
    040000               tree
  (these are the only valid modes)

* Blobs don't point at anything.

The doc will be updated in a later patch.

Acked-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 builtin/replace.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/builtin/replace.c b/builtin/replace.c
index 59d3115..9a94769 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -85,6 +85,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 			  int force)
 {
 	unsigned char object[20], prev[20], repl[20];
+	enum object_type obj_type, repl_type;
 	char ref[PATH_MAX];
 	struct ref_lock *lock;
 
@@ -100,6 +101,15 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	if (check_refname_format(ref, 0))
 		die("'%s' is not a valid ref name.", ref);
 
+	obj_type = sha1_object_info(object, NULL);
+	repl_type = sha1_object_info(repl, NULL);
+	if (obj_type != repl_type)
+		die("Objects must be of the same type.\n"
+		    "'%s' points to a replaced object of type '%s'\n"
+		    "while '%s' points to a replacement object of type '%s'.",
+		    object_ref, typename(obj_type),
+		    replace_ref, typename(repl_type));
+
 	if (read_ref(ref, prev))
 		hashclr(prev);
 	else if (!force)
-- 
1.8.4.rc1.26.gdd51ee9

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> Users replacing an object with one of a different type were not
> prevented to do so, even if it was obvious, and stated in the doc,
> that bad things would result from doing that.
>
> To avoid mistakes, it is better to just forbid that though.
>
> There is no case where one object can be replaced with one of a
> different type while keeping the history valid, because:
>
> * Annotated tags contain the type of the tagged object.

If you replace the tagged object and the tag at the same time,
wouldn't that make the resulting history valid again?

Granted, there may not be a strong reason to reuse the object name
of the tagged object in such a case, but this "there may not be" is
merely "I do not think of offhand", so I am not sure what workflow
of other people we are breaking with this change.  A light-weight
tag may already point at the tagged object (in other words, the
object name of the tagged object is known to the outside world) and
that could be a reason why you would need to reuse the object name
of that object while changing its type.

I dunno.

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Thomas Rast]

Junio C Hamano <gitster@pobox.com> writes:

> Christian Couder <chriscool@tuxfamily.org> writes:
>
>> Users replacing an object with one of a different type were not
>> prevented to do so, even if it was obvious, and stated in the doc,
>> that bad things would result from doing that.
>>
>> To avoid mistakes, it is better to just forbid that though.
>>
>> There is no case where one object can be replaced with one of a
>> different type while keeping the history valid, because:
>>
>> * Annotated tags contain the type of the tagged object.
>
> If you replace the tagged object and the tag at the same time,
> wouldn't that make the resulting history valid again?
>
> Granted, there may not be a strong reason to reuse the object name
> of the tagged object in such a case, but this "there may not be" is
> merely "I do not think of offhand", so I am not sure what workflow
> of other people we are breaking with this change.  A light-weight
> tag may already point at the tagged object (in other words, the
> object name of the tagged object is known to the outside world) and
> that could be a reason why you would need to reuse the object name
> of that object while changing its type.
>
> I dunno.

Hrm, you're right, that's a flaw in my logic.  You could do the same in
all other cases too, e.g. replace a tree so that an entry is of a
different type and at the same time change the type of the object
itself.  You however have to carefully go through all objects that refer
to the one that was replaced, and fix the type in all of them.

It still seems an extremely unsafe thing to do with trees: especially
for small trees there is a small probability that you will generate the
same tree again in the future (by having the same blobs in the directory
again) and record it as a subtree or the 'tree' field of a commit.  The
history would then again be invalid.

Should we add a --force flag of some sort to allow the user to do this,
while keeping the normal safety checks?

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Junio C Hamano]

Thomas Rast <trast@inf.ethz.ch> writes:

> Hrm, you're right, that's a flaw in my logic.  You could do the same in
> all other cases too, e.g. replace a tree so that an entry is of a
> different type and at the same time change the type of the object
> itself.  You however have to carefully go through all objects that refer
> to the one that was replaced, and fix the type in all of them.
>
> It still seems an extremely unsafe thing to do with trees...
>  ...
> Should we add a --force flag of some sort to allow the user to do this,
> while keeping the normal safety checks?

As long as we do not forbid such an unusual replacement on the
reading side, we won't break people who are more inventive than we
are (I am not convinced that we know people's workflow well enough
to definitively say that no sane workflow, which benefits from being
able to replace an object with another from a different type,
exists).

Preventing "git replace" wrapper from creating such a replacement by
default will make it harder to do and may reduce mistakes, without
breaking them too much, I think.

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Christian Couder]

From: Junio C Hamano <gitster@pobox.com>
>
> Thomas Rast <trast@inf.ethz.ch> writes:
> 
>> Hrm, you're right, that's a flaw in my logic.  You could do the same in
>> all other cases too, e.g. replace a tree so that an entry is of a
>> different type and at the same time change the type of the object
>> itself.  You however have to carefully go through all objects that refer
>> to the one that was replaced, and fix the type in all of them.
>>
>> It still seems an extremely unsafe thing to do with trees...
>>  ...
>> Should we add a --force flag of some sort to allow the user to do this,
>> while keeping the normal safety checks?
> 
> As long as we do not forbid such an unusual replacement on the
> reading side, we won't break people who are more inventive than we
> are (I am not convinced that we know people's workflow well enough
> to definitively say that no sane workflow, which benefits from being
> able to replace an object with another from a different type,
> exists).
> 
> Preventing "git replace" wrapper from creating such a replacement by
> default will make it harder to do and may reduce mistakes, without
> breaking them too much, I think.

I agree. It is always possible to create replacement refs using "git
update-ref" if one really wants to.

What about using the following in the commit message or in the
documentation:

------------------------------------------

DISCUSSION

If one object is replaced with one of a different type, the only way
to keep the history valid is to also replace all the other objects
that point to the replaced object. That's because:

* Annotated tags contain the type of the tagged object.

* The tree/parent lines in commits must be a tree and commits, resp.

* The object types referred to by trees are specified in the 'mode'
  field:
    100644 and 100755    blob
    160000               commit
    040000               tree
  (these are the only valid modes)

* Blobs don't point at anything.

But if all the objects that point to an object, called O, are to be
replaced, then in most cases object O probably doesn't need to be
replaced. It's probably sufficient to create the new object, called
O2, that would replace object O and to replace all the objects
pointing to object O with objects pointing to O2.

The only case where someone might really want to replace object 0,
with an object O2 of a different type, and all the objects pointing to
it, is if it's really important, perhaps for external reasons, to have
object O's SHA1 point to O2.

And anyway, if one really wants to do that, it can still be done using
"git update-ref".

------------------------------------------

Thanks,
Christian.

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> But if all the objects that point to an object, called O, are to be
> replaced, then in most cases object O probably doesn't need to be
> replaced. It's probably sufficient to create the new object, called
> O2, that would replace object O and to replace all the objects
> pointing to object O with objects pointing to O2.

Hmmmm.

What the above says, with "probably" and "most cases", can easily
inferred by anybody remotely intelligent, and the only reason to
have something like the above paragraph would be if it assures that
the statement holds without these qualifications to weaken it, which
it doesn't.  I am not sure this paragraph adds much value.

> The only case where someone might really want to replace object 0,
> with an object O2 of a different type, and all the objects pointing to
> it, is if it's really important, perhaps for external reasons, to have
> object O's SHA1 point to O2.

The same comment applies here.

> And anyway, if one really wants to do that, it can still be done using
> "git update-ref".

And I really do not think this sentence is right---you can justify
with the same sentence to remove "git replace" wrapper.

The earlier suggestion to bypass the new hand-holding with "--force"
is more sensible, I think.

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Christian Couder]

From: Junio C Hamano <gitster@pobox.com>
>
> Christian Couder <chriscool@tuxfamily.org> writes:
> 
>> But if all the objects that point to an object, called O, are to be
>> replaced, then in most cases object O probably doesn't need to be
>> replaced. It's probably sufficient to create the new object, called
>> O2, that would replace object O and to replace all the objects
>> pointing to object O with objects pointing to O2.
> 
> Hmmmm.
> 
> What the above says, with "probably" and "most cases", can easily
> inferred by anybody remotely intelligent, and the only reason to
> have something like the above paragraph would be if it assures that
> the statement holds without these qualifications to weaken it, which
> it doesn't.  I am not sure this paragraph adds much value.
> 
>> The only case where someone might really want to replace object 0,
>> with an object O2 of a different type, and all the objects pointing to
>> it, is if it's really important, perhaps for external reasons, to have
>> object O's SHA1 point to O2.
> 
> The same comment applies here.
> 
>> And anyway, if one really wants to do that, it can still be done using
>> "git update-ref".
> 
> And I really do not think this sentence is right---you can justify
> with the same sentence to remove "git replace" wrapper.

Ok, so the commit message (excluding the Acked-by and Signed-off-by)
will be only:

------------------------------------------

Users replacing an object with one of a different type were not
prevented to do so, even if it was obvious, and stated in the doc,
that bad things would result from doing that.

To avoid mistakes, it is better to just forbid that though.

If one object is replaced with one of a different type, the only way
to keep the history valid is to also replace all the other objects
that point to the replaced object. That's because:

* Annotated tags contain the type of the tagged object.

* The tree/parent lines in commits must be a tree and commits, resp.

* The object types referred to by trees are specified in the 'mode'
  field:
    100644 and 100755    blob
    160000               commit
    040000               tree
  (these are the only valid modes)

* Blobs don't point at anything.

The doc will be updated in a later patch.

------------------------------------------

> The earlier suggestion to bypass the new hand-holding with "--force"
> is more sensible, I think.

There is already a --force option, but I can add a --force-type in a
another patch.

Thanks,
Christian.

Re: [PATCH v2 1/5] replace: forbid replacing an object with one of a different type

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> There is already a --force option, but I can add a --force-type in a
> another patch.

Oh, that was not what I meant.  As long as existing --force override
this check, that should be sufficient and more preferrable than yet
another kind of "force".

Thanks.

[PATCH v2 2/5] Documentation/replace: state that objects must be of the same type

[Christian Couder]

A previous patch ensures that both the replaced and the replacement
objects passed to git replace must be of the same type.

While at it state that there is no other restriction on both objects.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/git-replace.txt | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt
index e0b4057..aa66d27 100644
--- a/Documentation/git-replace.txt
+++ b/Documentation/git-replace.txt
@@ -20,6 +20,9 @@ The name of the 'replace' reference is the SHA-1 of the object that is
 replaced. The content of the 'replace' reference is the SHA-1 of the
 replacement object.
 
+The replaced object and the replacement object must be of the same type.
+There is no other restriction on them.
+
 Unless `-f` is given, the 'replace' reference must not yet exist.
 
 Replacement references will be used by default by all Git commands
@@ -69,9 +72,7 @@ go back to a replaced commit will move the branch to the replacement
 commit instead of the replaced commit.
 
 There may be other problems when using 'git rev-list' related to
-pending objects. And of course things may break if an object of one
-type is replaced by an object of another type (for example a blob
-replaced by a commit).
+pending objects.
 
 SEE ALSO
 --------
-- 
1.8.4.rc1.26.gdd51ee9

[PATCH v2 3/5] t6050-replace: test that objects are of the same type

[Christian Couder]

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/t6050-replace.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/t/t6050-replace.sh b/t/t6050-replace.sh
index decdc33..5c352c4 100755
--- a/t/t6050-replace.sh
+++ b/t/t6050-replace.sh
@@ -263,4 +263,17 @@ test_expect_success 'not just commits' '
 	test_cmp file.replaced file
 '
 
+test_expect_success 'replaced and replacement objects must be of the same type' '
+	test_must_fail git replace mytag $HASH1 2>err &&
+	grep "mytag. points to a replaced object of type .tag" err &&
+	grep "$HASH1. points to a replacement object of type .commit" err &&
+	test_must_fail git replace HEAD^{tree} HEAD~1 2>err &&
+	grep "HEAD^{tree}. points to a replaced object of type .tree" err &&
+	grep "HEAD~1. points to a replacement object of type .commit" err &&
+	BLOB=$(git rev-parse :file) &&
+	test_must_fail git replace HEAD^ $BLOB 2>err &&
+	grep "HEAD^. points to a replaced object of type .commit" err &&
+	grep "$BLOB. points to a replacement object of type .blob" err
+'
+
 test_done
-- 
1.8.4.rc1.26.gdd51ee9

[PATCH v2 4/5] t6050-replace: add test to clean up all the replace refs

[Christian Couder]

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/t6050-replace.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/t/t6050-replace.sh b/t/t6050-replace.sh
index 5c352c4..05be228 100755
--- a/t/t6050-replace.sh
+++ b/t/t6050-replace.sh
@@ -276,4 +276,10 @@ test_expect_success 'replaced and replacement objects must be of the same type'
 	grep "$BLOB. points to a replacement object of type .blob" err
 '
 
+test_expect_success 'replace ref cleanup' '
+	test -n "$(git replace)" &&
+	git replace -d $(git replace) &&
+	test -z "$(git replace)"
+'
+
 test_done
-- 
1.8.4.rc1.26.gdd51ee9

[PATCH v2 5/5] Documentation/replace: add Creating Replacement Objects section

[Christian Couder]

There were no hints in the documentation about how to create
replacement objects.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/git-replace.txt | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt
index aa66d27..736b48c 100644
--- a/Documentation/git-replace.txt
+++ b/Documentation/git-replace.txt
@@ -64,6 +64,19 @@ OPTIONS
 	Typing "git replace" without arguments, also lists all replace
 	refs.
 
+CREATING REPLACEMENT OBJECTS
+----------------------------
+
+linkgit:git-filter-branch[1], linkgit:git-hash-object[1] and
+linkgit:git-rebase[1], among other git commands, can be used to create
+replacement objects from existing objects.
+
+If you want to replace many blobs, trees or commits that are part of a
+string of commits, you may just want to create a replacement string of
+commits and then only replace the commit at the tip of the target
+string of commits with the commit at the tip of the replacement string
+of commits.
+
 BUGS
 ----
 Comparing blobs or trees that have been replaced with those that
@@ -76,6 +89,9 @@ pending objects.
 
 SEE ALSO
 --------
+linkgit:git-hash-object[1]
+linkgit:git-filter-branch[1]
+linkgit:git-rebase[1]
 linkgit:git-tag[1]
 linkgit:git-branch[1]
 linkgit:git[1]
-- 
1.8.4.rc1.26.gdd51ee9