git apply vs. renamed files index mismatch (was: Re: [PATCH 1/4] POWERPC: Rename dma_64.c to dma.c)

git apply vs. renamed files index mismatch (was: Re: [PATCH 1/4] POWERPC: Rename dma_64.c to dma.c)

[Anton Vorontsov]

On Mon, Sep 08, 2008 at 02:18:42PM -0500, Scott Wood wrote:
> Becky Bruce wrote:
>> This is in preparation for the merge of the 32 and 64-bit
>> dma code in arch/powerpc.
>>
>> Signed-off-by: Becky Bruce <becky.bruce@freescale.com>
>> ---
>>  arch/powerpc/kernel/Makefile |    2 +-
>>  arch/powerpc/kernel/dma.c    |  200 ++++++++++++++++++++++++++++++++++++++++++
>>  arch/powerpc/kernel/dma_64.c |  200 ------------------------------------------
>>  3 files changed, 201 insertions(+), 201 deletions(-)
>>  create mode 100644 arch/powerpc/kernel/dma.c
>>  delete mode 100644 arch/powerpc/kernel/dma_64.c
>
> Passing -M to git format-patch makes it much easier

I always thought that posting "-M" patches to the public lists is
discouraged since it is quite difficult to apply them via patch(1).
Also think of non-git users...

> to see whether  
> anything changed between the old file and the new file.

This is still possible by comparing the hashes:

diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
new file mode 100644
index 0000000..ae5708e
[...]
diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
deleted file mode 100644
index ae5708e..0000000

That is, if hashes match then it was pure rename.

Though, too bad git {apply,am} does not produce any warnings if there
are any hidden changes...

Cc'ing git mailing list.

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: git apply vs. renamed files index mismatch

[Scott Wood]

Anton Vorontsov wrote:
> I always thought that posting "-M" patches to the public lists is
> discouraged since it is quite difficult to apply them via patch(1).
> Also think of non-git users...

I think the substantially enhanced reviewability trumps non-git-users 
who can follow the rename instructions manually (or fix up their patch 
utility) if they insist on shunning tools that would make their life easier.

> diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
> new file mode 100644
> index 0000000..ae5708e
> [...]
> diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
> deleted file mode 100644
> index ae5708e..0000000
> 
> That is, if hashes match then it was pure rename.

I suppose, though it's not as easy to spot, and won't help in showing 
what the differences are if there are any.

-Scott

Re: git apply vs. renamed files index mismatch

[Anton Vorontsov]

On Mon, Sep 08, 2008 at 04:38:57PM -0500, Scott Wood wrote:
> Anton Vorontsov wrote:
>> I always thought that posting "-M" patches to the public lists is
>> discouraged since it is quite difficult to apply them via patch(1).
>> Also think of non-git users...
>
> I think the substantially enhanced reviewability trumps non-git-users  
> who can follow the rename instructions manually (or fix up their patch  
> utility) if they insist on shunning tools that would make their life 
> easier.

Can't disagree, I myself use git. ;-) But I'm also using standard
patch(1) to test occasional patches... And sometimes linux-X tree
isn't tracked by git (e.g. -mm otm snapshots).

>> diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
>> new file mode 100644
>> index 0000000..ae5708e
>> [...]
>> diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
>> deleted file mode 100644
>> index ae5708e..0000000
>>
>> That is, if hashes match then it was pure rename.
>
> I suppose, though it's not as easy to spot, and won't help in showing  
> what the differences are if there are any.

..rename and changes ideally go in separate patches.

IIRC this also helps git to track renames (it can easily compare
hashes instead of guessing).

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: git apply vs. renamed files index mismatch (was: Re: [PATCH 1/4] POWERPC: Rename dma_64.c to dma.c)

[Christoph Hellwig]

On Tue, Sep 09, 2008 at 01:27:17AM +0400, Anton Vorontsov wrote:
> I always thought that posting "-M" patches to the public lists is
> discouraged since it is quite difficult to apply them via patch(1).
> Also think of non-git users...

Yes, it's a horrible idea.

Re: git apply vs. renamed files index mismatch

[Junio C Hamano]

Anton Vorontsov <avorontsov@ru.mvista.com> writes:

>>>  3 files changed, 201 insertions(+), 201 deletions(-)
>>>  create mode 100644 arch/powerpc/kernel/dma.c
>>>  delete mode 100644 arch/powerpc/kernel/dma_64.c
>>
>> Passing -M to git format-patch makes it much easier
>
> I always thought that posting "-M" patches to the public lists is
> discouraged since it is quite difficult to apply them via patch(1).
> Also think of non-git users...

My understanding has been that it is encouraged on the kernel mailing
list, because the rename format is far easier to review by showing the
differences that matter to reviewers, than showing a big chunk of text
deleted and another big chunk of text that is similar added elsewhere.

I won't comment on this any further; the use of it is strictly a list and
community policy issue.

> This is still possible by comparing the hashes:
> ...
> That is, if hashes match then it was pure rename.
>
> Though, too bad git {apply,am} does not produce any warnings if there
> are any hidden changes...

But I _do_ want to know what you mean by this comment.  Your statement
makes it sounds as if apply/am happily and silently accept "hidden
changes" and it is a bad thing.

Now what do you exactly mean by "any hidden changes"?  Do you mean "the
sender did not use renaming format, the patch you fed was a one that
removes a huge chunk of text from one file, and adds a similarly huge
chunk of text to another file.  The changes to these files looked similar
but was not quite the same"?  It is all there for you to review, and
especially if you prefer non-renaming format, then that is what you get.
So I do not think that is what you are complaining about.  It must be
something else --- what is it?

Re: git apply vs. renamed files index mismatch

[Junio C Hamano]

Anton Vorontsov <avorontsov@ru.mvista.com> writes:

> ..rename and changes ideally go in separate patches.
>
> IIRC this also helps git to track renames (it can easily compare
> hashes instead of guessing).

It does not help much, and it is frowned upon (at least by well educated
users in git circle) because such a split patch hurts reviewability.

Re: git apply vs. renamed files index mismatch

[Geert Uytterhoeven]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1975 bytes --]

On Tue, 9 Sep 2008, Anton Vorontsov wrote:
> On Mon, Sep 08, 2008 at 04:38:57PM -0500, Scott Wood wrote:
> > Anton Vorontsov wrote:
> >> I always thought that posting "-M" patches to the public lists is
> >> discouraged since it is quite difficult to apply them via patch(1).
> >> Also think of non-git users...
> >
> > I think the substantially enhanced reviewability trumps non-git-users  
> > who can follow the rename instructions manually (or fix up their patch  
> > utility) if they insist on shunning tools that would make their life 
> > easier.
> 
> Can't disagree, I myself use git. ;-) But I'm also using standard
> patch(1) to test occasional patches... And sometimes linux-X tree
> isn't tracked by git (e.g. -mm otm snapshots).
> 
> >> diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
> >> new file mode 100644
> >> index 0000000..ae5708e
> >> [...]
> >> diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
> >> deleted file mode 100644
> >> index ae5708e..0000000
> >>
> >> That is, if hashes match then it was pure rename.
> >
> > I suppose, though it's not as easy to spot, and won't help in showing  
> > what the differences are if there are any.
> 
> ..rename and changes ideally go in separate patches.

Except if the rename also requires some small changes (cfr. the move of
include/asm-*). But if no changes are required to fix breakage caused by the
rename, then make the changes separate.

> IIRC this also helps git to track renames (it can easily compare
> hashes instead of guessing).

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

Re: git apply vs. renamed files index mismatch

[Anton Vorontsov]

On Mon, Sep 08, 2008 at 05:53:41PM -0700, Junio C Hamano wrote:
> Anton Vorontsov <avorontsov@ru.mvista.com> writes:
> 
> >>>  3 files changed, 201 insertions(+), 201 deletions(-)
> >>>  create mode 100644 arch/powerpc/kernel/dma.c
> >>>  delete mode 100644 arch/powerpc/kernel/dma_64.c
> >>
> >> Passing -M to git format-patch makes it much easier
> >
> > I always thought that posting "-M" patches to the public lists is
> > discouraged since it is quite difficult to apply them via patch(1).
> > Also think of non-git users...
> 
> My understanding has been that it is encouraged on the kernel mailing
> list, because the rename format is far easier to review by showing the
> differences that matter to reviewers, than showing a big chunk of text
> deleted and another big chunk of text that is similar added elsewhere.
> 
> I won't comment on this any further; the use of it is strictly a list and
> community policy issue.
> 
> > This is still possible by comparing the hashes:
> > ...
> > That is, if hashes match then it was pure rename.
> >
> > Though, too bad git {apply,am} does not produce any warnings if there
> > are any hidden changes...
> 
> But I _do_ want to know what you mean by this comment.  Your statement
> makes it sounds as if apply/am happily and silently accept "hidden
> changes" and it is a bad thing.
> 
> Now what do you exactly mean by "any hidden changes"?  Do you mean "the
> sender did not use renaming format, the patch you fed was a one that
> removes a huge chunk of text from one file, and adds a similarly huge
> chunk of text to another file.  The changes to these files looked similar
> but was not quite the same"?  It is all there for you to review, and
> especially if you prefer non-renaming format, then that is what you get.

As I said, "index .." lines that git puts into patches are useful to see
if there any changes has been made to a renamed file. So usually I don't
have to look through the whole patch to see if there are any changes,
I can just look into the patch and conclude: "this is git patch, and the
overhead information says that it is rename-only patch. It should
be safe."

Now consider the following patch (modified by hand: it should say
+foo, but I changed it to +bar).

diff --git a/file b/file
deleted file mode 100644
index 257cc56..0000000
--- a/file
+++ /dev/null
@@ -1 +0,0 @@
-foo
diff --git a/file_renamed b/file_renamed
new file mode 100644
index 0000000..257cc56
--- /dev/null
+++ b/file_renamed
@@ -0,0 +1 @@
+bar


The "index ..." stuff says that there are no changes and it is
pure rename, but obviously there is a change.

What would be great is to have is some warning (or error), that
is: "git-am: patch claims that it would only rename the file %s,
but it also changes things in that file. Somebody tried to make
a fool of you."

Makes sense?

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: git apply vs. renamed files index mismatch

[Junio C Hamano]

Anton Vorontsov <avorontsov@ru.mvista.com> writes:

> Now consider the following patch (modified by hand: it should say
> +foo, but I changed it to +bar).
> ...
> The "index ..." stuff says that there are no changes and it is
> pure rename, but obviously there is a change.

Ah, I see what you mean.  But in general, it is not obvious at all.

If you have the identical preimage (recorded on the LHS of the index line)
or the patch reproduces the postimage in full (i.e. "create a new file"),
you *could* notice.  It's an interesting idea from git person's point of
view (i.e. "would be fun to implement"), but I doubt it would be useful in
practice, because:

 (1) You often do not have the identically matching preimage;

 (2) More importantly, it is not unusual for people to *edit* the patch in
     their MUA (think of typofixes), after getting it out of git.

 (3) Even more importantly, even if you notice there is some difference,
     you cannot produce the postimage by only looking at the hash (this is
     obvious -- otherwise by definition you broke SHA-1), so you cannot
     tell *how* the patch was modified.

What is of much more practical value to learn here in the context of this
topic would be that after accepting such a patch that does not use -M (so
that non-git people can use patch(1) to apply), a git person can still
verify the result with "git show -M" to see what changes other than a pure
rename was made by the patch.

Re: git apply vs. renamed files index mismatch

[Anton Vorontsov]

On Tue, Sep 09, 2008 at 07:45:19AM -0700, Junio C Hamano wrote:
> Anton Vorontsov <avorontsov@ru.mvista.com> writes:
> 
> > Now consider the following patch (modified by hand: it should say
> > +foo, but I changed it to +bar).
> > ...
> > The "index ..." stuff says that there are no changes and it is
> > pure rename, but obviously there is a change.
> 
> Ah, I see what you mean.  But in general, it is not obvious at all.
> 
> If you have the identical preimage (recorded on the LHS of the index line)
> or the patch reproduces the postimage in full (i.e. "create a new file"),
> you *could* notice.  It's an interesting idea from git person's point of
> view (i.e. "would be fun to implement"), but I doubt it would be useful in
> practice, because:
> 
>  (1) You often do not have the identically matching preimage;
> 
>  (2) More importantly, it is not unusual for people to *edit* the patch in
>      their MUA (think of typofixes), after getting it out of git.

Not for rename patches...

>  (3) Even more importantly, even if you notice there is some difference,

Just noticing that there is a difference is enough.

As for implementing, isn't this as simple as this pseudo code:

if (index_deleted_file == index_new_file)
	if (deleted_file != new_file)
		printk("warning\n");

In the git-apply?

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: git apply vs. renamed files index mismatch

[Junio C Hamano]

Anton Vorontsov <avorontsov@ru.mvista.com> writes:

> On Tue, Sep 09, 2008 at 07:45:19AM -0700, Junio C Hamano wrote:
> ...
>> ...  It's an interesting idea from git person's point of
>> view (i.e. "would be fun to implement"), but I doubt it would be useful in
>> practice, because:
>> 
>>  (1) You often do not have the identically matching preimage;
>> 
>>  (2) More importantly, it is not unusual for people to *edit* the patch in
>>      their MUA (think of typofixes), after getting it out of git.
>
> Not for rename patches...

a. Why not?  Even if your patch is (totally uninteresting) pure rename, it
   is natural to review the patch before you send out, and it also is
   natural to get tempted to fix typoes, just for a straight normal patch.

b. If you can expect good behaviour out of people, by declaring "Not for
   rename patches" as your guarantee, what's the point of this discussion?

> As for implementing, isn't this as simple as this pseudo code:
>
> if (index_deleted_file == index_new_file)
> 	if (deleted_file != new_file)
> 		printk("warning\n");
>
> In the git-apply?

Implementation is easy (I said "would be fun to code", didn't I? --- by the
way, how did you match "index_deleted_file" with "index_new_file"?).

My point was that it would not be reliable enough to be useful in
practice.