linux-next: stackprotector tree build failure

linux-next: stackprotector tree build failure

[Stephen Rothwell]

Hi all,

Today's linux-next build (powerpc ppc64_defconfig) failed like this:

kernel/fork.c: In function 'copy_process':
kernel/fork.c:1026: error: 'struct task_struct' has no member named 'default_timer_slack_ns'
kernel/fork.c:1026: error: 'struct task_struct' has no member named 'timer_slack_ns'

Caused by commit 929c12f25feeec881fd67eaa9b551f98588a2931 ("manual merge
of stackprotector") which introduced the above line.

I applied the following patch.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

From: Stephen Rothwell <sfr@canb.auug.org.au>
Date: Wed, 22 Oct 2008 13:09:17 +1100
Subject: [PATCH] stackprotector: fix up bad merge

Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
---
 kernel/fork.c |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/kernel/fork.c b/kernel/fork.c
index 194dc04..3bc1f86 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1023,8 +1023,6 @@ static struct task_struct *copy_process(unsigned long clone_flags,
 	p->prev_utime = cputime_zero;
 	p->prev_stime = cputime_zero;
 
-	p->default_timer_slack_ns = current->timer_slack_ns;
-
 #ifdef CONFIG_DETECT_SOFTLOCKUP
 	p->last_switch_count = 0;
 	p->last_switch_timestamp = 0;
-- 
1.5.6.5

Re: linux-next: stackprotector tree build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi all,
> 
> Today's linux-next build (powerpc ppc64_defconfig) failed like this:
> 
> kernel/fork.c: In function 'copy_process':
> kernel/fork.c:1026: error: 'struct task_struct' has no member named 'default_timer_slack_ns'
> kernel/fork.c:1026: error: 'struct task_struct' has no member named 'timer_slack_ns'
> 
> Caused by commit 929c12f25feeec881fd67eaa9b551f98588a2931 ("manual merge
> of stackprotector") which introduced the above line.
> 
> I applied the following patch.
> -- 
> Cheers,
> Stephen Rothwell                    sfr@canb.auug.org.au
> http://www.canb.auug.org.au/~sfr/
> 
> From: Stephen Rothwell <sfr@canb.auug.org.au>
> Date: Wed, 22 Oct 2008 13:09:17 +1100
> Subject: [PATCH] stackprotector: fix up bad merge
> 
> Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
> ---
>  kernel/fork.c |    2 --
>  1 files changed, 0 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/fork.c b/kernel/fork.c
> index 194dc04..3bc1f86 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -1023,8 +1023,6 @@ static struct task_struct *copy_process(unsigned long clone_flags,
>  	p->prev_utime = cputime_zero;
>  	p->prev_stime = cputime_zero;
>  
> -	p->default_timer_slack_ns = current->timer_slack_ns;
> -
>  #ifdef CONFIG_DETECT_SOFTLOCKUP
>  	p->last_switch_count = 0;
>  	p->last_switch_timestamp = 0;
> -- 
> 1.5.6.5

thanks Stephen!

This seems to have been caused by a git-rerere bug - it mis-matched a 
timers tree conflict resolution. I cleared out that resolution (it had 
nothing to do with stackprotector), re-did the conflict resolution 
(which was about overlapping additions of header files), and pushed out 
a new stackprotector tree - the delta below has the expected result.

	Ingo

diff --git a/kernel/fork.c b/kernel/fork.c
index 194dc04..3bc1f86 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1023,8 +1023,6 @@ static struct task_struct *copy_process(unsigned long clone_flags,
 	p->prev_utime = cputime_zero;
 	p->prev_stime = cputime_zero;
 
-	p->default_timer_slack_ns = current->timer_slack_ns;
-
 #ifdef CONFIG_DETECT_SOFTLOCKUP
 	p->last_switch_count = 0;
 	p->last_switch_timestamp = 0;

Re: linux-next: stackprotector tree build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

Hi Ingo,

On Wed, 22 Oct 2008 06:32:27 +0200 Ingo Molnar <mingo@elte.hu> wrote:
>
> This seems to have been caused by a git-rerere bug - it mis-matched a 
> timers tree conflict resolution. I cleared out that resolution (it had 
> nothing to do with stackprotector), re-did the conflict resolution 
> (which was about overlapping additions of header files), and pushed out 
> a new stackprotector tree - the delta below has the expected result.

I wondered how you could have possibly got that result - aren't computers
wonderful! :-)

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: stackprotector tree build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Ingo,
> 
> On Wed, 22 Oct 2008 06:32:27 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> >
> > This seems to have been caused by a git-rerere bug - it mis-matched a 
> > timers tree conflict resolution. I cleared out that resolution (it had 
> > nothing to do with stackprotector), re-did the conflict resolution 
> > (which was about overlapping additions of header files), and pushed out 
> > a new stackprotector tree - the delta below has the expected result.
> 
> I wondered how you could have possibly got that result - aren't 
> computers wonderful! :-)

heh, yes :)

this is the second time i met a git-rerere mismatch - the first one was 
half a year ago.

Unfortunately i failed at generating a reproducer back then and even now 
- as to resolve this problem i manually removed the preimage and 
postimage, so it's gone now.

I've Cc:-ed Junio and the Git list as a general FYI - but it must be 
frustrating to get such a bugreport, because i have no reproducer.

git-rerere sometimes seems to be picking up the wrong resolution. VERY 
rarely.

It seems random and content dependent. Once it happened to 
arch/x86/kernel/traps_32.c and now to kernel/fork.c. Along the ~170 
successful resolutions i have in my tree right now. And i do many 
conflict resolutions every day - and it happened only once every 6 
months or so.

(the arch/x86/kernel/traps_32.c one happened regularly, that's why i 
thought it's content sha1 dependent, and not some corruption.)

Next time it happens i'll be on the watchout and will save the complete 
tree.

	Ingo

Re: linux-next: stackprotector tree build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1292 bytes --]

On Wed, 22 Oct 2008 09:29:23 +0200 Ingo Molnar <mingo@elte.hu> wrote:
>
> I've Cc:-ed Junio and the Git list as a general FYI - but it must be 
> frustrating to get such a bugreport, because i have no reproducer.
> 
> git-rerere sometimes seems to be picking up the wrong resolution. VERY 
> rarely.
> 
> It seems random and content dependent. Once it happened to 
> arch/x86/kernel/traps_32.c and now to kernel/fork.c. Along the ~170 
> successful resolutions i have in my tree right now. And i do many 
> conflict resolutions every day - and it happened only once every 6 
> months or so.
> 
> (the arch/x86/kernel/traps_32.c one happened regularly, that's why i 
> thought it's content sha1 dependent, and not some corruption.)
> 
> Next time it happens i'll be on the watchout and will save the complete 
> tree.

I think rerere matches preimages on the SHA1 of the conflict (or its
reverse), so sufficiently similar pieces of code will match.  I would
expect things like ext2/3/4 to be candidates.  Did the traps_32.c one
match one for traps_64.c?

I may be mistaken, but I once followed the code in rerere to try to
figure out how to fix a resolution.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: stackprotector tree build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

On Wed, 22 Oct 2008 19:27:25 +1100 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> I think rerere matches preimages on the SHA1 of the conflict (or its
> reverse), so sufficiently similar pieces of code will match.  I would
> expect things like ext2/3/4 to be candidates.  Did the traps_32.c one
> match one for traps_64.c?

Or, of course similar conflicts in different merges in different branches
of the same tree are really asking for it ...

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: stackprotector tree build failure

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> On Wed, 22 Oct 2008 09:29:23 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> >
> > I've Cc:-ed Junio and the Git list as a general FYI - but it must be 
> > frustrating to get such a bugreport, because i have no reproducer.
> > 
> > git-rerere sometimes seems to be picking up the wrong resolution. VERY 
> > rarely.
> > 
> > It seems random and content dependent. Once it happened to 
> > arch/x86/kernel/traps_32.c and now to kernel/fork.c. Along the ~170 
> > successful resolutions i have in my tree right now. And i do many 
> > conflict resolutions every day - and it happened only once every 6 
> > months or so.
> > 
> > (the arch/x86/kernel/traps_32.c one happened regularly, that's why i 
> > thought it's content sha1 dependent, and not some corruption.)
> > 
> > Next time it happens i'll be on the watchout and will save the complete 
> > tree.
> 
> I think rerere matches preimages on the SHA1 of the conflict (or its 
> reverse), so sufficiently similar pieces of code will match.  I would 
> expect things like ext2/3/4 to be candidates.  Did the traps_32.c one 
> match one for traps_64.c?
> 
> I may be mistaken, but I once followed the code in rerere to try to 
> figure out how to fix a resolution.

the traps_32.c one was that git-rerere put in a traps_64.c end result. 
So i ended up with a 32-bit kernel that tried to build a 64-bit piece of 
code - fireworks. That condition persisted - i had to fix it up manually 
all the time i integrated that portion of the tree. That too was i think 
centered around a header file chunk - perhaps the #include section of 
traps_32.c and traps_64.c was similar enough in that section?

	Ingo

Re: linux-next: stackprotector tree build failure

[Johannes Schindelin]

Hi,

On Wed, 22 Oct 2008, Ingo Molnar wrote:

> 
> * Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> 
> > On Wed, 22 Oct 2008 09:29:23 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > >
> > > I've Cc:-ed Junio and the Git list as a general FYI - but it must be 
> > > frustrating to get such a bugreport, because i have no reproducer.
> > > 
> > > git-rerere sometimes seems to be picking up the wrong resolution. VERY 
> > > rarely.
> > > 
> > > It seems random and content dependent. Once it happened to 
> > > arch/x86/kernel/traps_32.c and now to kernel/fork.c. Along the ~170 
> > > successful resolutions i have in my tree right now. And i do many 
> > > conflict resolutions every day - and it happened only once every 6 
> > > months or so.
> > > 
> > > (the arch/x86/kernel/traps_32.c one happened regularly, that's why i 
> > > thought it's content sha1 dependent, and not some corruption.)
> > > 
> > > Next time it happens i'll be on the watchout and will save the complete 
> > > tree.
> > 
> > I think rerere matches preimages on the SHA1 of the conflict (or its 
> > reverse), so sufficiently similar pieces of code will match.  I would 
> > expect things like ext2/3/4 to be candidates.  Did the traps_32.c one 
> > match one for traps_64.c?
> > 
> > I may be mistaken, but I once followed the code in rerere to try to 
> > figure out how to fix a resolution.
> 
> the traps_32.c one was that git-rerere put in a traps_64.c end result. 
> So i ended up with a 32-bit kernel that tried to build a 64-bit piece of 
> code - fireworks. That condition persisted - i had to fix it up manually 
> all the time i integrated that portion of the tree. That too was i think 
> centered around a header file chunk - perhaps the #include section of 
> traps_32.c and traps_64.c was similar enough in that section?

I think it might be in order to explain how git-rerere works internally:

- in case of a conflicts, the files' conflicts are written into the 
  rr-cache _per file_.

  First, the human readable stuff after the "<<<", "|||" and ">>>" lines 
  is removed.

  Then, the "||| ... ===" part is removed, too (in effect turning the 
  diff3-style conflicts into an RCS merge-style conflicts).

  The result is recorded as "preimage".  The conflict is identified by the 
  SHA-1 of _just_ the conflicts (including the "<<<", "===" and ">>>" 
  lines).

  After all the conflicted files have been handled, a list is written to 
  the file "MERGE_RR" containing the SHA-1s of the conflicts together with 
  the file names.

- when committing, the files whose conflicts were resolved are recorded 
  verbatim in the file "postimage".

Now, when rerere is called again and there are conflicted files, again the 
files' conflicts are identified by their SHA-1.  If a resolution exists, a 
3-way merge is performed with the recorded preimage (the 
original file with conflict lines) as base, the postimage (the originally 
resolved file) and the current file with conflict lines.

The idea being: the diff between the recorded preimage and postimage gives 
the resolution that you want, and the diff between the recorded preimage 
and the current file with conflict lines gives you the changes that 
happened to the file in-between.

So I think that you might hit the unfortunate case where two files 
happened to have the same conflicts, but you needed to resolve them one 
way for one file, and another way for the other file.

Ciao,
Dscho

Re: linux-next: stackprotector tree build failure

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

Hi Johannes,

On Wed, 22 Oct 2008 19:41:18 +0200 (CEST) Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>
> I think it might be in order to explain how git-rerere works internally:

Thanks for the explanation.

> So I think that you might hit the unfortunate case where two files 
> happened to have the same conflicts, but you needed to resolve them one 
> way for one file, and another way for the other file.

Which makes perfect sense, unfortunately.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]