ckpt-16-dev BUG() - bisected

ckpt-16-dev BUG() - bisected

[Nathan Lynch]

Pulled latest updates today, started seeing the bug below when doing
simple bash shell checkpoint/restart.  Git bisect blames:

commit 505d9949cac385c5111bbd8ff3d2e123d5c468d8
Author: Oren Laadan <orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org>
Date:   Sun May 31 12:59:40 2009 -0400

    c/r: [X86_32] validate task's cpu registers and tls during restart
    
    Check the eflags, segment registers and tls entries from the
    checkpoint image to prevent breakpoints inside the kernel code.
    
    Fixes in restore_thread() and load_cpu_regs():
            /* TODO: ADD SANITY CHECKS TO VERIFY VALIDITY OF VALUES */
    
    Based on patch from Alexey Dobriyan:
    https://lists.linux-foundation.org/pipermail/containers/2009-May/017886.html
        Segment registers are abstracted to allow i386 => x86_64
        migration (BTW, I'm not so sure if just making 32-bit selectors
        the same will achieve same effect)


kernel BUG at arch/x86/mm/checkpoint.c:86!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
last sysfs file: /sys/block/sda/size
Modules linked in:

Pid: 2236, comm: ckpt Not tainted (2.6.30-rc7-00054-g505d994 #55) 
EIP: 0060:[<c021c3dd>] EFLAGS: 00010202 CPU: 0
EIP is at encode_segment+0x51/0x57
EAX: 1fffe009 EBX: df03a720 ECX: 00000130 EDX: 1fffe00f
ESI: df996fb4 EDI: def3f250 EBP: de8abf18 ESP: de8abf18
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ckpt (pid: 2236, ti=de8ab000 task=de8e28e0 task.ti=de8ab000)
Stack:
 de8abf38 c021cc07 decf51c0 def3f250 00000062 dc99a440 00000000 def3f250
 de8abf58 c038eb3c decf51c0 decf54e8 00000007 00000000 00000000 def3f250
 de8abf98 c038d7bf c07a69e0 00000001 00000001 00000000 def73ee0 c07ad224
Call Trace:
 [<c021cc07>] ? checkpoint_cpu+0x11a/0x426
 [<c038eb3c>] ? checkpoint_task+0x226/0x239
 [<c038d7bf>] ? do_checkpoint+0x404/0x511
 [<c038c669>] ? sys_checkpoint+0x6d/0x83
 [<c0202ce5>] ? syscall_call+0x7/0xb
Code: 74 2d 83 fa 7b b8 02 00 00 00 74 23 f6 c2 04 74 0a 89 d0 c1 e8 03 80 cc 80 eb 14 c1 ea 03 8d 42 fa 83 f8 02 77 05 80 cc 40 eb 04 <0f> 0b eb fe 5d c3 55 89 e5 e8 f9 72 fe ff 85 c0 74 40 83 f8 01 
EIP: [<c021c3dd>] encode_segment+0x51/0x57 SS:ESP 0068:de8abf18
---[ end trace 270dba57a69e79e1 ]---

Re: ckpt-16-dev BUG() - bisected

[Oren Laadan]

Hi,

I couldn't reproduce the problem on my system - your script
works fine (checkpoint and restart) on my kernel.

The actual logic is borrowed from Alexey's i386 patches.

My best guess is that either the issue occurs because you are
running under KVM which uses different segments (?)

Either that, or something is wrong with Alexey's original code.

Alexey - any idea ?

Also, you can you apply this patch so we know what was the
segment's value:

----------------------------------
diff --git a/arch/x86/mm/checkpoint.c b/arch/x86/mm/checkpoint.c
index 7242762..18e957a 100644
--- a/arch/x86/mm/checkpoint.c
+++ b/arch/x86/mm/checkpoint.c
@@ -83,6 +83,8 @@ static __u32 encode_segment(unsigned long seg)
 	seg >>= 3;
 	if (GDT_ENTRY_TLS_MIN <= seg && seg <= GDT_ENTRY_TLS_MAX)
 		return CKPT_X86_SEG_TLS | (seg - GDT_ENTRY_TLS_MIN);
+
+	printk(KERN_ERR "bad segment %#lx\n", seg);
 	BUG();
 }
 
@@ -103,6 +105,8 @@ static unsigned long decode_segment(__u32 seg)
 		seg &= ~CKPT_X86_SEG_LDT;
 		return (seg << 3) | 7;
 	}
+
+	printk(KERN_ERR "bad segment %#lx\n", seg);
 	BUG();
 }
 
----------------------------------

Oren.


On Wed, 3 Jun 2009, Nathan Lynch wrote:

> Pulled latest updates today, started seeing the bug below when doing
> simple bash shell checkpoint/restart.  Git bisect blames:
> 
> commit 505d9949cac385c5111bbd8ff3d2e123d5c468d8
> Author: Oren Laadan <orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org>
> Date:   Sun May 31 12:59:40 2009 -0400
> 
>     c/r: [X86_32] validate task's cpu registers and tls during restart
>     
>     Check the eflags, segment registers and tls entries from the
>     checkpoint image to prevent breakpoints inside the kernel code.
>     
>     Fixes in restore_thread() and load_cpu_regs():
>             /* TODO: ADD SANITY CHECKS TO VERIFY VALIDITY OF VALUES */
>     
>     Based on patch from Alexey Dobriyan:
>     https://lists.linux-foundation.org/pipermail/containers/2009-May/017886.html
>         Segment registers are abstracted to allow i386 => x86_64
>         migration (BTW, I'm not so sure if just making 32-bit selectors
>         the same will achieve same effect)
> 
> 
> kernel BUG at arch/x86/mm/checkpoint.c:86!
> invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> last sysfs file: /sys/block/sda/size
> Modules linked in:
> 
> Pid: 2236, comm: ckpt Not tainted (2.6.30-rc7-00054-g505d994 #55) 
> EIP: 0060:[<c021c3dd>] EFLAGS: 00010202 CPU: 0
> EIP is at encode_segment+0x51/0x57
> EAX: 1fffe009 EBX: df03a720 ECX: 00000130 EDX: 1fffe00f
> ESI: df996fb4 EDI: def3f250 EBP: de8abf18 ESP: de8abf18
>  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
> Process ckpt (pid: 2236, ti=de8ab000 task=de8e28e0 task.ti=de8ab000)
> Stack:
>  de8abf38 c021cc07 decf51c0 def3f250 00000062 dc99a440 00000000 def3f250
>  de8abf58 c038eb3c decf51c0 decf54e8 00000007 00000000 00000000 def3f250
>  de8abf98 c038d7bf c07a69e0 00000001 00000001 00000000 def73ee0 c07ad224
> Call Trace:
>  [<c021cc07>] ? checkpoint_cpu+0x11a/0x426
>  [<c038eb3c>] ? checkpoint_task+0x226/0x239
>  [<c038d7bf>] ? do_checkpoint+0x404/0x511
>  [<c038c669>] ? sys_checkpoint+0x6d/0x83
>  [<c0202ce5>] ? syscall_call+0x7/0xb
> Code: 74 2d 83 fa 7b b8 02 00 00 00 74 23 f6 c2 04 74 0a 89 d0 c1 e8 03 80 cc 80 eb 14 c1 ea 03 8d 42 fa 83 f8 02 77 05 80 cc 40 eb 04 <0f> 0b eb fe 5d c3 55 89 e5 e8 f9 72 fe ff 85 c0 74 40 83 f8 01 
> EIP: [<c021c3dd>] encode_segment+0x51/0x57 SS:ESP 0068:de8abf18
> ---[ end trace 270dba57a69e79e1 ]---
> 
> 

Re: ckpt-16-dev BUG() - bisected

[Nathan Lynch]

Oren Laadan <orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org> writes:
> Hi,
>
> I couldn't reproduce the problem on my system - your script
> works fine (checkpoint and restart) on my kernel.
>
> The actual logic is borrowed from Alexey's i386 patches.
>
> My best guess is that either the issue occurs because you are
> running under KVM which uses different segments (?)
>
> Either that, or something is wrong with Alexey's original code.
>
> Alexey - any idea ?
>
> Also, you can you apply this patch so we know what was the
> segment's value:
>
> ----------------------------------
> diff --git a/arch/x86/mm/checkpoint.c b/arch/x86/mm/checkpoint.c
> index 7242762..18e957a 100644
> --- a/arch/x86/mm/checkpoint.c
> +++ b/arch/x86/mm/checkpoint.c
> @@ -83,6 +83,8 @@ static __u32 encode_segment(unsigned long seg)
>  	seg >>= 3;
>  	if (GDT_ENTRY_TLS_MIN <= seg && seg <= GDT_ENTRY_TLS_MAX)
>  		return CKPT_X86_SEG_TLS | (seg - GDT_ENTRY_TLS_MIN);
> +
> +	printk(KERN_ERR "bad segment %#lx\n", seg);
>  	BUG();
>  }
>  
> @@ -103,6 +105,8 @@ static unsigned long decode_segment(__u32 seg)
>  		seg &= ~CKPT_X86_SEG_LDT;
>  		return (seg << 3) | 7;
>  	}
> +
> +	printk(KERN_ERR "bad segment %#lx\n", seg);
>  	BUG();
>  }

bad segment 0x1fffe00f
------------[ cut here ]------------
kernel BUG at arch/x86/mm/checkpoint.c:88!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
last sysfs file: /sys/class/net/br0/ifindex
Modules linked in:

Pid: 3117, comm: ckpt Not tainted (2.6.30-rc7-00058-gb0ddfc4-dirty #58) 
EIP: 0060:[<c021c622>] EFLAGS: 00010246 CPU: 0
EIP is at encode_segment+0x5c/0x64
EAX: 0000001a EBX: de86f300 ECX: 00000000 EDX: 00000003
ESI: df1f8fb4 EDI: dfad7020 EBP: df1faf18 ESP: df1faf10
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ckpt (pid: 3117, ti=df1fa000 task=df9f0000 task.ti=df1fa000)
Stack:
 c06f9756 1fffe00f df1faf38 c021cc21 de810000 dfad7020 00000044 df280200
 00000000 dfad7020 df1faf58 c038edee de810000 de810328 00000007 00000000
 00000000 dfad7020 df1faf98 c038d88c c07ab9e0 00000001 00000001 00000000
Call Trace:
 [<c021cc21>] ? checkpoint_cpu+0x11a/0x426
 [<c038edee>] ? checkpoint_task+0x226/0x239
 [<c038d88c>] ? do_checkpoint+0x459/0x551
 [<c038c689>] ? sys_checkpoint+0x6d/0x83
 [<c0202ce5>] ? syscall_call+0x7/0xb
Code: 30 f6 c2 04 74 0a 89 d0 c1 e8 03 80 cc 80 eb 21 c1 ea 03 8d 42 fa 83 f8 02 77 05 80 cc 40 eb 11 52 68 56 97 6f c0 e8 76 c2 3c 00 <0f> 0b 58 5a eb fe c9 c3 55 89 e5 57 56 53 83 ec 10 e8 ac 70 fe 
EIP: [<c021c622>] encode_segment+0x5c/0x64 SS:ESP 0068:df1faf10
---[ end trace 0c5f24ea38f32c3f ]---

Re: ckpt-16-dev BUG() - bisected

[Nathan Lynch]

Nathan Lynch <ntl-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org> writes:

> Pulled latest updates today, started seeing the bug below when doing
> simple bash shell checkpoint/restart.  Git bisect blames:
>
> commit 505d9949cac385c5111bbd8ff3d2e123d5c468d8
> Author: Oren Laadan <orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org>
> Date:   Sun May 31 12:59:40 2009 -0400
>
>     c/r: [X86_32] validate task's cpu registers and tls during restart

...

> kernel BUG at arch/x86/mm/checkpoint.c:86!

This seems to be fixed in latest ckpt-v16-dev, presumably by
f5867c9c8383c2d2d5361512e79e70df4b730fe9 ("c/r: [X86_32] used
__u16/short to encode/decode segment registers").