PV guest of Linux 2.6.34-rc1 hangs..rip: ffffffff817d8759 free_early_partial+0x2d

PV guest of Linux 2.6.34-rc1 hangs..rip: ffffffff817d8759 free_early_partial+0x2d

[Konrad Rzeszutek Wilk]

For the fun of it, I tried to boot 2.6.34-rc1 as PV guest and it
just hangs (1 CPU, 512MB). xenctx shows this:

sh-3.1# xenctx 1008 -s /System.map 
rip: ffffffff817d8759 free_early_partial+0x2d 
flags: 00001202 i nz 
rsp: ffffffff816ffe58
rax: ffff880000016300   rcx: 0000000001946668   rdx: ffff880000016000
rbx: 0000000007620000   rsi: 0000000000000040   rdi: 0000000000000019
rbp: ffffffff816ffe68    r8: 0000000007604000    r9: 0000000000000fff
r10: 0000000007620000   r11: ffff880000016400   r12: 0000000007620000
r13: ffffffffffffffff   r14: 000000000001c000   r15: ffff880001953050
 cs: e033        ss: e02b        ds: 0000        es: 0000 
 fs: 0000 @ 0000000000000000
 gs: 0000 @ ffffffff817ab000/0000000000000000
Code (instr addr ffffffff817d8759)
03 00 31 ff 48 89 d0 eb 10 4c 3b 20 76 05 48 39 cb 72 13 ff c7 <48> 83
c0 20 39 f7 7d 39 48 8b 48 


Stack:
 0000000000000000 ffff880001954000 ffffffff816ffe88 ffffffff817cacd2
 ffffffffffffffff ffff880001953000 ffffffff816fff28 ffffffff817de3c5
 ffffffff816ffec8 ffffffff817fb960 ffffffff817fce90 ffff880007604000
 ffff880007604000 ffff880001953040 ffffffff817cacd6 ffff880001953000

Call Trace:
  [<ffffffff817d8759>] free_early_partial+0x2d  <--  
  [<ffffffff817cacd2>] pcpu_fc_free+0x1d 
  [<ffffffff817de3c5>] pcpu_embed_first_chunk+0x159 
  [<ffffffff817cacd6>] pcpu_fc_free+0x21 
  [<ffffffff817cacb5>] pcpup_populate_pte+0xb 
  [<ffffffff817caaf3>] setup_per_cpu_areas+0x82 
  [<ffffffff817c0af0>] start_kernel+0x14e 
  [<ffffffff817c02bc>] x86_64_start_reservations+0xa7 
  [<ffffffff817c333d>] xen_start_kernel+0x52b 

I haven't taken a look any further than that, has anybody seen this
or maybe is itching to dig in the kernel to figure this out?

Re: PV guest of Linux 2.6.34-rc1 hangs..rip: ffffffff817d8759 free_early_partial+0x2d

[Ian Campbell]

FWIW I fixed a hang in the tip tree (which may now be in 2.6.34-rc1) in
roughly this area with a very similar stack trace not so very long ago.
See https://patchwork.kernel.org/patch/83756/

Perhaps you are seeing a different manifestation of the same issue?

Ian.

On Fri, 2010-03-19 at 15:25 +0000, Konrad Rzeszutek Wilk wrote:
> For the fun of it, I tried to boot 2.6.34-rc1 as PV guest and it
> just hangs (1 CPU, 512MB). xenctx shows this:
> 
> sh-3.1# xenctx 1008 -s /System.map 
> rip: ffffffff817d8759 free_early_partial+0x2d 
> flags: 00001202 i nz 
> rsp: ffffffff816ffe58
> rax: ffff880000016300   rcx: 0000000001946668   rdx: ffff880000016000
> rbx: 0000000007620000   rsi: 0000000000000040   rdi: 0000000000000019
> rbp: ffffffff816ffe68    r8: 0000000007604000    r9: 0000000000000fff
> r10: 0000000007620000   r11: ffff880000016400   r12: 0000000007620000
> r13: ffffffffffffffff   r14: 000000000001c000   r15: ffff880001953050
>  cs: e033        ss: e02b        ds: 0000        es: 0000 
>  fs: 0000 @ 0000000000000000
>  gs: 0000 @ ffffffff817ab000/0000000000000000
> Code (instr addr ffffffff817d8759)
> 03 00 31 ff 48 89 d0 eb 10 4c 3b 20 76 05 48 39 cb 72 13 ff c7 <48> 83
> c0 20 39 f7 7d 39 48 8b 48 
> 
> 
> Stack:
>  0000000000000000 ffff880001954000 ffffffff816ffe88 ffffffff817cacd2
>  ffffffffffffffff ffff880001953000 ffffffff816fff28 ffffffff817de3c5
>  ffffffff816ffec8 ffffffff817fb960 ffffffff817fce90 ffff880007604000
>  ffff880007604000 ffff880001953040 ffffffff817cacd6 ffff880001953000
> 
> Call Trace:
>   [<ffffffff817d8759>] free_early_partial+0x2d  <--  
>   [<ffffffff817cacd2>] pcpu_fc_free+0x1d 
>   [<ffffffff817de3c5>] pcpu_embed_first_chunk+0x159 
>   [<ffffffff817cacd6>] pcpu_fc_free+0x21 
>   [<ffffffff817cacb5>] pcpup_populate_pte+0xb 
>   [<ffffffff817caaf3>] setup_per_cpu_areas+0x82 
>   [<ffffffff817c0af0>] start_kernel+0x14e 
>   [<ffffffff817c02bc>] x86_64_start_reservations+0xa7 
>   [<ffffffff817c333d>] xen_start_kernel+0x52b 
> 
> I haven't taken a look any further than that, has anybody seen this
> or maybe is itching to dig in the kernel to figure this out?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: PV guest of Linux 2.6.34-rc1 hangs..rip: ffffffff817d8759 free_early_partial+0x2d

[Konrad Rzeszutek Wilk]

On Fri, Mar 19, 2010 at 04:03:48PM +0000, Ian Campbell wrote:
> FWIW I fixed a hang in the tip tree (which may now be in 2.6.34-rc1) in
> roughly this area with a very similar stack trace not so very long ago.
> See https://patchwork.kernel.org/patch/83756/
> 
> Perhaps you are seeing a different manifestation of the same issue?

Yup. Your patch fixed it. Thanks!

Re: PV guest of Linux 2.6.34-rc1 hangs..rip: ffffffff817d8759 free_early_partial+0x2d

[Ian Campbell]

On Fri, 2010-03-19 at 16:46 +0000, Konrad Rzeszutek Wilk wrote:
> On Fri, Mar 19, 2010 at 04:03:48PM +0000, Ian Campbell wrote:
> > FWIW I fixed a hang in the tip tree (which may now be in 2.6.34-rc1) in
> > roughly this area with a very similar stack trace not so very long ago.
> > See https://patchwork.kernel.org/patch/83756/
> > 
> > Perhaps you are seeing a different manifestation of the same issue?
> 
> Yup. Your patch fixed it. Thanks!

It's probably worth bringing that to the attention of the x86
maintainers in case they weren't planning to push that fix in this
release.

Ian.