Re: [crash, bisected] Re: [PATCH 3/4] x86_64: Fold pda into per cpu area

[Jeremy Fitzhardinge <jeremy@goop.org>]

Mike Travis wrote:
> Jeremy Fitzhardinge wrote:
>   
>> Mike Travis wrote:
>>     
>>> @@ -132,6 +133,12 @@ ident_complete:
>>>  #ifdef CONFIG_SMP
>>>      addq    %rbp, trampoline_level4_pgt + 0(%rip)
>>>      addq    %rbp, trampoline_level4_pgt + (511*8)(%rip)
>>> +
>>> +    /*
>>> +     * Fix up per_cpu__gdt_page offset when basing percpu
>>> +     * variables at zero.  This is only needed for the boot cpu.
>>> +     */
>>> +    addq    $__per_cpu_load, early_gdt_descr_base
>>>   
>>>       
>> This needs to be rip-relative.  An absolute reference here will fail
>> because you're still running in physical addresses.
>>
>>    J
>>     
>
> Still bombs right at boot up... ;-(
>   

Yep.  I see the triple-fault at the "mov %eax,%ds", which means it's 
having trouble with the gdt.  Either 1) the lgdt pointed to a bad 
address, or 2) there's something wrong with the descriptor there.

The dump is:

(XEN) hvm.c:767:d14 Triple fault on VCPU0 - invoking HVM system reset.
(XEN) ----[ Xen-3.3-unstable  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    0010:[<ffffffff80200167>]
(XEN) RFLAGS: 0000000000010002   CONTEXT: hvm
(XEN) rax: 0000000000000018   rbx: 0000000000000000   rcx: ffffffff808d6000
(XEN) rdx: 0000000000000000   rsi: 0000000000092f40   rdi: 0000000020100800
(XEN) rbp: 0000000000000000   rsp: ffffffff80827ff8   r8:  0000000000208000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 00000000000000d8
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000000000a0
(XEN) cr3: 0000000000201000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0010


I loaded early_gdt_descr+2 into %rcx, which looks reasonable.  Hm, but 
loading the __KERNEL_DS descriptor into %rdx, which is all zero.

So it seems the problem is that the pre-initialized gdt_page is being 
lost and replaced with zero.  Linker script bug?

    J

--- a/arch/x86/kernel/head_64.S	Fri Jun 20 09:50:02 2008 -0700
+++ b/arch/x86/kernel/head_64.S	Fri Jun 20 10:19:20 2008 -0700
@@ -213,6 +213,8 @@
 	 * because in 32bit we couldn't load a 64bit linear address.
 	 */
 	lgdt	early_gdt_descr(%rip)
+	movq	early_gdt_descr+2(%rip), %rcx
+	movq	__KERNEL_DS(%rcx), %rdx
 
 	/* set up data segments. actually 0 would do too */
 	movl $__KERNEL_DS,%eax