Re: unified percpu stuff

[Jeremy Fitzhardinge <jeremy@goop.org>]

Brian Gerst wrote:
> On Mon, Jan 26, 2009 at 8:12 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>   
>> I'm really pleased to see the unified percpu stuff in the kernel, but
>> unfortunately its breaking Xen at the moment.
>> It looks like this is just a matter of initializing %gs properly in
>> xen_start_kernel.  Is there any problem with me doing a load_gs_base(0)
>> somewhere early in xen_start_kernel (arch/x86/xen/enlighten.c)?
>>     
>
> Some of the changes I did made the assumption that the percpu state is
> set up early in head_xx.S, which apparently is skipped for xen.  Is
> there documentation anywhere for the xen bootstrap process?
>   

Not really.  Its quite different from native because the guest kernel 
starts up in protected/long mode with paging enabled, using a 
Xen-provided pagetable.  It means that most of the normal head.S stuff 
is redundant; the kernel starts executing more or less exactly at 
xen_start_kernel (there's a 2 instruction asm part which stashes away 
the Xen info pointer from %[re]si, then jumps to xen_start_kernel).

> Try this patch (untested):
>
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index bef941f..b90d061 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -1647,6 +1647,8 @@ asmlinkage void __init xen_start_kernel(void)
>  	have_vcpu_info_placement = 0;
>  #endif
>
> +	switch_to_new_gdt();
> +
>  	xen_smp_init();
>
>  	/* Get mfn list */
> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
> index 7735e3d..00d9265 100644
> --- a/arch/x86/xen/smp.c
> +++ b/arch/x86/xen/smp.c
> @@ -235,6 +235,8 @@ cpu_initialize_context(unsigned int cpu, struct
> task_struct *idle)
>  	ctxt->user_regs.ss = __KERNEL_DS;
>  #ifdef CONFIG_X86_32
>  	ctxt->user_regs.fs = __KERNEL_PERCPU;
> +#else
> +	ctxt->gs_base_kernel = per_cpu_offset(cpu);
>  #endif
>  	ctxt->user_regs.eip = (unsigned long)cpu_bringup_and_idle;
>  	ctxt->user_regs.eflags = 0x1000; /* IOPL_RING1 */
>   

Thanks, I'll try this out.

BTW, does the initial cpu0 percpu area get reallocated and moved during 
boot, or does cpu0 keep using the same memory forever?

Thanks,
    J