[MODERATED] [boris.ostrovsky@oracle.com: Re: [speck@linutronix.de: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2]]

[MODERATED] [boris.ostrovsky@oracle.com: Re: [speck@linutronix.de: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2]]

[Konrad Rzeszutek Wilk]

[Forwarding and encrypting.]
----- Forwarded message from Boris Ostrovsky <boris.ostrovsky@oracle.com> -----

Date: Fri, 18 May 2018 17:43:26 -0400
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
To: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [speck@linutronix.de: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2]
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0

On 05/18/2018 05:37 PM, Boris Ostrovsky wrote:
> On 05/18/2018 05:28 PM, Konrad Rzeszutek Wilk wrote:
>> Prior to that.
>>
>> ----- Forwarded message from speck for Konrad Rzeszutek Wilk <speck@linutronix.de> -----
>>
>>
>> On Fri, May 18, 2018 at 01:40:39PM -0700, speck for Linus Torvalds wrote:
>>> On Fri, 18 May 2018, speck for Konrad Rzeszutek Wilk wrote:
>>>> Yup. B/c there is no early callback installed yet and it ends up using 'cli'
>>> Why isn't the default callback for cli (and restore-flags) just a no-op? 
>> The paravirt code hadn't yet run.
>>> Interrupts had better not be enabled before the callbacks are done, so 
>>> they *should* default to not doing anything.
>>>
>>>                Linus
>> ----- End forwarded message -----
> Here is the thing: yes, the patch is trying to work around a particular
> issue with cli. But even if cli was a nop we *can't* printk() at this
> point anyway! For example, I believe somewhere in bowels of printk code
> there will be a dereference of a per-CPU variable. We haven't set up
> per-CPU segments either.
>
> The point of that patch, really, is that we shouldn't be calling non-Xen
> code yet.


An equivalent of what xen_start_kernel() is currently doing is calling
get_cpu_cap() from arch/x86/kernel/head_64.S:startup_64.

-boris

----- End forwarded message -----

[MODERATED] [boris.ostrovsky@oracle.com: Re: [speck@linutronix.de: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2]]

[Konrad Rzeszutek Wilk]

Figured it out.

We have in our 4.14 kernel (aka UEK5) backports of this _and_ the
IBRS pile that never got accepted upstream. One of those patches is the:

 printk("FEATURE SPEC_CTRL Present\n")

which is called from 'init_speculation_control' and we end up with this
disaster as the infrastructure is not yet setup.

I confused which tree test results we had and ended coming to the wrong
conclusion.

<sigh>

My apologies for wasting your folks time.

----- Forwarded message from Boris Ostrovsky <boris.ostrovsky@oracle.com> -----

Date: Sat, 19 May 2018 13:49:29 -0400
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
To: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [speck@linutronix.de: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2]



On 05/19/2018 01:45 PM, Boris Ostrovsky wrote:
> 
> 
> On 05/19/2018 11:13 AM, Konrad Rzeszutek Wilk wrote:
> > You wouldn't have a call chain from your X6?
> 
> 
> I don't have upstream build but on uek5 port:
> 
> xen_start_kernel
>   get_cpu_cap
>    init_speculation_control



Unless, of course, upstream code does not call init_speculation_control() from
get_cpu_cap()?


-boris



>     pr_info
>      printk
>       vprintk_func
>        vprintk_default
>         vprintk_emit
>          logbuf_lock_irqsave
>           printk_safe_enter_irqsave
>            local_irq_save
> 
> 
> I manually unwrapped Xen splat but it is certainly a cli:
> 
> (XEN) RIP:    e033:[<ffffffff8106fcf4>]
> 
> [root@bur-virt-x6-2-100 linux-osv]# grep ffffffff8106fcf4 vmlinux.s
> ffffffff8106fcf4:    fa                       cli
> [root@bur-virt-x6-2-100 linux-osv]#
> 
> [root@bur-virt-x6-2-100 linux-osv]# addr2line -e vmlinux ffffffff8106fcf4
> /data/linux-osv/./arch/x86/include/asm/irqflags.h:44
> [root@bur-virt-x6-2-100 linux-osv]#
> 
> I can try upsteam tree is you can point me to it.
> 
> But again, the point here is not cli. It's that we are essentially calling
> printk before *anything* is set up.
> 
> 
> -boris
> 
> 
> 
> 
> > 
> > ----- Forwarded message from speck for Thomas Gleixner
> > <speck@linutronix.de> -----
> > 
> > Date: Sat, 19 May 2018 10:33:48 +0200 (CEST)
> > From: speck for Thomas Gleixner <speck@linutronix.de>
> > To: konrad.wilk@oracle.com
> > Subject: Re: [PATCH v17.1 2/2] [PATCH v17.1 2/2] SSB Fix #2
> > 
> > On Fri, 18 May 2018, speck for Konrad Rzeszutek Wilk wrote:
> > > On Fri, May 18, 2018 at 09:57:47PM +0200, speck for Thomas
> > > Gleixner wrote:
> > > > > (XEN)    0000000000000000 0000000000000000
> > > > > 0000000000000000 0000000000000000
> > > > > (XEN)    0000000000000000 0000000000000000
> > > > > 0000000000000000 0000000000000000
> > > > > (XEN)    0f00000060c0c748 ccccccccccccc305
> > > > > cccccccccccccccc cccccccccccccccc
> > > > > (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
> > > > > (XEN) APIC error on CPU0: 40(00)
> > > > 
> > > > The above is pretty useless and undecodable. So what makes
> > > > Dom0 crash in a
> > > > way that the machine needs to be rebooted? The call to get_cpu_cap()?
> > > 
> > > Yup. B/c there is no early callback installed yet and it ends up
> > > using 'cli'
> > 
> > Sigh, 'using CLI' is not helpful either. The callchain and where the
> > CLI is
> > invoked, is the interesting information.
> > 
> > But well, as you say its get_cpu_cap() I went and stared at it for while,
> > but there is absolutely NOTHING which might invoke CLI. It was nothing
> > before SSB and SSB did not add anything either.
> > 
> > So this patch is voodoo and makes exactly zero sense, unless you come up
> > with some sensible explanation.
> > 
> > Thanks,
> > 
> >     tglx
> > 
> > 
> > 
> > ----- End forwarded message -----
> > 

----- End forwarded message -----