* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Jann Horn @ 2018-08-30 17:59 UTC (permalink / raw)
To: yu-cheng.yu
Cc: Dave Hansen, the arch/x86 maintainers, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, kernel list, linux-doc, Linux-MM,
linux-arch, Linux API, Arnd Bergmann, Andy Lutomirski,
Balbir Singh, Cyrill Gorcunov, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov
In-Reply-To: <1535651666.27823.6.camel@intel.com>
On Thu, Aug 30, 2018 at 7:58 PM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
>
> On Thu, 2018-08-30 at 10:33 -0700, Dave Hansen wrote:
> > On 08/30/2018 10:26 AM, Yu-cheng Yu wrote:
> > >
> > > We don't have the guard page now, but there is a shadow stack
> > > token
> > > there, which cannot be used as a return address.
> > The overall concern is that we could overflow into a page that we
> > did
> > not intend. Either another actual shadow stack or something that a
> > page
> > that the attacker constructed, like the transient scenario Jann
> > described.
> >
>
> A task could go beyond the bottom of its shadow stack by doing either
> 'ret' or 'incssp'. If it is the 'ret' case, the token prevents it.
> If it is the 'incssp' case, a guard page cannot prevent it entirely,
> right?
I mean the other direction, on "call".
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Yu-cheng Yu @ 2018-08-30 17:54 UTC (permalink / raw)
To: Dave Hansen, Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Florian Weimer, hjl.tools, Jonathan Corbet,
keescook, Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek,
Peter Zijlstra
In-Reply-To: <33d45a12-513c-eba2-a2de-3d6b630e928e@linux.intel.com>
On Thu, 2018-08-30 at 10:33 -0700, Dave Hansen wrote:
> On 08/30/2018 10:26 AM, Yu-cheng Yu wrote:
> >
> > We don't have the guard page now, but there is a shadow stack
> > token
> > there, which cannot be used as a return address.
> The overall concern is that we could overflow into a page that we
> did
> not intend. Either another actual shadow stack or something that a
> page
> that the attacker constructed, like the transient scenario Jann
> described.
>
A task could go beyond the bottom of its shadow stack by doing either
'ret' or 'incssp'. If it is the 'ret' case, the token prevents it.
If it is the 'incssp' case, a guard page cannot prevent it entirely,
right?
Yu-cheng
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Andy Lutomirski @ 2018-08-30 17:34 UTC (permalink / raw)
To: Dave Hansen
Cc: Jann Horn, yu-cheng.yu, the arch/x86 maintainers, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, kernel list, linux-doc, Linux-MM,
linux-arch, Linux API, Arnd Bergmann, Balbir Singh,
Cyrill Gorcunov, Florian Weimer, hjl.tools, Jonathan Corbet,
keescook, Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek,
Peter Zijlstra <pe>
In-Reply-To: <ce051b5b-feef-376f-e085-11f65a5f2215@linux.intel.com>
> On Aug 30, 2018, at 10:19 AM, Dave Hansen <dave.hansen@linux.intel.com> wrote:
>
>> On 08/30/2018 09:23 AM, Jann Horn wrote:
>> Three threads (A, B, C) run with the same CR3.
>>
>> 1. a dirty+writable PTE is placed directly in front of B's shadow stack.
>> (this can happen, right? or is there a guard page?)
>> 2. C's TLB caches the dirty+writable PTE.
>> 3. A performs some syscall that triggers ptep_set_wrprotect().
>> 4. A's syscall calls clear_bit().
>> 5. B's TLB caches the transient shadow stack.
>> [now C has write access to B's transiently-extended shadow stack]
>> 6. B recurses into the transiently-extended shadow stack
>> 7. C overwrites the transiently-extended shadow stack area.
>> 8. B returns through the transiently-extended shadow stack, giving
>> the attacker instruction pointer control in B.
>> 9. A's syscall broadcasts a TLB flush.
>
> Heh, that's a good point. The shadow stack permissions are *not*
> strictly reduced because a page getting marked as shadow-stack has
> *increased* permissions when being used as a shadow stack. Fun.
>
> For general hardening, it seems like we want to ensure that there's a
> guard page at the bottom of the shadow stack. Yu-cheng, do we have a
> guard page?
>
> But, to keep B's TLB from picking up the entry, I think we can just make
> it !Present for a moment. No TLB can cache it, and I believe the same
> "don't set Dirty on a !Writable entry" logic also holds for !Present
> (modulo a weird erratum or two).
Can we get documentation? Pretty please?
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Dave Hansen @ 2018-08-30 17:33 UTC (permalink / raw)
To: Yu-cheng Yu, Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Florian Weimer, hjl.tools, Jonathan Corbet,
keescook, Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek,
Peter Zijlstra
In-Reply-To: <1535649960.26689.15.camel@intel.com>
On 08/30/2018 10:26 AM, Yu-cheng Yu wrote:
> We don't have the guard page now, but there is a shadow stack token
> there, which cannot be used as a return address.
The overall concern is that we could overflow into a page that we did
not intend. Either another actual shadow stack or something that a page
that the attacker constructed, like the transient scenario Jann described.
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Yu-cheng Yu @ 2018-08-30 17:26 UTC (permalink / raw)
To: Dave Hansen, Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Florian Weimer, hjl.tools, Jonathan Corbet,
keescook, Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek,
Peter Zijlstra
In-Reply-To: <ce051b5b-feef-376f-e085-11f65a5f2215@linux.intel.com>
On Thu, 2018-08-30 at 10:19 -0700, Dave Hansen wrote:
> On 08/30/2018 09:23 AM, Jann Horn wrote:
> >
> > Three threads (A, B, C) run with the same CR3.
> >
> > 1. a dirty+writable PTE is placed directly in front of B's shadow
> > stack.
> > (this can happen, right? or is there a guard page?)
> > 2. C's TLB caches the dirty+writable PTE.
> > 3. A performs some syscall that triggers ptep_set_wrprotect().
> > 4. A's syscall calls clear_bit().
> > 5. B's TLB caches the transient shadow stack.
> > [now C has write access to B's transiently-extended shadow stack]
> > 6. B recurses into the transiently-extended shadow stack
> > 7. C overwrites the transiently-extended shadow stack area.
> > 8. B returns through the transiently-extended shadow stack, giving
> > the attacker instruction pointer control in B.
> > 9. A's syscall broadcasts a TLB flush.
> Heh, that's a good point. The shadow stack permissions are *not*
> strictly reduced because a page getting marked as shadow-stack has
> *increased* permissions when being used as a shadow stack. Fun.
>
> For general hardening, it seems like we want to ensure that there's
> a
> guard page at the bottom of the shadow stack. Yu-cheng, do we have
> a
> guard page?
We don't have the guard page now, but there is a shadow stack token
there, which cannot be used as a return address.
Yu-cheng
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Dave Hansen @ 2018-08-30 17:19 UTC (permalink / raw)
To: Jann Horn
Cc: yu-cheng.yu, the arch/x86 maintainers, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, kernel list, linux-doc, Linux-MM,
linux-arch, Linux API, Arnd Bergmann, Andy Lutomirski,
Balbir Singh, Cyrill Gorcunov, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek
In-Reply-To: <CAG48ez2gHOD9hH4+0wek5vUOv9upj79XWoug2SXjdwfXWoQqxw@mail.gmail.com>
On 08/30/2018 09:23 AM, Jann Horn wrote:
> Three threads (A, B, C) run with the same CR3.
>
> 1. a dirty+writable PTE is placed directly in front of B's shadow stack.
> (this can happen, right? or is there a guard page?)
> 2. C's TLB caches the dirty+writable PTE.
> 3. A performs some syscall that triggers ptep_set_wrprotect().
> 4. A's syscall calls clear_bit().
> 5. B's TLB caches the transient shadow stack.
> [now C has write access to B's transiently-extended shadow stack]
> 6. B recurses into the transiently-extended shadow stack
> 7. C overwrites the transiently-extended shadow stack area.
> 8. B returns through the transiently-extended shadow stack, giving
> the attacker instruction pointer control in B.
> 9. A's syscall broadcasts a TLB flush.
Heh, that's a good point. The shadow stack permissions are *not*
strictly reduced because a page getting marked as shadow-stack has
*increased* permissions when being used as a shadow stack. Fun.
For general hardening, it seems like we want to ensure that there's a
guard page at the bottom of the shadow stack. Yu-cheng, do we have a
guard page?
But, to keep B's TLB from picking up the entry, I think we can just make
it !Present for a moment. No TLB can cache it, and I believe the same
"don't set Dirty on a !Writable entry" logic also holds for !Present
(modulo a weird erratum or two).
If we do that, we just need to make sure that the fault handler knows it
can get spurious faults from it, and might even run into the !Present
PTE for a moment. It might be a bit confusing because it won't be a
PROT_NONE, migration, or swap PTE, but will be !Present. We'll also
have to make sure that we're doing this in a way which is friendly to
the L1TF PTE handling.
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Jann Horn @ 2018-08-30 16:23 UTC (permalink / raw)
To: Dave Hansen
Cc: yu-cheng.yu, the arch/x86 maintainers, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, kernel list, linux-doc, Linux-MM,
linux-arch, Linux API, Arnd Bergmann, Andy Lutomirski,
Balbir Singh, Cyrill Gorcunov, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek
In-Reply-To: <079a55f2-4654-4adf-a6ef-6e480b594a2f@linux.intel.com>
On Thu, Aug 30, 2018 at 6:09 PM Dave Hansen <dave.hansen@linux.intel.com> wrote:
>
> On 08/30/2018 08:49 AM, Jann Horn wrote:
> >> @@ -1203,7 +1203,28 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
> >> static inline void ptep_set_wrprotect(struct mm_struct *mm,
> >> unsigned long addr, pte_t *ptep)
> >> {
> >> + pte_t pte;
> >> +
> >> clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
> >> + pte = *ptep;
> >> +
> >> + /*
> >> + * Some processors can start a write, but ending up seeing
> >> + * a read-only PTE by the time they get to the Dirty bit.
> >> + * In this case, they will set the Dirty bit, leaving a
> >> + * read-only, Dirty PTE which looks like a Shadow Stack PTE.
> >> + *
> >> + * However, this behavior has been improved and will not occur
> >> + * on processors supporting Shadow Stacks. Without this
> >> + * guarantee, a transition to a non-present PTE and flush the
> >> + * TLB would be needed.
> >> + *
> >> + * When change a writable PTE to read-only and if the PTE has
> >> + * _PAGE_DIRTY_HW set, we move that bit to _PAGE_DIRTY_SW so
> >> + * that the PTE is not a valid Shadow Stack PTE.
> >> + */
> >> + pte = pte_move_flags(pte, _PAGE_DIRTY_HW, _PAGE_DIRTY_SW);
> >> + set_pte_at(mm, addr, ptep, pte);
> >> }
> > I don't understand why it's okay that you first atomically clear the
> > RW bit, then atomically switch from DIRTY_HW to DIRTY_SW. Doesn't that
> > mean that between the two atomic writes, another core can incorrectly
> > see a shadow stack?
>
> Good point.
>
> This could result in a spurious shadow-stack fault, or allow a
> shadow-stack write to the page in the transient state.
>
> But, the shadow-stack permissions are more restrictive than what could
> be in the TLB at this point, so I don't think there's a real security
> implication here.
How about this:
Three threads (A, B, C) run with the same CR3.
1. a dirty+writable PTE is placed directly in front of B's shadow stack.
(this can happen, right? or is there a guard page?)
2. C's TLB caches the dirty+writable PTE.
3. A performs some syscall that triggers ptep_set_wrprotect().
4. A's syscall calls clear_bit().
5. B's TLB caches the transient shadow stack.
[now C has write access to B's transiently-extended shadow stack]
6. B recurses into the transiently-extended shadow stack
7. C overwrites the transiently-extended shadow stack area.
8. B returns through the transiently-extended shadow stack, giving
the attacker instruction pointer control in B.
9. A's syscall broadcasts a TLB flush.
Sure, it's not exactly an easy race and probably requires at least
some black timing magic to exploit, if it's exploitable at all - but
still. This seems suboptimal.
> The only trouble is handling the spurious shadow-stack fault. The
> alternative is to go !Present for a bit, which we would probably just
> handle fine in the existing page fault code.
^ permalink raw reply
* Re: [RFC PATCH v3 19/24] x86/cet/shstk: Introduce WRUSS instruction
From: Yu-cheng Yu @ 2018-08-30 16:22 UTC (permalink / raw)
To: Andy Lutomirski, Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Balbir Singh, Cyrill Gorcunov,
Dave Hansen, Florian Weimer, H. J. Lu, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
In-Reply-To: <CALCETrW78UKt6AQJeN8GkhtxjuASnH1PV5QSpzQtDz9-2d3Asw@mail.gmail.com>
On Thu, 2018-08-30 at 08:55 -0700, Andy Lutomirski wrote:
> On Thu, Aug 30, 2018 at 8:39 AM, Jann Horn <jannh@google.com> wrote:
> >
> > On Thu, Aug 30, 2018 at 4:44 PM Yu-cheng Yu <yu-cheng.yu@intel.com
> > > wrote:
> > >
> > >
> > > WRUSS is a new kernel-mode instruction but writes directly
> > > to user shadow stack memory. This is used to construct
> > > a return address on the shadow stack for the signal
> > > handler.
> > >
> > > This instruction can fault if the user shadow stack is
> > > invalid shadow stack memory. In that case, the kernel does
> > > fixup.
> > >
> > > Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
> > [...]
> > >
> > > +static inline int write_user_shstk_64(unsigned long addr,
> > > unsigned long val)
> > > +{
> > > + int err = 0;
> > > +
> > > + asm volatile("1: wrussq %1, (%0)\n"
> > > + "2:\n"
> > > + _ASM_EXTABLE_HANDLE(1b, 2b,
> > > ex_handler_wruss)
> > > + :
> > > + : "r" (addr), "r" (val));
> > > +
> > > + return err;
> > > +}
> > What's up with "err"? You set it to zero, and then you return it,
> > but
> > nothing can ever set it to non-zero, right?
> >
> > >
> > > +__visible bool ex_handler_wruss(const struct
> > > exception_table_entry *fixup,
> > > + struct pt_regs *regs, int
> > > trapnr)
> > > +{
> > > + regs->ip = ex_fixup_addr(fixup);
> > > + regs->ax = -1;
> > > + return true;
> > > +}
> > And here you just write into regs->ax, but your "asm volatile"
> > doesn't
> > reserve that register. This looks wrong to me.
> >
> > I think you probably want to add something like an explicit
> > `"+&a"(err)` output to the asm statements.
> We require asm goto support these days. How about using that? You
> won't even need a special exception handler.
>
> Also, please change the BUG to WARN in the you-did-it-wrong 32-bit
> case. And return -EFAULT.
>
> --Andy
I will look into that.
Yu-cheng
^ permalink raw reply
* Re: [RFC PATCH v3 18/24] x86/cet/shstk: User-mode shadow stack support
From: Yu-cheng Yu @ 2018-08-30 16:20 UTC (permalink / raw)
To: Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek <pave>
In-Reply-To: <CAG48ez0d8+E_O-9u6ZHZ6dQO55Ui2ydV_kia-EEhyYeB_w4r2g@mail.gmail.com>
On Thu, 2018-08-30 at 18:10 +0200, Jann Horn wrote:
> On Thu, Aug 30, 2018 at 4:44 PM Yu-cheng Yu <yu-cheng.yu@intel.com>
> wrote:
> >
> >
> > This patch adds basic shadow stack enabling/disabling routines.
> > A task's shadow stack is allocated from memory with VM_SHSTK
> > flag set and read-only protection. The shadow stack is
> > allocated to a fixed size of RLIMIT_STACK.
> >
> > Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
> [...]
> >
> > +static int set_shstk_ptr(unsigned long addr)
> > +{
> > + u64 r;
> > +
> > + if (!cpu_feature_enabled(X86_FEATURE_SHSTK))
> > + return -1;
> > +
> > + if ((addr >= TASK_SIZE_MAX) || (!IS_ALIGNED(addr, 4)))
> > + return -1;
> > +
> > + rdmsrl(MSR_IA32_U_CET, r);
> > + wrmsrl(MSR_IA32_PL3_SSP, addr);
> > + wrmsrl(MSR_IA32_U_CET, r | MSR_IA32_CET_SHSTK_EN);
> > + return 0;
> > +}
> Here's a really stupid question: Where is the logic for switching
> those MSRs on task switch? MSR_IA32_PL3_SSP contains a userspace
> pointer, so it has to be switched on task switch, right? I'm sure
> I'm
> missing something obvious, but grepping for places that set
> MSR_IA32_PL3_SSP to nonzero values through the entire patchset, I
> only
> see set_shstk_ptr(), which is called from:
>
> - cet_setup_shstk() (called from arch_setup_features(), which is
> called from load_elf_binary())
> - cet_restore_signal() (called on signal handler return)
> - cet_setup_signal() (called from signal handling code)
The MSR is in the XSAVES buffer and switched by XSAVES/XRSTORS.
Yu-cheng
^ permalink raw reply
* Re: [RFC PATCH v3 18/24] x86/cet/shstk: User-mode shadow stack support
From: Jann Horn @ 2018-08-30 16:10 UTC (permalink / raw)
To: yu-cheng.yu
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov
In-Reply-To: <20180830143904.3168-19-yu-cheng.yu@intel.com>
On Thu, Aug 30, 2018 at 4:44 PM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
>
> This patch adds basic shadow stack enabling/disabling routines.
> A task's shadow stack is allocated from memory with VM_SHSTK
> flag set and read-only protection. The shadow stack is
> allocated to a fixed size of RLIMIT_STACK.
>
> Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
[...]
> +static int set_shstk_ptr(unsigned long addr)
> +{
> + u64 r;
> +
> + if (!cpu_feature_enabled(X86_FEATURE_SHSTK))
> + return -1;
> +
> + if ((addr >= TASK_SIZE_MAX) || (!IS_ALIGNED(addr, 4)))
> + return -1;
> +
> + rdmsrl(MSR_IA32_U_CET, r);
> + wrmsrl(MSR_IA32_PL3_SSP, addr);
> + wrmsrl(MSR_IA32_U_CET, r | MSR_IA32_CET_SHSTK_EN);
> + return 0;
> +}
Here's a really stupid question: Where is the logic for switching
those MSRs on task switch? MSR_IA32_PL3_SSP contains a userspace
pointer, so it has to be switched on task switch, right? I'm sure I'm
missing something obvious, but grepping for places that set
MSR_IA32_PL3_SSP to nonzero values through the entire patchset, I only
see set_shstk_ptr(), which is called from:
- cet_setup_shstk() (called from arch_setup_features(), which is
called from load_elf_binary())
- cet_restore_signal() (called on signal handler return)
- cet_setup_signal() (called from signal handling code)
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Dave Hansen @ 2018-08-30 16:08 UTC (permalink / raw)
To: Jann Horn, yu-cheng.yu
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Florian Weimer, hjl.tools, Jonathan Corbet,
keescook, Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek,
Peter Zijlstra
In-Reply-To: <CAG48ez0Rca0XsdXJZ07c+iGPyep0Gpxw+sxQuACP5gyPaBgDKA@mail.gmail.com>
On 08/30/2018 08:49 AM, Jann Horn wrote:
>> @@ -1203,7 +1203,28 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
>> static inline void ptep_set_wrprotect(struct mm_struct *mm,
>> unsigned long addr, pte_t *ptep)
>> {
>> + pte_t pte;
>> +
>> clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
>> + pte = *ptep;
>> +
>> + /*
>> + * Some processors can start a write, but ending up seeing
>> + * a read-only PTE by the time they get to the Dirty bit.
>> + * In this case, they will set the Dirty bit, leaving a
>> + * read-only, Dirty PTE which looks like a Shadow Stack PTE.
>> + *
>> + * However, this behavior has been improved and will not occur
>> + * on processors supporting Shadow Stacks. Without this
>> + * guarantee, a transition to a non-present PTE and flush the
>> + * TLB would be needed.
>> + *
>> + * When change a writable PTE to read-only and if the PTE has
>> + * _PAGE_DIRTY_HW set, we move that bit to _PAGE_DIRTY_SW so
>> + * that the PTE is not a valid Shadow Stack PTE.
>> + */
>> + pte = pte_move_flags(pte, _PAGE_DIRTY_HW, _PAGE_DIRTY_SW);
>> + set_pte_at(mm, addr, ptep, pte);
>> }
> I don't understand why it's okay that you first atomically clear the
> RW bit, then atomically switch from DIRTY_HW to DIRTY_SW. Doesn't that
> mean that between the two atomic writes, another core can incorrectly
> see a shadow stack?
Good point.
This could result in a spurious shadow-stack fault, or allow a
shadow-stack write to the page in the transient state.
But, the shadow-stack permissions are more restrictive than what could
be in the TLB at this point, so I don't think there's a real security
implication here.
The only trouble is handling the spurious shadow-stack fault. The
alternative is to go !Present for a bit, which we would probably just
handle fine in the existing page fault code.
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Yu-cheng Yu @ 2018-08-30 16:02 UTC (permalink / raw)
To: Jann Horn
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek <pave>
In-Reply-To: <CAG48ez0Rca0XsdXJZ07c+iGPyep0Gpxw+sxQuACP5gyPaBgDKA@mail.gmail.com>
On Thu, 2018-08-30 at 17:49 +0200, Jann Horn wrote:
> On Thu, Aug 30, 2018 at 4:43 PM Yu-cheng Yu <yu-cheng.yu@intel.com>
> wrote:
> >
> >
> > When Shadow Stack is enabled, the read-only and PAGE_DIRTY_HW PTE
> > setting is reserved only for the Shadow Stack. To track dirty of
> > non-Shadow Stack read-only PTEs, we use PAGE_DIRTY_SW.
> >
> > Update ptep_set_wrprotect() and pmdp_set_wrprotect().
> >
> > Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
> > ---
> > arch/x86/include/asm/pgtable.h | 42
> > ++++++++++++++++++++++++++++++++++
> > 1 file changed, 42 insertions(+)
> >
> > diff --git a/arch/x86/include/asm/pgtable.h
> > b/arch/x86/include/asm/pgtable.h
> > index 4d50de77ea96..556ef258eeff 100644
> > --- a/arch/x86/include/asm/pgtable.h
> > +++ b/arch/x86/include/asm/pgtable.h
> > @@ -1203,7 +1203,28 @@ static inline pte_t
> > ptep_get_and_clear_full(struct mm_struct *mm,
> > static inline void ptep_set_wrprotect(struct mm_struct *mm,
> > unsigned long addr, pte_t
> > *ptep)
> > {
> > + pte_t pte;
> > +
> > clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
> > + pte = *ptep;
> > +
> > + /*
> > + * Some processors can start a write, but ending up seeing
> > + * a read-only PTE by the time they get to the Dirty bit.
> > + * In this case, they will set the Dirty bit, leaving a
> > + * read-only, Dirty PTE which looks like a Shadow Stack
> > PTE.
> > + *
> > + * However, this behavior has been improved and will not
> > occur
> > + * on processors supporting Shadow Stacks. Without this
> > + * guarantee, a transition to a non-present PTE and flush
> > the
> > + * TLB would be needed.
> > + *
> > + * When change a writable PTE to read-only and if the PTE
> > has
> > + * _PAGE_DIRTY_HW set, we move that bit to _PAGE_DIRTY_SW
> > so
> > + * that the PTE is not a valid Shadow Stack PTE.
> > + */
> > + pte = pte_move_flags(pte, _PAGE_DIRTY_HW, _PAGE_DIRTY_SW);
> > + set_pte_at(mm, addr, ptep, pte);
> > }
> I don't understand why it's okay that you first atomically clear the
> RW bit, then atomically switch from DIRTY_HW to DIRTY_SW. Doesn't
> that
> mean that between the two atomic writes, another core can
> incorrectly
> see a shadow stack?
Yes, we had that concern earlier and checked.
On processors supporting Shadow Stacks, that will not happen.
Yu-cheng
^ permalink raw reply
* Re: [RFC PATCH v3 19/24] x86/cet/shstk: Introduce WRUSS instruction
From: Andy Lutomirski @ 2018-08-30 15:55 UTC (permalink / raw)
To: Jann Horn
Cc: Yu-cheng Yu, the arch/x86 maintainers, H . Peter Anvin,
Thomas Gleixner, Ingo Molnar, kernel list, linux-doc, Linux-MM,
linux-arch, Linux API, Arnd Bergmann, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, H. J. Lu,
Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov <ole>
In-Reply-To: <CAG48ez3uZrC-9uJ0uMoVPTtxRXRN8D+3zs5FknZD2woTT6mazg@mail.gmail.com>
On Thu, Aug 30, 2018 at 8:39 AM, Jann Horn <jannh@google.com> wrote:
> On Thu, Aug 30, 2018 at 4:44 PM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
>>
>> WRUSS is a new kernel-mode instruction but writes directly
>> to user shadow stack memory. This is used to construct
>> a return address on the shadow stack for the signal
>> handler.
>>
>> This instruction can fault if the user shadow stack is
>> invalid shadow stack memory. In that case, the kernel does
>> fixup.
>>
>> Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
> [...]
>> +static inline int write_user_shstk_64(unsigned long addr, unsigned long val)
>> +{
>> + int err = 0;
>> +
>> + asm volatile("1: wrussq %1, (%0)\n"
>> + "2:\n"
>> + _ASM_EXTABLE_HANDLE(1b, 2b, ex_handler_wruss)
>> + :
>> + : "r" (addr), "r" (val));
>> +
>> + return err;
>> +}
>
> What's up with "err"? You set it to zero, and then you return it, but
> nothing can ever set it to non-zero, right?
>
>> +__visible bool ex_handler_wruss(const struct exception_table_entry *fixup,
>> + struct pt_regs *regs, int trapnr)
>> +{
>> + regs->ip = ex_fixup_addr(fixup);
>> + regs->ax = -1;
>> + return true;
>> +}
>
> And here you just write into regs->ax, but your "asm volatile" doesn't
> reserve that register. This looks wrong to me.
>
> I think you probably want to add something like an explicit
> `"+&a"(err)` output to the asm statements.
We require asm goto support these days. How about using that? You
won't even need a special exception handler.
Also, please change the BUG to WARN in the you-did-it-wrong 32-bit
case. And return -EFAULT.
--Andy
^ permalink raw reply
* Re: [RFC PATCH v3 12/24] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
From: Jann Horn @ 2018-08-30 15:49 UTC (permalink / raw)
To: yu-cheng.yu
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov
In-Reply-To: <20180830143904.3168-13-yu-cheng.yu@intel.com>
On Thu, Aug 30, 2018 at 4:43 PM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
>
> When Shadow Stack is enabled, the read-only and PAGE_DIRTY_HW PTE
> setting is reserved only for the Shadow Stack. To track dirty of
> non-Shadow Stack read-only PTEs, we use PAGE_DIRTY_SW.
>
> Update ptep_set_wrprotect() and pmdp_set_wrprotect().
>
> Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
> ---
> arch/x86/include/asm/pgtable.h | 42 ++++++++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
>
> diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
> index 4d50de77ea96..556ef258eeff 100644
> --- a/arch/x86/include/asm/pgtable.h
> +++ b/arch/x86/include/asm/pgtable.h
> @@ -1203,7 +1203,28 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
> static inline void ptep_set_wrprotect(struct mm_struct *mm,
> unsigned long addr, pte_t *ptep)
> {
> + pte_t pte;
> +
> clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
> + pte = *ptep;
> +
> + /*
> + * Some processors can start a write, but ending up seeing
> + * a read-only PTE by the time they get to the Dirty bit.
> + * In this case, they will set the Dirty bit, leaving a
> + * read-only, Dirty PTE which looks like a Shadow Stack PTE.
> + *
> + * However, this behavior has been improved and will not occur
> + * on processors supporting Shadow Stacks. Without this
> + * guarantee, a transition to a non-present PTE and flush the
> + * TLB would be needed.
> + *
> + * When change a writable PTE to read-only and if the PTE has
> + * _PAGE_DIRTY_HW set, we move that bit to _PAGE_DIRTY_SW so
> + * that the PTE is not a valid Shadow Stack PTE.
> + */
> + pte = pte_move_flags(pte, _PAGE_DIRTY_HW, _PAGE_DIRTY_SW);
> + set_pte_at(mm, addr, ptep, pte);
> }
I don't understand why it's okay that you first atomically clear the
RW bit, then atomically switch from DIRTY_HW to DIRTY_SW. Doesn't that
mean that between the two atomic writes, another core can incorrectly
see a shadow stack?
^ permalink raw reply
* Re: [RFC PATCH v3 19/24] x86/cet/shstk: Introduce WRUSS instruction
From: Jann Horn @ 2018-08-30 15:39 UTC (permalink / raw)
To: yu-cheng.yu
Cc: the arch/x86 maintainers, H . Peter Anvin, Thomas Gleixner,
Ingo Molnar, kernel list, linux-doc, Linux-MM, linux-arch,
Linux API, Arnd Bergmann, Andy Lutomirski, Balbir Singh,
Cyrill Gorcunov, Dave Hansen, Florian Weimer, hjl.tools,
Jonathan Corbet, keescook, Mike Kravetz, Nadav Amit,
Oleg Nesterov
In-Reply-To: <20180830143904.3168-20-yu-cheng.yu@intel.com>
On Thu, Aug 30, 2018 at 4:44 PM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
>
> WRUSS is a new kernel-mode instruction but writes directly
> to user shadow stack memory. This is used to construct
> a return address on the shadow stack for the signal
> handler.
>
> This instruction can fault if the user shadow stack is
> invalid shadow stack memory. In that case, the kernel does
> fixup.
>
> Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
[...]
> +static inline int write_user_shstk_64(unsigned long addr, unsigned long val)
> +{
> + int err = 0;
> +
> + asm volatile("1: wrussq %1, (%0)\n"
> + "2:\n"
> + _ASM_EXTABLE_HANDLE(1b, 2b, ex_handler_wruss)
> + :
> + : "r" (addr), "r" (val));
> +
> + return err;
> +}
What's up with "err"? You set it to zero, and then you return it, but
nothing can ever set it to non-zero, right?
> +__visible bool ex_handler_wruss(const struct exception_table_entry *fixup,
> + struct pt_regs *regs, int trapnr)
> +{
> + regs->ip = ex_fixup_addr(fixup);
> + regs->ax = -1;
> + return true;
> +}
And here you just write into regs->ax, but your "asm volatile" doesn't
reserve that register. This looks wrong to me.
I think you probably want to add something like an explicit
`"+&a"(err)` output to the asm statements.
> @@ -1305,6 +1305,15 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code,
> error_code |= X86_PF_USER;
> flags |= FAULT_FLAG_USER;
> } else {
> + /*
> + * WRUSS is a kernel instrcution and but writes
Nits: typo ("instrcution"), weird grammar ("and but writes")
^ permalink raw reply
* [RFC PATCH v3 8/8] x86/cet: Add PTRACE interface for CET
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
Add REGSET_CET64/REGSET_CET32 to get/set CET MSRs:
IA32_U_CET (user-mode CET settings),
IA32_PL3_SSP (user-mode shadow stack),
IA32_PL0_SSP (kernel-mode shadow stack),
IA32_PL1_SSP (ring-1 shadow stack),
IA32_PL2_SSP (ring-2 shadow stack).
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/asm/fpu/regset.h | 7 +++---
arch/x86/kernel/fpu/regset.c | 41 +++++++++++++++++++++++++++++++
arch/x86/kernel/ptrace.c | 16 ++++++++++++
include/uapi/linux/elf.h | 1 +
4 files changed, 62 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/fpu/regset.h b/arch/x86/include/asm/fpu/regset.h
index d5bdffb9d27f..edad0d889084 100644
--- a/arch/x86/include/asm/fpu/regset.h
+++ b/arch/x86/include/asm/fpu/regset.h
@@ -7,11 +7,12 @@
#include <linux/regset.h>
-extern user_regset_active_fn regset_fpregs_active, regset_xregset_fpregs_active;
+extern user_regset_active_fn regset_fpregs_active, regset_xregset_fpregs_active,
+ cetregs_active;
extern user_regset_get_fn fpregs_get, xfpregs_get, fpregs_soft_get,
- xstateregs_get;
+ xstateregs_get, cetregs_get;
extern user_regset_set_fn fpregs_set, xfpregs_set, fpregs_soft_set,
- xstateregs_set;
+ xstateregs_set, cetregs_set;
/*
* xstateregs_active == regset_fpregs_active. Please refer to the comment
diff --git a/arch/x86/kernel/fpu/regset.c b/arch/x86/kernel/fpu/regset.c
index bc02f5144b95..7008eb084d36 100644
--- a/arch/x86/kernel/fpu/regset.c
+++ b/arch/x86/kernel/fpu/regset.c
@@ -160,6 +160,47 @@ int xstateregs_set(struct task_struct *target, const struct user_regset *regset,
return ret;
}
+int cetregs_active(struct task_struct *target, const struct user_regset *regset)
+{
+#ifdef CONFIG_X86_INTEL_CET
+ if (target->thread.cet.shstk_enabled || target->thread.cet.ibt_enabled)
+ return regset->n;
+#endif
+ return 0;
+}
+
+int cetregs_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ struct fpu *fpu = &target->thread.fpu;
+ struct cet_user_state *cetregs;
+
+ if (!boot_cpu_has(X86_FEATURE_SHSTK))
+ return -ENODEV;
+
+ cetregs = get_xsave_addr(&fpu->state.xsave, XFEATURE_MASK_SHSTK_USER);
+
+ fpu__prepare_read(fpu);
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, cetregs, 0, -1);
+}
+
+int cetregs_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct fpu *fpu = &target->thread.fpu;
+ struct cet_user_state *cetregs;
+
+ if (!boot_cpu_has(X86_FEATURE_SHSTK))
+ return -ENODEV;
+
+ cetregs = get_xsave_addr(&fpu->state.xsave, XFEATURE_MASK_SHSTK_USER);
+
+ fpu__prepare_write(fpu);
+ return user_regset_copyin(&pos, &count, &kbuf, &ubuf, cetregs, 0, -1);
+}
+
#if defined CONFIG_X86_32 || defined CONFIG_IA32_EMULATION
/*
diff --git a/arch/x86/kernel/ptrace.c b/arch/x86/kernel/ptrace.c
index e2ee403865eb..ac2bc3a18427 100644
--- a/arch/x86/kernel/ptrace.c
+++ b/arch/x86/kernel/ptrace.c
@@ -49,7 +49,9 @@ enum x86_regset {
REGSET_IOPERM64 = REGSET_XFP,
REGSET_XSTATE,
REGSET_TLS,
+ REGSET_CET64 = REGSET_TLS,
REGSET_IOPERM32,
+ REGSET_CET32,
};
struct pt_regs_offset {
@@ -1276,6 +1278,13 @@ static struct user_regset x86_64_regsets[] __ro_after_init = {
.size = sizeof(long), .align = sizeof(long),
.active = ioperm_active, .get = ioperm_get
},
+ [REGSET_CET64] = {
+ .core_note_type = NT_X86_CET,
+ .n = sizeof(struct cet_user_state) / sizeof(u64),
+ .size = sizeof(u64), .align = sizeof(u64),
+ .active = cetregs_active, .get = cetregs_get,
+ .set = cetregs_set
+ },
};
static const struct user_regset_view user_x86_64_view = {
@@ -1331,6 +1340,13 @@ static struct user_regset x86_32_regsets[] __ro_after_init = {
.size = sizeof(u32), .align = sizeof(u32),
.active = ioperm_active, .get = ioperm_get
},
+ [REGSET_CET32] = {
+ .core_note_type = NT_X86_CET,
+ .n = sizeof(struct cet_user_state) / sizeof(u64),
+ .size = sizeof(u64), .align = sizeof(u64),
+ .active = cetregs_active, .get = cetregs_get,
+ .set = cetregs_set
+ },
};
static const struct user_regset_view user_x86_32_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index 5ef25a565e88..f4cdfdc59c0a 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -401,6 +401,7 @@ typedef struct elf64_shdr {
#define NT_386_TLS 0x200 /* i386 TLS slots (struct user_desc) */
#define NT_386_IOPERM 0x201 /* x86 io permission bitmap (1=deny) */
#define NT_X86_XSTATE 0x202 /* x86 extended state using xsave */
+#define NT_X86_CET 0x203 /* x86 cet state */
#define NT_S390_HIGH_GPRS 0x300 /* s390 upper register halves */
#define NT_S390_TIMER 0x301 /* s390 timer register */
#define NT_S390_TODCMP 0x302 /* s390 TOD clock comparator register */
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 7/8] x86: Insert endbr32/endbr64 to vDSO
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
From: "H.J. Lu" <hjl.tools@gmail.com>
When Intel indirect branch tracking is enabled, functions in vDSO which
may be called indirectly must have endbr32 or endbr64 as the first
instruction. Compiler must support -fcf-protection=branch so that it
can be used to compile vDSO.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
---
arch/x86/entry/vdso/.gitignore | 4 ++++
arch/x86/entry/vdso/Makefile | 12 +++++++++++-
arch/x86/entry/vdso/vdso-layout.lds.S | 1 +
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/x86/entry/vdso/.gitignore b/arch/x86/entry/vdso/.gitignore
index aae8ffdd5880..552941fdfae0 100644
--- a/arch/x86/entry/vdso/.gitignore
+++ b/arch/x86/entry/vdso/.gitignore
@@ -5,3 +5,7 @@ vdso32-sysenter-syms.lds
vdso32-int80-syms.lds
vdso-image-*.c
vdso2c
+vclock_gettime.S
+vgetcpu.S
+vclock_gettime.asm
+vgetcpu.asm
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index fa3f439f0a92..8694f70c08e6 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -102,13 +102,17 @@ vobjx32s := $(foreach F,$(vobjx32s-y),$(obj)/$F)
# Convert 64bit object file to x32 for x32 vDSO.
quiet_cmd_x32 = X32 $@
- cmd_x32 = $(OBJCOPY) -O elf32-x86-64 $< $@
+ cmd_x32 = $(OBJCOPY) -R .note.gnu.property -O elf32-x86-64 $< $@
$(obj)/%-x32.o: $(obj)/%.o FORCE
$(call if_changed,x32)
targets += vdsox32.lds $(vobjx32s-y)
+ifdef CONFIG_X86_INTEL_BRANCH_TRACKING_USER
+ $(obj)/vclock_gettime.o $(obj)/vgetcpu.o $(obj)/vdso32/vclock_gettime.o: KBUILD_CFLAGS += -fcf-protection=branch
+endif
+
$(obj)/%.so: OBJCOPYFLAGS := -S
$(obj)/%.so: $(obj)/%.so.dbg
$(call if_changed,objcopy)
@@ -160,6 +164,12 @@ quiet_cmd_vdso = VDSO $@
VDSO_LDFLAGS = -shared $(call ld-option, --hash-style=both) \
$(call ld-option, --build-id) -Bsymbolic
+ifdef CONFIG_X86_INTEL_BRANCH_TRACKING_USER
+ VDSO_LDFLAGS += $(call ldoption, -z$(comma)ibt)
+endif
+ifdef CONFIG_X86_INTEL_SHADOW_STACK_USER
+ VDSO_LDFLAGS += $(call ldoption, -z$(comma)shstk)
+endif
GCOV_PROFILE := n
#
diff --git a/arch/x86/entry/vdso/vdso-layout.lds.S b/arch/x86/entry/vdso/vdso-layout.lds.S
index acfd5ba7d943..cabaeedfed78 100644
--- a/arch/x86/entry/vdso/vdso-layout.lds.S
+++ b/arch/x86/entry/vdso/vdso-layout.lds.S
@@ -74,6 +74,7 @@ SECTIONS
.fake_shstrtab : { *(.fake_shstrtab) } :text
+ .note.gnu.property : { *(.note.gnu.property) } :text :note
.note : { *(.note.*) } :text :note
.eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 6/8] mm/mmap: Add IBT bitmap size to address space limit check
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
The indirect branch tracking legacy bitmap takes a large address
space. This causes may_expand_vm() failure on the address limit
check. For a IBT-enabled task, add the bitmap size to the
address limit.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/uapi/asm/resource.h | 5 +++++
arch/x86/kernel/cet.c | 10 ++++++++--
include/uapi/asm-generic/resource.h | 3 +++
mm/mmap.c | 12 +++++++++++-
4 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/uapi/asm/resource.h b/arch/x86/include/uapi/asm/resource.h
index 04bc4db8921b..0741b2a6101a 100644
--- a/arch/x86/include/uapi/asm/resource.h
+++ b/arch/x86/include/uapi/asm/resource.h
@@ -1 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+#ifdef CONFIG_X86_INTEL_CET
+#define rlimit_as_extra() current->thread.cet.ibt_bitmap_size
+#endif
+
#include <asm-generic/resource.h>
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index 071b9dd5bc5c..f700743f6f48 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -295,20 +295,26 @@ int cet_setup_ibt(void)
if (!cpu_feature_enabled(X86_FEATURE_IBT))
return -EOPNOTSUPP;
+ /*
+ * Calculate size and put in thread header.
+ * may_expand_vm() needs this information.
+ */
size = TASK_SIZE / PAGE_SIZE / BITS_PER_BYTE;
+ current->thread.cet.ibt_bitmap_size = size;
bitmap = do_mmap_locked(0, size, PROT_READ | PROT_WRITE,
MAP_ANONYMOUS | MAP_PRIVATE,
VM_DONTDUMP);
- if (bitmap >= TASK_SIZE)
+ if (bitmap >= TASK_SIZE) {
+ current->thread.cet.ibt_bitmap_size = 0;
return -ENOMEM;
+ }
rdmsrl(MSR_IA32_U_CET, r);
r |= (MSR_IA32_CET_ENDBR_EN | MSR_IA32_CET_NO_TRACK_EN);
wrmsrl(MSR_IA32_U_CET, r);
current->thread.cet.ibt_bitmap_addr = bitmap;
- current->thread.cet.ibt_bitmap_size = size;
current->thread.cet.ibt_enabled = 1;
return 0;
}
diff --git a/include/uapi/asm-generic/resource.h b/include/uapi/asm-generic/resource.h
index f12db7a0da64..8a7608a09700 100644
--- a/include/uapi/asm-generic/resource.h
+++ b/include/uapi/asm-generic/resource.h
@@ -58,5 +58,8 @@
# define RLIM_INFINITY (~0UL)
#endif
+#ifndef rlimit_as_extra
+#define rlimit_as_extra() 0
+#endif
#endif /* _UAPI_ASM_GENERIC_RESOURCE_H */
diff --git a/mm/mmap.c b/mm/mmap.c
index 5f2b2b184c60..6f6c722c1484 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -3228,7 +3228,17 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
*/
bool may_expand_vm(struct mm_struct *mm, vm_flags_t flags, unsigned long npages)
{
- if (mm->total_vm + npages > rlimit(RLIMIT_AS) >> PAGE_SHIFT)
+ unsigned long as_limit = rlimit(RLIMIT_AS);
+ unsigned long as_limit_plus = as_limit + rlimit_as_extra();
+
+ /* as_limit_plus overflowed */
+ if (as_limit_plus < as_limit)
+ as_limit_plus = RLIM_INFINITY;
+
+ if (as_limit_plus > as_limit)
+ as_limit = as_limit_plus;
+
+ if (mm->total_vm + npages > as_limit >> PAGE_SHIFT)
return false;
if (is_data_mapping(flags) &&
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 5/8] x86/cet/ibt: Add ENDBR to op-code-map
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
Add control transfer terminating instructions:
ENDBR64/ENDBR32:
Mark a valid 64/32-bit control transfer endpoint.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/lib/x86-opcode-map.txt | 13 +++++++++++--
tools/objtool/arch/x86/lib/x86-opcode-map.txt | 13 +++++++++++--
2 files changed, 22 insertions(+), 4 deletions(-)
diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt
index c5e825d44766..fbc53481bc59 100644
--- a/arch/x86/lib/x86-opcode-map.txt
+++ b/arch/x86/lib/x86-opcode-map.txt
@@ -620,7 +620,16 @@ ea: SAVEPREVSSP (f3)
# Skip 0xeb-0xff
EndTable
-Table: 3-byte opcode 2 (0x0f 0x38)
+Table: 3-byte opcode 2 (0x0f 0x1e)
+Referrer:
+AVXcode:
+# Skip 0x00-0xf9
+fa: ENDBR64 (f3)
+fb: ENDBR32 (f3)
+#skip 0xfc-0xff
+EndTable
+
+Table: 3-byte opcode 3 (0x0f 0x38)
Referrer: 3-byte escape 1
AVXcode: 2
# 0x0f 0x38 0x00-0x0f
@@ -804,7 +813,7 @@ f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSS Pq,Qq
f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v)
EndTable
-Table: 3-byte opcode 3 (0x0f 0x3a)
+Table: 3-byte opcode 4 (0x0f 0x3a)
Referrer: 3-byte escape 2
AVXcode: 3
# 0x0f 0x3a 0x00-0xff
diff --git a/tools/objtool/arch/x86/lib/x86-opcode-map.txt b/tools/objtool/arch/x86/lib/x86-opcode-map.txt
index c5e825d44766..fbc53481bc59 100644
--- a/tools/objtool/arch/x86/lib/x86-opcode-map.txt
+++ b/tools/objtool/arch/x86/lib/x86-opcode-map.txt
@@ -620,7 +620,16 @@ ea: SAVEPREVSSP (f3)
# Skip 0xeb-0xff
EndTable
-Table: 3-byte opcode 2 (0x0f 0x38)
+Table: 3-byte opcode 2 (0x0f 0x1e)
+Referrer:
+AVXcode:
+# Skip 0x00-0xf9
+fa: ENDBR64 (f3)
+fb: ENDBR32 (f3)
+#skip 0xfc-0xff
+EndTable
+
+Table: 3-byte opcode 3 (0x0f 0x38)
Referrer: 3-byte escape 1
AVXcode: 2
# 0x0f 0x38 0x00-0x0f
@@ -804,7 +813,7 @@ f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSS Pq,Qq
f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v)
EndTable
-Table: 3-byte opcode 3 (0x0f 0x3a)
+Table: 3-byte opcode 4 (0x0f 0x3a)
Referrer: 3-byte escape 2
AVXcode: 3
# 0x0f 0x3a 0x00-0xff
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 4/8] x86/cet/ibt: Add arch_prctl functions for IBT
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
Update ARCH_CET_STATUS and ARCH_CET_DISABLE to include Indirect
Branch Tracking features.
Introduce:
arch_prctl(ARCH_CET_LEGACY_BITMAP, unsigned long *addr)
Enable the Indirect Branch Tracking legacy code bitmap.
The parameter 'addr' is a pointer to a user buffer.
On returning to the caller, the kernel fills the following:
*addr = IBT bitmap base address
*(addr + 1) = IBT bitmap size
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/uapi/asm/prctl.h | 1 +
arch/x86/kernel/cet_prctl.c | 38 ++++++++++++++++++++++++++++++-
arch/x86/kernel/process.c | 1 +
3 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/uapi/asm/prctl.h b/arch/x86/include/uapi/asm/prctl.h
index 3aec1088e01d..31d2465f9caf 100644
--- a/arch/x86/include/uapi/asm/prctl.h
+++ b/arch/x86/include/uapi/asm/prctl.h
@@ -18,5 +18,6 @@
#define ARCH_CET_DISABLE 0x3002
#define ARCH_CET_LOCK 0x3003
#define ARCH_CET_ALLOC_SHSTK 0x3004
+#define ARCH_CET_LEGACY_BITMAP 0x3005
#endif /* _ASM_X86_PRCTL_H */
diff --git a/arch/x86/kernel/cet_prctl.c b/arch/x86/kernel/cet_prctl.c
index c4b7c19f5040..df47b5ebc3f4 100644
--- a/arch/x86/kernel/cet_prctl.c
+++ b/arch/x86/kernel/cet_prctl.c
@@ -20,6 +20,8 @@ static int handle_get_status(unsigned long arg2)
if (current->thread.cet.shstk_enabled)
features |= GNU_PROPERTY_X86_FEATURE_1_SHSTK;
+ if (current->thread.cet.ibt_enabled)
+ features |= GNU_PROPERTY_X86_FEATURE_1_IBT;
shstk_base = current->thread.cet.shstk_base;
shstk_size = current->thread.cet.shstk_size;
@@ -49,9 +51,35 @@ static int handle_alloc_shstk(unsigned long arg2)
return 0;
}
+static int handle_bitmap(unsigned long arg2)
+{
+ unsigned long addr, size;
+
+ if (current->thread.cet.ibt_enabled) {
+ int err;
+
+ err = cet_setup_ibt_bitmap();
+ if (err)
+ return err;
+
+ addr = current->thread.cet.ibt_bitmap_addr;
+ size = current->thread.cet.ibt_bitmap_size;
+ } else {
+ addr = 0;
+ size = 0;
+ }
+
+ if (put_user(addr, (unsigned long __user *)arg2) ||
+ put_user(size, (unsigned long __user *)arg2 + 1))
+ return -EFAULT;
+
+ return 0;
+}
+
int prctl_cet(int option, unsigned long arg2)
{
- if (!cpu_feature_enabled(X86_FEATURE_SHSTK))
+ if (!cpu_feature_enabled(X86_FEATURE_SHSTK) &&
+ !cpu_feature_enabled(X86_FEATURE_IBT))
return -EINVAL;
switch (option) {
@@ -63,6 +91,8 @@ int prctl_cet(int option, unsigned long arg2)
return -EPERM;
if (arg2 & GNU_PROPERTY_X86_FEATURE_1_SHSTK)
cet_disable_free_shstk(current);
+ if (arg2 & GNU_PROPERTY_X86_FEATURE_1_IBT)
+ cet_disable_ibt();
return 0;
@@ -73,6 +103,12 @@ int prctl_cet(int option, unsigned long arg2)
case ARCH_CET_ALLOC_SHSTK:
return handle_alloc_shstk(arg2);
+ /*
+ * Allocate legacy bitmap and return address & size to user.
+ */
+ case ARCH_CET_LEGACY_BITMAP:
+ return handle_bitmap(arg2);
+
default:
return -EINVAL;
}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index ac0ea9c7e89f..aea15a9b6a3e 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -797,6 +797,7 @@ long do_arch_prctl_common(struct task_struct *task, int option,
case ARCH_CET_DISABLE:
case ARCH_CET_LOCK:
case ARCH_CET_ALLOC_SHSTK:
+ case ARCH_CET_LEGACY_BITMAP:
return prctl_cet(option, cpuid_enabled);
}
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 3/8] x86/cet/ibt: ELF header parsing for IBT
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
Look in .note.gnu.property of an ELF file and check if Indirect
Branch Tracking needs to be enabled for the task.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/uapi/asm/elf_property.h | 1 +
arch/x86/kernel/elf.c | 8 +++++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/uapi/asm/elf_property.h b/arch/x86/include/uapi/asm/elf_property.h
index af361207718c..343a871b8fc1 100644
--- a/arch/x86/include/uapi/asm/elf_property.h
+++ b/arch/x86/include/uapi/asm/elf_property.h
@@ -11,5 +11,6 @@
* Bits for GNU_PROPERTY_X86_FEATURE_1_AND
*/
#define GNU_PROPERTY_X86_FEATURE_1_SHSTK (0x00000002)
+#define GNU_PROPERTY_X86_FEATURE_1_IBT (0x00000001)
#endif /* _UAPI_ASM_X86_ELF_PROPERTY_H */
diff --git a/arch/x86/kernel/elf.c b/arch/x86/kernel/elf.c
index a2c41bf39c58..41957f1bd9d0 100644
--- a/arch/x86/kernel/elf.c
+++ b/arch/x86/kernel/elf.c
@@ -298,7 +298,8 @@ int arch_setup_features(void *ehdr_p, void *phdr_p,
struct elf64_hdr *ehdr64 = ehdr_p;
- if (!cpu_feature_enabled(X86_FEATURE_SHSTK))
+ if (!cpu_feature_enabled(X86_FEATURE_SHSTK) &&
+ !cpu_feature_enabled(X86_FEATURE_IBT))
return 0;
if (ehdr64->e_ident[EI_CLASS] == ELFCLASS64) {
@@ -333,6 +334,11 @@ int arch_setup_features(void *ehdr_p, void *phdr_p,
}
}
+ if (cpu_feature_enabled(X86_FEATURE_IBT)) {
+ if (feature & GNU_PROPERTY_X86_FEATURE_1_IBT)
+ err = cet_setup_ibt();
+ }
+
out:
return err;
}
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 2/8] x86/cet/ibt: User-mode indirect branch tracking support
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
Add user-mode indirect branch tracking enabling/disabling
and supporting routines.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/asm/cet.h | 8 +++
arch/x86/include/asm/disabled-features.h | 8 ++-
arch/x86/kernel/cet.c | 68 ++++++++++++++++++++++++
arch/x86/kernel/cpu/common.c | 20 ++++++-
arch/x86/kernel/process.c | 1 +
5 files changed, 103 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h
index 212bd68e31d3..1fea93fd436a 100644
--- a/arch/x86/include/asm/cet.h
+++ b/arch/x86/include/asm/cet.h
@@ -12,8 +12,11 @@ struct task_struct;
struct cet_status {
unsigned long shstk_base;
unsigned long shstk_size;
+ unsigned long ibt_bitmap_addr;
+ unsigned long ibt_bitmap_size;
unsigned int locked:1;
unsigned int shstk_enabled:1;
+ unsigned int ibt_enabled:1;
};
#ifdef CONFIG_X86_INTEL_CET
@@ -25,6 +28,9 @@ void cet_disable_shstk(void);
void cet_disable_free_shstk(struct task_struct *p);
int cet_restore_signal(unsigned long ssp);
int cet_setup_signal(bool ia32, unsigned long rstor, unsigned long *new_ssp);
+int cet_setup_ibt(void);
+int cet_setup_ibt_bitmap(void);
+void cet_disable_ibt(void);
#else
static inline int prctl_cet(int option, unsigned long arg2) { return 0; }
static inline int cet_setup_shstk(void) { return 0; }
@@ -35,6 +41,8 @@ static inline void cet_disable_free_shstk(struct task_struct *p) {}
static inline int cet_restore_signal(unsigned long ssp) { return 0; }
static inline int cet_setup_signal(bool ia32, unsigned long rstor,
unsigned long *new_ssp) { return 0; }
+static inline int cet_setup_ibt(void) { return 0; }
+static inline void cet_disable_ibt(void) {}
#endif
#endif /* __ASSEMBLY__ */
diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h
index 3624a11e5ba6..ce5bdaf0f1ff 100644
--- a/arch/x86/include/asm/disabled-features.h
+++ b/arch/x86/include/asm/disabled-features.h
@@ -62,6 +62,12 @@
#define DISABLE_SHSTK (1<<(X86_FEATURE_SHSTK & 31))
#endif
+#ifdef CONFIG_X86_INTEL_BRANCH_TRACKING_USER
+#define DISABLE_IBT 0
+#else
+#define DISABLE_IBT (1<<(X86_FEATURE_IBT & 31))
+#endif
+
/*
* Make sure to add features to the correct mask
*/
@@ -72,7 +78,7 @@
#define DISABLED_MASK4 (DISABLE_PCID)
#define DISABLED_MASK5 0
#define DISABLED_MASK6 0
-#define DISABLED_MASK7 (DISABLE_PTI)
+#define DISABLED_MASK7 (DISABLE_PTI|DISABLE_IBT)
#define DISABLED_MASK8 0
#define DISABLED_MASK9 (DISABLE_MPX)
#define DISABLED_MASK10 0
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index 1c2689738604..071b9dd5bc5c 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -12,6 +12,8 @@
#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/sched/signal.h>
+#include <linux/vmalloc.h>
+#include <linux/bitops.h>
#include <asm/msr.h>
#include <asm/user.h>
#include <asm/fpu/xstate.h>
@@ -283,3 +285,69 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr,
set_shstk_ptr(ssp);
return 0;
}
+
+int cet_setup_ibt(void)
+{
+ u64 r;
+ unsigned long bitmap;
+ unsigned long size;
+
+ if (!cpu_feature_enabled(X86_FEATURE_IBT))
+ return -EOPNOTSUPP;
+
+ size = TASK_SIZE / PAGE_SIZE / BITS_PER_BYTE;
+ bitmap = do_mmap_locked(0, size, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE,
+ VM_DONTDUMP);
+
+ if (bitmap >= TASK_SIZE)
+ return -ENOMEM;
+
+ rdmsrl(MSR_IA32_U_CET, r);
+ r |= (MSR_IA32_CET_ENDBR_EN | MSR_IA32_CET_NO_TRACK_EN);
+ wrmsrl(MSR_IA32_U_CET, r);
+
+ current->thread.cet.ibt_bitmap_addr = bitmap;
+ current->thread.cet.ibt_bitmap_size = size;
+ current->thread.cet.ibt_enabled = 1;
+ return 0;
+}
+
+int cet_setup_ibt_bitmap(void)
+{
+ u64 r;
+ unsigned long bitmap;
+
+ if (!cpu_feature_enabled(X86_FEATURE_IBT))
+ return -EOPNOTSUPP;
+
+ /*
+ * Lower bits of MSR_IA32_CET_LEG_IW_EN are for IBT
+ * settings. Clear lower bits even bitmap is already
+ * page-aligned.
+ */
+ bitmap = current->thread.cet.ibt_bitmap_addr;
+ bitmap &= PAGE_MASK;
+
+ /*
+ * Turn on IBT legacy bitmap.
+ */
+ rdmsrl(MSR_IA32_U_CET, r);
+ r |= (MSR_IA32_CET_LEG_IW_EN | bitmap);
+ wrmsrl(MSR_IA32_U_CET, r);
+ return 0;
+}
+
+void cet_disable_ibt(void)
+{
+ u64 r;
+
+ if (!cpu_feature_enabled(X86_FEATURE_IBT))
+ return;
+
+ rdmsrl(MSR_IA32_U_CET, r);
+ r &= ~(MSR_IA32_CET_ENDBR_EN | MSR_IA32_CET_LEG_IW_EN |
+ MSR_IA32_CET_NO_TRACK_EN);
+ wrmsrl(MSR_IA32_U_CET, r);
+ current->thread.cet.ibt_enabled = 0;
+}
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index e7eb41830add..cd03c4db2270 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -413,7 +413,8 @@ __setup("nopku", setup_disable_pku);
static __always_inline void setup_cet(struct cpuinfo_x86 *c)
{
- if (cpu_feature_enabled(X86_FEATURE_SHSTK))
+ if (cpu_feature_enabled(X86_FEATURE_SHSTK) ||
+ cpu_feature_enabled(X86_FEATURE_IBT))
cr4_set_bits(X86_CR4_CET);
}
@@ -434,6 +435,23 @@ static __init int setup_disable_shstk(char *s)
__setup("no_cet_shstk", setup_disable_shstk);
#endif
+#ifdef CONFIG_X86_INTEL_BRANCH_TRACKING_USER
+static __init int setup_disable_ibt(char *s)
+{
+ /* require an exact match without trailing characters */
+ if (strlen(s))
+ return 0;
+
+ if (!boot_cpu_has(X86_FEATURE_IBT))
+ return 1;
+
+ setup_clear_cpu_cap(X86_FEATURE_IBT);
+ pr_info("x86: 'no_cet_ibt' specified, disabling Branch Tracking\n");
+ return 1;
+}
+__setup("no_cet_ibt", setup_disable_ibt);
+#endif
+
/*
* Some CPU features depend on higher CPUID levels, which may not always
* be available due to CPUID level capping or broken virtualization
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 251b8714f9a3..ac0ea9c7e89f 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -137,6 +137,7 @@ void flush_thread(void)
memset(tsk->thread.tls_array, 0, sizeof(tsk->thread.tls_array));
cet_disable_shstk();
+ cet_disable_ibt();
fpu__clear(&tsk->thread.fpu);
}
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 1/8] x86/cet/ibt: Add Kconfig option for user-mode Indirect Branch Tracking
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830144009.3314-1-yu-cheng.yu@intel.com>
The user-mode indirect branch tracking support is done mostly by
GCC to insert ENDBR64/ENDBR32 instructions at branch targets.
The kernel provides CPUID enumeration, feature MSR setup and
the allocation of legacy bitmap.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/Kconfig | 12 ++++++++++++
arch/x86/Makefile | 7 +++++++
2 files changed, 19 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 2cfe11e1cf7f..0d97b03f35f6 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1941,6 +1941,18 @@ config X86_INTEL_SHADOW_STACK_USER
If unsure, say y.
+config X86_INTEL_BRANCH_TRACKING_USER
+ prompt "Intel Indirect Branch Tracking for user-mode"
+ def_bool n
+ depends on CPU_SUP_INTEL && X86_64
+ select X86_INTEL_CET
+ select ARCH_HAS_PROGRAM_PROPERTIES
+ ---help---
+ Indirect Branch Tracking provides hardware protection against return-/jmp-
+ oriented programing attacks.
+
+ If unsure, say y
+
config EFI
bool "EFI runtime service support"
depends on ACPI
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 00927853e409..0da5121c30db 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -159,6 +159,13 @@ ifdef CONFIG_X86_INTEL_SHADOW_STACK_USER
endif
endif
+# Check compiler ibt support
+ifdef CONFIG_X86_INTEL_BRANCH_TRACKING_USER
+ ifeq ($(call cc-option-yn, -fcf-protection=branch), n)
+ $(error CONFIG_X86_INTEL_BRANCH_TRACKING_USER not supported by compiler)
+ endif
+endif
+
#
# If the function graph tracer is used with mcount instead of fentry,
# '-maccumulate-outgoing-args' is needed to prevent a GCC bug
--
2.17.1
^ permalink raw reply related
* [RFC PATCH v3 0/8] Control Flow Enforcement: Branch Tracking, PTRACE
From: Yu-cheng Yu @ 2018-08-30 14:40 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
This set includes CET Indirect Branch Tracking patches and a PTRACE patch.
Summary of changes from v2:
Allocate IBT bitmap at the start of a task.
H.J. Lu (1):
x86: Insert endbr32/endbr64 to vDSO
Yu-cheng Yu (7):
x86/cet/ibt: Add Kconfig option for user-mode Indirect Branch Tracking
x86/cet/ibt: User-mode indirect branch tracking support
x86/cet/ibt: ELF header parsing for IBT
x86/cet/ibt: Add arch_prctl functions for IBT
x86/cet/ibt: Add ENDBR to op-code-map
mm/mmap: Add IBT bitmap size to address space limit check
x86/cet: Add PTRACE interface for CET
arch/x86/Kconfig | 12 +++
arch/x86/Makefile | 7 ++
arch/x86/entry/vdso/.gitignore | 4 +
arch/x86/entry/vdso/Makefile | 12 ++-
arch/x86/entry/vdso/vdso-layout.lds.S | 1 +
arch/x86/include/asm/cet.h | 8 ++
arch/x86/include/asm/disabled-features.h | 8 +-
arch/x86/include/asm/fpu/regset.h | 7 +-
arch/x86/include/uapi/asm/elf_property.h | 1 +
arch/x86/include/uapi/asm/prctl.h | 1 +
arch/x86/include/uapi/asm/resource.h | 5 ++
arch/x86/kernel/cet.c | 74 +++++++++++++++++++
arch/x86/kernel/cet_prctl.c | 38 +++++++++-
arch/x86/kernel/cpu/common.c | 20 ++++-
arch/x86/kernel/elf.c | 8 +-
arch/x86/kernel/fpu/regset.c | 41 ++++++++++
arch/x86/kernel/process.c | 2 +
arch/x86/kernel/ptrace.c | 16 ++++
arch/x86/lib/x86-opcode-map.txt | 13 +++-
include/uapi/asm-generic/resource.h | 3 +
include/uapi/linux/elf.h | 1 +
mm/mmap.c | 12 ++-
tools/objtool/arch/x86/lib/x86-opcode-map.txt | 13 +++-
23 files changed, 294 insertions(+), 13 deletions(-)
--
2.17.1
^ permalink raw reply
* [RFC PATCH v3 24/24] x86/cet/shstk: Add Shadow Stack instructions to opcode map
From: Yu-cheng Yu @ 2018-08-30 14:39 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Cyrill Gorcunov, Dave Hansen,
Florian Weimer, H.J. Lu, Jann Horn, Jonathan Corbet, Kees Cook,
Mike Kravetz, Nadav Amit, Oleg Nesterov, Pavel Machek
Cc: Yu-cheng Yu
In-Reply-To: <20180830143904.3168-1-yu-cheng.yu@intel.com>
Add the following shadow stack management instructions.
INCSSP:
Increment shadow stack pointer by the steps specified.
RDSSP:
Read SSP register into a GPR.
SAVEPREVSSP:
Use "prev ssp" token at top of current shadow stack to
create a "restore token" on previous shadow stack.
RSTORSSP:
Restore from a "restore token" pointed by a GPR to SSP.
WRSS:
Write to kernel-mode shadow stack (kernel-mode instruction).
WRUSS:
Write to user-mode shadow stack (kernel-mode instruction).
SETSSBSY:
Verify the "supervisor token" pointed by IA32_PL0_SSP MSR,
if valid, set the token to busy, and set SSP to the value
of IA32_PL0_SSP MSR.
CLRSSBSY:
Verify the "supervisor token" pointed by a GPR, if valid,
clear the busy bit from the token.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/lib/x86-opcode-map.txt | 26 +++++++++++++------
tools/objtool/arch/x86/lib/x86-opcode-map.txt | 26 +++++++++++++------
2 files changed, 36 insertions(+), 16 deletions(-)
diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt
index e0b85930dd77..c5e825d44766 100644
--- a/arch/x86/lib/x86-opcode-map.txt
+++ b/arch/x86/lib/x86-opcode-map.txt
@@ -366,7 +366,7 @@ AVXcode: 1
1b: BNDCN Gv,Ev (F2) | BNDMOV Ev,Gv (66) | BNDMK Gv,Ev (F3) | BNDSTX Ev,Gv
1c:
1d:
-1e:
+1e: RDSSP Rd (F3),REX.W
1f: NOP Ev
# 0x0f 0x20-0x2f
20: MOV Rd,Cd
@@ -610,7 +610,17 @@ fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1)
ff: UD0
EndTable
-Table: 3-byte opcode 1 (0x0f 0x38)
+Table: 3-byte opcode 1 (0x0f 0x01)
+Referrer:
+AVXcode:
+# Skip 0x00-0xe7
+e8: SETSSBSY (f3)
+e9:
+ea: SAVEPREVSSP (f3)
+# Skip 0xeb-0xff
+EndTable
+
+Table: 3-byte opcode 2 (0x0f 0x38)
Referrer: 3-byte escape 1
AVXcode: 2
# 0x0f 0x38 0x00-0x0f
@@ -789,12 +799,12 @@ f0: MOVBE Gy,My | MOVBE Gw,Mw (66) | CRC32 Gd,Eb (F2) | CRC32 Gd,Eb (66&F2)
f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2)
f2: ANDN Gy,By,Ey (v)
f3: Grp17 (1A)
-f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v)
-f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v)
+f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) | WRUSS Pq,Qq (66),REX.W
+f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSS Pq,Qq (66),REX.W
f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v)
EndTable
-Table: 3-byte opcode 2 (0x0f 0x3a)
+Table: 3-byte opcode 3 (0x0f 0x3a)
Referrer: 3-byte escape 2
AVXcode: 3
# 0x0f 0x3a 0x00-0xff
@@ -948,7 +958,7 @@ GrpTable: Grp7
2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B)
3: LIDT Ms
4: SMSW Mw/Rv
-5: rdpkru (110),(11B) | wrpkru (111),(11B)
+5: rdpkru (110),(11B) | wrpkru (111),(11B) | RSTORSSP Mq (F3)
6: LMSW Ew
7: INVLPG Mb | SWAPGS (o64),(000),(11B) | RDTSCP (001),(11B)
EndTable
@@ -1019,8 +1029,8 @@ GrpTable: Grp15
2: vldmxcsr Md (v1) | WRFSBASE Ry (F3),(11B)
3: vstmxcsr Md (v1) | WRGSBASE Ry (F3),(11B)
4: XSAVE | ptwrite Ey (F3),(11B)
-5: XRSTOR | lfence (11B)
-6: XSAVEOPT | clwb (66) | mfence (11B)
+5: XRSTOR | lfence (11B) | INCSSP Rd (F3),REX.W
+6: XSAVEOPT | clwb (66) | mfence (11B) | CLRSSBSY Mq (F3)
7: clflush | clflushopt (66) | sfence (11B)
EndTable
diff --git a/tools/objtool/arch/x86/lib/x86-opcode-map.txt b/tools/objtool/arch/x86/lib/x86-opcode-map.txt
index e0b85930dd77..c5e825d44766 100644
--- a/tools/objtool/arch/x86/lib/x86-opcode-map.txt
+++ b/tools/objtool/arch/x86/lib/x86-opcode-map.txt
@@ -366,7 +366,7 @@ AVXcode: 1
1b: BNDCN Gv,Ev (F2) | BNDMOV Ev,Gv (66) | BNDMK Gv,Ev (F3) | BNDSTX Ev,Gv
1c:
1d:
-1e:
+1e: RDSSP Rd (F3),REX.W
1f: NOP Ev
# 0x0f 0x20-0x2f
20: MOV Rd,Cd
@@ -610,7 +610,17 @@ fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1)
ff: UD0
EndTable
-Table: 3-byte opcode 1 (0x0f 0x38)
+Table: 3-byte opcode 1 (0x0f 0x01)
+Referrer:
+AVXcode:
+# Skip 0x00-0xe7
+e8: SETSSBSY (f3)
+e9:
+ea: SAVEPREVSSP (f3)
+# Skip 0xeb-0xff
+EndTable
+
+Table: 3-byte opcode 2 (0x0f 0x38)
Referrer: 3-byte escape 1
AVXcode: 2
# 0x0f 0x38 0x00-0x0f
@@ -789,12 +799,12 @@ f0: MOVBE Gy,My | MOVBE Gw,Mw (66) | CRC32 Gd,Eb (F2) | CRC32 Gd,Eb (66&F2)
f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2)
f2: ANDN Gy,By,Ey (v)
f3: Grp17 (1A)
-f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v)
-f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v)
+f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) | WRUSS Pq,Qq (66),REX.W
+f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) | WRSS Pq,Qq (66),REX.W
f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v)
EndTable
-Table: 3-byte opcode 2 (0x0f 0x3a)
+Table: 3-byte opcode 3 (0x0f 0x3a)
Referrer: 3-byte escape 2
AVXcode: 3
# 0x0f 0x3a 0x00-0xff
@@ -948,7 +958,7 @@ GrpTable: Grp7
2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B)
3: LIDT Ms
4: SMSW Mw/Rv
-5: rdpkru (110),(11B) | wrpkru (111),(11B)
+5: rdpkru (110),(11B) | wrpkru (111),(11B) | RSTORSSP Mq (F3)
6: LMSW Ew
7: INVLPG Mb | SWAPGS (o64),(000),(11B) | RDTSCP (001),(11B)
EndTable
@@ -1019,8 +1029,8 @@ GrpTable: Grp15
2: vldmxcsr Md (v1) | WRFSBASE Ry (F3),(11B)
3: vstmxcsr Md (v1) | WRGSBASE Ry (F3),(11B)
4: XSAVE | ptwrite Ey (F3),(11B)
-5: XRSTOR | lfence (11B)
-6: XSAVEOPT | clwb (66) | mfence (11B)
+5: XRSTOR | lfence (11B) | INCSSP Rd (F3),REX.W
+6: XSAVEOPT | clwb (66) | mfence (11B) | CLRSSBSY Mq (F3)
7: clflush | clflushopt (66) | sfence (11B)
EndTable
--
2.17.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox