* Re: [PATCH v8 1/4] mm/cma: Add PF flag to force non cma alloc
From: Vlastimil Babka @ 2019-02-28 12:20 UTC (permalink / raw)
To: Aneesh Kumar K.V, akpm, Michal Hocko, Alexey Kardashevskiy,
David Gibson, Andrea Arcangeli, mpe
Cc: Peter Zijlstra, linux-kernel, Matthew Wilcox, linux-mm,
Ingo Molnar, linuxppc-dev
In-Reply-To: <20190227144736.5872-2-aneesh.kumar@linux.ibm.com>
On 2/27/19 3:47 PM, Aneesh Kumar K.V wrote:
> This patch adds PF_MEMALLOC_NOCMA which make sure any allocation in that context
> is marked non-movable and hence cannot be satisfied by CMA region.
>
> This is useful with get_user_pages_longterm where we want to take a page pin by
> migrating pages from CMA region. Marking the section PF_MEMALLOC_NOCMA ensures
> that we avoid unnecessary page migration later.
>
> Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
> Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
+CC scheduler guys
Do we really take the last available PF flag just so that "we avoid
unnecessary page migration later"?
If yes, that's a third PF_MEMALLOC flag, should we get separate variable
for gfp context at this point?
Also I don't like the name PF_MEMALLOC_NOCMA, as it's unnecessarily tied
to CMA. If anything it should be e.g. PF_MEMALLOC_NOMOVABLE.
Thanks.
> ---
> include/linux/sched.h | 1 +
> include/linux/sched/mm.h | 48 +++++++++++++++++++++++++++++++++-------
> 2 files changed, 41 insertions(+), 8 deletions(-)
>
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index f9b43c989577..dfa90088ba08 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1403,6 +1403,7 @@ extern struct pid *cad_pid;
> #define PF_UMH 0x02000000 /* I'm an Usermodehelper process */
> #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_allowed */
> #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */
> +#define PF_MEMALLOC_NOCMA 0x10000000 /* All allocation request will have _GFP_MOVABLE cleared */
> #define PF_MUTEX_TESTER 0x20000000 /* Thread belongs to the rt mutex tester */
> #define PF_FREEZER_SKIP 0x40000000 /* Freezer should not count it as freezable */
> #define PF_SUSPEND_TASK 0x80000000 /* This thread called freeze_processes() and should not be frozen */
> diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
> index 3bfa6a0cbba4..0cd9f10423fb 100644
> --- a/include/linux/sched/mm.h
> +++ b/include/linux/sched/mm.h
> @@ -148,17 +148,25 @@ static inline bool in_vfork(struct task_struct *tsk)
> * Applies per-task gfp context to the given allocation flags.
> * PF_MEMALLOC_NOIO implies GFP_NOIO
> * PF_MEMALLOC_NOFS implies GFP_NOFS
> + * PF_MEMALLOC_NOCMA implies no allocation from CMA region.
> */
> static inline gfp_t current_gfp_context(gfp_t flags)
> {
> - /*
> - * NOIO implies both NOIO and NOFS and it is a weaker context
> - * so always make sure it makes precedence
> - */
> - if (unlikely(current->flags & PF_MEMALLOC_NOIO))
> - flags &= ~(__GFP_IO | __GFP_FS);
> - else if (unlikely(current->flags & PF_MEMALLOC_NOFS))
> - flags &= ~__GFP_FS;
> + if (unlikely(current->flags &
> + (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS | PF_MEMALLOC_NOCMA))) {
> + /*
> + * NOIO implies both NOIO and NOFS and it is a weaker context
> + * so always make sure it makes precedence
> + */
> + if (current->flags & PF_MEMALLOC_NOIO)
> + flags &= ~(__GFP_IO | __GFP_FS);
> + else if (current->flags & PF_MEMALLOC_NOFS)
> + flags &= ~__GFP_FS;
> +#ifdef CONFIG_CMA
> + if (current->flags & PF_MEMALLOC_NOCMA)
> + flags &= ~__GFP_MOVABLE;
> +#endif
> + }
> return flags;
> }
>
> @@ -248,6 +256,30 @@ static inline void memalloc_noreclaim_restore(unsigned int flags)
> current->flags = (current->flags & ~PF_MEMALLOC) | flags;
> }
>
> +#ifdef CONFIG_CMA
> +static inline unsigned int memalloc_nocma_save(void)
> +{
> + unsigned int flags = current->flags & PF_MEMALLOC_NOCMA;
> +
> + current->flags |= PF_MEMALLOC_NOCMA;
> + return flags;
> +}
> +
> +static inline void memalloc_nocma_restore(unsigned int flags)
> +{
> + current->flags = (current->flags & ~PF_MEMALLOC_NOCMA) | flags;
> +}
> +#else
> +static inline unsigned int memalloc_nocma_save(void)
> +{
> + return 0;
> +}
> +
> +static inline void memalloc_nocma_restore(unsigned int flags)
> +{
> +}
> +#endif
> +
> #ifdef CONFIG_MEMCG
> /**
> * memalloc_use_memcg - Starts the remote memcg charging scope.
>
^ permalink raw reply
* Re: [PATCH 2/2] mm/dax: Don't enable huge dax mapping by default
From: Aneesh Kumar K.V @ 2019-02-28 12:32 UTC (permalink / raw)
To: Jan Kara
Cc: linux-kernel, linux-mm, Oliver O'Halloran, Ross Zwisler, akpm,
linuxppc-dev, Dan Williams, Kirill A . Shutemov
In-Reply-To: <20190228094011.GB22210@quack2.suse.cz>
On 2/28/19 3:10 PM, Jan Kara wrote:
> On Thu 28-02-19 14:05:22, Aneesh Kumar K.V wrote:
>> Add a flag to indicate the ability to do huge page dax mapping. On architecture
>> like ppc64, the hypervisor can disable huge page support in the guest. In
>> such a case, we should not enable huge page dax mapping. This patch adds
>> a flag which the architecture code will update to indicate huge page
>> dax mapping support.
>>
>> Architectures mostly do transparent_hugepage_flag = 0; if they can't
>> do hugepages. That also takes care of disabling dax hugepage mapping
>> with this change.
>>
>> Without this patch we get the below error with kvm on ppc64.
>>
>> [ 118.849975] lpar: Failed hash pte insert with error -4
>>
>> NOTE: The patch also use
>>
>> echo never > /sys/kernel/mm/transparent_hugepage/enabled
>> to disable dax huge page mapping.
>>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>
> Added Dan to CC for opinion. I kind of fail to see why you don't use
> TRANSPARENT_HUGEPAGE_FLAG for this. I know that technically DAX huge pages
> and normal THPs are different things but so far we've tried to avoid making
> that distinction visible to userspace.
I would also like to use the same flag. Was not sure whether it was ok.
In fact that is one of the reason I hooked this to
/sys/kernel/mm/transparent_hugepage/enabled. If we are ok with using
same flag, we can kill the vma_is_dax() check completely.
-aneesh
^ permalink raw reply
* Re: [PATCH 1/2] fs/dax: deposit pagetable even when installing zero page
From: Aneesh Kumar K.V @ 2019-02-28 12:34 UTC (permalink / raw)
To: Jan Kara
Cc: linux-kernel, linux-mm, Oliver O'Halloran, Ross Zwisler, akpm,
linuxppc-dev, Kirill A . Shutemov
In-Reply-To: <20190228092101.GA22210@quack2.suse.cz>
On 2/28/19 2:51 PM, Jan Kara wrote:
> On Thu 28-02-19 14:05:21, Aneesh Kumar K.V wrote:
>> Architectures like ppc64 use the deposited page table to store hardware
>> page table slot information. Make sure we deposit a page table when
>> using zero page at the pmd level for hash.
>>
>> Without this we hit
>>
>> Unable to handle kernel paging request for data at address 0x00000000
>> Faulting instruction address: 0xc000000000082a74
>> Oops: Kernel access of bad area, sig: 11 [#1]
>> ....
>>
>> NIP [c000000000082a74] __hash_page_thp+0x224/0x5b0
>> LR [c0000000000829a4] __hash_page_thp+0x154/0x5b0
>> Call Trace:
>> hash_page_mm+0x43c/0x740
>> do_hash_page+0x2c/0x3c
>> copy_from_iter_flushcache+0xa4/0x4a0
>> pmem_copy_from_iter+0x2c/0x50 [nd_pmem]
>> dax_copy_from_iter+0x40/0x70
>> dax_iomap_actor+0x134/0x360
>> iomap_apply+0xfc/0x1b0
>> dax_iomap_rw+0xac/0x130
>> ext4_file_write_iter+0x254/0x460 [ext4]
>> __vfs_write+0x120/0x1e0
>> vfs_write+0xd8/0x220
>> SyS_write+0x6c/0x110
>> system_call+0x3c/0x130
>>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>
> Thanks for the patch. It looks good to me. You can add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
>
>> ---
>> TODO:
>> * Add fixes tag
>
> Probably this is a problem since initial PPC PMEM support, isn't it?
>
Considering ppc64 is the only broken architecture here, I guess I will
use the commit that enabled PPC PMEM support here.
-aneesh
^ permalink raw reply
* Re: [PATCH 2/2] mm/dax: Don't enable huge dax mapping by default
From: Aneesh Kumar K.V @ 2019-02-28 12:43 UTC (permalink / raw)
To: Oliver, Dan Williams
Cc: Jan Kara, Linux Kernel Mailing List, Linux MM, Ross Zwisler,
Andrew Morton, linuxppc-dev, Kirill A . Shutemov
In-Reply-To: <CAOSf1CHjkyX2NTex7dc1AEHXSDcWA_UGYX8NoSyHpb5s_RkwXQ@mail.gmail.com>
On 2/28/19 3:10 PM, Oliver wrote:
> On Thu, Feb 28, 2019 at 7:35 PM Aneesh Kumar K.V
> <aneesh.kumar@linux.ibm.com> wrote:
>>
>> Add a flag to indicate the ability to do huge page dax mapping. On architecture
>> like ppc64, the hypervisor can disable huge page support in the guest. In
>> such a case, we should not enable huge page dax mapping. This patch adds
>> a flag which the architecture code will update to indicate huge page
>> dax mapping support.
>
> *groan*
>
>> Architectures mostly do transparent_hugepage_flag = 0; if they can't
>> do hugepages. That also takes care of disabling dax hugepage mapping
>> with this change.
>>
>> Without this patch we get the below error with kvm on ppc64.
>>
>> [ 118.849975] lpar: Failed hash pte insert with error -4
>>
>> NOTE: The patch also use
>>
>> echo never > /sys/kernel/mm/transparent_hugepage/enabled
>> to disable dax huge page mapping.
>>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>> ---
>> TODO:
>> * Add Fixes: tag
>>
>> include/linux/huge_mm.h | 4 +++-
>> mm/huge_memory.c | 4 ++++
>> 2 files changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> index 381e872bfde0..01ad5258545e 100644
>> --- a/include/linux/huge_mm.h
>> +++ b/include/linux/huge_mm.h
>> @@ -53,6 +53,7 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr,
>> pud_t *pud, pfn_t pfn, bool write);
>> enum transparent_hugepage_flag {
>> TRANSPARENT_HUGEPAGE_FLAG,
>> + TRANSPARENT_HUGEPAGE_DAX_FLAG,
>> TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG,
>> TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
>> TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG,
>> @@ -111,7 +112,8 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
>> if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG))
>> return true;
>>
>> - if (vma_is_dax(vma))
>> + if (vma_is_dax(vma) &&
>> + (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_DAX_FLAG)))
>> return true;
>
> Forcing PTE sized faults should be fine for fsdax, but it'll break
> devdax. The devdax driver requires the fault size be >= the namespace
> alignment since devdax tries to guarantee hugepage mappings will be
> used and PMD alignment is the default. We can probably have devdax
> fall back to the largest size the hypervisor has made available, but
> it does run contrary to the design. Ah well, I suppose it's better off
> being degraded rather than unusable.
>
Will fix that. I will make PFN_DEFAULT_ALIGNMENT arch specific.
>> if (transparent_hugepage_flags &
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index faf357eaf0ce..43d742fe0341 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -53,6 +53,7 @@ unsigned long transparent_hugepage_flags __read_mostly =
>> #ifdef CONFIG_TRANSPARENT_HUGEPAGE_MADVISE
>> (1<<TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)|
>> #endif
>> + (1 << TRANSPARENT_HUGEPAGE_DAX_FLAG) |
>> (1<<TRANSPARENT_HUGEPAGE_DEFRAG_REQ_MADV_FLAG)|
>> (1<<TRANSPARENT_HUGEPAGE_DEFRAG_KHUGEPAGED_FLAG)|
>> (1<<TRANSPARENT_HUGEPAGE_USE_ZERO_PAGE_FLAG);
>> @@ -475,6 +476,8 @@ static int __init setup_transparent_hugepage(char *str)
>> &transparent_hugepage_flags);
>> clear_bit(TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG,
>> &transparent_hugepage_flags);
>> + clear_bit(TRANSPARENT_HUGEPAGE_DAX_FLAG,
>> + &transparent_hugepage_flags);
>> ret = 1;
>> }
>> out:
>
>> @@ -753,6 +756,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr,
>> spinlock_t *ptl;
>>
>> ptl = pmd_lock(mm, pmd);
>> + /* should we check for none here again? */
>
> VM_WARN_ON() maybe? If THP is disabled and we're here then something
> has gone wrong.
I was wondering whether we can end up calling insert_pfn_pmd in parallel
and hence end up having a pmd entry here already. Usually we check for
if (!pmd_none(pmd)) after holding pmd_lock. Was not sure whether there
is anything preventing a parallel fault in case of dax.
>
>> entry = pmd_mkhuge(pfn_t_pmd(pfn, prot));
>> if (pfn_t_devmap(pfn))
>> entry = pmd_mkdevmap(entry);
>> --
>> 2.20.1
>>
>
^ permalink raw reply
* Re: [PATCH] powerpc: remove dead code in head_fsl_booke.S
From: Christophe Leroy @ 2019-02-28 13:00 UTC (permalink / raw)
To: Jason Yan, benh, paulus, mpe, diana.craciun, linuxppc-dev
In-Reply-To: <20190228083121.30598-1-yanaijie@huawei.com>
Le 28/02/2019 à 09:31, Jason Yan a écrit :
> This code is dead. Just remove it.
>
> Signed-off-by: Jason Yan <yanaijie@huawei.com>
Reviewed-by: Christophe Leroy <christophe.leroy@c-s.fr>
This review was easy :)
Christophe
> ---
> arch/powerpc/kernel/head_fsl_booke.S | 7 -------
> 1 file changed, 7 deletions(-)
>
> diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S
> index 2386ce2a9c6e..b8450f017d85 100644
> --- a/arch/powerpc/kernel/head_fsl_booke.S
> +++ b/arch/powerpc/kernel/head_fsl_booke.S
> @@ -194,13 +194,6 @@ set_ivor:
> #endif
> mtspr SPRN_MAS4, r2
>
> -#if 0
> - /* Enable DOZE */
> - mfspr r2,SPRN_HID0
> - oris r2,r2,HID0_DOZE@h
> - mtspr SPRN_HID0, r2
> -#endif
> -
> #if !defined(CONFIG_BDI_SWITCH)
> /*
> * The Abatron BDI JTAG debugger does not tolerate others
>
^ permalink raw reply
* Re: BUG: KASAN: stack-out-of-bounds
From: Christophe Leroy @ 2019-02-28 13:41 UTC (permalink / raw)
To: Andrey Ryabinin, Dmitry Vyukov
Cc: Linux-MM, Alexander Potapenko, linuxppc-dev, kasan-dev,
Daniel Axtens
In-Reply-To: <11314e32-6044-9207-a238-738e394ea2eb@virtuozzo.com>
Le 28/02/2019 à 10:47, Andrey Ryabinin a écrit :
>
>
> On 2/28/19 12:27 PM, Dmitry Vyukov wrote:
>> On Thu, Feb 28, 2019 at 10:22 AM Andrey Ryabinin
>> <aryabinin@virtuozzo.com> wrote:
>>>
>>>
>>>
>>> On 2/27/19 4:11 PM, Christophe Leroy wrote:
>>>>
>>>>
>>>> Le 27/02/2019 à 10:19, Andrey Ryabinin a écrit :
>>>>>
>>>>>
>>>>> On 2/27/19 11:25 AM, Christophe Leroy wrote:
>>>>>> With version v8 of the series implementing KASAN on 32 bits powerpc (https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=94309), I'm now able to activate KASAN on a mac99 is QEMU.
>>>>>>
>>>>>> Then I get the following reports at startup. Which of the two reports I get seems to depend on the option used to build the kernel, but for a given kernel I always get the same report.
>>>>>>
>>>>>> Is that a real bug, in which case how could I spot it ? Or is it something wrong in my implementation of KASAN ?
>>>>>>
>>>>>> I checked that after kasan_init(), the entire shadow memory is full of 0 only.
>>>>>>
>>>>>> I also made a try with the strong STACK_PROTECTOR compiled in, but no difference and nothing detected by the stack protector.
>>>>>>
>>>>>> ==================================================================
>>>>>> BUG: KASAN: stack-out-of-bounds in memchr+0x24/0x74
>>>>>> Read of size 1 at addr c0ecdd40 by task swapper/0
>>>>>>
>>>>>> CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc7+ #1133
>>>>>> Call Trace:
>>>>>> [c0e9dca0] [c01c42a0] print_address_description+0x64/0x2bc (unreliable)
>>>>>> [c0e9dcd0] [c01c4684] kasan_report+0xfc/0x180
>>>>>> [c0e9dd10] [c089579c] memchr+0x24/0x74
>>>>>> [c0e9dd30] [c00a9e38] msg_print_text+0x124/0x574
>>>>>> [c0e9dde0] [c00ab710] console_unlock+0x114/0x4f8
>>>>>> [c0e9de40] [c00adc60] vprintk_emit+0x188/0x1c4
>>>>>> --- interrupt: c0e9df00 at 0x400f330
>>>>>> LR = init_stack+0x1f00/0x2000
>>>>>> [c0e9de80] [c00ae3c4] printk+0xa8/0xcc (unreliable)
>>>>>> [c0e9df20] [c0c28e44] early_irq_init+0x38/0x108
>>>>>> [c0e9df50] [c0c16434] start_kernel+0x310/0x488
>>>>>> [c0e9dff0] [00003484] 0x3484
>>>>>>
>>>>>> The buggy address belongs to the variable:
>>>>>> __log_buf+0xec0/0x4020
>>>>>> The buggy address belongs to the page:
>>>>>> page:c6eac9a0 count:1 mapcount:0 mapping:00000000 index:0x0
>>>>>> flags: 0x1000(reserved)
>>>>>> raw: 00001000 c6eac9a4 c6eac9a4 00000000 00000000 00000000 ffffffff 00000001
>>>>>> page dumped because: kasan: bad access detected
>>>>>>
>>>>>> Memory state around the buggy address:
>>>>>> c0ecdc00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>>>> c0ecdc80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>>>>> c0ecdd00: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00
>>>>>> ^
>>>>>> c0ecdd80: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
>>>>>> c0ecde00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>>>> ==================================================================
>>>>>>
>>>>>
>>>>> This one doesn't look good. Notice that it says stack-out-of-bounds, but at the same time there is
>>>>> "The buggy address belongs to the variable: __log_buf+0xec0/0x4020"
>>>>> which is printed by following code:
>>>>> if (kernel_or_module_addr(addr) && !init_task_stack_addr(addr)) {
>>>>> pr_err("The buggy address belongs to the variable:\n");
>>>>> pr_err(" %pS\n", addr);
>>>>> }
>>>>>
>>>>> So the stack unrelated address got stack-related poisoning. This could be a stack overflow, did you increase THREAD_SHIFT?
>>>>> KASAN with stack instrumentation significantly increases stack usage.
>>>>>
>>>>
>>>> I get the above with THREAD_SHIFT set to 13 (default value).
>>>> If increasing it to 14, I get the following instead. That means that in that case the problem arises a lot earlier in the boot process (but still after the final kasan shadow setup).
>>>>
>>>
>>> We usually use 15 (with 4k pages), but I think 14 should be enough for the clean boot.
>>>
>>>> ==================================================================
>>>> BUG: KASAN: stack-out-of-bounds in pmac_nvram_init+0x1f8/0x5d0
>>>> Read of size 1 at addr f6f37de0 by task swapper/0
>>>>
>>>> CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc7+ #1143
>>>> Call Trace:
>>>> [c0e9fd60] [c01c43c0] print_address_description+0x164/0x2bc (unreliable)
>>>> [c0e9fd90] [c01c46a4] kasan_report+0xfc/0x180
>>>> [c0e9fdd0] [c0c226d4] pmac_nvram_init+0x1f8/0x5d0
>>>> [c0e9fef0] [c0c1f73c] pmac_setup_arch+0x298/0x314
>>>> [c0e9ff20] [c0c1ac40] setup_arch+0x250/0x268
>>>> [c0e9ff50] [c0c151dc] start_kernel+0xb8/0x488
>>>> [c0e9fff0] [00003484] 0x3484
>>>>
>>>>
>>>> Memory state around the buggy address:
>>>> f6f37c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>> f6f37d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>>> f6f37d80: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
>>>> ^
>>>> f6f37e00: 00 00 01 f4 f2 f2 f2 f2 00 00 00 00 f2 f2 f2 f2
>>>> f6f37e80: 00 00 00 00 f3 f3 f3 f3 00 00 00 00 00 00 00 00
>>>> ==================================================================
>>>
>>> Powerpc's show_stack() prints stack addresses, so we know that stack is something near 0xc0e9f... address.
>>> f6f37de0 is definitely not stack address and it's to far for the stack overflow.
>>> So it looks like shadow for stack - kasan_mem_to_shadow(0xc0e9f...) and shadow for address in report - kasan_mem_to_shadow(0xf6f37de0)
>>> point to the same physical page.
>>
>> Shouldn't shadow start at 0xf8 for powerpc32? I did some math
>> yesterday which I think lead me to 0xf8.
>
> Dunno, maybe. How is this relevant? In case you referring to the 0xf6f* addresses in the report,
> these are not shadow, but accessed addresses.
Thanks for your help. Indeed you made me realise here that the access is
to an IO Mapping, so being covered by the zero shadow page.
After some investigation I saw that the zero shadow page was being
poisonned allthough i confirmed it was mapped RO in every page table
entry referencing it.
What I finaly discovered is that in fact the HW still had some of the
early page table entries pointing to the zero page in RW.
The reason for the above is due to the PGD having multiple entries
pointing to kasan_early_shadow_pte[]. In powerpc hash32, a flag
_PAGE_HASHPTE is set to tell when a PTE has been given to HW. Then when
flush_tlb_kernel_range() is called, the kernel walks the page tables and
only really flushes the pages having the _PAGE_HASHPTE flag, then clear it.
The consequence is that when the kernel walk again that PTE from a
different PGD entry, it is seen as not needing flush anymore.
So, the conclusion to this that I'm finalising at the moment is to have
the final shadow page table layout set up as soon as memblock is
available and before switching from the early hash table to the final
hash table.
Christophe
>
>> This allows to cover at most 1GB of memory. Do you have more by any chance?
>>
^ permalink raw reply
* Re: [PATCH v8 1/4] mm/cma: Add PF flag to force non cma alloc
From: Michal Hocko @ 2019-02-28 14:34 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrea Arcangeli, Alexey Kardashevskiy, Aneesh Kumar K.V,
linux-kernel, Matthew Wilcox, linux-mm, Peter Zijlstra,
Ingo Molnar, akpm, linuxppc-dev, David Gibson
In-Reply-To: <1d083bf9-0beb-0c49-9aab-c6bc14da46ea@suse.cz>
On Thu 28-02-19 13:20:03, Vlastimil Babka wrote:
> On 2/27/19 3:47 PM, Aneesh Kumar K.V wrote:
> > This patch adds PF_MEMALLOC_NOCMA which make sure any allocation in that context
> > is marked non-movable and hence cannot be satisfied by CMA region.
> >
> > This is useful with get_user_pages_longterm where we want to take a page pin by
> > migrating pages from CMA region. Marking the section PF_MEMALLOC_NOCMA ensures
> > that we avoid unnecessary page migration later.
> >
> > Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
> > Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
> > Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>
> +CC scheduler guys
>
> Do we really take the last available PF flag just so that "we avoid
> unnecessary page migration later"?
> If yes, that's a third PF_MEMALLOC flag, should we get separate variable
> for gfp context at this point?
Yes, that sounds like a reasonable thing to do. Just note that xfs still
uses current_{set,restore}* api to handle PF_MEMALLOC_NOFS so that would
have to be moved over to the memalloc_nofs_{save,restore} API.
--
Michal Hocko
^ permalink raw reply
* Re: [PATCH] ASoC: cs42xx8: Remove S32_LE in format list
From: Timur Tabi @ 2019-02-28 14:33 UTC (permalink / raw)
To: S.j. Wang, nicoleotsuka@gmail.com, Xiubo.Lee@gmail.com,
festevam@gmail.com, broonie@kernel.org,
alsa-devel@alsa-project.org
Cc: linuxppc-dev@lists.ozlabs.org
In-Reply-To: <1551333389-22791-1-git-send-email-shengjiu.wang@nxp.com>
On 2/27/19 11:56 PM, S.j. Wang wrote:
> cs42xx8 is a 24-bit A/D and 24-bit D/A device, so the S32_LE
> should not be in the supported format list.
>
> Signed-off-by: Shengjiu Wang<shengjiu.wang@nxp.com>
Is the device capable of accepting 32-bit samples, even if it downgrades
it to 24-bit internally? If so, then maybe SNDRV_PCM_FMTBIT_S32_LE
should stay.
^ permalink raw reply
* [PATCH v4 1/9] powerpc/powernv/idle: Restore IAMR after idle
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
From: Russell Currey <ruscur@russell.cc>
Without restoring the IAMR after idle, execution prevention on POWER9
with Radix MMU is overwritten and the kernel can freely execute
userspace without faulting.
This is necessary when returning from any stop state that modifies
user state, as well as hypervisor state.
To test how this fails without this patch, load the lkdtm driver and
do the following:
$ echo EXEC_USERSPACE > /sys/kernel/debug/provoke-crash/DIRECT
which won't fault, then boot the kernel with powersave=off, where it
will fault. Applying this patch will fix this.
Fixes: 3b10d0095a1e ("powerpc/mm/radix: Prevent kernel execution of user space")
Cc: stable@vger.kernel.org # v4.10+
Signed-off-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Akshay Adiga <akshay.adiga@linux.vnet.ibm.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
v4: mpe: Use a #define for the stack slot.
arch/powerpc/kernel/idle_book3s.S | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index 7f5ac2e8581b..36178000a2f2 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -170,6 +170,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
bne- core_idle_lock_held
blr
+/* Reuse an unused pt_regs slot for IAMR */
+#define PNV_POWERSAVE_IAMR _DAR
+
/*
* Pass requested state in r3:
* r3 - PNV_THREAD_NAP/SLEEP/WINKLE in POWER8
@@ -200,6 +203,12 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
/* Continue saving state */
SAVE_GPR(2, r1)
SAVE_NVGPRS(r1)
+
+BEGIN_FTR_SECTION
+ mfspr r5, SPRN_IAMR
+ std r5, PNV_POWERSAVE_IAMR(r1)
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
+
mfcr r5
std r5,_CCR(r1)
std r1,PACAR1(r13)
@@ -924,6 +933,17 @@ BEGIN_FTR_SECTION
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
REST_NVGPRS(r1)
REST_GPR(2, r1)
+
+BEGIN_FTR_SECTION
+ /* IAMR was saved in pnv_powersave_common() */
+ ld r5, PNV_POWERSAVE_IAMR(r1)
+ mtspr SPRN_IAMR, r5
+ /*
+ * We don't need an isync here because the upcoming mtmsrd is
+ * execution synchronizing.
+ */
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
+
ld r4,PACAKMSR(r13)
ld r5,_LINK(r1)
ld r6,_CCR(r1)
--
2.20.1
^ permalink raw reply related
* [PATCH v4 2/9] powerpc/powernv/idle: Restore AMR/UAMOR/AMOR after idle
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
In order to implement KUAP (Kernel Userspace Access Protection) on
Power9 we will be using the AMR, and therefore indirectly the
UAMOR/AMOR.
So save/restore these regs in the idle code.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/idle_book3s.S | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
v4: New.
diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S
index 36178000a2f2..4a860d3b9229 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -170,8 +170,11 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
bne- core_idle_lock_held
blr
-/* Reuse an unused pt_regs slot for IAMR */
+/* Reuse some unused pt_regs slots for AMR/IAMR/UAMOR/UAMOR */
+#define PNV_POWERSAVE_AMR _TRAP
#define PNV_POWERSAVE_IAMR _DAR
+#define PNV_POWERSAVE_UAMOR _DSISR
+#define PNV_POWERSAVE_AMOR RESULT
/*
* Pass requested state in r3:
@@ -205,8 +208,16 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300)
SAVE_NVGPRS(r1)
BEGIN_FTR_SECTION
+ mfspr r4, SPRN_AMR
mfspr r5, SPRN_IAMR
+ mfspr r6, SPRN_UAMOR
+ std r4, PNV_POWERSAVE_AMR(r1)
std r5, PNV_POWERSAVE_IAMR(r1)
+ std r6, PNV_POWERSAVE_UAMOR(r1)
+BEGIN_FTR_SECTION_NESTED(42)
+ mfspr r7, SPRN_AMOR
+ std r7, PNV_POWERSAVE_AMOR(r1)
+END_FTR_SECTION_NESTED_IFSET(CPU_FTR_HVMODE, 42)
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
mfcr r5
@@ -935,12 +946,20 @@ END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
REST_GPR(2, r1)
BEGIN_FTR_SECTION
- /* IAMR was saved in pnv_powersave_common() */
+ /* These regs were saved in pnv_powersave_common() */
+ ld r4, PNV_POWERSAVE_AMR(r1)
ld r5, PNV_POWERSAVE_IAMR(r1)
+ ld r6, PNV_POWERSAVE_UAMOR(r1)
+ mtspr SPRN_AMR, r4
mtspr SPRN_IAMR, r5
+ mtspr SPRN_UAMOR, r6
+BEGIN_FTR_SECTION_NESTED(42)
+ ld r7, PNV_POWERSAVE_AMOR(r1)
+ mtspr SPRN_AMOR, r7
+END_FTR_SECTION_NESTED_IFSET(CPU_FTR_HVMODE, 42)
/*
- * We don't need an isync here because the upcoming mtmsrd is
- * execution synchronizing.
+ * We don't need an isync here after restoring IAMR because the upcoming
+ * mtmsrd is execution synchronizing.
*/
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
--
2.20.1
^ permalink raw reply related
* [PATCH v4 3/9] powerpc: Add framework for Kernel Userspace Protection
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Christophe Leroy <christophe.leroy@c-s.fr>
This patch adds a skeleton for Kernel Userspace Protection
functionnalities like Kernel Userspace Access Protection and Kernel
Userspace Execution Prevention
The subsequent implementation of KUAP for radix makes use of a MMU
feature in order to patch out assembly when KUAP is disabled or
unsupported. This won't work unless there's an entry point for KUP
support before the feature magic happens, so for PPC64 setup_kup() is
called early in setup.
On PPC32, feature_fixup() is done too early to allow the same.
Suggested-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/include/asm/kup.h | 11 +++++++++++
arch/powerpc/kernel/setup_64.c | 7 +++++++
arch/powerpc/mm/init-common.c | 5 +++++
arch/powerpc/mm/init_32.c | 3 +++
4 files changed, 26 insertions(+)
create mode 100644 arch/powerpc/include/asm/kup.h
v4: Unchanged.
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
new file mode 100644
index 000000000000..7a88b8b9b54d
--- /dev/null
+++ b/arch/powerpc/include/asm/kup.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_POWERPC_KUP_H_
+#define _ASM_POWERPC_KUP_H_
+
+#ifndef __ASSEMBLY__
+
+void setup_kup(void);
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _ASM_POWERPC_KUP_H_ */
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index daa361fc6a24..47ffa0885081 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -68,6 +68,7 @@
#include <asm/cputhreads.h>
#include <asm/hw_irq.h>
#include <asm/feature-fixups.h>
+#include <asm/kup.h>
#include "setup.h"
@@ -331,6 +332,12 @@ void __init early_setup(unsigned long dt_ptr)
*/
configure_exceptions();
+ /*
+ * Configure Kernel Userspace Protection. This needs to happen before
+ * feature fixups for platforms that implement this using features.
+ */
+ setup_kup();
+
/* Apply all the dynamic patching */
apply_feature_fixups();
setup_feature_keys();
diff --git a/arch/powerpc/mm/init-common.c b/arch/powerpc/mm/init-common.c
index 1e6910eb70ed..36d28e872289 100644
--- a/arch/powerpc/mm/init-common.c
+++ b/arch/powerpc/mm/init-common.c
@@ -24,6 +24,11 @@
#include <linux/string.h>
#include <asm/pgalloc.h>
#include <asm/pgtable.h>
+#include <asm/kup.h>
+
+void __init setup_kup(void)
+{
+}
#define CTOR(shift) static void ctor_##shift(void *addr) \
{ \
diff --git a/arch/powerpc/mm/init_32.c b/arch/powerpc/mm/init_32.c
index 41a3513cadc9..80cc97cd8878 100644
--- a/arch/powerpc/mm/init_32.c
+++ b/arch/powerpc/mm/init_32.c
@@ -45,6 +45,7 @@
#include <asm/tlb.h>
#include <asm/sections.h>
#include <asm/hugetlb.h>
+#include <asm/kup.h>
#include "mmu_decl.h"
@@ -178,6 +179,8 @@ void __init MMU_init(void)
btext_unmap();
#endif
+ setup_kup();
+
/* Shortly after that, the entire linear mapping will be available */
memblock_set_current_limit(lowmem_end_addr);
}
--
2.20.1
^ permalink raw reply related
* [PATCH v4 4/9] powerpc: Add skeleton for Kernel Userspace Execution Prevention
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Christophe Leroy <christophe.leroy@c-s.fr>
This patch adds a skeleton for Kernel Userspace Execution Prevention.
Then subarches implementing it have to define CONFIG_PPC_HAVE_KUEP
and provide setup_kuep() function.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
[mpe: Don't split strings, use pr_crit_ratelimited()]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
Documentation/admin-guide/kernel-parameters.txt | 2 +-
arch/powerpc/include/asm/kup.h | 6 ++++++
arch/powerpc/mm/fault.c | 9 ++++-----
arch/powerpc/mm/init-common.c | 11 +++++++++++
arch/powerpc/platforms/Kconfig.cputype | 12 ++++++++++++
5 files changed, 34 insertions(+), 6 deletions(-)
v4: mpe: Don't split strings, use pr_crit_ratelimited()
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b799bcf67d7b..f81d79de4de0 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2813,7 +2813,7 @@
Disable SMAP (Supervisor Mode Access Prevention)
even if it is supported by processor.
- nosmep [X86]
+ nosmep [X86,PPC]
Disable SMEP (Supervisor Mode Execution Prevention)
even if it is supported by processor.
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index 7a88b8b9b54d..a2a959cb4e36 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -6,6 +6,12 @@
void setup_kup(void);
+#ifdef CONFIG_PPC_KUEP
+void setup_kuep(bool disabled);
+#else
+static inline void setup_kuep(bool disabled) { }
+#endif /* CONFIG_PPC_KUEP */
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_KUP_H_ */
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 887f11bcf330..3384354abc1d 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -229,11 +229,10 @@ static bool bad_kernel_fault(bool is_exec, unsigned long error_code,
/* NX faults set DSISR_PROTFAULT on the 8xx, DSISR_NOEXEC_OR_G on others */
if (is_exec && (error_code & (DSISR_NOEXEC_OR_G | DSISR_KEYFAULT |
DSISR_PROTFAULT))) {
- printk_ratelimited(KERN_CRIT "kernel tried to execute"
- " exec-protected page (%lx) -"
- "exploit attempt? (uid: %d)\n",
- address, from_kuid(&init_user_ns,
- current_uid()));
+ pr_crit_ratelimited("kernel tried to execute %s page (%lx) - exploit attempt? (uid: %d)\n",
+ address >= TASK_SIZE ? "exec-protected" : "user",
+ address,
+ from_kuid(&init_user_ns, current_uid()));
}
return is_exec || (address >= TASK_SIZE);
}
diff --git a/arch/powerpc/mm/init-common.c b/arch/powerpc/mm/init-common.c
index 36d28e872289..83f95a5565d6 100644
--- a/arch/powerpc/mm/init-common.c
+++ b/arch/powerpc/mm/init-common.c
@@ -26,8 +26,19 @@
#include <asm/pgtable.h>
#include <asm/kup.h>
+static bool disable_kuep = !IS_ENABLED(CONFIG_PPC_KUEP);
+
+static int __init parse_nosmep(char *p)
+{
+ disable_kuep = true;
+ pr_warn("Disabling Kernel Userspace Execution Prevention\n");
+ return 0;
+}
+early_param("nosmep", parse_nosmep);
+
void __init setup_kup(void)
{
+ setup_kuep(disable_kuep);
}
#define CTOR(shift) static void ctor_##shift(void *addr) \
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 842b2c7e156a..7d30bbbaa3c1 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -345,6 +345,18 @@ config PPC_RADIX_MMU_DEFAULT
If you're unsure, say Y.
+config PPC_HAVE_KUEP
+ bool
+
+config PPC_KUEP
+ bool "Kernel Userspace Execution Prevention"
+ depends on PPC_HAVE_KUEP
+ default y
+ help
+ Enable support for Kernel Userspace Execution Prevention (KUEP)
+
+ If you're unsure, say Y.
+
config ARCH_ENABLE_HUGEPAGE_MIGRATION
def_bool y
depends on PPC_BOOK3S_64 && HUGETLB_PAGE && MIGRATION
--
2.20.1
^ permalink raw reply related
* [PATCH v4 5/9] powerpc: Add a framework for Kernel Userspace Access Protection
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Christophe Leroy <christophe.leroy@c-s.fr>
This patch implements a framework for Kernel Userspace Access
Protection.
Then subarches will have the possibility to provide their own
implementation by providing setup_kuap() and
allow/prevent_user_access().
Some platforms will need to know the area accessed and whether it is
accessed from read, write or both. Therefore source, destination and
size and handed over to the two functions.
mpe: Rename to allow/prevent rather than unlock/lock, and add
read/write wrappers. Drop the 32-bit code for now until we have an
implementation for it. Add kuap to pt_regs for 64-bit as well as
32-bit. Don't split strings, use pr_crit_ratelimited().
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
.../admin-guide/kernel-parameters.txt | 2 +-
arch/powerpc/include/asm/futex.h | 4 ++
arch/powerpc/include/asm/kup.h | 32 ++++++++++++++++
arch/powerpc/include/asm/ptrace.h | 11 +++++-
arch/powerpc/include/asm/uaccess.h | 38 +++++++++++++++----
arch/powerpc/kernel/asm-offsets.c | 4 ++
arch/powerpc/lib/checksum_wrappers.c | 4 ++
arch/powerpc/mm/fault.c | 19 ++++++++--
arch/powerpc/mm/init-common.c | 10 +++++
arch/powerpc/platforms/Kconfig.cputype | 12 ++++++
10 files changed, 121 insertions(+), 15 deletions(-)
v4: mpe: Rename to allow/prevent rather than unlock/lock, and add
read/write wrappers. Drop the 32-bit code for now until we have an
implementation for it. Add kuap to pt_regs for 64-bit as well as
32-bit. Don't split strings, use pr_crit_ratelimited().
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f81d79de4de0..16883f2a05fd 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2809,7 +2809,7 @@
noexec=on: enable non-executable mappings (default)
noexec=off: disable non-executable mappings
- nosmap [X86]
+ nosmap [X86,PPC]
Disable SMAP (Supervisor Mode Access Prevention)
even if it is supported by processor.
diff --git a/arch/powerpc/include/asm/futex.h b/arch/powerpc/include/asm/futex.h
index 88b38b37c21b..3a6aa57b9d90 100644
--- a/arch/powerpc/include/asm/futex.h
+++ b/arch/powerpc/include/asm/futex.h
@@ -35,6 +35,7 @@ static inline int arch_futex_atomic_op_inuser(int op, int oparg, int *oval,
{
int oldval = 0, ret;
+ allow_write_to_user(uaddr, sizeof(*uaddr));
pagefault_disable();
switch (op) {
@@ -62,6 +63,7 @@ static inline int arch_futex_atomic_op_inuser(int op, int oparg, int *oval,
if (!ret)
*oval = oldval;
+ prevent_write_to_user(uaddr, sizeof(*uaddr));
return ret;
}
@@ -75,6 +77,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
if (!access_ok(uaddr, sizeof(u32)))
return -EFAULT;
+ allow_write_to_user(uaddr, sizeof(*uaddr));
__asm__ __volatile__ (
PPC_ATOMIC_ENTRY_BARRIER
"1: lwarx %1,0,%3 # futex_atomic_cmpxchg_inatomic\n\
@@ -95,6 +98,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
: "cc", "memory");
*uval = prev;
+ prevent_write_to_user(uaddr, sizeof(*uaddr));
return ret;
}
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index a2a959cb4e36..4d78b9d8c99c 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -4,6 +4,8 @@
#ifndef __ASSEMBLY__
+#include <asm/pgtable.h>
+
void setup_kup(void);
#ifdef CONFIG_PPC_KUEP
@@ -12,6 +14,36 @@ void setup_kuep(bool disabled);
static inline void setup_kuep(bool disabled) { }
#endif /* CONFIG_PPC_KUEP */
+#ifdef CONFIG_PPC_KUAP
+void setup_kuap(bool disabled);
+#else
+static inline void setup_kuap(bool disabled) { }
+static inline void allow_user_access(void __user *to, const void __user *from,
+ unsigned long size) { }
+static inline void prevent_user_access(void __user *to, const void __user *from,
+ unsigned long size) { }
+#endif /* CONFIG_PPC_KUAP */
+
+static inline void allow_read_from_user(const void __user *from, unsigned long size)
+{
+ allow_user_access(NULL, from, size);
+}
+
+static inline void allow_write_to_user(void __user *to, unsigned long size)
+{
+ allow_user_access(to, NULL, size);
+}
+
+static inline void prevent_read_from_user(const void __user *from, unsigned long size)
+{
+ prevent_user_access(NULL, from, size);
+}
+
+static inline void prevent_write_to_user(void __user *to, unsigned long size)
+{
+ prevent_user_access(to, NULL, size);
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_KUP_H_ */
diff --git a/arch/powerpc/include/asm/ptrace.h b/arch/powerpc/include/asm/ptrace.h
index 64271e562fed..6f047730e642 100644
--- a/arch/powerpc/include/asm/ptrace.h
+++ b/arch/powerpc/include/asm/ptrace.h
@@ -52,10 +52,17 @@ struct pt_regs
};
};
+ union {
+ struct {
#ifdef CONFIG_PPC64
- unsigned long ppr;
- unsigned long __pad; /* Maintain 16 byte interrupt stack alignment */
+ unsigned long ppr;
+#endif
+#ifdef CONFIG_PPC_KUAP
+ unsigned long kuap;
#endif
+ };
+ unsigned long __pad[2]; /* Maintain 16 byte interrupt stack alignment */
+ };
};
#endif
diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h
index e3a731793ea2..fb7651a5488b 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -6,6 +6,7 @@
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/extable.h>
+#include <asm/kup.h>
/*
* The fs value determines whether argument validity checking should be
@@ -141,6 +142,7 @@ extern long __put_user_bad(void);
#define __put_user_size(x, ptr, size, retval) \
do { \
retval = 0; \
+ allow_write_to_user(ptr, size); \
switch (size) { \
case 1: __put_user_asm(x, ptr, retval, "stb"); break; \
case 2: __put_user_asm(x, ptr, retval, "sth"); break; \
@@ -148,6 +150,7 @@ do { \
case 8: __put_user_asm2(x, ptr, retval); break; \
default: __put_user_bad(); \
} \
+ prevent_write_to_user(ptr, size); \
} while (0)
#define __put_user_nocheck(x, ptr, size) \
@@ -240,6 +243,7 @@ do { \
__chk_user_ptr(ptr); \
if (size > sizeof(x)) \
(x) = __get_user_bad(); \
+ allow_read_from_user(ptr, size); \
switch (size) { \
case 1: __get_user_asm(x, ptr, retval, "lbz"); break; \
case 2: __get_user_asm(x, ptr, retval, "lhz"); break; \
@@ -247,6 +251,7 @@ do { \
case 8: __get_user_asm2(x, ptr, retval); break; \
default: (x) = __get_user_bad(); \
} \
+ prevent_read_from_user(ptr, size); \
} while (0)
/*
@@ -306,15 +311,21 @@ extern unsigned long __copy_tofrom_user(void __user *to,
static inline unsigned long
raw_copy_in_user(void __user *to, const void __user *from, unsigned long n)
{
- return __copy_tofrom_user(to, from, n);
+ unsigned long ret;
+
+ allow_user_access(to, from, n);
+ ret = __copy_tofrom_user(to, from, n);
+ prevent_user_access(to, from, n);
+ return ret;
}
#endif /* __powerpc64__ */
static inline unsigned long raw_copy_from_user(void *to,
const void __user *from, unsigned long n)
{
+ unsigned long ret;
if (__builtin_constant_p(n) && (n <= 8)) {
- unsigned long ret = 1;
+ ret = 1;
switch (n) {
case 1:
@@ -339,14 +350,18 @@ static inline unsigned long raw_copy_from_user(void *to,
}
barrier_nospec();
- return __copy_tofrom_user((__force void __user *)to, from, n);
+ allow_read_from_user(from, n);
+ ret = __copy_tofrom_user((__force void __user *)to, from, n);
+ prevent_read_from_user(from, n);
+ return ret;
}
static inline unsigned long raw_copy_to_user(void __user *to,
const void *from, unsigned long n)
{
+ unsigned long ret;
if (__builtin_constant_p(n) && (n <= 8)) {
- unsigned long ret = 1;
+ ret = 1;
switch (n) {
case 1:
@@ -366,17 +381,24 @@ static inline unsigned long raw_copy_to_user(void __user *to,
return 0;
}
- return __copy_tofrom_user(to, (__force const void __user *)from, n);
+ allow_write_to_user(to, n);
+ ret = __copy_tofrom_user(to, (__force const void __user *)from, n);
+ prevent_write_to_user(to, n);
+ return ret;
}
extern unsigned long __clear_user(void __user *addr, unsigned long size);
static inline unsigned long clear_user(void __user *addr, unsigned long size)
{
+ unsigned long ret = size;
might_fault();
- if (likely(access_ok(addr, size)))
- return __clear_user(addr, size);
- return size;
+ if (likely(access_ok(addr, size))) {
+ allow_write_to_user(addr, size);
+ ret = __clear_user(addr, size);
+ prevent_write_to_user(addr, size);
+ }
+ return ret;
}
extern long strncpy_from_user(char *dst, const char __user *src, long count);
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 86a61e5f8285..66202e02fee2 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -332,6 +332,10 @@ int main(void)
STACK_PT_REGS_OFFSET(_PPR, ppr);
#endif /* CONFIG_PPC64 */
+#ifdef CONFIG_PPC_KUAP
+ STACK_PT_REGS_OFFSET(STACK_REGS_KUAP, kuap);
+#endif
+
#if defined(CONFIG_PPC32)
#if defined(CONFIG_BOOKE) || defined(CONFIG_40x)
DEFINE(EXC_LVL_SIZE, STACK_EXC_LVL_FRAME_SIZE);
diff --git a/arch/powerpc/lib/checksum_wrappers.c b/arch/powerpc/lib/checksum_wrappers.c
index 890d4ddd91d6..bb9307ce2440 100644
--- a/arch/powerpc/lib/checksum_wrappers.c
+++ b/arch/powerpc/lib/checksum_wrappers.c
@@ -29,6 +29,7 @@ __wsum csum_and_copy_from_user(const void __user *src, void *dst,
unsigned int csum;
might_sleep();
+ allow_read_from_user(src, len);
*err_ptr = 0;
@@ -60,6 +61,7 @@ __wsum csum_and_copy_from_user(const void __user *src, void *dst,
}
out:
+ prevent_read_from_user(src, len);
return (__force __wsum)csum;
}
EXPORT_SYMBOL(csum_and_copy_from_user);
@@ -70,6 +72,7 @@ __wsum csum_and_copy_to_user(const void *src, void __user *dst, int len,
unsigned int csum;
might_sleep();
+ allow_write_to_user(dst, len);
*err_ptr = 0;
@@ -97,6 +100,7 @@ __wsum csum_and_copy_to_user(const void *src, void __user *dst, int len,
}
out:
+ prevent_write_to_user(dst, len);
return (__force __wsum)csum;
}
EXPORT_SYMBOL(csum_and_copy_to_user);
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 3384354abc1d..463d1e9d026e 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -223,9 +223,11 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr,
}
/* Is this a bad kernel fault ? */
-static bool bad_kernel_fault(bool is_exec, unsigned long error_code,
+static bool bad_kernel_fault(struct pt_regs *regs, unsigned long error_code,
unsigned long address)
{
+ int is_exec = TRAP(regs) == 0x400;
+
/* NX faults set DSISR_PROTFAULT on the 8xx, DSISR_NOEXEC_OR_G on others */
if (is_exec && (error_code & (DSISR_NOEXEC_OR_G | DSISR_KEYFAULT |
DSISR_PROTFAULT))) {
@@ -234,7 +236,15 @@ static bool bad_kernel_fault(bool is_exec, unsigned long error_code,
address,
from_kuid(&init_user_ns, current_uid()));
}
- return is_exec || (address >= TASK_SIZE);
+
+ if (!is_exec && address < TASK_SIZE && (error_code & DSISR_PROTFAULT) &&
+ !search_exception_tables(regs->nip)) {
+ pr_crit_ratelimited("Kernel attempted to access user page (%lx) - exploit attempt? (uid: %d)\n",
+ address,
+ from_kuid(&init_user_ns, current_uid()));
+ }
+
+ return is_exec || (address >= TASK_SIZE) || !search_exception_tables(regs->nip);
}
static bool bad_stack_expansion(struct pt_regs *regs, unsigned long address,
@@ -454,9 +464,10 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address,
/*
* The kernel should never take an execute fault nor should it
- * take a page fault to a kernel address.
+ * take a page fault to a kernel address or a page fault to a user
+ * address outside of dedicated places
*/
- if (unlikely(!is_user && bad_kernel_fault(is_exec, error_code, address)))
+ if (unlikely(!is_user && bad_kernel_fault(regs, error_code, address)))
return SIGSEGV;
/*
diff --git a/arch/powerpc/mm/init-common.c b/arch/powerpc/mm/init-common.c
index 83f95a5565d6..ecaedfff9992 100644
--- a/arch/powerpc/mm/init-common.c
+++ b/arch/powerpc/mm/init-common.c
@@ -27,6 +27,7 @@
#include <asm/kup.h>
static bool disable_kuep = !IS_ENABLED(CONFIG_PPC_KUEP);
+static bool disable_kuap = !IS_ENABLED(CONFIG_PPC_KUAP);
static int __init parse_nosmep(char *p)
{
@@ -36,9 +37,18 @@ static int __init parse_nosmep(char *p)
}
early_param("nosmep", parse_nosmep);
+static int __init parse_nosmap(char *p)
+{
+ disable_kuap = true;
+ pr_warn("Disabling Kernel Userspace Access Protection\n");
+ return 0;
+}
+early_param("nosmap", parse_nosmap);
+
void __init setup_kup(void)
{
setup_kuep(disable_kuep);
+ setup_kuap(disable_kuap);
}
#define CTOR(shift) static void ctor_##shift(void *addr) \
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 7d30bbbaa3c1..457fc3a5ed93 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -357,6 +357,18 @@ config PPC_KUEP
If you're unsure, say Y.
+config PPC_HAVE_KUAP
+ bool
+
+config PPC_KUAP
+ bool "Kernel Userspace Access Protection"
+ depends on PPC_HAVE_KUAP
+ default y
+ help
+ Enable support for Kernel Userspace Access Protection (KUAP)
+
+ If you're unsure, say Y.
+
config ARCH_ENABLE_HUGEPAGE_MIGRATION
def_bool y
depends on PPC_BOOK3S_64 && HUGETLB_PAGE && MIGRATION
--
2.20.1
^ permalink raw reply related
* [PATCH v4 6/9] powerpc/64: Setup KUP on secondary CPUs
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Russell Currey <ruscur@russell.cc>
Some platforms (i.e. Radix MMU) need per-CPU initialisation for KUP.
Any platforms that only want to do KUP initialisation once
globally can just check to see if they're running on the boot CPU, or
check if whatever setup they need has already been performed.
Note that this is only for 64-bit.
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/setup_64.c | 3 +++
1 file changed, 3 insertions(+)
v4: Unchanged.
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index 47ffa0885081..72358a5022fe 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -390,6 +390,9 @@ void early_setup_secondary(void)
/* Initialize the hash table or TLB handling */
early_init_mmu_secondary();
+ /* Perform any KUP setup that is per-cpu */
+ setup_kup();
+
/*
* At this point, we can let interrupts switch to virtual mode
* (the MMU has been setup), so adjust the MSR in the PACA to
--
2.20.1
^ permalink raw reply related
* [PATCH v4 7/9] powerpc/mm/radix: Use KUEP API for Radix MMU
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Russell Currey <ruscur@russell.cc>
Execution protection already exists on radix, this just refactors
the radix init to provide the KUEP setup function instead.
Thus, the only functional change is that it can now be disabled.
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/mm/pgtable-radix.c | 12 +++++++++---
arch/powerpc/platforms/Kconfig.cputype | 1 +
2 files changed, 10 insertions(+), 3 deletions(-)
v4: Unchanged.
diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c
index 931156069a81..224bcd4be5ae 100644
--- a/arch/powerpc/mm/pgtable-radix.c
+++ b/arch/powerpc/mm/pgtable-radix.c
@@ -535,8 +535,15 @@ static void radix_init_amor(void)
mtspr(SPRN_AMOR, (3ul << 62));
}
-static void radix_init_iamr(void)
+#ifdef CONFIG_PPC_KUEP
+void __init setup_kuep(bool disabled)
{
+ if (disabled || !early_radix_enabled())
+ return;
+
+ if (smp_processor_id() == boot_cpuid)
+ pr_info("Activating Kernel Userspace Execution Prevention\n");
+
/*
* Radix always uses key0 of the IAMR to determine if an access is
* allowed. We set bit 0 (IBM bit 1) of key0, to prevent instruction
@@ -544,6 +551,7 @@ static void radix_init_iamr(void)
*/
mtspr(SPRN_IAMR, (1ul << 62));
}
+#endif
void __init radix__early_init_mmu(void)
{
@@ -605,7 +613,6 @@ void __init radix__early_init_mmu(void)
memblock_set_current_limit(MEMBLOCK_ALLOC_ANYWHERE);
- radix_init_iamr();
radix_init_pgtable();
/* Switch to the guard PID before turning on MMU */
radix__switch_mmu_context(NULL, &init_mm);
@@ -627,7 +634,6 @@ void radix__early_init_mmu_secondary(void)
__pa(partition_tb) | (PATB_SIZE_SHIFT - 12));
radix_init_amor();
}
- radix_init_iamr();
radix__switch_mmu_context(NULL, &init_mm);
if (cpu_has_feature(CPU_FTR_HVMODE))
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 457fc3a5ed93..60371784c9f1 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -326,6 +326,7 @@ config PPC_RADIX_MMU
bool "Radix MMU Support"
depends on PPC_BOOK3S_64
select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
+ select PPC_HAVE_KUEP
default y
help
Enable support for the Power ISA 3.0 Radix style MMU. Currently this
--
2.20.1
^ permalink raw reply related
* [PATCH v4 8/9] powerpc/lib: Refactor __patch_instruction() to use __put_user_asm()
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Russell Currey <ruscur@russell.cc>
__patch_instruction() is called in early boot, and uses
__put_user_size(), which includes the allow/prevent calls to enforce
KUAP, which could either be called too early, or in the Radix case,
forced to use "early_" versions of functions just to safely handle
this one case.
__put_user_asm() does not do this, and thus is safe to use both in
early boot, and later on since in this case it should only ever be
touching kernel memory.
__patch_instruction() was previously refactored to use
__put_user_size() in order to be able to return -EFAULT, which would
allow the kernel to patch instructions in userspace, which should
never happen. This has the functional change of causing faults on
userspace addresses if KUAP is turned on, which should never happen in
practice.
A future enhancement could be to double check the patch address is
definitely allowed to be tampered with by the kernel.
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/lib/code-patching.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
v4: Unchanged.
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 506413a2c25e..42fdadac6587 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -26,9 +26,9 @@
static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
unsigned int *patch_addr)
{
- int err;
+ int err = 0;
- __put_user_size(instr, patch_addr, 4, err);
+ __put_user_asm(instr, patch_addr, err, "stw");
if (err)
return err;
--
2.20.1
^ permalink raw reply related
* [PATCH v4 9/9] powerpc/64s: Implement KUAP for Radix MMU
From: Michael Ellerman @ 2019-02-28 14:49 UTC (permalink / raw)
To: linuxppc-dev; +Cc: akshay.adiga, npiggin
In-Reply-To: <20190228144917.16876-1-mpe@ellerman.id.au>
From: Russell Currey <ruscur@russell.cc>
Kernel Userspace Access Prevention utilises a feature of the Radix MMU
which disallows read and write access to userspace addresses. By
utilising this, the kernel is prevented from accessing user data from
outside of trusted paths that perform proper safety checks, such as
copy_{to/from}_user() and friends.
Userspace access is disabled from early boot and is only enabled when
performing an operation like copy_{to/from}_user(). The register that
controls this (AMR) does not prevent userspace from accessing itself,
so there is no need to save and restore when entering and exiting
userspace.
This feature has a slight performance impact which I roughly measured
to be 3% slower in the worst case (performing 1GB of 1 byte
read()/write() syscalls), and is gated behind the CONFIG_PPC_KUAP
option for performance-critical builds.
This feature can be tested by using the lkdtm driver (CONFIG_LKDTM=y)
and performing the following:
# (echo ACCESS_USERSPACE) > [debugfs]/provoke-crash/DIRECT
If enabled, this should send SIGSEGV to the thread.
mpe: Drop the unused paca flags. Zero the UAMOR to be safe. Save the
AMR when we enter the kernel from the kernel and then lock it again.
Restore on the way back to the kernel. This means we handle nesting of
interrupts properly, ie. we are protected inside the page fault
handler caused by a user access. Add paranoid checking of AMR in
switch and syscall return. Add an isync() to prevent_user_access()
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
.../powerpc/include/asm/book3s/64/kup-radix.h | 79 +++++++++++++++++++
arch/powerpc/include/asm/exception-64s.h | 2 +
arch/powerpc/include/asm/feature-fixups.h | 3 +
arch/powerpc/include/asm/kup.h | 4 +
arch/powerpc/include/asm/mmu.h | 10 ++-
arch/powerpc/include/asm/reg.h | 1 +
arch/powerpc/kernel/entry_64.S | 22 +++++-
arch/powerpc/kernel/exceptions-64s.S | 3 +
arch/powerpc/mm/pgtable-radix.c | 19 +++++
arch/powerpc/mm/pkeys.c | 1 +
arch/powerpc/platforms/Kconfig.cputype | 8 ++
11 files changed, 149 insertions(+), 3 deletions(-)
create mode 100644 arch/powerpc/include/asm/book3s/64/kup-radix.h
v4: mpe: Drop the unused paca flags. Zero the UAMOR to be safe. Save the
AMR when we enter the kernel from the kernel and then lock it again.
Restore on the way back to the kernel. This means we handle nesting of
interrupts properly, ie. we are protected inside the page fault
handler caused by a user access. Add paranoid checking of AMR in
switch and syscall return. Add an isync() to prevent_user_access()
diff --git a/arch/powerpc/include/asm/book3s/64/kup-radix.h b/arch/powerpc/include/asm/book3s/64/kup-radix.h
new file mode 100644
index 000000000000..87f2af26764c
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/kup-radix.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H
+#define _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H
+
+#ifdef __ASSEMBLY__
+
+.macro kuap_restore_amr gpr
+#ifdef CONFIG_PPC_KUAP
+ BEGIN_MMU_FTR_SECTION_NESTED(67)
+ ld \gpr, STACK_REGS_KUAP(r1)
+ mtspr SPRN_AMR, \gpr
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+#endif
+.endm
+
+.macro kuap_check_amr gpr
+#ifdef CONFIG_PPC_KUAP_DEBUG
+ BEGIN_MMU_FTR_SECTION_NESTED(67)
+ mfspr \gpr, SPRN_AMR
+ sradi \gpr, \gpr, 48
+999: tdnei \gpr, RADIX_AMR_LOCKED@highest
+ EMIT_BUG_ENTRY 999b,__FILE__,__LINE__, \
+ (BUGFLAG_WARNING|BUGFLAG_ONCE)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+#endif
+.endm
+
+.macro kuap_save_amr_and_lock gpr, msr_pr_cr
+#ifdef CONFIG_PPC_KUAP
+ BEGIN_MMU_FTR_SECTION_NESTED(67)
+ .ifnb \msr_pr_cr
+ bne \msr_pr_cr, 99f
+ .endif
+ mfspr \gpr, SPRN_AMR
+ std \gpr, STACK_REGS_KUAP(r1)
+ li \gpr, (RADIX_AMR_LOCKED)@highest
+ sldi \gpr, \gpr, 48
+ mtspr SPRN_AMR,\gpr
+99:
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+#endif
+.endm
+
+#else /* !__ASSEMBLY__ */
+
+#ifdef CONFIG_PPC_KUAP
+
+#include <asm/reg.h>
+/*
+ * We do have the ability to individually lock/unlock reads and writes rather
+ * than both at once, however it's a significant performance hit due to needing
+ * to do a read-modify-write, which adds a mfspr, which is slow. As a result,
+ * locking/unlocking both at once is preferred.
+ */
+static inline void allow_user_access(void __user *to, const void __user *from,
+ unsigned long size)
+{
+ if (!mmu_has_feature(MMU_FTR_RADIX_KUAP))
+ return;
+
+ mtspr(SPRN_AMR, 0);
+ isync();
+}
+
+static inline void prevent_user_access(void __user *to, const void __user *from,
+ unsigned long size)
+{
+ if (!mmu_has_feature(MMU_FTR_RADIX_KUAP))
+ return;
+
+ mtspr(SPRN_AMR, RADIX_AMR_LOCKED);
+ isync();
+}
+
+#endif /* CONFIG_PPC_KUAP */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H */
diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h
index 937bb630093f..df7cbfc45952 100644
--- a/arch/powerpc/include/asm/exception-64s.h
+++ b/arch/powerpc/include/asm/exception-64s.h
@@ -497,6 +497,7 @@ END_FTR_SECTION_NESTED(ftr,ftr,943)
RESTORE_CTR(r1, area); \
b bad_stack; \
3: EXCEPTION_PROLOG_COMMON_1(); \
+ kuap_save_amr_and_lock r9, cr0; \
beq 4f; /* if from kernel mode */ \
ACCOUNT_CPU_USER_ENTRY(r13, r9, r10); \
SAVE_PPR(area, r9); \
@@ -691,6 +692,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_CTRL)
*/
#define EXCEPTION_COMMON_NORET_STACK(area, trap, label, hdlr, additions) \
EXCEPTION_PROLOG_COMMON_1(); \
+ kuap_save_amr_and_lock r9; \
EXCEPTION_PROLOG_COMMON_2(area); \
EXCEPTION_PROLOG_COMMON_3(trap); \
/* Volatile regs are potentially clobbered here */ \
diff --git a/arch/powerpc/include/asm/feature-fixups.h b/arch/powerpc/include/asm/feature-fixups.h
index 40a6c9261a6b..f6fc31f8baff 100644
--- a/arch/powerpc/include/asm/feature-fixups.h
+++ b/arch/powerpc/include/asm/feature-fixups.h
@@ -100,6 +100,9 @@ label##5: \
#define END_MMU_FTR_SECTION(msk, val) \
END_MMU_FTR_SECTION_NESTED(msk, val, 97)
+#define END_MMU_FTR_SECTION_NESTED_IFSET(msk, label) \
+ END_MMU_FTR_SECTION_NESTED((msk), (msk), label)
+
#define END_MMU_FTR_SECTION_IFSET(msk) END_MMU_FTR_SECTION((msk), (msk))
#define END_MMU_FTR_SECTION_IFCLR(msk) END_MMU_FTR_SECTION((msk), 0)
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index 4d78b9d8c99c..082f50ad41a7 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -2,6 +2,10 @@
#ifndef _ASM_POWERPC_KUP_H_
#define _ASM_POWERPC_KUP_H_
+#ifdef CONFIG_PPC_BOOK3S_64
+#include <asm/book3s/64/kup-radix.h>
+#endif
+
#ifndef __ASSEMBLY__
#include <asm/pgtable.h>
diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
index d34ad1657d7b..59acb4418164 100644
--- a/arch/powerpc/include/asm/mmu.h
+++ b/arch/powerpc/include/asm/mmu.h
@@ -107,6 +107,11 @@
*/
#define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000)
+/*
+ * Supports KUAP (key 0 controlling userspace addresses) on radix
+ */
+#define MMU_FTR_RADIX_KUAP ASM_CONST(0x80000000)
+
/* MMU feature bit sets for various CPUs */
#define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \
MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2
@@ -164,7 +169,10 @@ enum {
#endif
#ifdef CONFIG_PPC_RADIX_MMU
MMU_FTR_TYPE_RADIX |
-#endif
+#ifdef CONFIG_PPC_KUAP
+ MMU_FTR_RADIX_KUAP |
+#endif /* CONFIG_PPC_KUAP */
+#endif /* CONFIG_PPC_RADIX_MMU */
0,
};
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index c5b2aff0ce8e..6350873af4bc 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -246,6 +246,7 @@
#define SPRN_DSCR 0x11
#define SPRN_CFAR 0x1c /* Come From Address Register */
#define SPRN_AMR 0x1d /* Authority Mask Register */
+#define RADIX_AMR_LOCKED 0xC000000000000000UL /* Read & Write disabled */
#define SPRN_UAMOR 0x9d /* User Authority Mask Override Register */
#define SPRN_AMOR 0x15d /* Authority Mask Override Register */
#define SPRN_ACOP 0x1F /* Available Coprocessor Register */
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 15c67d2c0534..612cfa4bffd7 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -46,6 +46,7 @@
#include <asm/exception-64e.h>
#endif
#include <asm/feature-fixups.h>
+#include <asm/kup.h>
/*
* System calls.
@@ -275,6 +276,8 @@ END_FTR_SECTION_IFCLR(CPU_FTR_STCX_CHECKS_ADDRESS)
andi. r6,r8,MSR_PR
ld r4,_LINK(r1)
+ kuap_check_amr r11
+
#ifdef CONFIG_PPC_BOOK3S
/*
* Clear MSR_RI, MSR_EE is already and remains disabled. We could do
@@ -296,6 +299,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
std r8, PACATMSCRATCH(r13)
#endif
+ /*
+ * We don't need to restore AMR on the way back to userspace for KUAP.
+ * The value of AMR only matters while we're in the kernel.
+ */
ld r13,GPR13(r1) /* only restore r13 if returning to usermode */
ld r2,GPR2(r1)
ld r1,GPR1(r1)
@@ -306,8 +313,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
RFI_TO_USER
b . /* prevent speculative execution */
- /* exit to kernel */
-1: ld r2,GPR2(r1)
+1: /* exit to kernel */
+ kuap_restore_amr r2
+
+ ld r2,GPR2(r1)
ld r1,GPR1(r1)
mtlr r4
mtcr r5
@@ -594,6 +603,8 @@ _GLOBAL(_switch)
std r23,_CCR(r1)
std r1,KSP(r3) /* Set old stack pointer */
+ kuap_check_amr r6
+
FLUSH_COUNT_CACHE
/*
@@ -974,6 +985,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
ACCOUNT_CPU_USER_EXIT(r13, r2, r4)
REST_GPR(13, r1)
+ /*
+ * We don't need to restore AMR on the way back to userspace for KUAP.
+ * The value of AMR only matters while we're in the kernel.
+ */
mtspr SPRN_SRR1,r3
ld r2,_CCR(r1)
@@ -1006,6 +1021,9 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
ld r0,GPR0(r1)
ld r2,GPR2(r1)
ld r3,GPR3(r1)
+
+ kuap_restore_amr r4
+
ld r4,GPR4(r1)
ld r1,GPR1(r1)
RFI_TO_KERNEL
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index b179b8b5d3f0..e25cea13b928 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -19,6 +19,7 @@
#include <asm/cpuidle.h>
#include <asm/head-64.h>
#include <asm/feature-fixups.h>
+#include <asm/kup.h>
/*
* There are a few constraints to be concerned with.
@@ -301,6 +302,7 @@ TRAMP_REAL_BEGIN(machine_check_common_early)
mfspr r11,SPRN_DSISR /* Save DSISR */
std r11,_DSISR(r1)
std r9,_CCR(r1) /* Save CR in stackframe */
+ kuap_save_amr_and_lock r9
/* Save r9 through r13 from EXMC save area to stack frame. */
EXCEPTION_PROLOG_COMMON_2(PACA_EXMC)
mfmsr r11 /* get MSR value */
@@ -1037,6 +1039,7 @@ TRAMP_REAL_BEGIN(hmi_exception_early)
mfspr r11,SPRN_HSRR0 /* Save HSRR0 */
mfspr r12,SPRN_HSRR1 /* Save HSRR1 */
EXCEPTION_PROLOG_COMMON_1()
+ kuap_save_amr_and_lock r9
EXCEPTION_PROLOG_COMMON_2(PACA_EXGEN)
EXCEPTION_PROLOG_COMMON_3(0xe60)
addi r3,r1,STACK_FRAME_OVERHEAD
diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c
index 224bcd4be5ae..aa7d9de3ba3f 100644
--- a/arch/powerpc/mm/pgtable-radix.c
+++ b/arch/powerpc/mm/pgtable-radix.c
@@ -29,6 +29,7 @@
#include <asm/powernv.h>
#include <asm/sections.h>
#include <asm/trace.h>
+#include <asm/uaccess.h>
#include <trace/events/thp.h>
@@ -553,6 +554,24 @@ void __init setup_kuep(bool disabled)
}
#endif
+#ifdef CONFIG_PPC_KUAP
+void __init setup_kuap(bool disabled)
+{
+ if (disabled || !early_radix_enabled())
+ return;
+
+ if (smp_processor_id() == boot_cpuid) {
+ pr_info("Activating Kernel Userspace Access Prevention\n");
+ cur_cpu_spec->mmu_features |= MMU_FTR_RADIX_KUAP;
+ }
+
+ /* Make sure userspace can't change the AMR */
+ mtspr(SPRN_UAMOR, 0);
+
+ mtspr(SPRN_AMR, RADIX_AMR_LOCKED);
+}
+#endif
+
void __init radix__early_init_mmu(void)
{
unsigned long lpcr;
diff --git a/arch/powerpc/mm/pkeys.c b/arch/powerpc/mm/pkeys.c
index 587807763737..ae7fca40e5b3 100644
--- a/arch/powerpc/mm/pkeys.c
+++ b/arch/powerpc/mm/pkeys.c
@@ -7,6 +7,7 @@
#include <asm/mman.h>
#include <asm/mmu_context.h>
+#include <asm/mmu.h>
#include <asm/setup.h>
#include <linux/pkeys.h>
#include <linux/of_device.h>
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 60371784c9f1..5e53b9fd62aa 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -327,6 +327,7 @@ config PPC_RADIX_MMU
depends on PPC_BOOK3S_64
select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
select PPC_HAVE_KUEP
+ select PPC_HAVE_KUAP
default y
help
Enable support for the Power ISA 3.0 Radix style MMU. Currently this
@@ -370,6 +371,13 @@ config PPC_KUAP
If you're unsure, say Y.
+config PPC_KUAP_DEBUG
+ bool "Extra debugging for Kernel Userspace Access Protection"
+ depends on PPC_HAVE_KUAP && PPC_RADIX_MMU
+ help
+ Add extra debugging for Kernel Userspace Access Protection (KUAP)
+ If you're unsure, say N.
+
config ARCH_ENABLE_HUGEPAGE_MIGRATION
def_bool y
depends on PPC_BOOK3S_64 && HUGETLB_PAGE && MIGRATION
--
2.20.1
^ permalink raw reply related
* [PATCH AUTOSEL 4.20 12/81] soc: fsl: qbman: avoid race in clearing QMan interrupt
From: Sasha Levin @ 2019-02-28 15:07 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Sasha Levin, Madalin Bucur, Roy Pledge, Li Yang, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <20190228150813.10256-1-sashal@kernel.org>
From: Madalin Bucur <madalin.bucur@nxp.com>
[ Upstream commit 89857a8a5c89a406b967ab2be7bd2ccdbe75e73d ]
By clearing all interrupt sources, not only those that
already occurred, the existing code may acknowledge by
mistake interrupts that occurred after the code checks
for them.
Signed-off-by: Madalin Bucur <madalin.bucur@nxp.com>
Signed-off-by: Roy Pledge <roy.pledge@nxp.com>
Signed-off-by: Li Yang <leoyang.li@nxp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/soc/fsl/qbman/qman.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/soc/fsl/qbman/qman.c b/drivers/soc/fsl/qbman/qman.c
index 5ce24718c2fd1..d8b3ba047c28f 100644
--- a/drivers/soc/fsl/qbman/qman.c
+++ b/drivers/soc/fsl/qbman/qman.c
@@ -1124,18 +1124,19 @@ static void qm_mr_process_task(struct work_struct *work);
static irqreturn_t portal_isr(int irq, void *ptr)
{
struct qman_portal *p = ptr;
-
- u32 clear = QM_DQAVAIL_MASK | p->irq_sources;
u32 is = qm_in(&p->p, QM_REG_ISR) & p->irq_sources;
+ u32 clear = 0;
if (unlikely(!is))
return IRQ_NONE;
/* DQRR-handling if it's interrupt-driven */
- if (is & QM_PIRQ_DQRI)
+ if (is & QM_PIRQ_DQRI) {
__poll_portal_fast(p, QMAN_POLL_LIMIT);
+ clear = QM_DQAVAIL_MASK | QM_PIRQ_DQRI;
+ }
/* Handling of anything else that's interrupt-driven */
- clear |= __poll_portal_slow(p, is);
+ clear |= __poll_portal_slow(p, is) & QM_PIRQ_SLOW;
qm_out(&p->p, QM_REG_ISR, clear);
return IRQ_HANDLED;
}
--
2.19.1
^ permalink raw reply related
* [PATCH AUTOSEL 4.19 10/64] soc: fsl: qbman: avoid race in clearing QMan interrupt
From: Sasha Levin @ 2019-02-28 15:10 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Sasha Levin, Madalin Bucur, Roy Pledge, Li Yang, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <20190228151105.11277-1-sashal@kernel.org>
From: Madalin Bucur <madalin.bucur@nxp.com>
[ Upstream commit 89857a8a5c89a406b967ab2be7bd2ccdbe75e73d ]
By clearing all interrupt sources, not only those that
already occurred, the existing code may acknowledge by
mistake interrupts that occurred after the code checks
for them.
Signed-off-by: Madalin Bucur <madalin.bucur@nxp.com>
Signed-off-by: Roy Pledge <roy.pledge@nxp.com>
Signed-off-by: Li Yang <leoyang.li@nxp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/soc/fsl/qbman/qman.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/soc/fsl/qbman/qman.c b/drivers/soc/fsl/qbman/qman.c
index 8cc0151830433..a4ac6073c555d 100644
--- a/drivers/soc/fsl/qbman/qman.c
+++ b/drivers/soc/fsl/qbman/qman.c
@@ -1081,18 +1081,19 @@ static void qm_mr_process_task(struct work_struct *work);
static irqreturn_t portal_isr(int irq, void *ptr)
{
struct qman_portal *p = ptr;
-
- u32 clear = QM_DQAVAIL_MASK | p->irq_sources;
u32 is = qm_in(&p->p, QM_REG_ISR) & p->irq_sources;
+ u32 clear = 0;
if (unlikely(!is))
return IRQ_NONE;
/* DQRR-handling if it's interrupt-driven */
- if (is & QM_PIRQ_DQRI)
+ if (is & QM_PIRQ_DQRI) {
__poll_portal_fast(p, QMAN_POLL_LIMIT);
+ clear = QM_DQAVAIL_MASK | QM_PIRQ_DQRI;
+ }
/* Handling of anything else that's interrupt-driven */
- clear |= __poll_portal_slow(p, is);
+ clear |= __poll_portal_slow(p, is) & QM_PIRQ_SLOW;
qm_out(&p->p, QM_REG_ISR, clear);
return IRQ_HANDLED;
}
--
2.19.1
^ permalink raw reply related
* [PATCH AUTOSEL 4.14 03/36] soc: fsl: qbman: avoid race in clearing QMan interrupt
From: Sasha Levin @ 2019-02-28 15:13 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Sasha Levin, Madalin Bucur, Roy Pledge, Li Yang, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <20190228151337.12176-1-sashal@kernel.org>
From: Madalin Bucur <madalin.bucur@nxp.com>
[ Upstream commit 89857a8a5c89a406b967ab2be7bd2ccdbe75e73d ]
By clearing all interrupt sources, not only those that
already occurred, the existing code may acknowledge by
mistake interrupts that occurred after the code checks
for them.
Signed-off-by: Madalin Bucur <madalin.bucur@nxp.com>
Signed-off-by: Roy Pledge <roy.pledge@nxp.com>
Signed-off-by: Li Yang <leoyang.li@nxp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/soc/fsl/qbman/qman.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/soc/fsl/qbman/qman.c b/drivers/soc/fsl/qbman/qman.c
index 4f27e95efcdd3..90892a360c61c 100644
--- a/drivers/soc/fsl/qbman/qman.c
+++ b/drivers/soc/fsl/qbman/qman.c
@@ -1048,18 +1048,19 @@ static void qm_mr_process_task(struct work_struct *work);
static irqreturn_t portal_isr(int irq, void *ptr)
{
struct qman_portal *p = ptr;
-
- u32 clear = QM_DQAVAIL_MASK | p->irq_sources;
u32 is = qm_in(&p->p, QM_REG_ISR) & p->irq_sources;
+ u32 clear = 0;
if (unlikely(!is))
return IRQ_NONE;
/* DQRR-handling if it's interrupt-driven */
- if (is & QM_PIRQ_DQRI)
+ if (is & QM_PIRQ_DQRI) {
__poll_portal_fast(p, QMAN_POLL_LIMIT);
+ clear = QM_DQAVAIL_MASK | QM_PIRQ_DQRI;
+ }
/* Handling of anything else that's interrupt-driven */
- clear |= __poll_portal_slow(p, is);
+ clear |= __poll_portal_slow(p, is) & QM_PIRQ_SLOW;
qm_out(&p->p, QM_REG_ISR, clear);
return IRQ_HANDLED;
}
--
2.19.1
^ permalink raw reply related
* [PATCH AUTOSEL 4.9 01/19] soc: fsl: qbman: avoid race in clearing QMan interrupt
From: Sasha Levin @ 2019-02-28 15:14 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Sasha Levin, Madalin Bucur, Roy Pledge, Li Yang, linuxppc-dev,
linux-arm-kernel
From: Madalin Bucur <madalin.bucur@nxp.com>
[ Upstream commit 89857a8a5c89a406b967ab2be7bd2ccdbe75e73d ]
By clearing all interrupt sources, not only those that
already occurred, the existing code may acknowledge by
mistake interrupts that occurred after the code checks
for them.
Signed-off-by: Madalin Bucur <madalin.bucur@nxp.com>
Signed-off-by: Roy Pledge <roy.pledge@nxp.com>
Signed-off-by: Li Yang <leoyang.li@nxp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/soc/fsl/qbman/qman.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/drivers/soc/fsl/qbman/qman.c b/drivers/soc/fsl/qbman/qman.c
index 2cc82ed6433a0..91f5c951850f7 100644
--- a/drivers/soc/fsl/qbman/qman.c
+++ b/drivers/soc/fsl/qbman/qman.c
@@ -1073,18 +1073,19 @@ static void qm_mr_process_task(struct work_struct *work);
static irqreturn_t portal_isr(int irq, void *ptr)
{
struct qman_portal *p = ptr;
-
- u32 clear = QM_DQAVAIL_MASK | p->irq_sources;
u32 is = qm_in(&p->p, QM_REG_ISR) & p->irq_sources;
+ u32 clear = 0;
if (unlikely(!is))
return IRQ_NONE;
/* DQRR-handling if it's interrupt-driven */
- if (is & QM_PIRQ_DQRI)
+ if (is & QM_PIRQ_DQRI) {
__poll_portal_fast(p, QMAN_POLL_LIMIT);
+ clear = QM_DQAVAIL_MASK | QM_PIRQ_DQRI;
+ }
/* Handling of anything else that's interrupt-driven */
- clear |= __poll_portal_slow(p, is);
+ clear |= __poll_portal_slow(p, is) & QM_PIRQ_SLOW;
qm_out(&p->p, QM_REG_ISR, clear);
return IRQ_HANDLED;
}
--
2.19.1
^ permalink raw reply related
* Re: [PATCH 2/2] mm/dax: Don't enable huge dax mapping by default
From: Dan Williams @ 2019-02-28 16:45 UTC (permalink / raw)
To: Oliver
Cc: Jan Kara, Aneesh Kumar K.V, Linux Kernel Mailing List, Linux MM,
Ross Zwisler, Andrew Morton, linuxppc-dev, Kirill A . Shutemov
In-Reply-To: <CAOSf1CHjkyX2NTex7dc1AEHXSDcWA_UGYX8NoSyHpb5s_RkwXQ@mail.gmail.com>
On Thu, Feb 28, 2019 at 1:40 AM Oliver <oohall@gmail.com> wrote:
>
> On Thu, Feb 28, 2019 at 7:35 PM Aneesh Kumar K.V
> <aneesh.kumar@linux.ibm.com> wrote:
> >
> > Add a flag to indicate the ability to do huge page dax mapping. On architecture
> > like ppc64, the hypervisor can disable huge page support in the guest. In
> > such a case, we should not enable huge page dax mapping. This patch adds
> > a flag which the architecture code will update to indicate huge page
> > dax mapping support.
>
> *groan*
>
> > Architectures mostly do transparent_hugepage_flag = 0; if they can't
> > do hugepages. That also takes care of disabling dax hugepage mapping
> > with this change.
> >
> > Without this patch we get the below error with kvm on ppc64.
> >
> > [ 118.849975] lpar: Failed hash pte insert with error -4
> >
> > NOTE: The patch also use
> >
> > echo never > /sys/kernel/mm/transparent_hugepage/enabled
> > to disable dax huge page mapping.
> >
> > Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> > ---
> > TODO:
> > * Add Fixes: tag
> >
> > include/linux/huge_mm.h | 4 +++-
> > mm/huge_memory.c | 4 ++++
> > 2 files changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index 381e872bfde0..01ad5258545e 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -53,6 +53,7 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr,
> > pud_t *pud, pfn_t pfn, bool write);
> > enum transparent_hugepage_flag {
> > TRANSPARENT_HUGEPAGE_FLAG,
> > + TRANSPARENT_HUGEPAGE_DAX_FLAG,
> > TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG,
> > TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG,
> > TRANSPARENT_HUGEPAGE_DEFRAG_KSWAPD_FLAG,
> > @@ -111,7 +112,8 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma)
> > if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG))
> > return true;
> >
> > - if (vma_is_dax(vma))
> > + if (vma_is_dax(vma) &&
> > + (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_DAX_FLAG)))
> > return true;
>
> Forcing PTE sized faults should be fine for fsdax, but it'll break
> devdax. The devdax driver requires the fault size be >= the namespace
> alignment since devdax tries to guarantee hugepage mappings will be
> used and PMD alignment is the default. We can probably have devdax
> fall back to the largest size the hypervisor has made available, but
> it does run contrary to the design. Ah well, I suppose it's better off
> being degraded rather than unusable.
Given this is an explicit setting I think device-dax should explicitly
fail to enable in the presence of this flag to preserve the
application visible behavior.
I.e. if device-dax was enabled after this setting was made then I
think future faults should fail as well.
^ permalink raw reply
* [PATCH 2/6] ptrace: introduce ptrace_syscall_enter to consolidate PTRACE_SYSEMU handling
From: Sudeep Holla @ 2019-02-28 18:32 UTC (permalink / raw)
To: x86, linux-arm-kernel, linux-kernel, linuxppc-dev
Cc: Haibo Xu, Steve Capper, jdike, Sudeep Holla, Will Deacon,
Oleg Nesterov, Bin Lu, Richard Weinberger, Ingo Molnar,
Paul Mackerras, Catalin Marinas, Thomas Gleixner
In-Reply-To: <20190228183220.15626-1-sudeep.holla@arm.com>
Currently each architecture handles PTRACE_SYSEMU in very similar way.
It's completely arch independent and can be handled in the code helping
to consolidate PTRACE_SYSEMU handling.
Let's introduce a hook 'ptrace_syscall_enter' that arch specific syscall
entry code can call.
Cc: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
---
include/linux/ptrace.h | 1 +
kernel/ptrace.c | 16 ++++++++++++++++
2 files changed, 17 insertions(+)
diff --git a/include/linux/ptrace.h b/include/linux/ptrace.h
index edb9b040c94c..e30f51e3363e 100644
--- a/include/linux/ptrace.h
+++ b/include/linux/ptrace.h
@@ -407,6 +407,7 @@ static inline void user_single_step_report(struct pt_regs *regs)
#define current_user_stack_pointer() user_stack_pointer(current_pt_regs())
#endif
+extern long ptrace_syscall_enter(struct pt_regs *regs);
extern int task_current_syscall(struct task_struct *target, long *callno,
unsigned long args[6], unsigned int maxargs,
unsigned long *sp, unsigned long *pc);
diff --git a/kernel/ptrace.c b/kernel/ptrace.c
index 4fa3b7f4c3c7..6724eaf98e79 100644
--- a/kernel/ptrace.c
+++ b/kernel/ptrace.c
@@ -29,6 +29,7 @@
#include <linux/hw_breakpoint.h>
#include <linux/cn_proc.h>
#include <linux/compat.h>
+#include <linux/tracehook.h>
/*
* Access another process' address space via ptrace.
@@ -557,6 +558,21 @@ static int ptrace_detach(struct task_struct *child, unsigned int data)
return 0;
}
+/*
+ * Hook to check and report for PTRACE_SYSEMU, can be called from arch
+ * arch syscall entry code
+ */
+long ptrace_syscall_enter(struct pt_regs *regs)
+{
+#ifdef TIF_SYSCALL_EMU
+ if (test_thread_flag(TIF_SYSCALL_EMU)) {
+ if (tracehook_report_syscall_entry(regs));
+ return -1L;
+ }
+#endif
+ return 0;
+}
+
/*
* Detach all tasks we were using ptrace on. Called with tasklist held
* for writing.
--
2.17.1
^ permalink raw reply related
* [PATCH 3/6] x86: clean up _TIF_SYSCALL_EMU handling using ptrace_syscall_enter hook
From: Sudeep Holla @ 2019-02-28 18:32 UTC (permalink / raw)
To: x86, linux-arm-kernel, linux-kernel, linuxppc-dev
Cc: Haibo Xu, Steve Capper, jdike, Sudeep Holla, Will Deacon,
Oleg Nesterov, Bin Lu, Richard Weinberger, Ingo Molnar,
Paul Mackerras, Andy Lutomirski, Catalin Marinas, Borislav Petkov,
Thomas Gleixner
In-Reply-To: <20190228183220.15626-1-sudeep.holla@arm.com>
Now that we have a new hook ptrace_syscall_enter that can be called from
syscall entry code and it handles PTRACE_SYSEMU in generic code, we
can do some cleanup using the same in syscall_trace_enter.
Further the extra logic to find single stepping PTRACE_SYSEMU_SINGLESTEP
in syscall_slow_exit_work seems unnecessary. Let's remove the same.
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
---
arch/x86/entry/common.c | 22 ++++------------------
1 file changed, 4 insertions(+), 18 deletions(-)
diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 7bc105f47d21..36457c1f87d2 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -70,22 +70,16 @@ static long syscall_trace_enter(struct pt_regs *regs)
struct thread_info *ti = current_thread_info();
unsigned long ret = 0;
- bool emulated = false;
u32 work;
if (IS_ENABLED(CONFIG_DEBUG_ENTRY))
BUG_ON(regs != task_pt_regs(current));
- work = READ_ONCE(ti->flags) & _TIF_WORK_SYSCALL_ENTRY;
-
- if (unlikely(work & _TIF_SYSCALL_EMU))
- emulated = true;
-
- if ((emulated || (work & _TIF_SYSCALL_TRACE)) &&
- tracehook_report_syscall_entry(regs))
+ if (unlikely(ptrace_syscall_enter(regs)))
return -1L;
- if (emulated)
+ work = READ_ONCE(ti->flags) & _TIF_WORK_SYSCALL_ENTRY;
+ if ((work & _TIF_SYSCALL_TRACE) && tracehook_report_syscall_entry(regs))
return -1L;
#ifdef CONFIG_SECCOMP
@@ -227,15 +221,7 @@ static void syscall_slow_exit_work(struct pt_regs *regs, u32 cached_flags)
if (cached_flags & _TIF_SYSCALL_TRACEPOINT)
trace_sys_exit(regs, regs->ax);
- /*
- * If TIF_SYSCALL_EMU is set, we only get here because of
- * TIF_SINGLESTEP (i.e. this is PTRACE_SYSEMU_SINGLESTEP).
- * We already reported this syscall instruction in
- * syscall_trace_enter().
- */
- step = unlikely(
- (cached_flags & (_TIF_SINGLESTEP | _TIF_SYSCALL_EMU))
- == _TIF_SINGLESTEP);
+ step = unlikely((cached_flags & _TIF_SINGLESTEP));
if (step || cached_flags & _TIF_SYSCALL_TRACE)
tracehook_report_syscall_exit(regs, step);
}
--
2.17.1
^ permalink raw reply related
* [PATCH 0/6] ptrace: consolidate PTRACE_SYSEMU handling and add support for arm64
From: Sudeep Holla @ 2019-02-28 18:32 UTC (permalink / raw)
To: x86, linux-arm-kernel, linux-kernel, linuxppc-dev
Cc: Haibo Xu, Steve Capper, jdike, Sudeep Holla, Will Deacon,
Oleg Nesterov, Bin Lu, Richard Weinberger, Ingo Molnar,
Paul Mackerras, Catalin Marinas, Thomas Gleixner
Hi,
This patchset evolved from the discussion in the thread[0][1]. When we
wanted to add PTRACE_SYSEMU support to ARM64, we thought instead of
duplicating what other architectures like x86 and powerpc have done,
let consolidate the existing support and move it to the core as there's
nothing arch specific in it.
So this is the first attempt to the same.
Regards,
Sudeep
[0] https://patchwork.kernel.org/patch/10585505/
[1] https://patchwork.kernel.org/patch/10675237/
Sudeep Holla (6):
ptrace: move clearing of TIF_SYSCALL_EMU flag to core
ptrace: introduce ptrace_syscall_enter to consolidate PTRACE_SYSEMU handling
x86: clean up _TIF_SYSCALL_EMU handling using ptrace_syscall_enter hook
powerpc: use common ptrace_syscall_enter hook to handle _TIF_SYSCALL_EMU
arm64: add PTRACE_SYSEMU{,SINGLESTEP} definations to uapi headers
arm64: ptrace: add support for syscall emulation
arch/arm64/include/asm/thread_info.h | 5 ++-
arch/arm64/include/uapi/asm/ptrace.h | 3 ++
arch/arm64/kernel/ptrace.c | 3 ++
arch/powerpc/kernel/ptrace.c | 51 ++++++++++++----------------
arch/x86/entry/common.c | 22 +++---------
arch/x86/kernel/ptrace.c | 3 --
include/linux/ptrace.h | 1 +
kernel/ptrace.c | 20 +++++++++++
8 files changed, 57 insertions(+), 51 deletions(-)
--
2.17.1
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox