[PATCH 0/2] x86/mm: More KASAN per-CPU CEA mapping bug fixes

[PATCH 0/2] x86/mm: More KASAN per-CPU CEA mapping bug fixes

[Sean Christopherson]

Two more bug fixes from the gift that keeps on giving.  Both of these were
found by inspection, and both are lightly tested.  I'm pretty confident
that patch 01 fixes a real bug, somewhat less so about patch 02, though
given the other issues I debugged I don't see anything that prevents the
DS buffers from exploding.

Sean Christopherson (2):
  x86/mm: Recompute physical address for every page of per-CPU CEA
    mapping
  x86/mm: Populate KASAN shadow for per-CPU DS buffers in CPU entry area

 arch/x86/mm/cpu_entry_area.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)


base-commit: 3301badde43dee7c2a013fbd6479c258366519da
-- 
2.38.1.431.g37b22c650d-goog

[PATCH 1/2] x86/mm: Recompute physical address for every page of per-CPU CEA mapping

[Sean Christopherson]

Recompute the physical address for each per-CPU page in the CPU entry
area, a recent commit inadvertantly modified cea_map_percpu_pages() such
that every PTE is mapped to the physical address of the first page.

Fixes: 9fd429c28073 ("x86/kasan: Map shadow for percpu pages on demand")
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/mm/cpu_entry_area.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c
index dff9001e5e12..d831aae94b41 100644
--- a/arch/x86/mm/cpu_entry_area.c
+++ b/arch/x86/mm/cpu_entry_area.c
@@ -97,7 +97,7 @@ cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot)
 					early_pfn_to_nid(PFN_DOWN(pa)));
 
 	for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE)
-		cea_set_pte(cea_vaddr, pa, prot);
+		cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot);
 }
 
 static void __init percpu_setup_debug_store(unsigned int cpu)
-- 
2.38.1.431.g37b22c650d-goog

[PATCH 2/2] x86/mm: Populate KASAN shadow for per-CPU DS buffers in CPU entry area

[Sean Christopherson]

Bounce through cea_map_percpu_pages() when setting the initial
protections for per-CPU DS buffers so that KASAN populates a shadow for
said mapping.  Failure to populate the shadow will result in a
not-present #PF during KASAN validation if DS buffers are activated
later on.

Fixes: 9fd429c28073 ("x86/kasan: Map shadow for percpu pages on demand")
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/mm/cpu_entry_area.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c
index d831aae94b41..64ae557ceb22 100644
--- a/arch/x86/mm/cpu_entry_area.c
+++ b/arch/x86/mm/cpu_entry_area.c
@@ -91,13 +91,12 @@ void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags)
 static void __init
 cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot)
 {
-	phys_addr_t pa = per_cpu_ptr_to_phys(ptr);
+	int nid = ptr ? early_pfn_to_nid(PFN_DOWN(per_cpu_ptr_to_phys(ptr))) : 0;
 
-	kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE,
-					early_pfn_to_nid(PFN_DOWN(pa)));
+	kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE, nid);
 
 	for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE)
-		cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot);
+		cea_set_pte(cea_vaddr, ptr ? per_cpu_ptr_to_phys(ptr) : 0, prot);
 }
 
 static void __init percpu_setup_debug_store(unsigned int cpu)
@@ -121,8 +120,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu)
 	 * memory like debug store buffers.
 	 */
 	npages = sizeof(struct debug_store_buffers) / PAGE_SIZE;
-	for (; npages; npages--, cea += PAGE_SIZE)
-		cea_set_pte(cea, 0, PAGE_NONE);
+	cea_map_percpu_pages(cea, NULL, npages, PAGE_NONE);
 #endif
 }
 
-- 
2.38.1.431.g37b22c650d-goog

Re: [PATCH 2/2] x86/mm: Populate KASAN shadow for per-CPU DS buffers in CPU entry area

[Sean Christopherson]

On Fri, Nov 04, 2022, Sean Christopherson wrote:
> Bounce through cea_map_percpu_pages() when setting the initial
> protections for per-CPU DS buffers so that KASAN populates a shadow for
> said mapping.  Failure to populate the shadow will result in a
> not-present #PF during KASAN validation if DS buffers are activated
> later on.
> 
> Fixes: 9fd429c28073 ("x86/kasan: Map shadow for percpu pages on demand")
> Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  arch/x86/mm/cpu_entry_area.c | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c
> index d831aae94b41..64ae557ceb22 100644
> --- a/arch/x86/mm/cpu_entry_area.c
> +++ b/arch/x86/mm/cpu_entry_area.c
> @@ -91,13 +91,12 @@ void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags)
>  static void __init
>  cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot)
>  {
> -	phys_addr_t pa = per_cpu_ptr_to_phys(ptr);
> +	int nid = ptr ? early_pfn_to_nid(PFN_DOWN(per_cpu_ptr_to_phys(ptr))) : 0;
>  
> -	kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE,
> -					early_pfn_to_nid(PFN_DOWN(pa)));
> +	kasan_populate_shadow_for_vaddr(cea_vaddr, pages * PAGE_SIZE, nid);
>  
>  	for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE)
> -		cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot);
> +		cea_set_pte(cea_vaddr, ptr ? per_cpu_ptr_to_phys(ptr) : 0, prot);

Gah, this is broken.  If pages > 1, subsequent iterations will generate a non-NULL
ptr.  This is likely what the kernel test bot is complaining about[1].  Andrey's
suggestion to map the entire per-CPU area in one go[2] should obviate the need for
special casing the DS buffer.  I'll give that a whirl and smush all three of these
mini-series together.

[1] https://lore.kernel.org/all/202211092215.948a1cf3-oliver.sang@intel.com
[2] https://lore.kernel.org/all/b5e31093-ac80-595b-1127-2a3e35913d86@gmail.com