[Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

When SEV is active, memory is encrypted with guest-specific key, and if
guest OS wants to share the memory region with hypervisor then it must
clear the C-bit (i.e set unencrypted) before sharing it.

DEFINE_PER_CPU_UNENCRYPTED can be used to define the per-cpu variables
which will be shared between guest and hypervisor. Currently, KVM defines
three variables (steal-time, apf_reason, and avic_eio) which are shared
with hypervisor.

Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: linux-arch@vger.kernel.org
Cc: x86@kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 include/asm-generic/vmlinux.lds.h | 11 +++++++++++
 include/linux/percpu-defs.h       | 15 +++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 8acfc1e099e1..363858f43cbc 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -777,6 +777,16 @@
 #define INIT_RAM_FS
 #endif
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+#define PERCPU_UNENCRYPTED_SECTION					\
+	. = ALIGN(PAGE_SIZE);						\
+	*(.data..percpu..unencrypted)					\
+	. = ALIGN(PAGE_SIZE);
+#else
+#define PERCPU_UNENCRYPTED_SECTION
+#endif
+
+
 /*
  * Default discarded sections.
  *
@@ -815,6 +825,7 @@
 	. = ALIGN(cacheline);						\
 	*(.data..percpu)						\
 	*(.data..percpu..shared_aligned)				\
+	PERCPU_UNENCRYPTED_SECTION					\
 	VMLINUX_SYMBOL(__per_cpu_end) = .;
 
 /**
diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h
index 8f16299ca068..b2b99ad4b31d 100644
--- a/include/linux/percpu-defs.h
+++ b/include/linux/percpu-defs.h
@@ -173,6 +173,21 @@
 	DEFINE_PER_CPU_SECTION(type, name, "..read_mostly")
 
 /*
+ * Declaration/definition used for per-CPU variables that should be accessed
+ * as unencrypted when memory encryption is enabled in the guest.
+ */
+#if defined(CONFIG_VIRTUALIZATION) && defined(CONFIG_AMD_MEM_ENCRYPT)
+
+#define DECLARE_PER_CPU_UNENCRYPTED(type, name)				\
+	DECLARE_PER_CPU_SECTION(type, name, "..unencrypted")
+
+#define DEFINE_PER_CPU_UNENCRYPTED(type, name)				\
+	DEFINE_PER_CPU_SECTION(type, name, "..unencrypted")
+#else
+#define DEFINE_PER_CPU_UNENCRYPTED(type, name)	DEFINE_PER_CPU(type, name)
+#endif
+
+/*
  * Intermodule exports for per-CPU variables.  sparse forgets about
  * address space across EXPORT_SYMBOL(), change EXPORT_SYMBOL() to
  * noop if __CHECKER__.
-- 
2.9.5

[Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

When SEV is active, memory is encrypted with guest-specific key, and if
guest OS wants to share the memory region with hypervisor then it must
clear the C-bit (i.e set unencrypted) before sharing it.

DEFINE_PER_CPU_UNENCRYPTED can be used to define the per-cpu variables
which will be shared between guest and hypervisor. Currently, KVM defines
three variables (steal-time, apf_reason, and avic_eio) which are shared
with hypervisor.

Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: linux-arch@vger.kernel.org
Cc: x86@kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 include/asm-generic/vmlinux.lds.h | 11 +++++++++++
 include/linux/percpu-defs.h       | 15 +++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 8acfc1e099e1..363858f43cbc 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -777,6 +777,16 @@
 #define INIT_RAM_FS
 #endif
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+#define PERCPU_UNENCRYPTED_SECTION					\
+	. = ALIGN(PAGE_SIZE);						\
+	*(.data..percpu..unencrypted)					\
+	. = ALIGN(PAGE_SIZE);
+#else
+#define PERCPU_UNENCRYPTED_SECTION
+#endif
+
+
 /*
  * Default discarded sections.
  *
@@ -815,6 +825,7 @@
 	. = ALIGN(cacheline);						\
 	*(.data..percpu)						\
 	*(.data..percpu..shared_aligned)				\
+	PERCPU_UNENCRYPTED_SECTION					\
 	VMLINUX_SYMBOL(__per_cpu_end) = .;
 
 /**
diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h
index 8f16299ca068..b2b99ad4b31d 100644
--- a/include/linux/percpu-defs.h
+++ b/include/linux/percpu-defs.h
@@ -173,6 +173,21 @@
 	DEFINE_PER_CPU_SECTION(type, name, "..read_mostly")
 
 /*
+ * Declaration/definition used for per-CPU variables that should be accessed
+ * as unencrypted when memory encryption is enabled in the guest.
+ */
+#if defined(CONFIG_VIRTUALIZATION) && defined(CONFIG_AMD_MEM_ENCRYPT)
+
+#define DECLARE_PER_CPU_UNENCRYPTED(type, name)				\
+	DECLARE_PER_CPU_SECTION(type, name, "..unencrypted")
+
+#define DEFINE_PER_CPU_UNENCRYPTED(type, name)				\
+	DEFINE_PER_CPU_SECTION(type, name, "..unencrypted")
+#else
+#define DEFINE_PER_CPU_UNENCRYPTED(type, name)	DEFINE_PER_CPU(type, name)
+#endif
+
+/*
  * Intermodule exports for per-CPU variables.  sparse forgets about
  * address space across EXPORT_SYMBOL(), change EXPORT_SYMBOL() to
  * noop if __CHECKER__.
-- 
2.9.5

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Tejun Heo]

On Sat, Sep 16, 2017 at 07:34:16AM -0500, Brijesh Singh wrote:
> When SEV is active, memory is encrypted with guest-specific key, and if
> guest OS wants to share the memory region with hypervisor then it must
> clear the C-bit (i.e set unencrypted) before sharing it.
> 
> DEFINE_PER_CPU_UNENCRYPTED can be used to define the per-cpu variables
> which will be shared between guest and hypervisor. Currently, KVM defines
> three variables (steal-time, apf_reason, and avic_eio) which are shared
> with hypervisor.
> 
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Christoph Lameter <cl@linux.com>
> Cc: linux-arch@vger.kernel.org
> Cc: x86@kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>

Acked-by: Tejun Heo <tj@kernel.org>

Thanks.

-- 
tejun

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Borislav Petkov]

On Sat, Sep 16, 2017 at 07:34:16AM -0500, Brijesh Singh wrote:
> When SEV is active, memory is encrypted with guest-specific key, and if

					  with a

> guest OS wants to share the memory region with hypervisor then it must

the guest OS ...			    with the hypervisor ...

> clear the C-bit (i.e set unencrypted) before sharing it.
> 
> DEFINE_PER_CPU_UNENCRYPTED can be used to define the per-cpu variables
> which will be shared between guest and hypervisor. Currently, KVM defines

		       between a guest and a hypervisor.

> three variables (steal-time, apf_reason, and avic_eio) which are shared
> with hypervisor.

This sentence should be the opening sentence of your commit message.

> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Christoph Lameter <cl@linux.com>
> Cc: linux-arch@vger.kernel.org
> Cc: x86@kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> ---
>  include/asm-generic/vmlinux.lds.h | 11 +++++++++++
>  include/linux/percpu-defs.h       | 15 +++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 8acfc1e099e1..363858f43cbc 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -777,6 +777,16 @@
>  #define INIT_RAM_FS
>  #endif
>  
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +#define PERCPU_UNENCRYPTED_SECTION					\
> +	. = ALIGN(PAGE_SIZE);						\
> +	*(.data..percpu..unencrypted)					\
> +	. = ALIGN(PAGE_SIZE);
> +#else
> +#define PERCPU_UNENCRYPTED_SECTION
> +#endif
> +
> +
>  /*
>   * Default discarded sections.
>   *
> @@ -815,6 +825,7 @@
>  	. = ALIGN(cacheline);						\
>  	*(.data..percpu)						\
>  	*(.data..percpu..shared_aligned)				\
> +	PERCPU_UNENCRYPTED_SECTION					\
>  	VMLINUX_SYMBOL(__per_cpu_end) = .;

So looking at this more: I'm wondering if we can simply reuse the
PER_CPU_SHARED_ALIGNED_SECTION definition which is for shared per-CPU
sections. Instead of introducing a special section which is going to be
used only by SEV, practically.

Because "shared" also kinda implies that it is shared by multiple agents
and those agents can just as well be guest and hypervisor. And then that
patch is gone too.

Hmmm...?

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Borislav Petkov]

On Sat, Sep 16, 2017 at 07:34:16AM -0500, Brijesh Singh wrote:
> When SEV is active, memory is encrypted with guest-specific key, and if

					  with a

> guest OS wants to share the memory region with hypervisor then it must

the guest OS ...			    with the hypervisor ...

> clear the C-bit (i.e set unencrypted) before sharing it.
> 
> DEFINE_PER_CPU_UNENCRYPTED can be used to define the per-cpu variables
> which will be shared between guest and hypervisor. Currently, KVM defines

		       between a guest and a hypervisor.

> three variables (steal-time, apf_reason, and avic_eio) which are shared
> with hypervisor.

This sentence should be the opening sentence of your commit message.

> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Christoph Lameter <cl@linux.com>
> Cc: linux-arch@vger.kernel.org
> Cc: x86@kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> ---
>  include/asm-generic/vmlinux.lds.h | 11 +++++++++++
>  include/linux/percpu-defs.h       | 15 +++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 8acfc1e099e1..363858f43cbc 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -777,6 +777,16 @@
>  #define INIT_RAM_FS
>  #endif
>  
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
> +#define PERCPU_UNENCRYPTED_SECTION					\
> +	. = ALIGN(PAGE_SIZE);						\
> +	*(.data..percpu..unencrypted)					\
> +	. = ALIGN(PAGE_SIZE);
> +#else
> +#define PERCPU_UNENCRYPTED_SECTION
> +#endif
> +
> +
>  /*
>   * Default discarded sections.
>   *
> @@ -815,6 +825,7 @@
>  	. = ALIGN(cacheline);						\
>  	*(.data..percpu)						\
>  	*(.data..percpu..shared_aligned)				\
> +	PERCPU_UNENCRYPTED_SECTION					\
>  	VMLINUX_SYMBOL(__per_cpu_end) = .;

So looking at this more: I'm wondering if we can simply reuse the
PER_CPU_SHARED_ALIGNED_SECTION definition which is for shared per-CPU
sections. Instead of introducing a special section which is going to be
used only by SEV, practically.

Because "shared" also kinda implies that it is shared by multiple agents
and those agents can just as well be guest and hypervisor. And then that
patch is gone too.

Hmmm...?

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

Hi Boris,

On 09/19/2017 05:39 AM, Borislav Petkov wrote:
...

>> @@ -815,6 +825,7 @@
>>   	. = ALIGN(cacheline);						\
>>   	*(.data..percpu)						\
>>   	*(.data..percpu..shared_aligned)				\
>> +	PERCPU_UNENCRYPTED_SECTION					\
>>   	VMLINUX_SYMBOL(__per_cpu_end) = .;
> 
> So looking at this more: I'm wondering if we can simply reuse the
> PER_CPU_SHARED_ALIGNED_SECTION definition which is for shared per-CPU
> sections. Instead of introducing a special section which is going to be
> used only by SEV, practically.
> 
> Because "shared" also kinda implies that it is shared by multiple agents
> and those agents can just as well be guest and hypervisor. And then that
> patch is gone too.
> 
> Hmmm...?
> 

"..shared_aligned" section does not start and end with page-size alignment.
Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section
starts and ends with page-size alignment. The closest I can find is
"..page_aligned" but again it does not end with page-size alignment.

Additionally, since we clear the C-bit from unencrypted section hence we
should avoid overloading the existing section -- we don't want to expose more
than we wish.

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

Hi Boris,

On 09/19/2017 05:39 AM, Borislav Petkov wrote:
...

>> @@ -815,6 +825,7 @@
>>   	. = ALIGN(cacheline);						\
>>   	*(.data..percpu)						\
>>   	*(.data..percpu..shared_aligned)				\
>> +	PERCPU_UNENCRYPTED_SECTION					\
>>   	VMLINUX_SYMBOL(__per_cpu_end) = .;
> 
> So looking at this more: I'm wondering if we can simply reuse the
> PER_CPU_SHARED_ALIGNED_SECTION definition which is for shared per-CPU
> sections. Instead of introducing a special section which is going to be
> used only by SEV, practically.
> 
> Because "shared" also kinda implies that it is shared by multiple agents
> and those agents can just as well be guest and hypervisor. And then that
> patch is gone too.
> 
> Hmmm...?
> 

"..shared_aligned" section does not start and end with page-size alignment.
Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section
starts and ends with page-size alignment. The closest I can find is
"..page_aligned" but again it does not end with page-size alignment.

Additionally, since we clear the C-bit from unencrypted section hence we
should avoid overloading the existing section -- we don't want to expose more
than we wish.

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Borislav Petkov]

On Tue, Sep 19, 2017 at 08:50:20AM -0500, Brijesh Singh wrote:
> "..shared_aligned" section does not start and end with page-size alignment.

Nowhere in the code there's a comment saying: "This percpu section really must
be page-size aligned because <reasons>." You need to be more verbose
with requirements like that.

Also, you're ending up needing a whole page per-CPU for those variables.
And now with the alignment before and after, you have worst-case two
pages fragmentation of percpu memory and percpu memory is a rather
limited resource AFAIR.

If only there were a alloc_percpu_page()...

> Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section

Btw, call that section "..decrypted" and everywhere do
s/unencrypted/decrypted/g.

> starts and ends with page-size alignment. The closest I can find is
> "..page_aligned" but again it does not end with page-size alignment.
> 
> Additionally, since we clear the C-bit from unencrypted section hence we
> should avoid overloading the existing section -- we don't want to expose more
> than we wish.

Add that to the comment too.

Thx.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Borislav Petkov]

On Tue, Sep 19, 2017 at 08:50:20AM -0500, Brijesh Singh wrote:
> "..shared_aligned" section does not start and end with page-size alignment.

Nowhere in the code there's a comment saying: "This percpu section really must
be page-size aligned because <reasons>." You need to be more verbose
with requirements like that.

Also, you're ending up needing a whole page per-CPU for those variables.
And now with the alignment before and after, you have worst-case two
pages fragmentation of percpu memory and percpu memory is a rather
limited resource AFAIR.

If only there were a alloc_percpu_page()...

> Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section

Btw, call that section "..decrypted" and everywhere do
s/unencrypted/decrypted/g.

> starts and ends with page-size alignment. The closest I can find is
> "..page_aligned" but again it does not end with page-size alignment.
> 
> Additionally, since we clear the C-bit from unencrypted section hence we
> should avoid overloading the existing section -- we don't want to expose more
> than we wish.

Add that to the comment too.

Thx.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

On 09/20/2017 02:34 AM, Borislav Petkov wrote:
> On Tue, Sep 19, 2017 at 08:50:20AM -0500, Brijesh Singh wrote:
>> "..shared_aligned" section does not start and end with page-size alignment.
> 
> Nowhere in the code there's a comment saying: "This percpu section really must
> be page-size aligned because <reasons>." You need to be more verbose
> with requirements like that.
> 

I will add that comment.


> Also, you're ending up needing a whole page per-CPU for those variables.
> And now with the alignment before and after, you have worst-case two
> pages fragmentation of percpu memory and percpu memory is a rather
> limited resource AFAIR.
> 
> If only there were a alloc_percpu_page()...
> 
>> Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section
> 
> Btw, call that section "..decrypted" and everywhere do
> s/unencrypted/decrypted/g.
> 

Will do


>> starts and ends with page-size alignment. The closest I can find is
>> "..page_aligned" but again it does not end with page-size alignment.
>>
>> Additionally, since we clear the C-bit from unencrypted section hence we
>> should avoid overloading the existing section -- we don't want to expose more
>> than we wish.
> 
> Add that to the comment too.
> 

Will do

Re: [Part1 PATCH v4 15/17] percpu: introduce DEFINE_PER_CPU_UNENCRYPTED

[Brijesh Singh]

On 09/20/2017 02:34 AM, Borislav Petkov wrote:
> On Tue, Sep 19, 2017 at 08:50:20AM -0500, Brijesh Singh wrote:
>> "..shared_aligned" section does not start and end with page-size alignment.
> 
> Nowhere in the code there's a comment saying: "This percpu section really must
> be page-size aligned because <reasons>." You need to be more verbose
> with requirements like that.
> 

I will add that comment.


> Also, you're ending up needing a whole page per-CPU for those variables.
> And now with the alignment before and after, you have worst-case two
> pages fragmentation of percpu memory and percpu memory is a rather
> limited resource AFAIR.
> 
> If only there were a alloc_percpu_page()...
> 
>> Since the C-bit works on PAGE_SIZE alignment hence the "..unencrypted" section
> 
> Btw, call that section "..decrypted" and everywhere do
> s/unencrypted/decrypted/g.
> 

Will do


>> starts and ends with page-size alignment. The closest I can find is
>> "..page_aligned" but again it does not end with page-size alignment.
>>
>> Additionally, since we clear the C-bit from unencrypted section hence we
>> should avoid overloading the existing section -- we don't want to expose more
>> than we wish.
> 
> Add that to the comment too.
> 

Will do