[PATCH V2 0/1] x86/sgx: Add code to inject hwpoison into SGX memory

[PATCH V2 0/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

This patch adds code to inject hwpoison into SGX memory. The SGX memory
is processor reserved memory that cannot be directly accessed by system
software.

Changes since v1:
- Add a comment in hwpoison_inject as suggested by Miaohe
- v1: Reviewed-by: Tony Luck <tony.luck@intel.com>
- v1: Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

Thomas Tai (1):
  x86/sgx: Add code to inject hwpoison into SGX memory

 Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
 mm/hwpoison-inject.c          |  4 ++++
 2 files changed, 48 insertions(+)

-- 
2.31.1

[PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
address validation), add a similar code in hwpoison_inject function to
check if the address is located in SGX Memory. The error will then be
handled by the arch_memory_failure function in the SGX driver.

Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
---
 Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
 mm/hwpoison-inject.c          |  4 ++++
 2 files changed, 48 insertions(+)

diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
index b9d5253c1305..8a542aca4744 100644
--- a/Documentation/mm/hwpoison.rst
+++ b/Documentation/mm/hwpoison.rst
@@ -162,6 +162,50 @@ Testing
 
   Some portable hwpoison test programs in mce-test, see below.
 
+* Special notes for injection into SGX enclaves
+
+  1) Determine physical address of enclave page
+
+	dmesg | grep "sgx: EPC"
+
+	sgx: EPC section 0x8000c00000-0x807f7fffff
+	sgx: EPC section 0x10000c00000-0x1007fffffff
+
+  2) Convert the EPC address to page frame number.
+
+	For 4K page size, the page frame number for 0x8000c00000 is
+	0x8000c00000 / 0x1000 = 0x8000c00.
+
+  3) Trace memory_failure
+
+	echo nop > /sys/kernel/tracing/current_tracer
+	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
+	echo function > /sys/kernel/tracing/current_tracer
+
+  4) Inject a memory error
+
+	modprobe hwpoison-inject
+	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
+
+  5) Check the trace output
+
+	cat /sys/kernel/tracing/trace
+
+	# tracer: function
+	#
+	# entries-in-buffer/entries-written: 2/2   #P:128
+	#
+	#                            _-----=> irqs-off
+	#                           / _----=> need-resched
+	#                          | / _---=> hardirq/softirq
+	#                          || / _--=> preempt-depth
+	#                          ||| / _-=> migrate-disable
+	#                          |||| /     delay
+	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
+	#          | |         |   |||||     |         |
+	        bash-12167   [002] .....   113.136808: memory_failure<-simple_attr_write
+	        bash-12167   [002] .....   113.136810: arch_memory_failure<-memory_failure
+
 References
 ==========
 
diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
index 65e242b5a432..bf83111c1d9b 100644
--- a/mm/hwpoison-inject.c
+++ b/mm/hwpoison-inject.c
@@ -21,6 +21,10 @@ static int hwpoison_inject(void *data, u64 val)
 	if (!capable(CAP_SYS_ADMIN))
 		return -EPERM;
 
+	/* Inject the error if the page is part of the processor reserved memory */
+	if (arch_is_platform_page(pfn << PAGE_SHIFT))
+		goto inject;
+
 	if (!pfn_valid(pfn))
 		return -ENXIO;
 
-- 
2.31.1

Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Jarkko Sakkinen]

On Wed, Sep 28, 2022 at 11:38:32AM -0400, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> address validation), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> ---
>  Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
>  mm/hwpoison-inject.c          |  4 ++++
>  2 files changed, 48 insertions(+)

Hmm.. no change log.

What was the change in-between v1 and v2?

BR, Jarkko

RE: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

> -----Original Message-----
> From: Jarkko Sakkinen <jarkko@kernel.org>
> Sent: September 30, 2022 5:51 PM
> To: Thomas Tai <thomas.tai@oracle.com>
> Cc: tony.luck@intel.com; dave.hansen@linux.intel.com;
> reinette.chatre@intel.co; naoya.horiguchi@nec.com; linmiaohe@huawei.com;
> akpm@linux-foundation.org; linux-mm@kvack.org; linux-
> kernel@vger.kernel.org
> Subject: Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX
> memory
> 
> On Wed, Sep 28, 2022 at 11:38:32AM -0400, Thomas Tai wrote:
> > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > address validation), add a similar code in hwpoison_inject function to
> > check if the address is located in SGX Memory. The error will then be
> > handled by the arch_memory_failure function in the SGX driver.
> >
> > Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> > ---
> >  Documentation/mm/hwpoison.rst | 44
> +++++++++++++++++++++++++++++++++++
> >  mm/hwpoison-inject.c          |  4 ++++
> >  2 files changed, 48 insertions(+)
> 
> Hmm.. no change log.
> 
> What was the change in-between v1 and v2?

Hi Jarkko,
I put the change log in [PATCH V2 0/1]. Following is the details, hope you find it useful.

Changes since v1:
- Add a comment in hwpoison_inject as suggested by Miaohe
- v1: Reviewed-by: Tony Luck <tony.luck@intel.com>
- v1: Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

Thanks,
Thomas
> 
> BR, Jarkko

Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Jarkko Sakkinen]

On Sun, Oct 02, 2022 at 02:51:22PM +0000, Thomas Tai wrote:
> 
> > -----Original Message-----
> > From: Jarkko Sakkinen <jarkko@kernel.org>
> > Sent: September 30, 2022 5:51 PM
> > To: Thomas Tai <thomas.tai@oracle.com>
> > Cc: tony.luck@intel.com; dave.hansen@linux.intel.com;
> > reinette.chatre@intel.co; naoya.horiguchi@nec.com; linmiaohe@huawei.com;
> > akpm@linux-foundation.org; linux-mm@kvack.org; linux-
> > kernel@vger.kernel.org
> > Subject: Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX
> > memory
> > 
> > On Wed, Sep 28, 2022 at 11:38:32AM -0400, Thomas Tai wrote:
> > > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > > address validation), add a similar code in hwpoison_inject function to
> > > check if the address is located in SGX Memory. The error will then be
> > > handled by the arch_memory_failure function in the SGX driver.
> > >
> > > Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> > > ---
> > >  Documentation/mm/hwpoison.rst | 44
> > +++++++++++++++++++++++++++++++++++
> > >  mm/hwpoison-inject.c          |  4 ++++
> > >  2 files changed, 48 insertions(+)
> > 
> > Hmm.. no change log.
> > 
> > What was the change in-between v1 and v2?
> 
> Hi Jarkko,
> I put the change log in [PATCH V2 0/1]. Following is the details, hope you find it useful.
> 
> Changes since v1:
> - Add a comment in hwpoison_inject as suggested by Miaohe
> - v1: Reviewed-by: Tony Luck <tony.luck@intel.com>
> - v1: Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> 
> Thanks,
> Thomas

Also,


Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

BR, Jarkko

Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[HORIGUCHI NAOYA(堀口 直也)]

On Wed, Sep 28, 2022 at 11:38:32AM -0400, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> address validation), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>

Thank you for sending patch.

> ---
>  Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
>  mm/hwpoison-inject.c          |  4 ++++
>  2 files changed, 48 insertions(+)
> 
> diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
> index b9d5253c1305..8a542aca4744 100644
> --- a/Documentation/mm/hwpoison.rst
> +++ b/Documentation/mm/hwpoison.rst
> @@ -162,6 +162,50 @@ Testing
>  
>    Some portable hwpoison test programs in mce-test, see below.
>  
> +* Special notes for injection into SGX enclaves
> +
> +  1) Determine physical address of enclave page
> +
> +	dmesg | grep "sgx: EPC"
> +
> +	sgx: EPC section 0x8000c00000-0x807f7fffff
> +	sgx: EPC section 0x10000c00000-0x1007fffffff
> +
> +  2) Convert the EPC address to page frame number.
> +
> +	For 4K page size, the page frame number for 0x8000c00000 is
> +	0x8000c00000 / 0x1000 = 0x8000c00.
> +
> +  3) Trace memory_failure
> +
> +	echo nop > /sys/kernel/tracing/current_tracer
> +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> +	echo function > /sys/kernel/tracing/current_tracer
> +
> +  4) Inject a memory error
> +
> +	modprobe hwpoison-inject
> +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> +
> +  5) Check the trace output
> +
> +	cat /sys/kernel/tracing/trace
> +
> +	# tracer: function
> +	#
> +	# entries-in-buffer/entries-written: 2/2   #P:128
> +	#
> +	#                            _-----=> irqs-off
> +	#                           / _----=> need-resched
> +	#                          | / _---=> hardirq/softirq
> +	#                          || / _--=> preempt-depth
> +	#                          ||| / _-=> migrate-disable
> +	#                          |||| /     delay
> +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> +	#          | |         |   |||||     |         |
> +	        bash-12167   [002] .....   113.136808: memory_failure<-simple_attr_write
> +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-memory_failure

In other cases of page types, memory_failure() leaves some kernel message
like "Memory failure: 0x10cf09: recovery action for free buddy page: Recovered",
which is printed out by action_result().  So I think it's better to adjust to
this convention also in SGX page's case.  Then, you don't have to use ftrace
to confirm the result of error injection.

Thanks,
Naoya Horiguchi

> +
>  References
>  ==========
>  
> diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> index 65e242b5a432..bf83111c1d9b 100644
> --- a/mm/hwpoison-inject.c
> +++ b/mm/hwpoison-inject.c
> @@ -21,6 +21,10 @@ static int hwpoison_inject(void *data, u64 val)
>  	if (!capable(CAP_SYS_ADMIN))
>  		return -EPERM;
>  
> +	/* Inject the error if the page is part of the processor reserved memory */
> +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> +		goto inject;
> +
>  	if (!pfn_valid(pfn))
>  		return -ENXIO;
>  
> -- 
> 2.31.1

RE: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX memory

[Thomas Tai]

> -----Original Message-----
> From: HORIGUCHI NAOYA(堀口　直也) <naoya.horiguchi@nec.com>
> Sent: October 7, 2022 2:34 AM
> To: Thomas Tai <thomas.tai@oracle.com>
> Cc: tony.luck@intel.com; dave.hansen@linux.intel.com; jarkko@kernel.org;
> reinette.chatre@intel.co; linmiaohe@huawei.com; akpm@linux-
> foundation.org; linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH V2 1/1] x86/sgx: Add code to inject hwpoison into SGX
> memory
> 
> On Wed, Sep 28, 2022 at 11:38:32AM -0400, Thomas Tai wrote:
> > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > address validation), add a similar code in hwpoison_inject function to
> > check if the address is located in SGX Memory. The error will then be
> > handled by the arch_memory_failure function in the SGX driver.
> >
> > Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> 
> Thank you for sending patch.
> 
> > ---
> >  Documentation/mm/hwpoison.rst | 44
> +++++++++++++++++++++++++++++++++++
> >  mm/hwpoison-inject.c          |  4 ++++
> >  2 files changed, 48 insertions(+)
> >
> > diff --git a/Documentation/mm/hwpoison.rst
> b/Documentation/mm/hwpoison.rst
> > index b9d5253c1305..8a542aca4744 100644
> > --- a/Documentation/mm/hwpoison.rst
> > +++ b/Documentation/mm/hwpoison.rst
> > @@ -162,6 +162,50 @@ Testing
> >
> >    Some portable hwpoison test programs in mce-test, see below.
> >
> > +* Special notes for injection into SGX enclaves
> > +
> > +  1) Determine physical address of enclave page
> > +
> > +	dmesg | grep "sgx: EPC"
> > +
> > +	sgx: EPC section 0x8000c00000-0x807f7fffff
> > +	sgx: EPC section 0x10000c00000-0x1007fffffff
> > +
> > +  2) Convert the EPC address to page frame number.
> > +
> > +	For 4K page size, the page frame number for 0x8000c00000 is
> > +	0x8000c00000 / 0x1000 = 0x8000c00.
> > +
> > +  3) Trace memory_failure
> > +
> > +	echo nop > /sys/kernel/tracing/current_tracer
> > +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> > +	echo function > /sys/kernel/tracing/current_tracer
> > +
> > +  4) Inject a memory error
> > +
> > +	modprobe hwpoison-inject
> > +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> > +
> > +  5) Check the trace output
> > +
> > +	cat /sys/kernel/tracing/trace
> > +
> > +	# tracer: function
> > +	#
> > +	# entries-in-buffer/entries-written: 2/2   #P:128
> > +	#
> > +	#                            _-----=> irqs-off
> > +	#                           / _----=> need-resched
> > +	#                          | / _---=> hardirq/softirq
> > +	#                          || / _--=> preempt-depth
> > +	#                          ||| / _-=> migrate-disable
> > +	#                          |||| /     delay
> > +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> > +	#          | |         |   |||||     |         |
> > +	        bash-12167   [002] .....   113.136808: memory_failure<-
> simple_attr_write
> > +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-
> memory_failure
> 
> In other cases of page types, memory_failure() leaves some kernel message
> like "Memory failure: 0x10cf09: recovery action for free buddy page:
> Recovered",
> which is printed out by action_result().  So I think it's better to adjust to
> this convention also in SGX page's case.  Then, you don't have to use ftrace
> to confirm the result of error injection.

Hi Naoya,
Thanks for your suggestion, I will look into it.

Thomas

> 
> Thanks,
> Naoya Horiguchi
> 
> > +
> >  References
> >  ==========
> >
> > diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> > index 65e242b5a432..bf83111c1d9b 100644
> > --- a/mm/hwpoison-inject.c
> > +++ b/mm/hwpoison-inject.c
> > @@ -21,6 +21,10 @@ static int hwpoison_inject(void *data, u64 val)
> >  	if (!capable(CAP_SYS_ADMIN))
> >  		return -EPERM;
> >
> > +	/* Inject the error if the page is part of the processor reserved memory
> */
> > +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> > +		goto inject;
> > +
> >  	if (!pfn_valid(pfn))
> >  		return -ENXIO;
> >
> > --
> > 2.31.1