[PATCHv2] x86/sgx: Avoid softlockup from sgx_vepc_release

[PATCHv2] x86/sgx: Avoid softlockup from sgx_vepc_release

[Jack Wang]

We hit softlocup with following call trace:
? asm_sysvec_apic_timer_interrupt+0x16/0x20
xa_erase+0x21/0xb0
? sgx_free_epc_page+0x20/0x50
sgx_vepc_release+0x75/0x220
__fput+0x89/0x250
task_work_run+0x59/0x90
do_exit+0x337/0x9a0

Similar like commit
8795359e35bc ("x86/sgx: Silence softlockup detection when releasing large enclaves")
The test system has 64GB of enclave memory, and all assigned to a single
VM. Release vepc take longer time and triggers the softlockup warning.

Add a cond_resched() to give other tasks a chance to run and placate
the softlockup detector.

Cc: Jarkko Sakkinen <jarkko@kernel.org>
Cc: Haitao Huang <haitao.huang@linux.intel.com>
Cc: stable@vger.kernel.org
Fixes: 540745ddbc70 ("x86/sgx: Introduce virtual EPC for use by KVM guests")
Reported-by: Yu Zhang <yu.zhang@ionos.com>
Tested-by: Yu Zhang <yu.zhang@ionos.com>
Acked-by: Haitao Huang <haitao.huang@linux.intel.com>
Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
v2: 
* collects review and test by.
* add fixes tag
* trim call trace.

 arch/x86/kernel/cpu/sgx/virt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c
index c3e37eaec8ec..01d2795792cc 100644
--- a/arch/x86/kernel/cpu/sgx/virt.c
+++ b/arch/x86/kernel/cpu/sgx/virt.c
@@ -204,6 +204,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
 			continue;
 
 		xa_erase(&vepc->page_array, index);
+		cond_resched();
 	}
 
 	/*
@@ -222,6 +223,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
 			list_add_tail(&epc_page->list, &secs_pages);
 
 		xa_erase(&vepc->page_array, index);
+		cond_resched();
 	}
 
 	/*
-- 
2.34.1

Re: [PATCHv2] x86/sgx: Avoid softlockup from sgx_vepc_release

[Huang, Kai]

On Tue, 2023-08-15 at 20:22 +0200, Jack Wang wrote:
> We hit softlocup with following call trace:

Please add an empty line for better readability.

> ? asm_sysvec_apic_timer_interrupt+0x16/0x20
> xa_erase+0x21/0xb0
> ? sgx_free_epc_page+0x20/0x50
> sgx_vepc_release+0x75/0x220
> __fput+0x89/0x250
> task_work_run+0x59/0x90
> do_exit+0x337/0x9a0
> 
> Similar like commit
> 8795359e35bc ("x86/sgx: Silence softlockup detection when releasing large enclaves")

Please wrap text properly.

> The test system has 64GB of enclave memory, and all assigned to a single
> VM. Release vepc take longer time and triggers the softlockup warning.
> 
> Add a cond_resched() to give other tasks a chance to run and placate
      ^

You are adding more than one 'cond_resched()', so remove 'a'.

> the softlockup detector.
> 
> Cc: Jarkko Sakkinen <jarkko@kernel.org>
> Cc: Haitao Huang <haitao.huang@linux.intel.com>
> Cc: stable@vger.kernel.org
> Fixes: 540745ddbc70 ("x86/sgx: Introduce virtual EPC for use by KVM guests")
> Reported-by: Yu Zhang <yu.zhang@ionos.com>
> Tested-by: Yu Zhang <yu.zhang@ionos.com>
> Acked-by: Haitao Huang <haitao.huang@linux.intel.com>
> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
> Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
> ---
> v2: 
> * collects review and test by.
> * add fixes tag
> * trim call trace.
> 
>  arch/x86/kernel/cpu/sgx/virt.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c
> index c3e37eaec8ec..01d2795792cc 100644
> --- a/arch/x86/kernel/cpu/sgx/virt.c
> +++ b/arch/x86/kernel/cpu/sgx/virt.c
> @@ -204,6 +204,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
>  			continue;
>  
>  		xa_erase(&vepc->page_array, index);
> +		cond_resched();
>  	}
>  
>  	/*
> @@ -222,6 +223,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
>  			list_add_tail(&epc_page->list, &secs_pages);
>  
>  		xa_erase(&vepc->page_array, index);
> +		cond_resched();
>  	}
>  
> 

Comments ignored without explanation:

https://lore.kernel.org/linux-sgx/op.19p011pawjvjmi@hhuan26-mobl.amr.corp.intel.com/T/#mf004deb81f9eb1eab7ad3c1c88746bfa23d5109c

Re: [PATCHv2] x86/sgx: Avoid softlockup from sgx_vepc_release

[Jinpu Wang]

On Wed, Aug 16, 2023 at 12:13 PM Huang, Kai <kai.huang@intel.com> wrote:
>
> On Tue, 2023-08-15 at 20:22 +0200, Jack Wang wrote:
> > We hit softlocup with following call trace:
>
> Please add an empty line for better readability.
>
> > ? asm_sysvec_apic_timer_interrupt+0x16/0x20
> > xa_erase+0x21/0xb0
> > ? sgx_free_epc_page+0x20/0x50
> > sgx_vepc_release+0x75/0x220
> > __fput+0x89/0x250
> > task_work_run+0x59/0x90
> > do_exit+0x337/0x9a0
> >
> > Similar like commit
> > 8795359e35bc ("x86/sgx: Silence softlockup detection when releasing large enclaves")
>
> Please wrap text properly.
>
> > The test system has 64GB of enclave memory, and all assigned to a single
> > VM. Release vepc take longer time and triggers the softlockup warning.
> >
> > Add a cond_resched() to give other tasks a chance to run and placate
>       ^
>
> You are adding more than one 'cond_resched()', so remove 'a'.
>
> > the softlockup detector.

Hi Kai,

Thx for the comments, sorry for slow reply. was busy with other high
prio tasks. I've answered your initial input in v1 patch.

I will wait a bit see if there is other comments, maybe the maintainer
want to fix them while apply

Thx!
> >
> > Cc: Jarkko Sakkinen <jarkko@kernel.org>
> > Cc: Haitao Huang <haitao.huang@linux.intel.com>
> > Cc: stable@vger.kernel.org
> > Fixes: 540745ddbc70 ("x86/sgx: Introduce virtual EPC for use by KVM guests")
> > Reported-by: Yu Zhang <yu.zhang@ionos.com>
> > Tested-by: Yu Zhang <yu.zhang@ionos.com>
> > Acked-by: Haitao Huang <haitao.huang@linux.intel.com>
> > Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
> > Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
> > ---
> > v2:
> > * collects review and test by.
> > * add fixes tag
> > * trim call trace.
> >
> >  arch/x86/kernel/cpu/sgx/virt.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/arch/x86/kernel/cpu/sgx/virt.c b/arch/x86/kernel/cpu/sgx/virt.c
> > index c3e37eaec8ec..01d2795792cc 100644
> > --- a/arch/x86/kernel/cpu/sgx/virt.c
> > +++ b/arch/x86/kernel/cpu/sgx/virt.c
> > @@ -204,6 +204,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
> >                       continue;
> >
> >               xa_erase(&vepc->page_array, index);
> > +             cond_resched();
> >       }
> >
> >       /*
> > @@ -222,6 +223,7 @@ static int sgx_vepc_release(struct inode *inode, struct file *file)
> >                       list_add_tail(&epc_page->list, &secs_pages);
> >
> >               xa_erase(&vepc->page_array, index);
> > +             cond_resched();
> >       }
> >
> >
>
> Comments ignored without explanation:
>
> https://lore.kernel.org/linux-sgx/op.19p011pawjvjmi@hhuan26-mobl.amr.corp.intel.com/T/#mf004deb81f9eb1eab7ad3c1c88746bfa23d5109c
>