Re: [PATCH v14 3/3] ACPI: APEI: handle synchronous exceptions in task work

["Jarkko Sakkinen" <jarkko@kernel.org>]

On Tue Oct 22, 2024 at 4:11 AM EEST, Shuai Xue wrote:
> Hi, Jarkko,
>
>
> 在 2024/10/14 16:42, Shuai Xue 写道:
> > The memory uncorrected error could be signaled by asynchronous interrupt
> > (specifically, SPI in arm64 platform), e.g. when an error is detected by
> > a background scrubber, or signaled by synchronous exception
> > (specifically, data abort excepction in arm64 platform), e.g. when a CPU
> > tries to access a poisoned cache line. Currently, both synchronous and
> > asynchronous error use memory_failure_queue() to schedule
> > memory_failure() exectute in kworker context.
> > 
> > As a result, when a user-space process is accessing a poisoned data, a
> > data abort is taken and the memory_failure() is executed in the kworker
> > context:
> > 
> >    - will send wrong si_code by SIGBUS signal in early_kill mode, and
> >    - can not kill the user-space in some cases resulting a synchronous
> >      error infinite loop
> > 
> > Issue 1: send wrong si_code in early_kill mode
> > 
> > Since commit a70297d22132 ("ACPI: APEI: set memory failure flags as
> > MF_ACTION_REQUIRED on synchronous events")', the flag MF_ACTION_REQUIRED
> > could be used to determine whether a synchronous exception occurs on
> > ARM64 platform.  When a synchronous exception is detected, the kernel is
> > expected to terminate the current process which has accessed poisoned
> > page. This is done by sending a SIGBUS signal with an error code
> > BUS_MCEERR_AR, indicating an action-required machine check error on
> > read.
> > 
> > However, when kill_proc() is called to terminate the processes who have
> > the poisoned page mapped, it sends the incorrect SIGBUS error code
> > BUS_MCEERR_AO because the context in which it operates is not the one
> > where the error was triggered.
> > 
> > To reproduce this problem:
> > 
> >    #sysctl -w vm.memory_failure_early_kill=1
> >    vm.memory_failure_early_kill = 1
> > 
> >    # STEP2: inject an UCE error and consume it to trigger a synchronous error
> >    #einj_mem_uc single
> >    0: single   vaddr = 0xffffb0d75400 paddr = 4092d55b400
> >    injecting ...
> >    triggering ...
> >    signal 7 code 5 addr 0xffffb0d75000
> >    page not present
> >    Test passed
> > 
> > The si_code (code 5) from einj_mem_uc indicates that it is BUS_MCEERR_AO
> > error and it is not fact.
> > 
> > After this patch:
> > 
> >    # STEP1: enable early kill mode
> >    #sysctl -w vm.memory_failure_early_kill=1
> >    vm.memory_failure_early_kill = 1
> >    # STEP2: inject an UCE error and consume it to trigger a synchronous error
> >    #einj_mem_uc single
> >    0: single   vaddr = 0xffffb0d75400 paddr = 4092d55b400
> >    injecting ...
> >    triggering ...
> >    signal 7 code 4 addr 0xffffb0d75000
> >    page not present
> >    Test passed
> > 
> > The si_code (code 4) from einj_mem_uc indicates that it is BUS_MCEERR_AR
> > error as we expected.
> > 
> > Issue 2: a synchronous error infinite loop
> > 
> > If a user-space process, e.g. devmem, a poisoned page which has been set
> > HWPosion flag, kill_accessing_process() is called to send SIGBUS to the
> > current processs with error info. Because the memory_failure() is
> > executed in the kworker contex, it will just do nothing but return
> > EFAULT. So, devmem will access the posioned page and trigger an
> > excepction again, resulting in a synchronous error infinite loop. Such
> > loop may cause platform firmware to exceed some threshold and reboot
> > when Linux could have recovered from this error.
> > 
> > To reproduce this problem:
> > 
> >    # STEP 1: inject an UCE error, and kernel will set HWPosion flag for related page
> >    #einj_mem_uc single
> >    0: single   vaddr = 0xffffb0d75400 paddr = 4092d55b400
> >    injecting ...
> >    triggering ...
> >    signal 7 code 4 addr 0xffffb0d75000
> >    page not present
> >    Test passed
> > 
> >    # STEP 2: access the same page and it will trigger a synchronous error infinite loop
> >    devmem 0x4092d55b400
> > 
> > To fix above two issues, queue memory_failure() as a task_work so that it runs in
> > the context of the process that is actually consuming the poisoned data.
> > 
> > Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
> > Tested-by: Ma Wupeng <mawupeng1@huawei.com>
> > Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> > Reviewed-by: Xiaofei Tan <tanxiaofei@huawei.com>
> > Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> > ---
> >   drivers/acpi/apei/ghes.c | 78 +++++++++++++++++++++++-----------------
> >   include/acpi/ghes.h      |  3 --
> >   include/linux/mm.h       |  1 -
> >   mm/memory-failure.c      | 13 -------
> >   4 files changed, 45 insertions(+), 50 deletions(-)
> > 
> > diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
> > index f2ee28c44d7a..95e9520eb803 100644
> > --- a/drivers/acpi/apei/ghes.c
> > +++ b/drivers/acpi/apei/ghes.c
> > @@ -467,28 +467,42 @@ static void ghes_clear_estatus(struct ghes *ghes,
> >   }
> >   
> >   /*
> > - * Called as task_work before returning to user-space.
> > - * Ensure any queued work has been done before we return to the context that
> > - * triggered the notification.
> > + * struct ghes_task_work - for synchronous RAS event
> > + *
> > + * @twork:                callback_head for task work
> > + * @pfn:                  page frame number of corrupted page
> > + * @flags:                work control flags
> > + *
> > + * Structure to pass task work to be handled before
> > + * returning to user-space via task_work_add().
> >    */
>
>
> Do you have any futer comments about this patch? Any comments are
> welcomed. If not, are you happy to explictly give the reveiwed-by tag?

Sorry I've been busy switching to a new job.

I read this now through and both commit messages and the code changes
look sane to me so I guess I don't have any problem with that:

Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

>
> Best Regard,
> Shuai

BR, Jarkko