From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0BB8BC433EF for ; Thu, 19 May 2022 10:40:05 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4L3mZq2dxCz3c24 for ; Thu, 19 May 2022 20:40:03 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=huawei.com (client-ip=45.249.212.187; helo=szxga01-in.huawei.com; envelope-from=tongtiangen@huawei.com; receiver=) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4L3mZG3whvz3004 for ; Thu, 19 May 2022 20:39:29 +1000 (AEST) Received: from kwepemi100004.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4L3mWj3ZpKzgYNX; Thu, 19 May 2022 18:37:21 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi100004.china.huawei.com (7.221.188.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:44 +0800 Received: from [10.174.179.234] (10.174.179.234) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:42 +0800 Message-ID: <151148fe-cd9d-eb81-6aeb-3aae1691b3dd@huawei.com> Date: Thu, 19 May 2022 18:38:42 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [PATCH -next v4 7/7] arm64: add cow to machine check safe To: Mark Rutland References: <20220420030418.3189040-1-tongtiangen@huawei.com> <20220420030418.3189040-8-tongtiangen@huawei.com> From: Tong Tiangen In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.179.234] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kefeng Wang , Dave Hansen , linux-mm@kvack.org, Paul Mackerras , Guohanjun , Will Deacon , "H . Peter Anvin" , x86@kernel.org, Ingo Molnar , Catalin Marinas , Xie XiuQi , Borislav Petkov , Alexander Viro , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Robin Murphy , linux-kernel@vger.kernel.org, James Morse , Andrew Morton , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" 在 2022/5/13 23:44, Mark Rutland 写道: > On Wed, Apr 20, 2022 at 03:04:18AM +0000, Tong Tiangen wrote: >> In the cow(copy on write) processing, the data of the user process is >> copied, when hardware memory error is encountered during copy, only the >> relevant processes are affected, so killing the user process and isolate >> the user page with hardware memory errors is a more reasonable choice than >> kernel panic. > > There are plenty of other places we'll access user pages via a kernel > alias (e.g. when performing IO), so why is this special? > > To be clear, I am not entirely averse to this, but it seems like this is > being done because it's easy to do rather than necessarily being all > that useful, and I'm not keen on having to duplicate a bunch of logic > for this. Yeah, There are lots of cases, COW is selected because it is more general. In addition, this provides the machine check safe capability of page copy(copy_highpage_mc), valuable cases can be based on this step by step[1]. [1]https://lore.kernel.org/all/20220429000947.2172219-1-jiaqiyan@google.com/T/ Thanks, Tong. > >> Add new helper copy_page_mc() which provide a page copy implementation with >> machine check safe. At present, only used in cow. In future, we can expand >> more scenes. As long as the consequences of page copy failure are not >> fatal(eg: only affect user process), we can use this helper. >> >> The copy_page_mc() in copy_page_mc.S is largely borrows from copy_page() >> in copy_page.S and the main difference is copy_page_mc() add extable entry >> to every load/store insn to support machine check safe. largely to keep the >> patch simple. If needed those optimizations can be folded in. >> >> Add new extable type EX_TYPE_COPY_PAGE_MC which used in copy_page_mc(). >> >> This type only be processed in fixup_exception_mc(), The reason is that >> copy_page_mc() is consistent with copy_page() except machine check safe is >> considered, and copy_page() do not need to consider exception fixup. >> >> Signed-off-by: Tong Tiangen >> --- >> arch/arm64/include/asm/asm-extable.h | 5 ++ >> arch/arm64/include/asm/page.h | 10 ++++ >> arch/arm64/lib/Makefile | 2 + >> arch/arm64/lib/copy_page_mc.S | 86 ++++++++++++++++++++++++++++ >> arch/arm64/mm/copypage.c | 36 ++++++++++-- >> arch/arm64/mm/extable.c | 2 + >> include/linux/highmem.h | 8 +++ >> mm/memory.c | 2 +- >> 8 files changed, 144 insertions(+), 7 deletions(-) >> create mode 100644 arch/arm64/lib/copy_page_mc.S >> >> diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h >> index 80410899a9ad..74c056ddae15 100644 >> --- a/arch/arm64/include/asm/asm-extable.h >> +++ b/arch/arm64/include/asm/asm-extable.h >> @@ -14,6 +14,7 @@ >> /* _MC indicates that can fixup from machine check errors */ >> #define EX_TYPE_UACCESS_MC 5 >> #define EX_TYPE_UACCESS_MC_ERR_ZERO 6 >> +#define EX_TYPE_COPY_PAGE_MC 7 >> >> #ifdef __ASSEMBLY__ >> >> @@ -42,6 +43,10 @@ >> __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_UACCESS_MC, 0) >> .endm >> >> + .macro _asm_extable_copy_page_mc, insn, fixup >> + __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_COPY_PAGE_MC, 0) >> + .endm >> + >> /* >> * Create an exception table entry for `insn` if `fixup` is provided. Otherwise >> * do nothing. >> diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h >> index 993a27ea6f54..832571a7dddb 100644 >> --- a/arch/arm64/include/asm/page.h >> +++ b/arch/arm64/include/asm/page.h >> @@ -29,6 +29,16 @@ void copy_user_highpage(struct page *to, struct page *from, >> void copy_highpage(struct page *to, struct page *from); >> #define __HAVE_ARCH_COPY_HIGHPAGE >> >> +#ifdef CONFIG_ARCH_HAS_COPY_MC >> +extern void copy_page_mc(void *to, const void *from); >> +void copy_highpage_mc(struct page *to, struct page *from); >> +#define __HAVE_ARCH_COPY_HIGHPAGE_MC >> + >> +void copy_user_highpage_mc(struct page *to, struct page *from, >> + unsigned long vaddr, struct vm_area_struct *vma); >> +#define __HAVE_ARCH_COPY_USER_HIGHPAGE_MC >> +#endif >> + >> struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, >> unsigned long vaddr); >> #define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE >> diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile >> index 29490be2546b..0d9f292ef68a 100644 >> --- a/arch/arm64/lib/Makefile >> +++ b/arch/arm64/lib/Makefile >> @@ -15,6 +15,8 @@ endif >> >> lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o >> >> +lib-$(CONFIG_ARCH_HAS_COPY_MC) += copy_page_mc.o >> + >> obj-$(CONFIG_CRC32) += crc32.o >> >> obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o >> diff --git a/arch/arm64/lib/copy_page_mc.S b/arch/arm64/lib/copy_page_mc.S >> new file mode 100644 >> index 000000000000..655161363dcf >> --- /dev/null >> +++ b/arch/arm64/lib/copy_page_mc.S >> @@ -0,0 +1,86 @@ >> +/* SPDX-License-Identifier: GPL-2.0-only */ >> +/* >> + * Copyright (C) 2012 ARM Ltd. >> + */ >> + >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> + >> +#define CPY_MC(l, x...) \ >> +9999: x; \ >> + _asm_extable_copy_page_mc 9999b, l >> + >> +/* >> + * Copy a page from src to dest (both are page aligned) with machine check >> + * >> + * Parameters: >> + * x0 - dest >> + * x1 - src >> + */ >> +SYM_FUNC_START(__pi_copy_page_mc) >> +alternative_if ARM64_HAS_NO_HW_PREFETCH >> + // Prefetch three cache lines ahead. >> + prfm pldl1strm, [x1, #128] >> + prfm pldl1strm, [x1, #256] >> + prfm pldl1strm, [x1, #384] >> +alternative_else_nop_endif >> + >> +CPY_MC(9998f, ldp x2, x3, [x1]) >> +CPY_MC(9998f, ldp x4, x5, [x1, #16]) >> +CPY_MC(9998f, ldp x6, x7, [x1, #32]) >> +CPY_MC(9998f, ldp x8, x9, [x1, #48]) >> +CPY_MC(9998f, ldp x10, x11, [x1, #64]) >> +CPY_MC(9998f, ldp x12, x13, [x1, #80]) >> +CPY_MC(9998f, ldp x14, x15, [x1, #96]) >> +CPY_MC(9998f, ldp x16, x17, [x1, #112]) >> + >> + add x0, x0, #256 >> + add x1, x1, #128 >> +1: >> + tst x0, #(PAGE_SIZE - 1) >> + >> +alternative_if ARM64_HAS_NO_HW_PREFETCH >> + prfm pldl1strm, [x1, #384] >> +alternative_else_nop_endif >> + >> +CPY_MC(9998f, stnp x2, x3, [x0, #-256]) >> +CPY_MC(9998f, ldp x2, x3, [x1]) >> +CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256]) >> +CPY_MC(9998f, ldp x4, x5, [x1, #16]) >> +CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256]) >> +CPY_MC(9998f, ldp x6, x7, [x1, #32]) >> +CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256]) >> +CPY_MC(9998f, ldp x8, x9, [x1, #48]) >> +CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256]) >> +CPY_MC(9998f, ldp x10, x11, [x1, #64]) >> +CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256]) >> +CPY_MC(9998f, ldp x12, x13, [x1, #80]) >> +CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256]) >> +CPY_MC(9998f, ldp x14, x15, [x1, #96]) >> +CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256]) >> +CPY_MC(9998f, ldp x16, x17, [x1, #112]) >> + >> + add x0, x0, #128 >> + add x1, x1, #128 >> + >> + b.ne 1b >> + >> +CPY_MC(9998f, stnp x2, x3, [x0, #-256]) >> +CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256]) >> +CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256]) >> +CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256]) >> +CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256]) >> +CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256]) >> +CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256]) >> +CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256]) >> + >> +9998: ret >> + >> +SYM_FUNC_END(__pi_copy_page_mc) >> +SYM_FUNC_ALIAS(copy_page_mc, __pi_copy_page_mc) >> +EXPORT_SYMBOL(copy_page_mc) >> diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c >> index 0dea80bf6de4..0f28edfcb234 100644 >> --- a/arch/arm64/mm/copypage.c >> +++ b/arch/arm64/mm/copypage.c >> @@ -14,13 +14,8 @@ >> #include >> #include >> >> -void copy_highpage(struct page *to, struct page *from) >> +static void do_mte(struct page *to, struct page *from, void *kto, void *kfrom) >> { >> - void *kto = page_address(to); >> - void *kfrom = page_address(from); >> - >> - copy_page(kto, kfrom); >> - >> if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) { >> set_bit(PG_mte_tagged, &to->flags); >> page_kasan_tag_reset(to); >> @@ -35,6 +30,15 @@ void copy_highpage(struct page *to, struct page *from) >> mte_copy_page_tags(kto, kfrom); >> } >> } >> + >> +void copy_highpage(struct page *to, struct page *from) >> +{ >> + void *kto = page_address(to); >> + void *kfrom = page_address(from); >> + >> + copy_page(kto, kfrom); >> + do_mte(to, from, kto, kfrom); >> +} >> EXPORT_SYMBOL(copy_highpage); >> >> void copy_user_highpage(struct page *to, struct page *from, >> @@ -44,3 +48,23 @@ void copy_user_highpage(struct page *to, struct page *from, >> flush_dcache_page(to); >> } >> EXPORT_SYMBOL_GPL(copy_user_highpage); >> + >> +#ifdef CONFIG_ARCH_HAS_COPY_MC >> +void copy_highpage_mc(struct page *to, struct page *from) >> +{ >> + void *kto = page_address(to); >> + void *kfrom = page_address(from); >> + >> + copy_page_mc(kto, kfrom); >> + do_mte(to, from, kto, kfrom); >> +} >> +EXPORT_SYMBOL(copy_highpage_mc); > > IIUC the do_mte() portion won't handle mermoy errors, so this isn't > actually going to recover safely. > > Thanks, > Mark. OK, Missing that, do_mte needs to be handled. Thanks, Tong. > >> + >> +void copy_user_highpage_mc(struct page *to, struct page *from, >> + unsigned long vaddr, struct vm_area_struct *vma) >> +{ >> + copy_highpage_mc(to, from); >> + flush_dcache_page(to); >> +} >> +EXPORT_SYMBOL_GPL(copy_user_highpage_mc); >> +#endif >> diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c >> index 1023ccdb2f89..4c882d36dd64 100644 >> --- a/arch/arm64/mm/extable.c >> +++ b/arch/arm64/mm/extable.c >> @@ -110,6 +110,8 @@ bool fixup_exception_mc(struct pt_regs *regs) >> return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_MC); >> case EX_TYPE_UACCESS_MC_ERR_ZERO: >> return ex_handler_uaccess_err_zero(ex, regs); >> + case EX_TYPE_COPY_PAGE_MC: >> + return ex_handler_fixup(ex, regs); >> >> } >> >> diff --git a/include/linux/highmem.h b/include/linux/highmem.h >> index 39bb9b47fa9c..a9dbf331b038 100644 >> --- a/include/linux/highmem.h >> +++ b/include/linux/highmem.h >> @@ -283,6 +283,10 @@ static inline void copy_user_highpage(struct page *to, struct page *from, >> >> #endif >> >> +#ifndef __HAVE_ARCH_COPY_USER_HIGHPAGE_MC >> +#define copy_user_highpage_mc copy_user_highpage >> +#endif >> + >> #ifndef __HAVE_ARCH_COPY_HIGHPAGE >> >> static inline void copy_highpage(struct page *to, struct page *from) >> @@ -298,6 +302,10 @@ static inline void copy_highpage(struct page *to, struct page *from) >> >> #endif >> >> +#ifndef __HAVE_ARCH_COPY_HIGHPAGE_MC >> +#define cop_highpage_mc copy_highpage >> +#endif >> + >> static inline void memcpy_page(struct page *dst_page, size_t dst_off, >> struct page *src_page, size_t src_off, >> size_t len) >> diff --git a/mm/memory.c b/mm/memory.c >> index 76e3af9639d9..d5f62234152d 100644 >> --- a/mm/memory.c >> +++ b/mm/memory.c >> @@ -2767,7 +2767,7 @@ static inline bool cow_user_page(struct page *dst, struct page *src, >> unsigned long addr = vmf->address; >> >> if (likely(src)) { >> - copy_user_highpage(dst, src, addr, vma); >> + copy_user_highpage_mc(dst, src, addr, vma); >> return true; >> } >> >> -- >> 2.25.1 >> > . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98556C433FE for ; Thu, 19 May 2022 10:40:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:CC:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9ObrpIq8wPS2+YZ1OGqPBfWe591SxAPAkEavwKTidYE=; b=xMplf3GouJRRX8 ii4qBOlkaUjOW4gcrXXy2xrylw+qZdMTekIt86gzptn+KlA/+xyrHuV3heSMqRQyeZozla84zKGdQ 1/ZWEyRgWTKICA3pzWrI0ibD4hXkJxBFeBwDoBGg1f8iZedrtNEB3FOnbHCLkKzTAkGVr97BC6HSK 6vNmkjKPh89f6q6CbfYf40IgfHy+bgX3sxqH4e4gP9taLe5PMJWouRsQzcNpoKChZm/7D5kGBjIXq 0rEqmdt6gR0xJrJMMLorY/HMszgE/ajPGQLtOyIgU679FtTnQ6EElAaW+frawWmYihZ6s7gSD0NMG JXaXAxvtHP0QoXXCfZAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrdYZ-006KGh-Bg; Thu, 19 May 2022 10:38:59 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrdYT-006KDU-IW for linux-arm-kernel@lists.infradead.org; Thu, 19 May 2022 10:38:57 +0000 Received: from kwepemi100004.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4L3mWj3ZpKzgYNX; Thu, 19 May 2022 18:37:21 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi100004.china.huawei.com (7.221.188.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:44 +0800 Received: from [10.174.179.234] (10.174.179.234) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:42 +0800 Message-ID: <151148fe-cd9d-eb81-6aeb-3aae1691b3dd@huawei.com> Date: Thu, 19 May 2022 18:38:42 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [PATCH -next v4 7/7] arm64: add cow to machine check safe To: Mark Rutland CC: James Morse , Andrew Morton , Thomas Gleixner , "Ingo Molnar" , Borislav Petkov , Robin Murphy , Dave Hansen , "Catalin Marinas" , Will Deacon , "Alexander Viro" , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , , "H . Peter Anvin" , , , , , Kefeng Wang , Xie XiuQi , Guohanjun References: <20220420030418.3189040-1-tongtiangen@huawei.com> <20220420030418.3189040-8-tongtiangen@huawei.com> From: Tong Tiangen In-Reply-To: X-Originating-IP: [10.174.179.234] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220519_033853_985494_969AB67B X-CRM114-Status: GOOD ( 27.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CgrlnKggMjAyMi81LzEzIDIzOjQ0LCBNYXJrIFJ1dGxhbmQg5YaZ6YGTOgo+IE9uIFdlZCwgQXBy IDIwLCAyMDIyIGF0IDAzOjA0OjE4QU0gKzAwMDAsIFRvbmcgVGlhbmdlbiB3cm90ZToKPj4gSW4g dGhlIGNvdyhjb3B5IG9uIHdyaXRlKSBwcm9jZXNzaW5nLCB0aGUgZGF0YSBvZiB0aGUgdXNlciBw cm9jZXNzIGlzCj4+IGNvcGllZCwgd2hlbiBoYXJkd2FyZSBtZW1vcnkgZXJyb3IgaXMgZW5jb3Vu dGVyZWQgZHVyaW5nIGNvcHksIG9ubHkgdGhlCj4+IHJlbGV2YW50IHByb2Nlc3NlcyBhcmUgYWZm ZWN0ZWQsIHNvIGtpbGxpbmcgdGhlIHVzZXIgcHJvY2VzcyBhbmQgaXNvbGF0ZQo+PiB0aGUgdXNl ciBwYWdlIHdpdGggaGFyZHdhcmUgbWVtb3J5IGVycm9ycyBpcyBhIG1vcmUgcmVhc29uYWJsZSBj aG9pY2UgdGhhbgo+PiBrZXJuZWwgcGFuaWMuCj4gCj4gVGhlcmUgYXJlIHBsZW50eSBvZiBvdGhl ciBwbGFjZXMgd2UnbGwgYWNjZXNzIHVzZXIgcGFnZXMgdmlhIGEga2VybmVsCj4gYWxpYXMgKGUu Zy4gd2hlbiBwZXJmb3JtaW5nIElPKSwgc28gd2h5IGlzIHRoaXMgc3BlY2lhbD8KPiAKPiBUbyBi ZSBjbGVhciwgSSBhbSBub3QgZW50aXJlbHkgYXZlcnNlIHRvIHRoaXMsIGJ1dCBpdCBzZWVtcyBs aWtlIHRoaXMgaXMKPiBiZWluZyBkb25lIGJlY2F1c2UgaXQncyBlYXN5IHRvIGRvIHJhdGhlciB0 aGFuIG5lY2Vzc2FyaWx5IGJlaW5nIGFsbAo+IHRoYXQgdXNlZnVsLCBhbmQgSSdtIG5vdCBrZWVu IG9uIGhhdmluZyB0byBkdXBsaWNhdGUgYSBidW5jaCBvZiBsb2dpYwo+IGZvciB0aGlzLgoKWWVh aCwgVGhlcmUgYXJlIGxvdHMgb2YgY2FzZXMsIENPVyBpcyBzZWxlY3RlZCBiZWNhdXNlIGl0IGlz IG1vcmUgCmdlbmVyYWwuIEluIGFkZGl0aW9uLCB0aGlzIHByb3ZpZGVzIHRoZSBtYWNoaW5lIGNo ZWNrIHNhZmUgY2FwYWJpbGl0eSBvZiAKcGFnZSBjb3B5KGNvcHlfaGlnaHBhZ2VfbWMpLCB2YWx1 YWJsZSBjYXNlcyBjYW4gYmUgYmFzZWQgb24gdGhpcyBzdGVwIGJ5IApzdGVwWzFdLgoKWzFdaHR0 cHM6Ly9sb3JlLmtlcm5lbC5vcmcvYWxsLzIwMjIwNDI5MDAwOTQ3LjIxNzIyMTktMS1qaWFxaXlh bkBnb29nbGUuY29tL1QvCgpUaGFua3MsClRvbmcuCgo+IAo+PiBBZGQgbmV3IGhlbHBlciBjb3B5 X3BhZ2VfbWMoKSB3aGljaCBwcm92aWRlIGEgcGFnZSBjb3B5IGltcGxlbWVudGF0aW9uIHdpdGgK Pj4gbWFjaGluZSBjaGVjayBzYWZlLiBBdCBwcmVzZW50LCBvbmx5IHVzZWQgaW4gY293LiBJbiBm dXR1cmUsIHdlIGNhbiBleHBhbmQKPj4gbW9yZSBzY2VuZXMuIEFzIGxvbmcgYXMgdGhlIGNvbnNl cXVlbmNlcyBvZiBwYWdlIGNvcHkgZmFpbHVyZSBhcmUgbm90Cj4+IGZhdGFsKGVnOiBvbmx5IGFm ZmVjdCB1c2VyIHByb2Nlc3MpLCB3ZSBjYW4gdXNlIHRoaXMgaGVscGVyLgo+Pgo+PiBUaGUgY29w eV9wYWdlX21jKCkgaW4gY29weV9wYWdlX21jLlMgaXMgbGFyZ2VseSBib3Jyb3dzIGZyb20gY29w eV9wYWdlKCkKPj4gaW4gY29weV9wYWdlLlMgYW5kIHRoZSBtYWluIGRpZmZlcmVuY2UgaXMgY29w eV9wYWdlX21jKCkgYWRkIGV4dGFibGUgZW50cnkKPj4gdG8gZXZlcnkgbG9hZC9zdG9yZSBpbnNu IHRvIHN1cHBvcnQgbWFjaGluZSBjaGVjayBzYWZlLiBsYXJnZWx5IHRvIGtlZXAgdGhlCj4+IHBh dGNoIHNpbXBsZS4gSWYgbmVlZGVkIHRob3NlIG9wdGltaXphdGlvbnMgY2FuIGJlIGZvbGRlZCBp bi4KPj4KPj4gQWRkIG5ldyBleHRhYmxlIHR5cGUgRVhfVFlQRV9DT1BZX1BBR0VfTUMgd2hpY2gg dXNlZCBpbiBjb3B5X3BhZ2VfbWMoKS4KPj4KPj4gVGhpcyB0eXBlIG9ubHkgYmUgcHJvY2Vzc2Vk IGluIGZpeHVwX2V4Y2VwdGlvbl9tYygpLCBUaGUgcmVhc29uIGlzIHRoYXQKPj4gY29weV9wYWdl X21jKCkgaXMgY29uc2lzdGVudCB3aXRoIGNvcHlfcGFnZSgpIGV4Y2VwdCBtYWNoaW5lIGNoZWNr IHNhZmUgaXMKPj4gY29uc2lkZXJlZCwgYW5kIGNvcHlfcGFnZSgpIGRvIG5vdCBuZWVkIHRvIGNv bnNpZGVyIGV4Y2VwdGlvbiBmaXh1cC4KPj4KPj4gU2lnbmVkLW9mZi1ieTogVG9uZyBUaWFuZ2Vu IDx0b25ndGlhbmdlbkBodWF3ZWkuY29tPgo+PiAtLS0KPj4gICBhcmNoL2FybTY0L2luY2x1ZGUv YXNtL2FzbS1leHRhYmxlLmggfCAgNSArKwo+PiAgIGFyY2gvYXJtNjQvaW5jbHVkZS9hc20vcGFn ZS5oICAgICAgICB8IDEwICsrKysKPj4gICBhcmNoL2FybTY0L2xpYi9NYWtlZmlsZSAgICAgICAg ICAgICAgfCAgMiArCj4+ICAgYXJjaC9hcm02NC9saWIvY29weV9wYWdlX21jLlMgICAgICAgIHwg ODYgKysrKysrKysrKysrKysrKysrKysrKysrKysrKwo+PiAgIGFyY2gvYXJtNjQvbW0vY29weXBh Z2UuYyAgICAgICAgICAgICB8IDM2ICsrKysrKysrKystLQo+PiAgIGFyY2gvYXJtNjQvbW0vZXh0 YWJsZS5jICAgICAgICAgICAgICB8ICAyICsKPj4gICBpbmNsdWRlL2xpbnV4L2hpZ2htZW0uaCAg ICAgICAgICAgICAgfCAgOCArKysKPj4gICBtbS9tZW1vcnkuYyAgICAgICAgICAgICAgICAgICAg ICAgICAgfCAgMiArLQo+PiAgIDggZmlsZXMgY2hhbmdlZCwgMTQ0IGluc2VydGlvbnMoKyksIDcg ZGVsZXRpb25zKC0pCj4+ICAgY3JlYXRlIG1vZGUgMTAwNjQ0IGFyY2gvYXJtNjQvbGliL2NvcHlf cGFnZV9tYy5TCj4+Cj4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL2FzbS1l eHRhYmxlLmggYi9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL2FzbS1leHRhYmxlLmgKPj4gaW5kZXgg ODA0MTA4OTlhOWFkLi43NGMwNTZkZGFlMTUgMTAwNjQ0Cj4+IC0tLSBhL2FyY2gvYXJtNjQvaW5j bHVkZS9hc20vYXNtLWV4dGFibGUuaAo+PiArKysgYi9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL2Fz bS1leHRhYmxlLmgKPj4gQEAgLTE0LDYgKzE0LDcgQEAKPj4gICAvKiBfTUMgaW5kaWNhdGVzIHRo YXQgY2FuIGZpeHVwIGZyb20gbWFjaGluZSBjaGVjayBlcnJvcnMgKi8KPj4gICAjZGVmaW5lIEVY X1RZUEVfVUFDQ0VTU19NQwkJNQo+PiAgICNkZWZpbmUgRVhfVFlQRV9VQUNDRVNTX01DX0VSUl9a RVJPCTYKPj4gKyNkZWZpbmUgRVhfVFlQRV9DT1BZX1BBR0VfTUMJCTcKPj4gICAKPj4gICAjaWZk ZWYgX19BU1NFTUJMWV9fCj4+ICAgCj4+IEBAIC00Miw2ICs0MywxMCBAQAo+PiAgIAlfX0FTTV9F WFRBQkxFX1JBVyhcaW5zbiwgXGZpeHVwLCBFWF9UWVBFX1VBQ0NFU1NfTUMsIDApCj4+ICAgCS5l bmRtCj4+ICAgCj4+ICsJLm1hY3JvICAgICAgICAgIF9hc21fZXh0YWJsZV9jb3B5X3BhZ2VfbWMs IGluc24sIGZpeHVwCj4+ICsJX19BU01fRVhUQUJMRV9SQVcoXGluc24sIFxmaXh1cCwgRVhfVFlQ RV9DT1BZX1BBR0VfTUMsIDApCj4+ICsJLmVuZG0KPj4gKwo+PiAgIC8qCj4+ICAgICogQ3JlYXRl IGFuIGV4Y2VwdGlvbiB0YWJsZSBlbnRyeSBmb3IgYGluc25gIGlmIGBmaXh1cGAgaXMgcHJvdmlk ZWQuIE90aGVyd2lzZQo+PiAgICAqIGRvIG5vdGhpbmcuCj4+IGRpZmYgLS1naXQgYS9hcmNoL2Fy bTY0L2luY2x1ZGUvYXNtL3BhZ2UuaCBiL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20vcGFnZS5oCj4+ IGluZGV4IDk5M2EyN2VhNmY1NC4uODMyNTcxYTdkZGRiIDEwMDY0NAo+PiAtLS0gYS9hcmNoL2Fy bTY0L2luY2x1ZGUvYXNtL3BhZ2UuaAo+PiArKysgYi9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL3Bh Z2UuaAo+PiBAQCAtMjksNiArMjksMTYgQEAgdm9pZCBjb3B5X3VzZXJfaGlnaHBhZ2Uoc3RydWN0 IHBhZ2UgKnRvLCBzdHJ1Y3QgcGFnZSAqZnJvbSwKPj4gICB2b2lkIGNvcHlfaGlnaHBhZ2Uoc3Ry dWN0IHBhZ2UgKnRvLCBzdHJ1Y3QgcGFnZSAqZnJvbSk7Cj4+ICAgI2RlZmluZSBfX0hBVkVfQVJD SF9DT1BZX0hJR0hQQUdFCj4+ICAgCj4+ICsjaWZkZWYgQ09ORklHX0FSQ0hfSEFTX0NPUFlfTUMK Pj4gK2V4dGVybiB2b2lkIGNvcHlfcGFnZV9tYyh2b2lkICp0bywgY29uc3Qgdm9pZCAqZnJvbSk7 Cj4+ICt2b2lkIGNvcHlfaGlnaHBhZ2VfbWMoc3RydWN0IHBhZ2UgKnRvLCBzdHJ1Y3QgcGFnZSAq ZnJvbSk7Cj4+ICsjZGVmaW5lIF9fSEFWRV9BUkNIX0NPUFlfSElHSFBBR0VfTUMKPj4gKwo+PiAr dm9pZCBjb3B5X3VzZXJfaGlnaHBhZ2VfbWMoc3RydWN0IHBhZ2UgKnRvLCBzdHJ1Y3QgcGFnZSAq ZnJvbSwKPj4gKwkJdW5zaWduZWQgbG9uZyB2YWRkciwgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2 bWEpOwo+PiArI2RlZmluZSBfX0hBVkVfQVJDSF9DT1BZX1VTRVJfSElHSFBBR0VfTUMKPj4gKyNl bmRpZgo+PiArCj4+ICAgc3RydWN0IHBhZ2UgKmFsbG9jX3plcm9lZF91c2VyX2hpZ2hwYWdlX21v dmFibGUoc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEsCj4+ICAgCQkJCQkJdW5zaWduZWQgbG9u ZyB2YWRkcik7Cj4+ICAgI2RlZmluZSBfX0hBVkVfQVJDSF9BTExPQ19aRVJPRURfVVNFUl9ISUdI UEFHRV9NT1ZBQkxFCj4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2xpYi9NYWtlZmlsZSBiL2Fy Y2gvYXJtNjQvbGliL01ha2VmaWxlCj4+IGluZGV4IDI5NDkwYmUyNTQ2Yi4uMGQ5ZjI5MmVmNjhh IDEwMDY0NAo+PiAtLS0gYS9hcmNoL2FybTY0L2xpYi9NYWtlZmlsZQo+PiArKysgYi9hcmNoL2Fy bTY0L2xpYi9NYWtlZmlsZQo+PiBAQCAtMTUsNiArMTUsOCBAQCBlbmRpZgo+PiAgIAo+PiAgIGxp Yi0kKENPTkZJR19BUkNIX0hBU19VQUNDRVNTX0ZMVVNIQ0FDSEUpICs9IHVhY2Nlc3NfZmx1c2hj YWNoZS5vCj4+ICAgCj4+ICtsaWItJChDT05GSUdfQVJDSF9IQVNfQ09QWV9NQykgKz0gY29weV9w YWdlX21jLm8KPj4gKwo+PiAgIG9iai0kKENPTkZJR19DUkMzMikgKz0gY3JjMzIubwo+PiAgIAo+ PiAgIG9iai0kKENPTkZJR19GVU5DVElPTl9FUlJPUl9JTkpFQ1RJT04pICs9IGVycm9yLWluamVj dC5vCj4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2xpYi9jb3B5X3BhZ2VfbWMuUyBiL2FyY2gv YXJtNjQvbGliL2NvcHlfcGFnZV9tYy5TCj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0Cj4+IGluZGV4 IDAwMDAwMDAwMDAwMC4uNjU1MTYxMzYzZGNmCj4+IC0tLSAvZGV2L251bGwKPj4gKysrIGIvYXJj aC9hcm02NC9saWIvY29weV9wYWdlX21jLlMKPj4gQEAgLTAsMCArMSw4NiBAQAo+PiArLyogU1BE WC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAtb25seSAqLwo+PiArLyoKPj4gKyAqIENvcHly aWdodCAoQykgMjAxMiBBUk0gTHRkLgo+PiArICovCj4+ICsKPj4gKyNpbmNsdWRlIDxsaW51eC9s aW5rYWdlLmg+Cj4+ICsjaW5jbHVkZSA8bGludXgvY29uc3QuaD4KPj4gKyNpbmNsdWRlIDxhc20v YXNzZW1ibGVyLmg+Cj4+ICsjaW5jbHVkZSA8YXNtL3BhZ2UuaD4KPj4gKyNpbmNsdWRlIDxhc20v Y3B1ZmVhdHVyZS5oPgo+PiArI2luY2x1ZGUgPGFzbS9hbHRlcm5hdGl2ZS5oPgo+PiArI2luY2x1 ZGUgPGFzbS9hc20tZXh0YWJsZS5oPgo+PiArCj4+ICsjZGVmaW5lIENQWV9NQyhsLCB4Li4uKQkJ XAo+PiArOTk5OTogICB4OwkJCVwKPj4gKwlfYXNtX2V4dGFibGVfY29weV9wYWdlX21jICAgIDk5 OTliLCBsCj4+ICsKPj4gKy8qCj4+ICsgKiBDb3B5IGEgcGFnZSBmcm9tIHNyYyB0byBkZXN0IChi b3RoIGFyZSBwYWdlIGFsaWduZWQpIHdpdGggbWFjaGluZSBjaGVjawo+PiArICoKPj4gKyAqIFBh cmFtZXRlcnM6Cj4+ICsgKgl4MCAtIGRlc3QKPj4gKyAqCXgxIC0gc3JjCj4+ICsgKi8KPj4gK1NZ TV9GVU5DX1NUQVJUKF9fcGlfY29weV9wYWdlX21jKQo+PiArYWx0ZXJuYXRpdmVfaWYgQVJNNjRf SEFTX05PX0hXX1BSRUZFVENICj4+ICsJLy8gUHJlZmV0Y2ggdGhyZWUgY2FjaGUgbGluZXMgYWhl YWQuCj4+ICsJcHJmbQlwbGRsMXN0cm0sIFt4MSwgIzEyOF0KPj4gKwlwcmZtCXBsZGwxc3RybSwg W3gxLCAjMjU2XQo+PiArCXByZm0JcGxkbDFzdHJtLCBbeDEsICMzODRdCj4+ICthbHRlcm5hdGl2 ZV9lbHNlX25vcF9lbmRpZgo+PiArCj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4MiwgeDMsIFt4MV0p Cj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4NCwgeDUsIFt4MSwgIzE2XSkKPj4gK0NQWV9NQyg5OTk4 ZiwgbGRwCXg2LCB4NywgW3gxLCAjMzJdKQo+PiArQ1BZX01DKDk5OThmLCBsZHAJeDgsIHg5LCBb eDEsICM0OF0pCj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4MTAsIHgxMSwgW3gxLCAjNjRdKQo+PiAr Q1BZX01DKDk5OThmLCBsZHAJeDEyLCB4MTMsIFt4MSwgIzgwXSkKPj4gK0NQWV9NQyg5OTk4Ziwg bGRwCXgxNCwgeDE1LCBbeDEsICM5Nl0pCj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4MTYsIHgxNywg W3gxLCAjMTEyXSkKPj4gKwo+PiArCWFkZAl4MCwgeDAsICMyNTYKPj4gKwlhZGQJeDEsIHgxLCAj MTI4Cj4+ICsxOgo+PiArCXRzdAl4MCwgIyhQQUdFX1NJWkUgLSAxKQo+PiArCj4+ICthbHRlcm5h dGl2ZV9pZiBBUk02NF9IQVNfTk9fSFdfUFJFRkVUQ0gKPj4gKwlwcmZtCXBsZGwxc3RybSwgW3gx LCAjMzg0XQo+PiArYWx0ZXJuYXRpdmVfZWxzZV9ub3BfZW5kaWYKPj4gKwo+PiArQ1BZX01DKDk5 OThmLCBzdG5wCXgyLCB4MywgW3gwLCAjLTI1Nl0pCj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4Miwg eDMsIFt4MV0pCj4+ICtDUFlfTUMoOTk5OGYsIHN0bnAJeDQsIHg1LCBbeDAsICMxNiAtIDI1Nl0p Cj4+ICtDUFlfTUMoOTk5OGYsIGxkcAl4NCwgeDUsIFt4MSwgIzE2XSkKPj4gK0NQWV9NQyg5OTk4 Ziwgc3RucAl4NiwgeDcsIFt4MCwgIzMyIC0gMjU2XSkKPj4gK0NQWV9NQyg5OTk4ZiwgbGRwCXg2 LCB4NywgW3gxLCAjMzJdKQo+PiArQ1BZX01DKDk5OThmLCBzdG5wCXg4LCB4OSwgW3gwLCAjNDgg LSAyNTZdKQo+PiArQ1BZX01DKDk5OThmLCBsZHAJeDgsIHg5LCBbeDEsICM0OF0pCj4+ICtDUFlf TUMoOTk5OGYsIHN0bnAJeDEwLCB4MTEsIFt4MCwgIzY0IC0gMjU2XSkKPj4gK0NQWV9NQyg5OTk4 ZiwgbGRwCXgxMCwgeDExLCBbeDEsICM2NF0pCj4+ICtDUFlfTUMoOTk5OGYsIHN0bnAJeDEyLCB4 MTMsIFt4MCwgIzgwIC0gMjU2XSkKPj4gK0NQWV9NQyg5OTk4ZiwgbGRwCXgxMiwgeDEzLCBbeDEs ICM4MF0pCj4+ICtDUFlfTUMoOTk5OGYsIHN0bnAJeDE0LCB4MTUsIFt4MCwgIzk2IC0gMjU2XSkK Pj4gK0NQWV9NQyg5OTk4ZiwgbGRwCXgxNCwgeDE1LCBbeDEsICM5Nl0pCj4+ICtDUFlfTUMoOTk5 OGYsIHN0bnAJeDE2LCB4MTcsIFt4MCwgIzExMiAtIDI1Nl0pCj4+ICtDUFlfTUMoOTk5OGYsIGxk cAl4MTYsIHgxNywgW3gxLCAjMTEyXSkKPj4gKwo+PiArCWFkZAl4MCwgeDAsICMxMjgKPj4gKwlh ZGQJeDEsIHgxLCAjMTI4Cj4+ICsKPj4gKwliLm5lCTFiCj4+ICsKPj4gK0NQWV9NQyg5OTk4Ziwg c3RucAl4MiwgeDMsIFt4MCwgIy0yNTZdKQo+PiArQ1BZX01DKDk5OThmLCBzdG5wCXg0LCB4NSwg W3gwLCAjMTYgLSAyNTZdKQo+PiArQ1BZX01DKDk5OThmLCBzdG5wCXg2LCB4NywgW3gwLCAjMzIg LSAyNTZdKQo+PiArQ1BZX01DKDk5OThmLCBzdG5wCXg4LCB4OSwgW3gwLCAjNDggLSAyNTZdKQo+ PiArQ1BZX01DKDk5OThmLCBzdG5wCXgxMCwgeDExLCBbeDAsICM2NCAtIDI1Nl0pCj4+ICtDUFlf TUMoOTk5OGYsIHN0bnAJeDEyLCB4MTMsIFt4MCwgIzgwIC0gMjU2XSkKPj4gK0NQWV9NQyg5OTk4 Ziwgc3RucAl4MTQsIHgxNSwgW3gwLCAjOTYgLSAyNTZdKQo+PiArQ1BZX01DKDk5OThmLCBzdG5w CXgxNiwgeDE3LCBbeDAsICMxMTIgLSAyNTZdKQo+PiArCj4+ICs5OTk4OglyZXQKPj4gKwo+PiAr U1lNX0ZVTkNfRU5EKF9fcGlfY29weV9wYWdlX21jKQo+PiArU1lNX0ZVTkNfQUxJQVMoY29weV9w YWdlX21jLCBfX3BpX2NvcHlfcGFnZV9tYykKPj4gK0VYUE9SVF9TWU1CT0woY29weV9wYWdlX21j KQo+PiBkaWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9tbS9jb3B5cGFnZS5jIGIvYXJjaC9hcm02NC9t bS9jb3B5cGFnZS5jCj4+IGluZGV4IDBkZWE4MGJmNmRlNC4uMGYyOGVkZmNiMjM0IDEwMDY0NAo+ PiAtLS0gYS9hcmNoL2FybTY0L21tL2NvcHlwYWdlLmMKPj4gKysrIGIvYXJjaC9hcm02NC9tbS9j b3B5cGFnZS5jCj4+IEBAIC0xNCwxMyArMTQsOCBAQAo+PiAgICNpbmNsdWRlIDxhc20vY3B1ZmVh dHVyZS5oPgo+PiAgICNpbmNsdWRlIDxhc20vbXRlLmg+Cj4+ICAgCj4+IC12b2lkIGNvcHlfaGln aHBhZ2Uoc3RydWN0IHBhZ2UgKnRvLCBzdHJ1Y3QgcGFnZSAqZnJvbSkKPj4gK3N0YXRpYyB2b2lk IGRvX210ZShzdHJ1Y3QgcGFnZSAqdG8sIHN0cnVjdCBwYWdlICpmcm9tLCB2b2lkICprdG8sIHZv aWQgKmtmcm9tKQo+PiAgIHsKPj4gLQl2b2lkICprdG8gPSBwYWdlX2FkZHJlc3ModG8pOwo+PiAt CXZvaWQgKmtmcm9tID0gcGFnZV9hZGRyZXNzKGZyb20pOwo+PiAtCj4+IC0JY29weV9wYWdlKGt0 bywga2Zyb20pOwo+PiAtCj4+ICAgCWlmIChzeXN0ZW1fc3VwcG9ydHNfbXRlKCkgJiYgdGVzdF9i aXQoUEdfbXRlX3RhZ2dlZCwgJmZyb20tPmZsYWdzKSkgewo+PiAgIAkJc2V0X2JpdChQR19tdGVf dGFnZ2VkLCAmdG8tPmZsYWdzKTsKPj4gICAJCXBhZ2Vfa2FzYW5fdGFnX3Jlc2V0KHRvKTsKPj4g QEAgLTM1LDYgKzMwLDE1IEBAIHZvaWQgY29weV9oaWdocGFnZShzdHJ1Y3QgcGFnZSAqdG8sIHN0 cnVjdCBwYWdlICpmcm9tKQo+PiAgIAkJbXRlX2NvcHlfcGFnZV90YWdzKGt0bywga2Zyb20pOwo+ PiAgIAl9Cj4+ICAgfQo+PiArCj4+ICt2b2lkIGNvcHlfaGlnaHBhZ2Uoc3RydWN0IHBhZ2UgKnRv LCBzdHJ1Y3QgcGFnZSAqZnJvbSkKPj4gK3sKPj4gKwl2b2lkICprdG8gPSBwYWdlX2FkZHJlc3Mo dG8pOwo+PiArCXZvaWQgKmtmcm9tID0gcGFnZV9hZGRyZXNzKGZyb20pOwo+PiArCj4+ICsJY29w eV9wYWdlKGt0bywga2Zyb20pOwo+PiArCWRvX210ZSh0bywgZnJvbSwga3RvLCBrZnJvbSk7Cj4+ ICt9Cj4+ICAgRVhQT1JUX1NZTUJPTChjb3B5X2hpZ2hwYWdlKTsKPj4gICAKPj4gICB2b2lkIGNv cHlfdXNlcl9oaWdocGFnZShzdHJ1Y3QgcGFnZSAqdG8sIHN0cnVjdCBwYWdlICpmcm9tLAo+PiBA QCAtNDQsMyArNDgsMjMgQEAgdm9pZCBjb3B5X3VzZXJfaGlnaHBhZ2Uoc3RydWN0IHBhZ2UgKnRv LCBzdHJ1Y3QgcGFnZSAqZnJvbSwKPj4gICAJZmx1c2hfZGNhY2hlX3BhZ2UodG8pOwo+PiAgIH0K Pj4gICBFWFBPUlRfU1lNQk9MX0dQTChjb3B5X3VzZXJfaGlnaHBhZ2UpOwo+PiArCj4+ICsjaWZk ZWYgQ09ORklHX0FSQ0hfSEFTX0NPUFlfTUMKPj4gK3ZvaWQgY29weV9oaWdocGFnZV9tYyhzdHJ1 Y3QgcGFnZSAqdG8sIHN0cnVjdCBwYWdlICpmcm9tKQo+PiArewo+PiArCXZvaWQgKmt0byA9IHBh Z2VfYWRkcmVzcyh0byk7Cj4+ICsJdm9pZCAqa2Zyb20gPSBwYWdlX2FkZHJlc3MoZnJvbSk7Cj4+ ICsKPj4gKwljb3B5X3BhZ2VfbWMoa3RvLCBrZnJvbSk7Cj4+ICsJZG9fbXRlKHRvLCBmcm9tLCBr dG8sIGtmcm9tKTsKPj4gK30KPj4gK0VYUE9SVF9TWU1CT0woY29weV9oaWdocGFnZV9tYyk7Cj4g Cj4gSUlVQyB0aGUgZG9fbXRlKCkgcG9ydGlvbiB3b24ndCBoYW5kbGUgbWVybW95IGVycm9ycywg c28gdGhpcyBpc24ndAo+IGFjdHVhbGx5IGdvaW5nIHRvIHJlY292ZXIgc2FmZWx5Lgo+IAo+IFRo YW5rcywKPiBNYXJrLgoKT0ssIE1pc3NpbmcgdGhhdCwgZG9fbXRlIG5lZWRzIHRvIGJlIGhhbmRs ZWQuCgpUaGFua3MsClRvbmcuCgo+IAo+PiArCj4+ICt2b2lkIGNvcHlfdXNlcl9oaWdocGFnZV9t YyhzdHJ1Y3QgcGFnZSAqdG8sIHN0cnVjdCBwYWdlICpmcm9tLAo+PiArCQkJdW5zaWduZWQgbG9u ZyB2YWRkciwgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEpCj4+ICt7Cj4+ICsJY29weV9oaWdo cGFnZV9tYyh0bywgZnJvbSk7Cj4+ICsJZmx1c2hfZGNhY2hlX3BhZ2UodG8pOwo+PiArfQo+PiAr RVhQT1JUX1NZTUJPTF9HUEwoY29weV91c2VyX2hpZ2hwYWdlX21jKTsKPj4gKyNlbmRpZgo+PiBk aWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9tbS9leHRhYmxlLmMgYi9hcmNoL2FybTY0L21tL2V4dGFi bGUuYwo+PiBpbmRleCAxMDIzY2NkYjJmODkuLjRjODgyZDM2ZGQ2NCAxMDA2NDQKPj4gLS0tIGEv YXJjaC9hcm02NC9tbS9leHRhYmxlLmMKPj4gKysrIGIvYXJjaC9hcm02NC9tbS9leHRhYmxlLmMK Pj4gQEAgLTExMCw2ICsxMTAsOCBAQCBib29sIGZpeHVwX2V4Y2VwdGlvbl9tYyhzdHJ1Y3QgcHRf cmVncyAqcmVncykKPj4gICAJCXJldHVybiBleF9oYW5kbGVyX3VhY2Nlc3NfdHlwZShleCwgcmVn cywgRklYVVBfVFlQRV9NQyk7Cj4+ICAgCWNhc2UgRVhfVFlQRV9VQUNDRVNTX01DX0VSUl9aRVJP Ogo+PiAgIAkJcmV0dXJuIGV4X2hhbmRsZXJfdWFjY2Vzc19lcnJfemVybyhleCwgcmVncyk7Cj4+ ICsJY2FzZSBFWF9UWVBFX0NPUFlfUEFHRV9NQzoKPj4gKwkJcmV0dXJuIGV4X2hhbmRsZXJfZml4 dXAoZXgsIHJlZ3MpOwo+PiAgIAo+PiAgIAl9Cj4+ICAgCj4+IGRpZmYgLS1naXQgYS9pbmNsdWRl L2xpbnV4L2hpZ2htZW0uaCBiL2luY2x1ZGUvbGludXgvaGlnaG1lbS5oCj4+IGluZGV4IDM5YmI5 YjQ3ZmE5Yy4uYTlkYmYzMzFiMDM4IDEwMDY0NAo+PiAtLS0gYS9pbmNsdWRlL2xpbnV4L2hpZ2ht ZW0uaAo+PiArKysgYi9pbmNsdWRlL2xpbnV4L2hpZ2htZW0uaAo+PiBAQCAtMjgzLDYgKzI4Mywx MCBAQCBzdGF0aWMgaW5saW5lIHZvaWQgY29weV91c2VyX2hpZ2hwYWdlKHN0cnVjdCBwYWdlICp0 bywgc3RydWN0IHBhZ2UgKmZyb20sCj4+ICAgCj4+ICAgI2VuZGlmCj4+ICAgCj4+ICsjaWZuZGVm IF9fSEFWRV9BUkNIX0NPUFlfVVNFUl9ISUdIUEFHRV9NQwo+PiArI2RlZmluZSBjb3B5X3VzZXJf aGlnaHBhZ2VfbWMgY29weV91c2VyX2hpZ2hwYWdlCj4+ICsjZW5kaWYKPj4gKwo+PiAgICNpZm5k ZWYgX19IQVZFX0FSQ0hfQ09QWV9ISUdIUEFHRQo+PiAgIAo+PiAgIHN0YXRpYyBpbmxpbmUgdm9p ZCBjb3B5X2hpZ2hwYWdlKHN0cnVjdCBwYWdlICp0bywgc3RydWN0IHBhZ2UgKmZyb20pCj4+IEBA IC0yOTgsNiArMzAyLDEwIEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBjb3B5X2hpZ2hwYWdlKHN0cnVj dCBwYWdlICp0bywgc3RydWN0IHBhZ2UgKmZyb20pCj4+ICAgCj4+ICAgI2VuZGlmCj4+ICAgCj4+ ICsjaWZuZGVmIF9fSEFWRV9BUkNIX0NPUFlfSElHSFBBR0VfTUMKPj4gKyNkZWZpbmUgY29wX2hp Z2hwYWdlX21jIGNvcHlfaGlnaHBhZ2UKPj4gKyNlbmRpZgo+PiArCj4+ICAgc3RhdGljIGlubGlu ZSB2b2lkIG1lbWNweV9wYWdlKHN0cnVjdCBwYWdlICpkc3RfcGFnZSwgc2l6ZV90IGRzdF9vZmYs Cj4+ICAgCQkJICAgICAgIHN0cnVjdCBwYWdlICpzcmNfcGFnZSwgc2l6ZV90IHNyY19vZmYsCj4+ ICAgCQkJICAgICAgIHNpemVfdCBsZW4pCj4+IGRpZmYgLS1naXQgYS9tbS9tZW1vcnkuYyBiL21t L21lbW9yeS5jCj4+IGluZGV4IDc2ZTNhZjk2MzlkOS4uZDVmNjIyMzQxNTJkIDEwMDY0NAo+PiAt LS0gYS9tbS9tZW1vcnkuYwo+PiArKysgYi9tbS9tZW1vcnkuYwo+PiBAQCAtMjc2Nyw3ICsyNzY3 LDcgQEAgc3RhdGljIGlubGluZSBib29sIGNvd191c2VyX3BhZ2Uoc3RydWN0IHBhZ2UgKmRzdCwg c3RydWN0IHBhZ2UgKnNyYywKPj4gICAJdW5zaWduZWQgbG9uZyBhZGRyID0gdm1mLT5hZGRyZXNz Owo+PiAgIAo+PiAgIAlpZiAobGlrZWx5KHNyYykpIHsKPj4gLQkJY29weV91c2VyX2hpZ2hwYWdl KGRzdCwgc3JjLCBhZGRyLCB2bWEpOwo+PiArCQljb3B5X3VzZXJfaGlnaHBhZ2VfbWMoZHN0LCBz cmMsIGFkZHIsIHZtYSk7Cj4+ICAgCQlyZXR1cm4gdHJ1ZTsKPj4gICAJfQo+PiAgIAo+PiAtLSAK Pj4gMi4yNS4xCj4+Cj4gLgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBs aXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlz dGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DC99C433F5 for ; Thu, 19 May 2022 10:38:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B21646B0072; Thu, 19 May 2022 06:38:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AD2436B0073; Thu, 19 May 2022 06:38:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9997E6B0074; Thu, 19 May 2022 06:38:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8B7896B0072 for ; Thu, 19 May 2022 06:38:50 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4547A3332D for ; Thu, 19 May 2022 10:38:50 +0000 (UTC) X-FDA: 79482144420.15.FE4E0B0 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf10.hostedemail.com (Postfix) with ESMTP id 1923EC0009 for ; Thu, 19 May 2022 10:38:17 +0000 (UTC) Received: from kwepemi100004.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4L3mWj3ZpKzgYNX; Thu, 19 May 2022 18:37:21 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi100004.china.huawei.com (7.221.188.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:44 +0800 Received: from [10.174.179.234] (10.174.179.234) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 19 May 2022 18:38:42 +0800 Message-ID: <151148fe-cd9d-eb81-6aeb-3aae1691b3dd@huawei.com> Date: Thu, 19 May 2022 18:38:42 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [PATCH -next v4 7/7] arm64: add cow to machine check safe To: Mark Rutland CC: James Morse , Andrew Morton , Thomas Gleixner , "Ingo Molnar" , Borislav Petkov , Robin Murphy , Dave Hansen , "Catalin Marinas" , Will Deacon , "Alexander Viro" , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , , "H . Peter Anvin" , , , , , Kefeng Wang , Xie XiuQi , Guohanjun References: <20220420030418.3189040-1-tongtiangen@huawei.com> <20220420030418.3189040-8-tongtiangen@huawei.com> From: Tong Tiangen In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.179.234] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Stat-Signature: oozcpz4kyyidbmb8k44q1xk5edfwg5i6 Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of tongtiangen@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=tongtiangen@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1923EC0009 X-HE-Tag: 1652956697-989868 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2022/5/13 23:44, Mark Rutland 写道: > On Wed, Apr 20, 2022 at 03:04:18AM +0000, Tong Tiangen wrote: >> In the cow(copy on write) processing, the data of the user process is >> copied, when hardware memory error is encountered during copy, only the >> relevant processes are affected, so killing the user process and isolate >> the user page with hardware memory errors is a more reasonable choice than >> kernel panic. > > There are plenty of other places we'll access user pages via a kernel > alias (e.g. when performing IO), so why is this special? > > To be clear, I am not entirely averse to this, but it seems like this is > being done because it's easy to do rather than necessarily being all > that useful, and I'm not keen on having to duplicate a bunch of logic > for this. Yeah, There are lots of cases, COW is selected because it is more general. In addition, this provides the machine check safe capability of page copy(copy_highpage_mc), valuable cases can be based on this step by step[1]. [1]https://lore.kernel.org/all/20220429000947.2172219-1-jiaqiyan@google.com/T/ Thanks, Tong. > >> Add new helper copy_page_mc() which provide a page copy implementation with >> machine check safe. At present, only used in cow. In future, we can expand >> more scenes. As long as the consequences of page copy failure are not >> fatal(eg: only affect user process), we can use this helper. >> >> The copy_page_mc() in copy_page_mc.S is largely borrows from copy_page() >> in copy_page.S and the main difference is copy_page_mc() add extable entry >> to every load/store insn to support machine check safe. largely to keep the >> patch simple. If needed those optimizations can be folded in. >> >> Add new extable type EX_TYPE_COPY_PAGE_MC which used in copy_page_mc(). >> >> This type only be processed in fixup_exception_mc(), The reason is that >> copy_page_mc() is consistent with copy_page() except machine check safe is >> considered, and copy_page() do not need to consider exception fixup. >> >> Signed-off-by: Tong Tiangen >> --- >> arch/arm64/include/asm/asm-extable.h | 5 ++ >> arch/arm64/include/asm/page.h | 10 ++++ >> arch/arm64/lib/Makefile | 2 + >> arch/arm64/lib/copy_page_mc.S | 86 ++++++++++++++++++++++++++++ >> arch/arm64/mm/copypage.c | 36 ++++++++++-- >> arch/arm64/mm/extable.c | 2 + >> include/linux/highmem.h | 8 +++ >> mm/memory.c | 2 +- >> 8 files changed, 144 insertions(+), 7 deletions(-) >> create mode 100644 arch/arm64/lib/copy_page_mc.S >> >> diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h >> index 80410899a9ad..74c056ddae15 100644 >> --- a/arch/arm64/include/asm/asm-extable.h >> +++ b/arch/arm64/include/asm/asm-extable.h >> @@ -14,6 +14,7 @@ >> /* _MC indicates that can fixup from machine check errors */ >> #define EX_TYPE_UACCESS_MC 5 >> #define EX_TYPE_UACCESS_MC_ERR_ZERO 6 >> +#define EX_TYPE_COPY_PAGE_MC 7 >> >> #ifdef __ASSEMBLY__ >> >> @@ -42,6 +43,10 @@ >> __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_UACCESS_MC, 0) >> .endm >> >> + .macro _asm_extable_copy_page_mc, insn, fixup >> + __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_COPY_PAGE_MC, 0) >> + .endm >> + >> /* >> * Create an exception table entry for `insn` if `fixup` is provided. Otherwise >> * do nothing. >> diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h >> index 993a27ea6f54..832571a7dddb 100644 >> --- a/arch/arm64/include/asm/page.h >> +++ b/arch/arm64/include/asm/page.h >> @@ -29,6 +29,16 @@ void copy_user_highpage(struct page *to, struct page *from, >> void copy_highpage(struct page *to, struct page *from); >> #define __HAVE_ARCH_COPY_HIGHPAGE >> >> +#ifdef CONFIG_ARCH_HAS_COPY_MC >> +extern void copy_page_mc(void *to, const void *from); >> +void copy_highpage_mc(struct page *to, struct page *from); >> +#define __HAVE_ARCH_COPY_HIGHPAGE_MC >> + >> +void copy_user_highpage_mc(struct page *to, struct page *from, >> + unsigned long vaddr, struct vm_area_struct *vma); >> +#define __HAVE_ARCH_COPY_USER_HIGHPAGE_MC >> +#endif >> + >> struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, >> unsigned long vaddr); >> #define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE >> diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile >> index 29490be2546b..0d9f292ef68a 100644 >> --- a/arch/arm64/lib/Makefile >> +++ b/arch/arm64/lib/Makefile >> @@ -15,6 +15,8 @@ endif >> >> lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o >> >> +lib-$(CONFIG_ARCH_HAS_COPY_MC) += copy_page_mc.o >> + >> obj-$(CONFIG_CRC32) += crc32.o >> >> obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o >> diff --git a/arch/arm64/lib/copy_page_mc.S b/arch/arm64/lib/copy_page_mc.S >> new file mode 100644 >> index 000000000000..655161363dcf >> --- /dev/null >> +++ b/arch/arm64/lib/copy_page_mc.S >> @@ -0,0 +1,86 @@ >> +/* SPDX-License-Identifier: GPL-2.0-only */ >> +/* >> + * Copyright (C) 2012 ARM Ltd. >> + */ >> + >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> +#include >> + >> +#define CPY_MC(l, x...) \ >> +9999: x; \ >> + _asm_extable_copy_page_mc 9999b, l >> + >> +/* >> + * Copy a page from src to dest (both are page aligned) with machine check >> + * >> + * Parameters: >> + * x0 - dest >> + * x1 - src >> + */ >> +SYM_FUNC_START(__pi_copy_page_mc) >> +alternative_if ARM64_HAS_NO_HW_PREFETCH >> + // Prefetch three cache lines ahead. >> + prfm pldl1strm, [x1, #128] >> + prfm pldl1strm, [x1, #256] >> + prfm pldl1strm, [x1, #384] >> +alternative_else_nop_endif >> + >> +CPY_MC(9998f, ldp x2, x3, [x1]) >> +CPY_MC(9998f, ldp x4, x5, [x1, #16]) >> +CPY_MC(9998f, ldp x6, x7, [x1, #32]) >> +CPY_MC(9998f, ldp x8, x9, [x1, #48]) >> +CPY_MC(9998f, ldp x10, x11, [x1, #64]) >> +CPY_MC(9998f, ldp x12, x13, [x1, #80]) >> +CPY_MC(9998f, ldp x14, x15, [x1, #96]) >> +CPY_MC(9998f, ldp x16, x17, [x1, #112]) >> + >> + add x0, x0, #256 >> + add x1, x1, #128 >> +1: >> + tst x0, #(PAGE_SIZE - 1) >> + >> +alternative_if ARM64_HAS_NO_HW_PREFETCH >> + prfm pldl1strm, [x1, #384] >> +alternative_else_nop_endif >> + >> +CPY_MC(9998f, stnp x2, x3, [x0, #-256]) >> +CPY_MC(9998f, ldp x2, x3, [x1]) >> +CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256]) >> +CPY_MC(9998f, ldp x4, x5, [x1, #16]) >> +CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256]) >> +CPY_MC(9998f, ldp x6, x7, [x1, #32]) >> +CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256]) >> +CPY_MC(9998f, ldp x8, x9, [x1, #48]) >> +CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256]) >> +CPY_MC(9998f, ldp x10, x11, [x1, #64]) >> +CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256]) >> +CPY_MC(9998f, ldp x12, x13, [x1, #80]) >> +CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256]) >> +CPY_MC(9998f, ldp x14, x15, [x1, #96]) >> +CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256]) >> +CPY_MC(9998f, ldp x16, x17, [x1, #112]) >> + >> + add x0, x0, #128 >> + add x1, x1, #128 >> + >> + b.ne 1b >> + >> +CPY_MC(9998f, stnp x2, x3, [x0, #-256]) >> +CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256]) >> +CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256]) >> +CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256]) >> +CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256]) >> +CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256]) >> +CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256]) >> +CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256]) >> + >> +9998: ret >> + >> +SYM_FUNC_END(__pi_copy_page_mc) >> +SYM_FUNC_ALIAS(copy_page_mc, __pi_copy_page_mc) >> +EXPORT_SYMBOL(copy_page_mc) >> diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c >> index 0dea80bf6de4..0f28edfcb234 100644 >> --- a/arch/arm64/mm/copypage.c >> +++ b/arch/arm64/mm/copypage.c >> @@ -14,13 +14,8 @@ >> #include >> #include >> >> -void copy_highpage(struct page *to, struct page *from) >> +static void do_mte(struct page *to, struct page *from, void *kto, void *kfrom) >> { >> - void *kto = page_address(to); >> - void *kfrom = page_address(from); >> - >> - copy_page(kto, kfrom); >> - >> if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) { >> set_bit(PG_mte_tagged, &to->flags); >> page_kasan_tag_reset(to); >> @@ -35,6 +30,15 @@ void copy_highpage(struct page *to, struct page *from) >> mte_copy_page_tags(kto, kfrom); >> } >> } >> + >> +void copy_highpage(struct page *to, struct page *from) >> +{ >> + void *kto = page_address(to); >> + void *kfrom = page_address(from); >> + >> + copy_page(kto, kfrom); >> + do_mte(to, from, kto, kfrom); >> +} >> EXPORT_SYMBOL(copy_highpage); >> >> void copy_user_highpage(struct page *to, struct page *from, >> @@ -44,3 +48,23 @@ void copy_user_highpage(struct page *to, struct page *from, >> flush_dcache_page(to); >> } >> EXPORT_SYMBOL_GPL(copy_user_highpage); >> + >> +#ifdef CONFIG_ARCH_HAS_COPY_MC >> +void copy_highpage_mc(struct page *to, struct page *from) >> +{ >> + void *kto = page_address(to); >> + void *kfrom = page_address(from); >> + >> + copy_page_mc(kto, kfrom); >> + do_mte(to, from, kto, kfrom); >> +} >> +EXPORT_SYMBOL(copy_highpage_mc); > > IIUC the do_mte() portion won't handle mermoy errors, so this isn't > actually going to recover safely. > > Thanks, > Mark. OK, Missing that, do_mte needs to be handled. Thanks, Tong. > >> + >> +void copy_user_highpage_mc(struct page *to, struct page *from, >> + unsigned long vaddr, struct vm_area_struct *vma) >> +{ >> + copy_highpage_mc(to, from); >> + flush_dcache_page(to); >> +} >> +EXPORT_SYMBOL_GPL(copy_user_highpage_mc); >> +#endif >> diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c >> index 1023ccdb2f89..4c882d36dd64 100644 >> --- a/arch/arm64/mm/extable.c >> +++ b/arch/arm64/mm/extable.c >> @@ -110,6 +110,8 @@ bool fixup_exception_mc(struct pt_regs *regs) >> return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_MC); >> case EX_TYPE_UACCESS_MC_ERR_ZERO: >> return ex_handler_uaccess_err_zero(ex, regs); >> + case EX_TYPE_COPY_PAGE_MC: >> + return ex_handler_fixup(ex, regs); >> >> } >> >> diff --git a/include/linux/highmem.h b/include/linux/highmem.h >> index 39bb9b47fa9c..a9dbf331b038 100644 >> --- a/include/linux/highmem.h >> +++ b/include/linux/highmem.h >> @@ -283,6 +283,10 @@ static inline void copy_user_highpage(struct page *to, struct page *from, >> >> #endif >> >> +#ifndef __HAVE_ARCH_COPY_USER_HIGHPAGE_MC >> +#define copy_user_highpage_mc copy_user_highpage >> +#endif >> + >> #ifndef __HAVE_ARCH_COPY_HIGHPAGE >> >> static inline void copy_highpage(struct page *to, struct page *from) >> @@ -298,6 +302,10 @@ static inline void copy_highpage(struct page *to, struct page *from) >> >> #endif >> >> +#ifndef __HAVE_ARCH_COPY_HIGHPAGE_MC >> +#define cop_highpage_mc copy_highpage >> +#endif >> + >> static inline void memcpy_page(struct page *dst_page, size_t dst_off, >> struct page *src_page, size_t src_off, >> size_t len) >> diff --git a/mm/memory.c b/mm/memory.c >> index 76e3af9639d9..d5f62234152d 100644 >> --- a/mm/memory.c >> +++ b/mm/memory.c >> @@ -2767,7 +2767,7 @@ static inline bool cow_user_page(struct page *dst, struct page *src, >> unsigned long addr = vmf->address; >> >> if (likely(src)) { >> - copy_user_highpage(dst, src, addr, vma); >> + copy_user_highpage_mc(dst, src, addr, vma); >> return true; >> } >> >> -- >> 2.25.1 >> > .