From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752870AbcADThJ (ORCPT ); Mon, 4 Jan 2016 14:37:09 -0500 Received: from mail-db3on0065.outbound.protection.outlook.com ([157.55.234.65]:23469 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753122AbcADTfo (ORCPT ); Mon, 4 Jan 2016 14:35:44 -0500 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; ezchip.com; dkim=none (message not signed) header.d=none;ezchip.com; dmarc=none action=none header.from=ezchip.com; From: Chris Metcalf To: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , "Rik van Riel" , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , , CC: Chris Metcalf Subject: [PATCH v9 09/13] arch/arm64: enable task isolation functionality Date: Mon, 4 Jan 2016 14:34:47 -0500 Message-ID: <1451936091-29247-10-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1451936091-29247-1-git-send-email-cmetcalf@ezchip.com> References: <1451936091-29247-1-git-send-email-cmetcalf@ezchip.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD010;1:PomZDMCupG8u9Fa74aIQtV8LcvBOS5dY+HBqMpNzvnHDWiZVTFE5u7EIjgK22lT+6H0o+CQxS2LxpIGNYIJNAkrqXleSoUc3wJvNVtd4SywEGfRZ4XvoWX845siIf8dOg0xWxHlzIzSQLM6ppTyLUDmEGwjAw2IN83XNiOeYnxjC3oQG7afhBI/X3VNfvbEsZrMMSm/APZtjkUpekfPjwTim932A6WN/uhrEQbdpFATfbhNwqvh5GeNFvPhV1YQI4L7Zc+ZO2gW9ceWdmTJo5LGfHnskT66/z/Z87T20dk6ECMpiv2zpEnWlcyTp6fX+6yn0F56kU3JD6QNU4aWF+7R4vBQlKLaC+pso+Vy1ROoDLuKsuE7HR1zcCgCyP4c7 X-Forefront-Antispam-Report: CIP:12.216.194.146;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(1110001)(1109001)(339900001)(199003)(189002)(2201001)(5003940100001)(2950100001)(86362001)(76176999)(106466001)(50226001)(586003)(1096002)(50466002)(1220700001)(4326007)(42186005)(105606002)(48376002)(85426001)(36756003)(5001970100001)(5001770100001)(19580405001)(6806005)(50986999)(19580395003)(229853001)(11100500001)(4001430100002)(104016004)(189998001)(33646002)(92566002)(107886002)(87936001)(47776003)(5008740100001)(921003)(2101003)(83996005)(1121003);DIR:OUT;SFP:1101;SCL:1;SRVR:AM2PR02MB0420;H:ld-1.internal.tilera.com;FPR:;SPF:Fail;PTR:wb-fw1.tilera.com;A:1;MX:1;LANG:en; MIME-Version: 1.0 Content-Type: text/plain X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;2:TGNq93cpO2qgs6BOVPZdHK1yLh7/1S26DWlmB8TLSvSTAScrdCEKcEcCzHLKuPHq6QtINMlH/A7EJsThEbtv+4Q3qMFVW1TD2wW77fxRVRiT1yig3LbLtF/O3DYIDxIUP2I4OS8ETv+5mQR5YaWgJg==;3:Prw7si1mf3ICP3cQOkEPq4OeOWYoF/AkLMAPBsJRzhLWUu419QPVt3NnZjgz5sulLIFKBpMMTPDRIvUuPFkvwan379HsrK+f4/JTx5hF7EF+hFXcsV3hVzvS4EJ5NFqXlyltbbhxqI9CsmyJToxAtFSaqH6yjpt32PfHkAM3k12eCnZl/5IpvXcCJgjU4dQsCC7QRGZ76Slo8/m4xN/BCEspX1Ykb4koWRYb+O2EWOM=;25:RQrEb3V6X1swPqGRWN2pyuff34C1/ybedNmXiKONB5qWdeSqHJoUQEb8x8CZrc08qO4po8PmBD0xA1wi3ZmWt+V/cgx/vbAkT+vF1/AyX0X858ZCEcSLog5UkJpQBNzNpLqOV+rhojsMwU0noQhzrFJjJ4a4Dna7iDjD3ZAEUvc7dt2FxCZV6cNhLZZ65v+WOyM1/gj/ejS2qxbBEU4vb0b0utCFseCsfbn69ghDPvvxhlwnkdbhfnT7gF1IGZ3zoWL1jK7qs1Dg7mb7AJTolA==;20:rK0IaDXhixv5l6IjVK9wD3Ss7k3jObYHpRXyXyd1FTIG0GkQONlqI51gJkJQkqSFaI7cU9ahptnLOAWasUaWC1mWSQ+BOecOHlyyEPkRjGWOWg9JNEBVoTPG2KxMRaspYj3yZ7y7WuATcK5FE99uhCEha1ug+rp34zM8INptjSc= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0420; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(121898900299872); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(520078)(5005006)(8121501046)(10201501046)(3002001);SRVR:AM2PR02MB0420;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0420; X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;4:yiw9old9lgLvK0D5nAVZhVK77tViBM+SRTkVye/Z7IK7IlsMO7+3gFFPMUBrX3D5fSv3pwsYmeckWj7a+MWMoPnPUzstx+GOxpzgslQw2cb53X2eP8m/7ckeWLOUqS6ug/xausG3kIouSORKVY5YQK/XwbI5Uo37z/ajbbt1ETKe938YitNTS+3Tgazo0TwJz06Cuv6lORyi5DVOk3KUqJlOf300w4z5tlwceozS2pSHHc4+ShWfoSL7MEbdkIzcnc6S7fndS+D0BVMzs2XDGsLfLpwDK9S+AOdxObl1tS9+ExS+RqbhIIrN9H+nGj17rPkMfpAHfKoDl//14IsuL0Zci6xn+ogw9dhaiuBDgaNglFjsf6/fEXFBzoXRnUddeghhYf1v6yx8P+cI/Pz6jlAbUBZfF5Z6qyvL6phzR3voZl66cdmIVFFs2VAskSXS X-Forefront-PRVS: 08118EFC2B X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM2PR02MB0420;23:O6Albygn4fMGTJ31pmIoO8O9qK6HqeOId1Iuey/ff?= =?us-ascii?Q?MQSJv11yFSVyN6OKpw62Zw+mhBG3J44trQgjzgkFsEAZn7gqscLXyxa+Zpr1?= =?us-ascii?Q?jP+JLlw8F58F1d1YKCaIym8xduIxyjZ9u/2hBAVwNEzJtPcRw5yOAaE2kDGx?= =?us-ascii?Q?iQ6W5cpbPzGgEElOaZe8Oo5K1i2rJlzctlVZiyLmliXDf8m1JdNvthcnYkiP?= =?us-ascii?Q?z7aoz9TL/ULGyzzmmyN37tmhCoZdYGsPROyXEOAM8zSv92wZXPMFDb1Ln9up?= =?us-ascii?Q?jcPzv4ckVHdotplkDiUz6rSv38wnHinqyNcHCqnhfWWKuT/+qxR7z+970pBZ?= =?us-ascii?Q?YXcXiHTCcfFPEKCAizAMarqe9AFJzUzBheD1s0cqpkhEt9m1k7zsjTRnWLGc?= =?us-ascii?Q?bStqBBteICv0zinU7rcmjsaegmI+c31UFWeZVUX1IknQAkSl82h9G2SMqvHn?= =?us-ascii?Q?ywysNwEf11jHHkIWjnucLCMqBLiy9IpgoWj3i4remHMQF5VIXveeY2o663H2?= =?us-ascii?Q?/DpyIDZMMshRzx19UTrtg/vKMnE4SmCgAW3+uZuIi/8XrccKXABUarAWDTUQ?= =?us-ascii?Q?970TjFI6Mn8esL3QfjmeVeZXLV44TzDA9gw9KvhYLCoZbTvFr34DqhCijiP3?= =?us-ascii?Q?mU7n4hGtNMU/j0BNuwQnEJlXnCJ99PYeNrnkIvbmqf4rSOKUwH9So2lRd/ls?= =?us-ascii?Q?Erb/Fig/fbOYmHLUvK8yIZwyAHzqg91O80OjxdK4P0jrCRE/uwLUIMUosH7l?= =?us-ascii?Q?U4fIYoXUm5pLJI0zXGXOuAry4cZJSVt3JeR4T3XkRMuYwXGDrYpG86CjO4zt?= =?us-ascii?Q?h1Fml7xOQxg3EOMY4Q5r2wGdyKVx3MzIUdyKpwBGrLCeSHF5B54SEb7bRQRa?= =?us-ascii?Q?YoY6s8WTDV4uhdWvy6V1GIRe9eZxWlZp+Akhho0uzDIrc2gHS1MSnL8BIMPL?= =?us-ascii?Q?gW9YKX3iARdSBeyScjwr2NN14sMYbUXXxGeIGUvVKtV095paD+jOhG2vRtG3?= =?us-ascii?Q?wAd7LMEk0u4+acEuY8EP6MatpOv+rYR4XMWPcotjxyu0G/LgzP7xFNwCAFmr?= =?us-ascii?Q?6C4jZvKeZMA4hfJ3u2m3ORBBvlclS7fA9JN6a+6w0+Dl5Gx2ghnSTY4dyiSb?= =?us-ascii?Q?ukbkjLH/Mg7kKyRg7ACniXQSJ73u3g89nRFZFUya6gxqTLVUHLr8Eu3SQ15H?= =?us-ascii?Q?+2MnxwomMITdiI=3D?= X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;5:LGByb6jnCFXtMPwTphZ4kLYoY2c+/8iGq03XEhZQdr7OTfrGgzK17FvqrYHB9Mf3HGCPSTFzgSB6up0Z7wZWY7b6TN159wczJ6sRrQXjwcrjB/VePCveDDi9gvqqUZ/beNA/GPflLSagmO/C/ntT0w==;24:kiNruWaFL/L8Im5i1yh7ybipqAd+svqUol0mGOiYFJT3xD8MO8QUNS3ZGBaQO11LlqOjJCWK36ikVoSUN40saSUJANuNb2ThXzoIyW1xCSE= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2016 19:35:40.5110 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3;Ip=[12.216.194.146];Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR02MB0420 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We need to call task_isolation_enter() from prepare_exit_to_usermode(), so that we can both ensure we do it last before returning to userspace, and we also are able to re-run signal handling, etc., if something occurs while task_isolation_enter() has interrupts enabled. To do this we add _TIF_NOHZ to the _TIF_WORK_MASK if we have CONFIG_TASK_ISOLATION enabled, which brings us into prepare_exit_to_usermode() on all return to userspace. But we don't put _TIF_NOHZ in the flags that we use to loop back and recheck, since we don't need to loop back only because the flag is set. Instead we unconditionally call task_isolation_enter() at the end of the loop if any other work is done. To make the assembly code continue to be as optimized as before, we renumber the _TIF flags so that both _TIF_WORK_MASK and _TIF_SYSCALL_WORK still have contiguous runs of bits in the immediate operand for the "and" instruction, as required by the ARM64 ISA. Since TIF_NOHZ is in both masks, it must be the middle bit in the contiguous run that starts with the _TIF_WORK_MASK bits and ends with the _TIF_SYSCALL_WORK bits. We tweak syscall_trace_enter() slightly to carry the "flags" value from current_thread_info()->flags for each of the tests, rather than doing a volatile read from memory for each one. This avoids a small overhead for each test, and in particular avoids that overhead for TIF_NOHZ when TASK_ISOLATION is not enabled. We instrument the smp_cross_call() routine so that it checks for isolated tasks and generates a suitable warning if we are about to disturb one of them in strict or debug mode. Finally, add an explicit check for STRICT mode in do_mem_abort() to handle the case of page faults. Signed-off-by: Chris Metcalf --- arch/arm64/include/asm/thread_info.h | 18 ++++++++++++------ arch/arm64/kernel/ptrace.c | 12 +++++++++--- arch/arm64/kernel/signal.c | 7 +++++-- arch/arm64/kernel/smp.c | 2 ++ arch/arm64/mm/fault.c | 4 ++++ 5 files changed, 32 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 90c7ff233735..94a98e9e29ef 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -103,11 +103,11 @@ static inline struct thread_info *current_thread_info(void) #define TIF_NEED_RESCHED 1 #define TIF_NOTIFY_RESUME 2 /* callback before returning to user */ #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ -#define TIF_NOHZ 7 -#define TIF_SYSCALL_TRACE 8 -#define TIF_SYSCALL_AUDIT 9 -#define TIF_SYSCALL_TRACEPOINT 10 -#define TIF_SECCOMP 11 +#define TIF_NOHZ 4 +#define TIF_SYSCALL_TRACE 5 +#define TIF_SYSCALL_AUDIT 6 +#define TIF_SYSCALL_TRACEPOINT 7 +#define TIF_SECCOMP 8 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_FREEZE 19 #define TIF_RESTORE_SIGMASK 20 @@ -125,9 +125,15 @@ static inline struct thread_info *current_thread_info(void) #define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_32BIT (1 << TIF_32BIT) -#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ +#define _TIF_WORK_LOOP_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE) +#ifdef CONFIG_TASK_ISOLATION +# define _TIF_WORK_MASK (_TIF_WORK_LOOP_MASK | _TIF_NOHZ) +#else +# define _TIF_WORK_MASK _TIF_WORK_LOOP_MASK +#endif + #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ _TIF_NOHZ) diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 1971f491bb90..69ed3ba81650 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -1240,14 +1241,19 @@ static void tracehook_report_syscall(struct pt_regs *regs, asmlinkage int syscall_trace_enter(struct pt_regs *regs) { - /* Do the secure computing check first; failures should be fast. */ + unsigned long work = ACCESS_ONCE(current_thread_info()->flags); + + if ((work & _TIF_NOHZ) && task_isolation_check_syscall(regs->syscallno)) + return -1; + + /* Do the secure computing check early; failures should be fast. */ if (secure_computing() == -1) return -1; - if (test_thread_flag(TIF_SYSCALL_TRACE)) + if (work & _TIF_SYSCALL_TRACE) tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER); - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + if (work & _TIF_SYSCALL_TRACEPOINT) trace_sys_enter(regs, regs->syscallno); audit_syscall_entry(regs->syscallno, regs->orig_x0, regs->regs[1], diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index fde59c1139a9..641c828653c7 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -419,10 +420,12 @@ asmlinkage void prepare_exit_to_usermode(struct pt_regs *regs, if (thread_flags & _TIF_FOREIGN_FPSTATE) fpsimd_restore_current_state(); + task_isolation_enter(); + local_irq_disable(); thread_flags = READ_ONCE(current_thread_info()->flags) & - _TIF_WORK_MASK; + _TIF_WORK_LOOP_MASK; - } while (thread_flags); + } while (thread_flags || !task_isolation_ready()); } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index b1adc51b2c2e..dcb3282d04a2 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -632,6 +633,7 @@ static const char *ipi_types[NR_IPI] __tracepoint_string = { static void smp_cross_call(const struct cpumask *target, unsigned int ipinr) { trace_ipi_raise(target, ipi_types[ipinr]); + task_isolation_debug_cpumask(target); __smp_cross_call(target, ipinr); } diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 92ddac1e8ca2..fbc78035b2af 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -466,6 +467,9 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, const struct fault_info *inf = fault_info + (esr & 63); struct siginfo info; + if (user_mode(regs)) + task_isolation_check_exception("%s at %#lx", inf->name, addr); + if (!inf->fn(addr, esr, regs)) return; -- 2.1.2