From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1B4DC433FE for ; Sat, 5 Dec 2020 23:26:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BA8B2310E for ; Sat, 5 Dec 2020 23:26:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725933AbgLEX0a (ORCPT ); Sat, 5 Dec 2020 18:26:30 -0500 Received: from Galois.linutronix.de ([193.142.43.55]:55620 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725270AbgLEX03 (ORCPT ); Sat, 5 Dec 2020 18:26:29 -0500 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1607210746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OF/yxcOl24sCZMNn4pFXh470oPoxUajiPWkLNQAbERU=; b=qM/BWflaQxEXLQ8tIgzhYUuf1xoW7bdG4CvFJrmU0VDLsTE391cb0fqh1cULmil6JPkN5H CjM9f71zxVI6Sk6dpZdpMv1/Dmw/Ki6KPTrX209JefSyxBKR9gikRtVtEd2SDYLG5WPI34 6CcwxqoVWHvlbSCVv7N6hjn1WunbYGvZ0LdHs9tCkU7kmNqqKPy+2SusNfTxHqzjDH8xHD Y1qThnsAJvud0e+1SToR7En8O3qIzL05tGCeqq6uPq9qkX5Rgy3UCVdCx6Yovo0QV/UY4c vLzoVpeqNJv/og+qJE/8OA0XGq+IVUZNcsbicbnWcoQYUKcC8m2Dk6fgKORH0Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1607210746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OF/yxcOl24sCZMNn4pFXh470oPoxUajiPWkLNQAbERU=; b=F0Wpl6o+XvAhNAela/BGvtsgKXboiTn2PGy6xEd0AhejBgQ9J5NvG+DtqLBW5WYakJVmng 7IXzd1Pi4pU5X8CQ== To: Pavel Machek , Alex Belits Cc: "nitesh\@redhat.com" , "frederic\@kernel.org" , Prasun Kapoor , "linux-api\@vger.kernel.org" , "davem\@davemloft.net" , "trix\@redhat.com" , "mingo\@kernel.org" , "catalin.marinas\@arm.com" , "rostedt\@goodmis.org" , "linux-kernel\@vger.kernel.org" , "peterx\@redhat.com" , "linux-arch\@vger.kernel.org" , "mtosatti\@redhat.com" , "will\@kernel.org" , "peterz\@infradead.org" , "leon\@sidebranch.com" , "linux-arm-kernel\@lists.infradead.org" , "pauld\@redhat.com" , "netdev\@vger.kernel.org" Subject: Re: [PATCH v5 0/9] "Task_isolation" mode In-Reply-To: <20201205204049.GA8578@amd> References: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> <20201205204049.GA8578@amd> Date: Sun, 06 Dec 2020 00:25:45 +0100 Message-ID: <87h7oz96o6.fsf@nanos.tec.linutronix.de> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-api@vger.kernel.org Pavel, On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote: > So... what kind of guarantees does this aim to provide / what tasks it > is useful for? > > For real time response, we have other approaches. Depends on your requirements. Some problems are actually better solved with busy polling. See below. > If you want to guarantee performnace of the "isolated" task... I don't > see how that works. Other tasks on the system still compete for DRAM > bandwidth, caches, etc... Applications which want to run as undisturbed as possible. There is quite a range of those: - Hardware in the loop simulation is today often done with that crude approach of "offlining" a CPU and then instead of playing dead jumping to a preloaded bare metal executable. That's a horrible hack and impossible to debug, but gives them the results they need to achieve. These applications are well optimized vs. cache and memory foot print, so they don't worry about these things too much and they surely don't run on SMI and BIOS value add inflicted machines. Don't even think about waiting for an interrupt to achieve what these folks are doing. So no, there are problems which a general purpose realtime OS cannot solve ever. - HPC computations on large data sets. While the memory foot print is large the access patterns are cache optimized. The problem there is that any unnecessary IPI, tick interrupt or whatever nuisance is disturbing the carefully optimized cache usage and alone getting rid of the timer interrupt gained them measurable performance. Even very low single digit percentage of runtime saving is valuable for these folks because the compute time on such beasts is expensive. - Realtime guests in KVM. With posted interrupts and a fully populated host side page table there is no point in running host side interrupts or IPIs for random accounting or whatever purposes as they affect the latency in the guest. With all the side effects mitigated and a properly set up guest and host it is possible to get to a zero exit situation after the bootup phase which means pretty much matching bare metal behaviour. Yes, you can do that with e.g. Jailhouse as well, but you lose lots of the fancy things KVM provides. And people care about these not just because they are fancy. They care because their application scenario needs them. There are more reasons why people want to be able to get as much isolation from the OS as possible but at the same time have a sane execution environment, debugging, performance monitoring and the OS provided protection mechanisms instead of horrible hacks. Isolation makes sense for a range of applications and there is no reason why Linux should not support them. Thanks, tglx From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15FF8C433FE for ; Sat, 5 Dec 2020 23:27:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C483F2310B for ; Sat, 5 Dec 2020 23:27:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C483F2310B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linutronix.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References:In-Reply-To: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=K1FiD2Nbzw5Wi9fmHSsNFbFHtPkhTK0rNR5Xn+kgql8=; b=2jNBIPw20FevzFPt93YCeRfS2 1RzY5FVtRg5O8hkTKXvJeab90865CL3IwpyddLwLURhpSyNWw7rzaxOFPeB2Zmkxf2+31CSjXQ6eu pdtPz1SrwwiKKxeVmlSLc4KKmi29OVHYrgfDa6dlGBGBZCUHmWAXhjGVrddYiX/9opQsPuW2Zf9Vc Ilt1P8BwsPMilSTFIYMv/5y/gTJFKYaoLWCC3uJYhz4mZimS1yUMn30p/ZhMohzRTrXWnYW1EMzyx 0MS9gt1p2WNpalWWzjsM5RLrALEXSR72NPV8hV/zxO8oVTaGz+LZh4U8OHFnCkSoG3/LC5uXi+xw0 CFyy5JOTg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1klgw7-0001wH-51; Sat, 05 Dec 2020 23:25:55 +0000 Received: from galois.linutronix.de ([193.142.43.55]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1klgw1-0001vh-6O for linux-arm-kernel@lists.infradead.org; Sat, 05 Dec 2020 23:25:51 +0000 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1607210746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OF/yxcOl24sCZMNn4pFXh470oPoxUajiPWkLNQAbERU=; b=qM/BWflaQxEXLQ8tIgzhYUuf1xoW7bdG4CvFJrmU0VDLsTE391cb0fqh1cULmil6JPkN5H CjM9f71zxVI6Sk6dpZdpMv1/Dmw/Ki6KPTrX209JefSyxBKR9gikRtVtEd2SDYLG5WPI34 6CcwxqoVWHvlbSCVv7N6hjn1WunbYGvZ0LdHs9tCkU7kmNqqKPy+2SusNfTxHqzjDH8xHD Y1qThnsAJvud0e+1SToR7En8O3qIzL05tGCeqq6uPq9qkX5Rgy3UCVdCx6Yovo0QV/UY4c vLzoVpeqNJv/og+qJE/8OA0XGq+IVUZNcsbicbnWcoQYUKcC8m2Dk6fgKORH0Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1607210746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OF/yxcOl24sCZMNn4pFXh470oPoxUajiPWkLNQAbERU=; b=F0Wpl6o+XvAhNAela/BGvtsgKXboiTn2PGy6xEd0AhejBgQ9J5NvG+DtqLBW5WYakJVmng 7IXzd1Pi4pU5X8CQ== To: Pavel Machek , Alex Belits Subject: Re: [PATCH v5 0/9] "Task_isolation" mode In-Reply-To: <20201205204049.GA8578@amd> References: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> <20201205204049.GA8578@amd> Date: Sun, 06 Dec 2020 00:25:45 +0100 Message-ID: <87h7oz96o6.fsf@nanos.tec.linutronix.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201205_182550_310766_CCF11EC1 X-CRM114-Status: GOOD ( 11.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-arch@vger.kernel.org" , "nitesh@redhat.com" , "pauld@redhat.com" , "catalin.marinas@arm.com" , "peterz@infradead.org" , "linux-api@vger.kernel.org" , "frederic@kernel.org" , "mtosatti@redhat.com" , "linux-kernel@vger.kernel.org" , "rostedt@goodmis.org" , "mingo@kernel.org" , "leon@sidebranch.com" , "netdev@vger.kernel.org" , "peterx@redhat.com" , "trix@redhat.com" , Prasun Kapoor , "will@kernel.org" , "davem@davemloft.net" , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Pavel, On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote: > So... what kind of guarantees does this aim to provide / what tasks it > is useful for? > > For real time response, we have other approaches. Depends on your requirements. Some problems are actually better solved with busy polling. See below. > If you want to guarantee performnace of the "isolated" task... I don't > see how that works. Other tasks on the system still compete for DRAM > bandwidth, caches, etc... Applications which want to run as undisturbed as possible. There is quite a range of those: - Hardware in the loop simulation is today often done with that crude approach of "offlining" a CPU and then instead of playing dead jumping to a preloaded bare metal executable. That's a horrible hack and impossible to debug, but gives them the results they need to achieve. These applications are well optimized vs. cache and memory foot print, so they don't worry about these things too much and they surely don't run on SMI and BIOS value add inflicted machines. Don't even think about waiting for an interrupt to achieve what these folks are doing. So no, there are problems which a general purpose realtime OS cannot solve ever. - HPC computations on large data sets. While the memory foot print is large the access patterns are cache optimized. The problem there is that any unnecessary IPI, tick interrupt or whatever nuisance is disturbing the carefully optimized cache usage and alone getting rid of the timer interrupt gained them measurable performance. Even very low single digit percentage of runtime saving is valuable for these folks because the compute time on such beasts is expensive. - Realtime guests in KVM. With posted interrupts and a fully populated host side page table there is no point in running host side interrupts or IPIs for random accounting or whatever purposes as they affect the latency in the guest. With all the side effects mitigated and a properly set up guest and host it is possible to get to a zero exit situation after the bootup phase which means pretty much matching bare metal behaviour. Yes, you can do that with e.g. Jailhouse as well, but you lose lots of the fancy things KVM provides. And people care about these not just because they are fancy. They care because their application scenario needs them. There are more reasons why people want to be able to get as much isolation from the OS as possible but at the same time have a sane execution environment, debugging, performance monitoring and the OS provided protection mechanisms instead of horrible hacks. Isolation makes sense for a range of applications and there is no reason why Linux should not support them. Thanks, tglx _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel