From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76FB2C433DF for ; Fri, 24 Jul 2020 03:03:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3130220737 for ; Fri, 24 Jul 2020 03:03:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Zd9/5EPx"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=marvell.com header.i=@marvell.com header.b="BVXchFXy"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.b="hOA3F8wX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3130220737 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Content-ID:In-Reply-To:References: Message-ID:Date:Subject:To:From:Reply-To:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=y1Vbjk+iehhuuAhz1ZUAGaQiXslRrU/unTQ72m0D7UY=; b=Zd9/5EPx+Dl780qEm9IDVRuZZ y6MmOyCsKNlNt9V/J4eGltyX3H27YfxY3b6tDEopFr2W69OyBB6VOE7HEmif/VwG0VoOqnEDe47uO j0C/3YCs0TQdQRQqTLUcOeAm0Ujft5SyBLGsGxvhiqv+BrpERFr9PKq/EmRGxsT7HStcM0jJyqHl6 KeeNJYwnkmc6umpe++DRjqcYAsiPsmZNNRxwCPUUi7u1fC472wCOlXCgkEZZbzhmBFYl+g3X5ng30 atXlkOOwNkcc4/ICGs3lgV5MY1WyC98ic50MXDKTXg4PoRRrbeyS/+Ip/8c8RFZRUU+cy0gIvVYMd KFxOykXXw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jynx7-0005oV-DW; Fri, 24 Jul 2020 03:00:53 +0000 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jynx4-0005o2-92 for linux-arm-kernel@lists.infradead.org; Fri, 24 Jul 2020 03:00:51 +0000 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06O2t7Cx020561; Thu, 23 Jul 2020 20:00:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=vYfsp45yYWNA/Lpc8SbdEh+FFnGkPC2DS9BBu/YbUMg=; b=BVXchFXyr7eM5VEtbHLMhFMlQutoBsrQtxTYgZf0qFkwfzUTNUvpveZtKsGJlaw2sk+k PcldHS3j4isUmngD9k0sSW0PcMCnyUni39TCXfGvRQH2Tre+f1ZMgTMjzt8W55GW5ABP 8QNEbm4Je8ucTbE4v97z6JH05OCeRDXqgJL95iIa8DpZJBsNokdlUq51xtOkgqNS+0Um v2nDCXE7NIUIkbkdClqetfEy6SG4F6geuxA01V5ExWN/9ZXrWVzVsMlbsZQoBoL+2m/J S0oPfAtrKa/E3WDq13qwQJsWXTsvx1fUpG2Y2oL9A1GGK71/YGLhIQ0Ar2BDLIQgEJkD Sg== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkytbw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 23 Jul 2020 20:00:07 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 23 Jul 2020 20:00:06 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.176) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Thu, 23 Jul 2020 20:00:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=REJfCxpjO1JFxTRnDjq3j+SyujCW3JUUgLEWcNB/qlqDO2sDjq1BCK45v9GwwdyNlGaeGXpEAZZMiPBvOODmDEh6wMHouGN6+scR+F4Eo6BY4dOW1G6fkrOOgD6d6uoplO5YRR+wuRPD8HNESj7/OlwMTb/Xf9Ntp7bdTQKnMYJHMX5iXjaZdLU+9UpyWZMIihdINP499lo2pmx3OozmrCFvydq3B42Lj1exRKIFWF5H7rPj3AT9uE4d60jNdOWVfKRFm1B/7JBAIk8mLLCxzY9SP1KSr2ST/lIMDqFnA90GfQzXen9MW1PCqckrunF2v1VomPFts6loDHuZ63HI4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vYfsp45yYWNA/Lpc8SbdEh+FFnGkPC2DS9BBu/YbUMg=; b=csUUvQJaeN2ZcTDrux/2e39b7ngur0wgguPeDaOBk5ucJHOi5G+np5m4jDVnWxXCzcy4s45vE5bUCXWfJuM6SfmS3lCSVlC+wOhk7fFo3oG8Am/zThxGN80Q/roT07k0I3U+C5zC2ySvmV2lwGObDJmYByu4eg6kI7MahB2M7AtjgFQO6EBanXb3+a3hCIpuh/g738SD7ge+SBtQGIJOFRh1YX8EYe12jXUzrHubalAAeI2fNJnEmubIaUBdB4Pr2QvYE3ispjQmuPPZbCL6F+rsN/CNSp878fST+t3t7OYSa8/t9T7XyI6fysr2cJLMuwDpof7KiidKm9u/SPqhCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vYfsp45yYWNA/Lpc8SbdEh+FFnGkPC2DS9BBu/YbUMg=; b=hOA3F8wXz/sB/JtRGYtcQpGcvJhuQHhf0I0ehTklert2uY2myvXUATUDDhHTv5Plf/isGsRL1H2MH8IlZqMLyH4zQDQe/cIvvTaW5aD8Q2Y8cRcHIuRU+HF0beTSnpUGChZ/B6uRd8nGpfmUBaAxjqPrpM5Ei3jRzLn+b9CBb+I= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MWHPR1801MB1872.namprd18.prod.outlook.com (2603:10b6:301:68::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.24; Fri, 24 Jul 2020 03:00:03 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Fri, 24 Jul 2020 03:00:03 +0000 From: Alex Belits To: "tglx@linutronix.de" , "peterz@infradead.org" Subject: Re: [EXT] Re: [PATCH v4 00/13] "Task_isolation" mode Thread-Topic: [EXT] Re: [PATCH v4 00/13] "Task_isolation" mode Thread-Index: AQHWYQSN+bm+zChO70Wsfble6+WsKakVT66AgAARDwCAAFIWAIAAWC+A Date: Fri, 24 Jul 2020 03:00:03 +0000 Message-ID: <851ee54e8317cd186338a76a045f738476144fcc.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> <87imeextf3.fsf@nanos.tec.linutronix.de> <831e023422aa0e4cb3da37ceef6fdcd5bc854682.camel@marvell.com> <20200723154933.GB709@worktop.programming.kicks-ass.net> <3ff1383e669b543462737b0d12c0d1fb7d409e3e.camel@marvell.com> <877dutx5xj.fsf@nanos.tec.linutronix.de> In-Reply-To: <877dutx5xj.fsf@nanos.tec.linutronix.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: linutronix.de; dkim=none (message not signed) header.d=none;linutronix.de; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 42f8da24-f141-4586-c20f-08d82f7da96f x-ms-traffictypediagnostic: MWHPR1801MB1872: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Mg1uhG+nb3XGujFKmdXdCzNLpFbknb2O+B47pwB7a59h4UII8LViKSavE/MA0lM5xcQmDDnOEg8CxhP2lGdsE0NVurfiL6kS0VJaonflW96WPLDSqsRnsTsMn7Osm4r5+GUnxDdktwM/4wSBFsu5k29SmbESpVzZBM1CSEprMA2wIGzv/4GW/6IsWMBkik6TCcInkKHm3l+P2hhKyPo/6aZr1i2WzntDrRH6b7qQJ1CbJmn9omVWYN5CfFLRLoOFZ/keR1MAlYpwVPSa1asql3jU5F5Wp5JZpwbn39usBfbs5UZDxgXs8cboloG8RlH5 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR18MB2267.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(39860400002)(396003)(376002)(346002)(136003)(2906002)(5660300002)(6506007)(71200400001)(478600001)(83380400001)(7416002)(64756008)(4326008)(66476007)(54906003)(86362001)(6512007)(8936002)(66446008)(66556008)(8676002)(26005)(2616005)(66946007)(316002)(76116006)(36756003)(6486002)(186003)(91956017)(110136005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: m0RkrEFXiV5CzMeGy/wkWsJeK7RgiR9rxvZMlvof9Hmf4pzuvJKSL71GiDjjAEM9X2UfmFuZ1cbmN0ydNzIX2iD0btlPmjJYmfJxu4Zptn6DiYOTpnmJ3XFOaEA0iQu15RgVUv+suQZuto/2o+GmsPtdjS6J4N2lNZ6ZVuqK7lP6itj+BFCvCTe3pWtxmrJ941dX43hJUjkXBTsJwzACDzUWyE8qqPnXSyrv27A/3J8/7Gp2Fh55D431+bP6D6zPltsW8hr+rdom7viX3H2e1A3rQTSuRP303mlxNcqcTLGgnv26OO/m+e1kq0V2BxieSmG9RKrnTDo2zGTuiDltjEgOTWTCFY7XpRjw1FGUFsMpogduB2lbLWzgpIsXGcFCJo7YIKPALUHiZ2GVveuhCyyaE97O60cP/MSGQ94qk6NRk1lER8If8hHMBYXJgBmHuarCa/Z0r5aoYup8SEhOvVnBcWP2yj1pYURvjvEmr6Q= Content-ID: <36CA89562C99C5498F674EEDF12CF56E@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42f8da24-f141-4586-c20f-08d82f7da96f X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jul 2020 03:00:03.1837 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 5FhgtPTVdDEMbWodDnr5xN/W4SKUwuKHUd5szGU0opgbvbmFKJpuYx9VlmeT7ETDRigEScki1AFCs0PapnJQ7w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1801MB1872 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-07-23_20:2020-07-23, 2020-07-23 signatures=0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200723_230050_421496_6D374E52 X-CRM114-Status: GOOD ( 33.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-arch@vger.kernel.org" , Prasun Kapoor , "frederic@kernel.org" , "linux-kernel@vger.kernel.org" , "rostedt@goodmis.org" , "davem@davemloft.net" , "netdev@vger.kernel.org" , "catalin.marinas@arm.com" , "linux-api@vger.kernel.org" , "will@kernel.org" , "mingo@kernel.org" , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 2020-07-23 at 23:44 +0200, Thomas Gleixner wrote: > External Email > > ------------------------------------------------------------------- > --- > Alex Belits writes: > > On Thu, 2020-07-23 at 17:49 +0200, Peter Zijlstra wrote: > > > 'What does noinstr mean? and why do we have it" -- don't dare > > > touch > > > the > > > entry code until you can answer that. > > > > noinstr disables instrumentation, so there would not be calls and > > dependencies on other parts of the kernel when it's not yet safe to > > call them. Relevant functions already have it, and I add an inline > > call > > to perform flags update and synchronization. Unless something else > > is > > involved, those operations are safe, so I am not adding anything > > that > > can break those. > > Sure. > > 1) That inline function can be put out of line by the compiler and > placed into the regular text section which makes it subject to > instrumentation > > 2) That inline function invokes local_irq_save() which is subject to > instrumentation _before_ the entry state for the instrumentation > mechanisms is established. > > 3) That inline function invokes sync_core() before important state > has > been established, which is especially interesting in NMI like > exceptions. > > As you clearly documented why all of the above is safe and does not > cause any problems, it's just me and Peter being silly, right? > > Try again. I don't think, accusations and mockery are really necessary here. I am trying to do the right thing here. In particular, I am trying to port the code that was developed on platforms that have not yet implemented those useful instrumentation safety features of x86 arch support. For most of the development time I had to figure out, where the synchronization can be safely inserted into kernel entry code on three platforms and tens of interrupt controller drivers, with some of those presenting unusual exceptions (forgive me the pun) from platform- wide conventions. I really appreciate the work you did cleaning up kernel entry procedures, my 5.6 version of this patch had to follow a much more complex and I would say, convoluted entry handling on x86, and now I don't have to do that, thanks to you. Unfortunately, most of my mental effort recently had to be spent on three things: 1. (small): finding a way to safely enable events and synchronize state on kernel entry, so it will not have a race condition between isolation-breaking kernel entry and an event that was disabled while the task was isolated. 2. (big): trying to derive any useful rules applicable to kernel entry in various architectures, finding that there is very little consistency across architectures, and whatever exists, can be broken by interrupt controller drivers that don't all follow the same rules as the rest of the platform. 3. (medium): introducing calls to synchronization on all kernel entry procedures, in places where it is guaranteed to not normally yet have done any calls to parts of the kernel that may be affected by "stale" state, and do it in a manner as consistent and generalized as possible. The current state of kernel entry handling on arm and arm64 architectures has significant differences from x86 and from each other. There is also a matter of interrupt controllers. As can be seen in interrupt controller-specific patch, I had to accommodate some variety of custom interrupt entry code. What can not be seen, is that I had to check that all other interrupt controller drivers and architecture- specific entry procedures, and find that they _do_ follow some understandable rules -- unfortunately architecture-specific and not documented in any manner. I have no valid reasons for complaining about it. I could not expect that authors of all kernel entry procedures would have any foreknowledge that someone at some point may have a reason to establish any kind of synchronization point for CPU cores. And this is why I had to do my research by manually drawing call trees and sequences, separately for every entry on every supported architecture, and across two or three versions of kernel, as those were changing along the way. The result of this may be not a "design" per se, but an understanding of how things are implemented, and what rules are being followed, so I could add my code in a manner consistent with what is done, and document the whole thing. Then there will be some written rules to check for, when anything of this kind will be necessary again (say, with TLB, but considering how much now is done in userspace, possibly to accommodate more exotic CPU features that may have state messed up by userspace). I am afraid, this task, kernel entry documentation, would take me some time, and I did not want to delay my task isolation patch for this reason. As I followed whatever rules I have determined to be applicable, I have produced code that introduces hooks in multiple seemingly unrelated to each other places. Whenever there was a, forgive me the pun, exception to those rules, another special case had to be handled. So no, I did not just add entry hooks randomly, and your accusations of having no design are unwarranted. My patches reflect what is already in code and in its design, I have added one simple rule that entry hook runs at the point when no dependency on something that requires synchronization, exists yet. The entry hook is small, you have already seen all of it while listing things that are not compatible with noinst. Its mechanism and purpose are explained in general description of task isolation. I don't think, I can provide a better explanation. I have to apologize for not taking into account all your carefully built instrumentation safety support. That was one thing I have missed. However at this point the only way for me to find out that I am wrong about it, and my code does not comply with expectations defined by advanced state of x86 architecture development, was to present whatever I could do right, based on experience with other platforms. I don't think, this warrants such hostility. Another issue that you have asked me to defend is the existence and scope of task isolation itself. I have provided long explanation in changelog and previous discussions of this patch, and before me so did Chris Metcalf and occasionally Yuri Norov. While I understand that this is an unusual feature and by its nature it affects kernel in multiple places, it does not deserve to be called a "mess" and other synonyms of "mess". It's an attempt to introduce a feature that turns Linux userspace into superior replacement of RTOS. Considering current state of CPU and SoC development, it is becoming very difficult even for vendor-specific RTOS to keep up with advanced hardware features. Linux keeps up with them just fine, however it lacks the ability to truly leave the CPU core alone, to run the performance-critical and latency- critical part of a task in a manner that RTOS user would expect. Very close but not yet. Task isolation provides the last step for this RTOS replacement. It is implemented in a manner that allows the user to combine Linux resource handling and initialization with RTOS isolation and latency. The decision about page faults is a part of this design, as well as many other decisions implemented in this code. Many may disagree with either those decisions, or the validity of a goal, some may even argue that it's a bad thing to provide a reason to stop RTOS development (I think, this is a good thing but that's not the point). However most definitely this is not a "mess", and it I do not believe that I have to defend the validity of this direction of development, or be accused of general incompetence every time someone finds a frustrating mistake in my code. As I said, I am trying to do the right thing, and want to bring my code not only to the state where x86 support is on par with other platforms (that is, working when instrumentation is disabled), but also make it fully compliant with current requirements of x86 platform. -- Alex _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel