From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C552CD128A for ; Wed, 3 Apr 2024 13:03:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2zB4ywXHyLk0VpcyNJKbo65TCBEYwZoUVHrPNQHtNzA=; b=r6QM9Qsvgo7gqB 1o8KKFdTIC03snx9+3RnmWAGVJv0GMrHwaiuZmQnqCeujxWiQl1iGVGw52XNApAu/hkIV3hOi46Ea vOyLCpE0Rg1b2a9xJCi8dyoea6P1DqrCgo4sqlTjxBqMV09xLK5HpBoULPgdU+uAoJ/BPuzEkW185 BMlU4ABQpAjVmr7RSiTP+eCNSW/Q+i9bLrn2txFvkdoCgPmmaw0/hzn1ewSiKS2efTzjhZYZ+ei5Y zTu454YeMw75qJS9U+M+/fWGgqXU9vf7pgagqI7q9ORI8mD28Q+LYOwfCW8HcpbVhjTcYC6gaxnJK rnn+bgkY/nIsZj6so9zg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs0H7-0000000G4kb-03bp; Wed, 03 Apr 2024 13:03:33 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs0H2-0000000G4iJ-2qIZ for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 13:03:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3259D1007; Wed, 3 Apr 2024 06:03:56 -0700 (PDT) Received: from bogus (e103737-lin.cambridge.arm.com [10.1.197.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 28AB33F7B4; Wed, 3 Apr 2024 06:03:24 -0700 (PDT) Date: Wed, 3 Apr 2024 14:03:21 +0100 From: Sudeep Holla To: Jens Wiklander Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Marc Bonnici , Olivier Deprez , Lorenzo Pieralisi , Bertrand Marquis Subject: Re: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm Message-ID: References: <20240325081335.2326979-1-jens.wiklander@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_060328_828951_98DAC780 X-CRM114-Status: GOOD ( 42.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBNYXIgMjcsIDIwMjQgYXQgMTA6MjM6MzVBTSArMDEwMCwgSmVucyBXaWtsYW5kZXIg d3JvdGU6Cj4gT24gVHVlLCBNYXIgMjYsIDIwMjQgYXQgNDozNeKAr1BNIFN1ZGVlcCBIb2xsYSA8 c3VkZWVwLmhvbGxhQGFybS5jb20+IHdyb3RlOgo+ID4KPiA+IE9uIE1vbiwgTWFyIDI1LCAyMDI0 IGF0IDA5OjEzOjM1QU0gKzAxMDAsIEplbnMgV2lrbGFuZGVyIHdyb3RlOgo+ID4gPiBBZGQgc3Vw cG9ydCBmb3IgcnVubmluZyB0aGUgZHJpdmVyIGluIGEgZ3Vlc3QgdG8gYSBoeXBlcnZpc29yLiBU aGUgbWFpbgo+ID4gPiBkaWZmZXJlbmNlIGlzIGludHJvZHVjaW5nIG5vdGlmaWNhdGlvbiBwZW5k aW5nIGludGVycnVwdCBhbmQgdGhhdAo+ID4gPiBGRkFfTk9USUZJQ0FUSU9OX0JJVE1BUF9DUkVB VEUgZG9lc24ndCBuZWVkIHRvIGJlIGNhbGxlZC4KPiA+ID4KPiA+ID4gVGhlIGd1ZXN0IG1heSBu ZWVkIHRvIHVzZSBhIG5vdGlmaWNhdGlvbiBwZW5kaW5nIGludGVycnVwdCBpbnN0ZWFkIG9mIG9y Cj4gPiA+IGluIGFkZGl0aW9uIHRvIHRoZSBzY2hlZHVsZSByZWNlaXZlciBpbnRlcnJ1cHQuCj4g Pgo+ID4gVGhlIGFib3ZlIHN0YXRlbWVudCBtYWtlcyBtZSB3b3JyeSBhIGJpdCB0aGF0IHdlIGFy ZSBzdGlsbCBub3Qgb24gdGhlIHNhbWUKPiA+IHBhZ2UgYWJvdXQgTlBJIHZzIFNSSS4gTlBJIG5l ZWQgbm90IGV4aXN0IGluIGFkZGl0aW9uIHRvIFNSSS4gQW5kIGluIHYxCj4gPiB5b3UgZGlkIG1l bnRpb24geW91IGhhdmUgU1JJIGluIHRoZSBndWVzdCBhcyB3ZWxsLiBUaGVuIHdoeSBkbyB3ZSBu ZWVkCj4gPiBOUEkgaW4gYWRkaXRpb24gdG8gdGhhdC4gQXMgcGFydCBvZiBTUkksIHRoZSBjYWxs YmFjayAgZmZhX3NlbGZfbm90aWZfaGFuZGxlCj4gPiBnZXRzIHJlZ2lzdGVyZWQgYW5kIHdpbGwg YmUgY2FsbGVkIGFzIHBhcnQgb2YgU1JJIGhhbmRsaW5nLiBXaGF0IHlvdQo+ID4gZG8gaW4gIG5v dGlmX3BlbmRfaXJxX2hhbmRsZXIoKSwgZXhhY3RseSB3aGF0IGZmYV9zZWxmX25vdGlmX2hhbmRs ZSgpCj4gPiBhbHJlYWR5IGRvZXMuCj4gCj4gVGhhdCdzIG15IHVuZGVyc3RhbmRpbmcgb2Ygd2hh dCBhbiBOUEkgaGFuZGxlciBzaG91bGQgZG8gdG8gYmUgYWJsZSB0bwo+IHJlY2VpdmUgcGVyLXZD UFUgbm90aWZpY2F0aW9ucy4KPiAKPiA+Cj4gPiBJIGFtIHN0aWxsIHN0cnVnZ2xpbmcgdG8gdW5k ZXJzdGFuZCB0aGUgdXNlY2FzZSBoZXJlLiBJZiB5b3UganVzdCBoYXZlCj4gPiBOUEkgYW5kIG5v IFNSSSB3aGVuIHJ1bm5pbmcgdGhlIGRyaXZlciBpbiB0aGUgVk0sIHRoZW4gaXQgYWxpZ25zIHdp dGgKPiA+IG15IHVuZGVyc3RhbmRpbmcgb2YgcG9zc2libGUgdXNlLWNhc2Uobm90IHRoZSBvbmUg eW91IG1lbnRpb25lZCBpbiB2MToKPiA+IHdoZXJlIEZGLUEgZHJpdmVyIGluIFZNIHdpbGwgaGF2 ZSBTUkkgYXMgT1BURUUgaXMgdGhlIHNlY29uZGFyeSBzY2hlZHVsZXIpCj4gCj4gT1AtVEVFIGlz IG5vdCBhIHNlY29uZGFyeSBzY2hlZHVsZXIuIE9QLVRFRSAodGhlIFNQKSBpcyBzY2hlZHVsZWQg YXMKPiB1c3VhbCBieSB0aGUgbm9ybWFsIHdvcmxkIHVzaW5nIGRpcmVjdCByZXF1ZXN0LiBPUC1U RUUgZG9lc24ndCByZWNlaXZlCj4gRkYtQSBub3RpZmljYXRpb25zIGFuZCBJJ20gbm90IHN1cmUg aXQgd2lsbCBldmVyIGJlIG5lZWRlZC4KPgoKU29ycnkgZm9yIG15IHBvb3IgY2hvaWNlIG9mIHdv cmRzIHlldCBhZ2Fpbi4gSSBtZWFudCBWTSBrZXJuZWwocnVubmluZwphcyBOUyB2aXJ0dWFsIGVu ZHBvaW50KSB3aXRoIE9QVEVFIGRyaXZlciBydW5uaW5nIGluIGl0IGFzIHNlY29uZGFyeQpzY2hl ZHVsZXIuIElJVUMsIHRoZXJlIHdpbGwgYmUgYW5vdGhlciBpbnN0YW5jZSBvZiBPUFRFRSBkcml2 ZXIgaW4gdGhlCnByaW1hcnkgc2NoZWR1bGVyIGVuZHBvaW50KGkuZS4gaG9zdCBrZXJuZWwpIGFu ZCBpdCB3aWxsIHRha2UgY2FyZSBvZgpydW5uaW5nIFNSSSBoYW5kbGVyID8KCj4gPgo+ID4gSWYg d2UgYXJlIHN1cHBvcnRpbmcgTlBJIG9yIFNSSSwgSSB0aGluayB3ZSBjYW4gc2VlIGlmIHdlIGNh biBmdXJ0aGVyCj4gPiBzaW1wbGlmeSB0aGlzIGNoYW5nZSwgYnV0IEkgd2FudCB0byBnZXQgdG8g YW4gYWdyZWVtZW50IHdpdGggdXNhZ2UgbW9kZWwKPiA+IGJlZm9yZSB3ZSBkaWcgaW50byBpbXBs ZW1lbnRhdGlvbiBkZXRhaWxzIGluIHRoaXMgcGF0Y2guCj4KPiBUaGUgc3BlYyBkb2Vzbid0IGFz IGZhciBhcyBJIGtub3cgZXhwbGljaXRseSBtYWtlIE5QSSBhbmQgU1JJIG11dHVhbGx5Cj4gZXhj bHVzaXZlLCBpdCBkb2Vzbid0IG1ha2Ugc2Vuc2UgdG8gdXNlIGJvdGggaW4gYWxsIGNvbmZpZ3Vy YXRpb25zLgo+IEknbSB0cnlpbmcgdG8gYmUgYXMgZHluYW1pYyBhcyBwb3NzaWJsZSB3aGVuIGNv bmZpZ3VyaW5nIHRoZSBOUEkgYW5kCj4gU1JJIGhhbmRsZXJzLgo+CgpGYWlyIGVub3VnaAoKPiBJ ZiB0aGUga2VybmVsIGlzIGEgcGh5c2ljYWwgZW5kcG9pbnQsIGl0J3MgZWFzeSwgaXQgb25seSB1 c2VzIFNSSSBhbmQKPiB0aGUgU1BNQyB3aWxsIG5vdCBnaXZlIGFuIE5QSSB3aGVuIGFza2VkLgo+ CgpBZ3JlZWQuCgo+IElmIHRoZSBrZXJuZWwgaXMgYSB2aXJ0dWFsIGVuZHBvaW50IGl0IG1pZ2h0 IGJlIG1vcmUgY29tcGxpY2F0ZWQgc2luY2UKPiBhIFZNIG1heSBuZWVkIHRvIGFjdCBhcyBhIHNl Y29uZGFyeSBzY2hlZHVsZXIuIFRoYXQncyBub3QgZnVsbHkKPiBzdXBwb3J0ZWQgaW4gdGhpcyBw YXRjaCwgc2luY2UgaXQgY2FuIG9ubHkgc2NoZWR1bGUgaXRzZWxmLiBTUkkgaXMgbm90Cj4gdXNl ZCBpbiBteSBjdXJyZW50IGNvbmZpZ3VyYXRpb24uIElmIGEgaHlwZXJ2aXNvciBpbmplY3RzIGFu IFNSSSBJCj4gZXhwZWN0IGl0IHRvIGZpbHRlciB3aGF0J3MgcmV0dXJuZWQgYnkgRkZBX05PVElG SUNBVElPTl9JTkZPX0dFVCBmb3IKPiB0aGlzIFZNIHNvIGl0IGRvZXNuJ3QgaW50ZXJmZXJlIHdp dGggbm90aWZpY2F0aW9ucyBmb3Igb3RoZXIgVk1zLgo+CgpPSwoKPiBJbiBteSBjdXJyZW50IGNv bmZpZ3VyYXRpb24sIHRoZSBoeXBlcnZpc29yIHVzZXMgTlBJIHRvIHNpZ25hbCBwZW5kaW5nCj4g bm90aWZpY2F0aW9ucyB0byB0aGUgZ3Vlc3QuIEkgZG8gbm90IG5lZWQgYSBzZWNvbmRhcnkgc2No ZWR1bGVyIHNpbmNlCj4gT1AtVEVFIGRvZXNuJ3QgcmVjZWl2ZSBub3RpZmljYXRpb25zLiBBdCBh IGxhdGVyIHRpbWUsIHdlIG1heSBoYXZlIFNQcwo+IHRoYXQgbmVlZCB0byBiZSBzY2hlZHVsZWQs IGJ1dCB0aGF0J3Mgbm90IGEgcHJvYmxlbSBJJ20gdHJ5aW5nIHRvCj4gc29sdmUgaGVyZS4KClVu ZGVyc3Rvb2QuIEkgd2lsbCB0YWtlIGEgbG9vayBhdCB0aGUgcGF0Y2ggd2l0aCB0aGUgYWJvdmUg aW5mb3JtYXRpb24uCgotLQpSZWdhcmRzLApTdWRlZXAKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0Cmxp bnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A855D146D4A for ; Wed, 3 Apr 2024 13:03:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712149407; cv=none; b=cQ9LNWo64mgPfoUZZ7a/c+9sgrfyEaOQOYXXdbl6+LmBtDqN3CWy7BXfPVtFo1KbfDk4WLTN7qEkpLFBb+4aQQ5I8n0QKrdmvAFIvE/hlhiyFOrHH/CAPwSnHBtrsetiezwRkqoJsSE4NGvcZHXAn94Tp3b8mNVNz9ox9BGebpw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712149407; c=relaxed/simple; bh=kIOhYZcFjaultZWhhUObnw752oNQT7/I+/PK/rbt4Wg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oDHuAViTleXYS+srS+0AOpu3JmE3orba4Kxfnn6xA5dXpWdNhAxBbIx2+jNM99eCHA7doJcvbJAx6nyv6ePclKHnjVRXJvJoUlwm50nCbsIT10wvTbVd8meU9z5+gW5oXi9ZJQP97W3Y00m2RmaZ7OpzsBCQSScoFaR0JdaPA4M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3259D1007; Wed, 3 Apr 2024 06:03:56 -0700 (PDT) Received: from bogus (e103737-lin.cambridge.arm.com [10.1.197.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 28AB33F7B4; Wed, 3 Apr 2024 06:03:24 -0700 (PDT) Date: Wed, 3 Apr 2024 14:03:21 +0100 From: Sudeep Holla To: Jens Wiklander Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Marc Bonnici , Olivier Deprez , Lorenzo Pieralisi , Bertrand Marquis Subject: Re: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm Message-ID: References: <20240325081335.2326979-1-jens.wiklander@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Wed, Mar 27, 2024 at 10:23:35AM +0100, Jens Wiklander wrote: > On Tue, Mar 26, 2024 at 4:35 PM Sudeep Holla wrote: > > > > On Mon, Mar 25, 2024 at 09:13:35AM +0100, Jens Wiklander wrote: > > > Add support for running the driver in a guest to a hypervisor. The main > > > difference is introducing notification pending interrupt and that > > > FFA_NOTIFICATION_BITMAP_CREATE doesn't need to be called. > > > > > > The guest may need to use a notification pending interrupt instead of or > > > in addition to the schedule receiver interrupt. > > > > The above statement makes me worry a bit that we are still not on the same > > page about NPI vs SRI. NPI need not exist in addition to SRI. And in v1 > > you did mention you have SRI in the guest as well. Then why do we need > > NPI in addition to that. As part of SRI, the callback ffa_self_notif_handle > > gets registered and will be called as part of SRI handling. What you > > do in notif_pend_irq_handler(), exactly what ffa_self_notif_handle() > > already does. > > That's my understanding of what an NPI handler should do to be able to > receive per-vCPU notifications. > > > > > I am still struggling to understand the usecase here. If you just have > > NPI and no SRI when running the driver in the VM, then it aligns with > > my understanding of possible use-case(not the one you mentioned in v1: > > where FF-A driver in VM will have SRI as OPTEE is the secondary scheduler) > > OP-TEE is not a secondary scheduler. OP-TEE (the SP) is scheduled as > usual by the normal world using direct request. OP-TEE doesn't receive > FF-A notifications and I'm not sure it will ever be needed. > Sorry for my poor choice of words yet again. I meant VM kernel(running as NS virtual endpoint) with OPTEE driver running in it as secondary scheduler. IIUC, there will be another instance of OPTEE driver in the primary scheduler endpoint(i.e. host kernel) and it will take care of running SRI handler ? > > > > If we are supporting NPI or SRI, I think we can see if we can further > > simplify this change, but I want to get to an agreement with usage model > > before we dig into implementation details in this patch. > > The spec doesn't as far as I know explicitly make NPI and SRI mutually > exclusive, it doesn't make sense to use both in all configurations. > I'm trying to be as dynamic as possible when configuring the NPI and > SRI handlers. > Fair enough > If the kernel is a physical endpoint, it's easy, it only uses SRI and > the SPMC will not give an NPI when asked. > Agreed. > If the kernel is a virtual endpoint it might be more complicated since > a VM may need to act as a secondary scheduler. That's not fully > supported in this patch, since it can only schedule itself. SRI is not > used in my current configuration. If a hypervisor injects an SRI I > expect it to filter what's returned by FFA_NOTIFICATION_INFO_GET for > this VM so it doesn't interfere with notifications for other VMs. > OK > In my current configuration, the hypervisor uses NPI to signal pending > notifications to the guest. I do not need a secondary scheduler since > OP-TEE doesn't receive notifications. At a later time, we may have SPs > that need to be scheduled, but that's not a problem I'm trying to > solve here. Understood. I will take a look at the patch with the above information. -- Regards, Sudeep