From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ozlabs.org (ozlabs.org [IPv6:2401:3900:2:1::2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xW5b62PjnzDr0d for ; Mon, 14 Aug 2017 16:40:34 +1000 (AEST) Received: from localhost.localdomain (localhost [127.0.0.1]) by ozlabs.org (Postfix) with ESMTP id 3xW5b61T3fz9t3h for ; Mon, 14 Aug 2017 16:40:34 +1000 (AEST) Message-ID: <1502692833.9844.4.camel@neuling.org> Subject: Re: [PATCH] powerpc/64s: Add support for ASB_Notify on POWER9 From: Michael Neuling To: benh@au1.ibm.com, Christophe Lombard , linuxppc-dev@lists.ozlabs.org, fbarrat@linux.vnet.ibm.com, vaibhav@linux.vnet.ibm.com, andrew.donnellan@au1.ibm.com Date: Mon, 14 Aug 2017 16:40:33 +1000 In-Reply-To: <1501907321.2664.126.camel@au1.ibm.com> References: <1501858572-29641-1-git-send-email-clombard@linux.vnet.ibm.com> <1501907321.2664.126.camel@au1.ibm.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Sat, 2017-08-05 at 14:28 +1000, Benjamin Herrenschmidt wrote: > On Fri, 2017-08-04 at 16:56 +0200, Christophe Lombard wrote: > > The POWER9 core supports a new feature: ASB_Notify which requires the > > support of the Special Purpose Register: TIDR. > >=20 > > The ASB_Notify command, generated by the AFU, will attempt to > > wake-up the host thread identified by the particular LPID:PID:TID. > >=20 > > The special register TIDR has to be updated to with the same value > > present in the process element. > >=20 > > If the length of the register TIDR is 64bits, the CAPI Translation > > Service Layer core (XSL9) for Power9 systems limits the size (16bits) o= f > > the Thread ID when it generates the ASB_Notify message adding > > PID:LPID:TID information from the context. > >=20 > > The content of the internal kernel Thread ID (32bits) can not therefore > > be used to fulfill the register TIDR. > >=20 > > This patch allows to avoid this limitation by adding a new interface > > for the user. The instructions mfspr/mtspr SPRN_TIDR are emulated, > > save/restore SPRs (context switch) are updated and a new feature > > (CPU_FTR_TIDR) is added to POWER9 system. >=20 > Those CPU_FTR_* are internal to the kernel. You probably also need a > feature in AT_HWCAP2 to indicate to userspace that this is supported. >=20 > Also you put the onus of allocating the TIDs onto userspace which is a > bit tricky. What happens if there are duplicate TIDs for example ? (ie, > userspace doesn't allocate it or uses a library that spawns a thread) I tend to agree. I don't want userspace knowing anything about TIDR allocations. If we want userspace to receive one of these as_notifys, ther= e should be some abstract handle (like a file descriptor) that the kernel giv= es out. That handle should be associated with an LPID/PID/TID tuple by the ker= nel. This is similar to the PE number in CAPI. Most of the time userspace doesn'= t need to know its PE number (there is some cases were a master needs to know= a slave PE, but that's the exception, not the rule). Similarly, guests don't= know their LPID. Processes don't know their PID. Threads shouldn't know the TID= . Also, Suka has posted a patch that does TID allocation in the kernel...=C2= =A0 http://patchwork.ozlabs.org/patch/799494/ Regards, Mikey