From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3DED3C001DF for ; Fri, 14 Jul 2023 01:57:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:CC:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YNevv5YVafbA7x27j9TWIuPa1dB8A8a3a5cQlDo+sJA=; b=WLqS89m0okiM+b XkgDvU+906YfT0QHS8JD0xTUTZ0KJJ9sbXEr2gjBCV9JhznVRfieXiwnoGxlzaNUdm8GvUczE+U1s H3r01uEm5zGU44FTl0YP98IYcWzH2z9uurjxLVk+VYIxJVn/HORHxUe2/JRqnu78BLlAxqaWli+V+ EYHSzrp7+pZUZiMlgy2ybIj24pj1sz4ce9Kpcdgxs+w6ttuOVYmfanOdXWUc0Pg6EAEJST5UpLCJn Lc3imoKOq++LXAxWM3HS8kSFZWIMhMY2qUFyRVkUViyLjB8YkGo9ZD3YppV5NruGBNnTA7WXRHWYR SBgW9KJ6RbdeGIQdWN1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qK832-004kOM-1o; Fri, 14 Jul 2023 01:56:44 +0000 Received: from mx0a-0031df01.pphosted.com ([205.220.168.131]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qK82z-004kNu-31 for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 01:56:43 +0000 Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36E1IpSv022241; Fri, 14 Jul 2023 01:56:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=qcppdkim1; bh=hEFYY1f/3z8thoYRinO1hnRR66bF9a/LRoOeYNqzDMY=; b=YPbSZIoSrdxZkqRXooOpvyWJkztdrEDPuI4Z66GfYSPEULY2ZeMvqpFEnpBYGjrLHSqL zqDB4Io+Jp2P9fFBvhs4MtiEbmgxq3NT/kSdW4SKXZUMmMp1iXOVAMTHU3suR0sBBAfA BZXNQsGSElexvRQh1AB92FWKdYH1brROdoN1lo1BpSkAC5wDLXSl5LND/ex5H3gqWare i02FmI6vgxpo8S2ZPI18dNoapx4cChQLznty7Qd+ljEP13dV5OnwTykDKldFFi8S2rzF oLwNzhqP2SGl1I5J7lC9+rwQYxk8qM4boF/c52I6n6/YKVJcsOdoniQW+PMfOqkR/3E1 AA== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3rtptw0m0f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jul 2023 01:56:35 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 36E1uZM4011232 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Jul 2023 01:56:35 GMT Received: from [10.239.133.73] (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Thu, 13 Jul 2023 18:56:31 -0700 Message-ID: <6d1a6691-f858-71bf-97fe-97e13fcb93b6@quicinc.com> Date: Fri, 14 Jul 2023 09:56:27 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.11.2 Subject: Re: [PATCH] arm64: Add the arm64.nolse_atomics command line option To: Mark Rutland CC: Will Deacon , , , , , , , , , , , References: <20230710093751.GC32673@willie-the-truck> <5cf15f85-0397-96f7-4110-13494551b53b@quicinc.com> <20230711082226.GA1554@willie-the-truck> <84f0994a-26de-c20a-a32f-ec8fe41df3a3@quicinc.com> <20230711102510.GA1809@willie-the-truck> <67c2621f-4cad-2495-9785-7737246d3e90@quicinc.com> <604ac52d-4336-744f-2ab8-44d1c93fbaa8@quicinc.com> From: "Aiqun(Maria) Yu" In-Reply-To: X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: Z00QfEMrs_mqRIbQ_hrhNZh7whI9c5II X-Proofpoint-GUID: Z00QfEMrs_mqRIbQ_hrhNZh7whI9c5II X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-13_12,2023-07-13_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 adultscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 spamscore=0 clxscore=1015 impostorscore=0 mlxscore=0 mlxlogscore=979 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307140015 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230713_185641_980061_9C44878A X-CRM114-Status: GOOD ( 32.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 7/14/2023 3:08 AM, Mark Rutland wrote: > On Thu, Jul 13, 2023 at 10:08:34PM +0800, Aiqun(Maria) Yu wrote: >> On 7/13/2023 7:20 PM, Mark Rutland wrote: >>> Are you saying that LSE atomics to *cacheable* mappings do not work on your >>> system? >>> >>> Specifically, when using a Normal Inner-Shareable Inner-Writeback >>> Outer-Writeback mapping, do the LSE atomics work or not work? >> *cacheable* mapping have the LSE atomic is not working if far atomic is >> performed. > > Thanks for confirming; the fact that this doesn't work on *cacheable* memory is > definitely a major issue. I think everyone is confused here because of the > earlier mention of non-cachable accesses (which don't matter). > Maybe I can have the information collected in a summary to see if that helps. > I know that some CPU implementations have EL3 control bits to force LSE atomics > to be performed near (e.g. in Cortex-A55, the CPUECTLR.ATOM control bits), > which would avoid the issue while still allowing the LSE atomics to be used. > > If those can be configured in EL3 firmware, that would be a preferable > workaround. > > Can you say which CPUs are integrated in this system? and/or can you check if > such control bits exist? We have CPUECTLR_EL1.ATOM bit can force LSE atomics to be perform near. CPUECTLR_EL1 is also an option to EL1 kernel drivers to be configuarable. Try to a detailed summarise of the whole discussions, anyone can ignore some part if you are already know. * Part 1: Solution for this issue. While we still want to have options to let third party and end users can have options: 1.Disable lse atomic cap. 2.*Disallow* far atomic by "CPUECTLR_EL1.atom force near atomic" and non-cachable mappling for lse atomic only. * Part 2: Why we need the solution 1. There is also some case far atomic is better performance than near atomic. end user may still can still try to do allow far atomic. while this driver is also use kerenl LSE ATOMIC macro, so it can be running on cpu don't support lse atomic and cpu support lse atomic already. while current system, cpu have feature register said lse atomic is supported, but memory controller is not supported is currently not yet supported. 2. cpu feature of lse atomic capbility can be controled via options for the same image. Can have GKI(generic kernel Image) + same third party drivers Images support multi systems. -- *New system* fully support lse atomic -- *Intermidiate support system* which only have cpu support lse atomic, but have memory control/bus don't support lse atomic.* (mainly issue are discussed in this thread.) -- *old system* have cpu don't have this cpu feature at all. 3. better for debugging purpose, it would be easier for verify if it is this feature related or not. 4. *Disallow* from the developer side is not easy to control, expecially when they have the same code working on *old system* or *new system*, but failed on current *Intermidiate support system*. > > Thanks, > Mark. > Thx for discussion in details. -- Thx and BRs, Aiqun(Maria) Yu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mx0a-0031df01.pphosted.com ([205.220.168.131]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qK82z-004kNu-31 for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 01:56:43 +0000 Message-ID: <6d1a6691-f858-71bf-97fe-97e13fcb93b6@quicinc.com> Date: Fri, 14 Jul 2023 09:56:27 +0800 MIME-Version: 1.0 Subject: Re: [PATCH] arm64: Add the arm64.nolse_atomics command line option References: <20230710093751.GC32673@willie-the-truck> <5cf15f85-0397-96f7-4110-13494551b53b@quicinc.com> <20230711082226.GA1554@willie-the-truck> <84f0994a-26de-c20a-a32f-ec8fe41df3a3@quicinc.com> <20230711102510.GA1809@willie-the-truck> <67c2621f-4cad-2495-9785-7737246d3e90@quicinc.com> <604ac52d-4336-744f-2ab8-44d1c93fbaa8@quicinc.com> From: "Aiqun(Maria) Yu" In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+lwn-linux-arm-kernel=archive.lwn.net@lists.infradead.org List-Archive: To: Mark Rutland Cc: Will Deacon , corbet@lwn.net, catalin.marinas@arm.com, maz@kernel.org, quic_pkondeti@quicinc.com, quic_kaushalk@quicinc.com, quic_satyap@quicinc.com, quic_shashim@quicinc.com, quic_songxue@quicinc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Message-ID: <20230714015627.Vy7BzqojSDM-g9rZUTzB4fLfRChcNwdEeQHgvhqWXec@z> On 7/14/2023 3:08 AM, Mark Rutland wrote: > On Thu, Jul 13, 2023 at 10:08:34PM +0800, Aiqun(Maria) Yu wrote: >> On 7/13/2023 7:20 PM, Mark Rutland wrote: >>> Are you saying that LSE atomics to *cacheable* mappings do not work on your >>> system? >>> >>> Specifically, when using a Normal Inner-Shareable Inner-Writeback >>> Outer-Writeback mapping, do the LSE atomics work or not work? >> *cacheable* mapping have the LSE atomic is not working if far atomic is >> performed. > > Thanks for confirming; the fact that this doesn't work on *cacheable* memory is > definitely a major issue. I think everyone is confused here because of the > earlier mention of non-cachable accesses (which don't matter). > Maybe I can have the information collected in a summary to see if that helps. > I know that some CPU implementations have EL3 control bits to force LSE atomics > to be performed near (e.g. in Cortex-A55, the CPUECTLR.ATOM control bits), > which would avoid the issue while still allowing the LSE atomics to be used. > > If those can be configured in EL3 firmware, that would be a preferable > workaround. > > Can you say which CPUs are integrated in this system? and/or can you check if > such control bits exist? We have CPUECTLR_EL1.ATOM bit can force LSE atomics to be perform near. CPUECTLR_EL1 is also an option to EL1 kernel drivers to be configuarable. Try to a detailed summarise of the whole discussions, anyone can ignore some part if you are already know. * Part 1: Solution for this issue. While we still want to have options to let third party and end users can have options: 1.Disable lse atomic cap. 2.*Disallow* far atomic by "CPUECTLR_EL1.atom force near atomic" and non-cachable mappling for lse atomic only. * Part 2: Why we need the solution 1. There is also some case far atomic is better performance than near atomic. end user may still can still try to do allow far atomic. while this driver is also use kerenl LSE ATOMIC macro, so it can be running on cpu don't support lse atomic and cpu support lse atomic already. while current system, cpu have feature register said lse atomic is supported, but memory controller is not supported is currently not yet supported. 2. cpu feature of lse atomic capbility can be controled via options for the same image. Can have GKI(generic kernel Image) + same third party drivers Images support multi systems. -- *New system* fully support lse atomic -- *Intermidiate support system* which only have cpu support lse atomic, but have memory control/bus don't support lse atomic.* (mainly issue are discussed in this thread.) -- *old system* have cpu don't have this cpu feature at all. 3. better for debugging purpose, it would be easier for verify if it is this feature related or not. 4. *Disallow* from the developer side is not easy to control, expecially when they have the same code working on *old system* or *new system*, but failed on current *Intermidiate support system*. > > Thanks, > Mark. > Thx for discussion in details. -- Thx and BRs, Aiqun(Maria) Yu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel