From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3AE3EE80A8C for ; Wed, 27 Sep 2023 05:52:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=6pLsyfdwHKKj37BNiw7P6j8WWAEdyuOs9yg5U9v/4O0=; b=szBTjK3bfI4PMA FQHvPcrf+31kGDKbE7J7MbZb077LBfzesdJfTeY5p4MDNNJOSVXhdOK5lPS2S1I0tUlw4YFDwt2j7 DfRBileqKV9nBpv9VZBjfrs/UU4YqgWhsklDwog4GVVIuA6fIfH0jZ2hukJBBoC37eG4WFRJgiwkn jEX1PdH2ju0PchgLpL3c/Fh/g5iaVDMXPJfhmvk7Bh1md21XiQubiYRthMEJncd7vGbC+w+DcTvhG DYWGaDz91u44iIvWhaBvpQKtBm5aHS2W8L2SJncHevJ5K8PtBDKMvcAWibr4c5j/e1GlQvp/UJlIT Vp6PZtSCqzUp3HvVcFHg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qlNSa-0002Bq-23; Wed, 27 Sep 2023 05:51:44 +0000 Received: from smtpbguseast3.qq.com ([54.243.244.52]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qlNSW-00029S-2z for linux-arm-kernel@lists.infradead.org; Wed, 27 Sep 2023 05:51:43 +0000 X-QQ-mid: bizesmtp80t1695793870t6nf2sfk Received: from localhost.localdomain ( [125.76.217.162]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 27 Sep 2023 13:51:08 +0800 (CST) X-QQ-SSF: 01400000000000D0H000000A0000000 X-QQ-FEAT: WZt+XiBtOcQ6wwqIH9+AjvE1k9s7W3/uJqFoy6lWAP9E+XxdgDJ+g6W1LJLty um3oa0vfR744Jbs46eUnL7iVxXFX7tO78fgtQ67+Vhi6Qz/EjMaOjaG0kI3cmahgnSH/9S0 uz3vVH+iId1qeakcFm2kv/7qHmKmD/j9GquSHtViqltiHBEUZCMn8h7ZPrBlfj1WCXhXPOI JLdj9lhqsjICn5S7VzlkH7Y2NZh/k2yE9TkjB+KQ7yWJXXt/jB5Nd15hcyeHB3kwTq3OEmd UdsMRTAM3EtNy1T/ovPdJCdx0BdESgFdUjrSTJZrrG3DJIR5CL+dgk+SG3It6XxZ91KJDkW XY+EAsXv2oiA0zpafao9w4wvQkuATlrSv000VdT8et7FDTWblgRd1xYi9dzmw== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 18351507345679951802 From: Guo Hui To: catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org Cc: quic_jiles@quicinc.com, mark.rutland@arm.com, wangxiaohua@uniontech.com, Guo Hui Subject: [PATCH] arm64: mm: Optimize querying asid from reserved_asids Date: Wed, 27 Sep 2023 13:51:06 +0800 Message-Id: <20230927055106.14635-1-guohui@uniontech.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:uniontech.com:qybglogicsvrsz:qybglogicsvrsz4a-0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230926_225141_456306_E0ECC049 X-CRM114-Status: GOOD ( 17.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move reserved_asids updates into function flush_context. When asid_generation increases, reserved_asids are updated synchronously. The execution frequency of function flush_context is far less than that of function check_update_reserved_asid. In function check_update_reserved_asid, you only need to query whether it is in reserved_asids based on the new newasid, and there is no need to update it. In the function check_update_reserved_asid, among all the times reserved_asids are hit, the probability that newasid is equal to the reserved_asids of the current CPU is about greater than 70%. Signed-off-by: Guo Hui --- arch/arm64/mm/context.c | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 188197590fc9..76e0beb14466 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -101,7 +101,7 @@ static void set_reserved_asid_bits(void) #define asid_gen_match(asid) \ (!(((asid) ^ atomic64_read(&asid_generation)) >> asid_bits)) -static void flush_context(void) +static void flush_context(u64 generation) { int i; u64 asid; @@ -120,6 +120,8 @@ static void flush_context(void) */ if (asid == 0) asid = per_cpu(reserved_asids, i); + + asid = generation | (asid & ~ASID_MASK); __set_bit(ctxid2asid(asid), asid_map); per_cpu(reserved_asids, i) = asid; } @@ -131,24 +133,21 @@ static void flush_context(void) cpumask_setall(&tlb_flush_pending); } -static bool check_update_reserved_asid(u64 asid, u64 newasid) +static bool check_update_reserved_asid(u64 newasid) { - int cpu; + int cpu, cur_cpu = smp_processor_id(); bool hit = false; - /* - * Iterate over the set of reserved ASIDs looking for a match. - * If we find one, then we can update our mm to use newasid - * (i.e. the same ASID in the current generation) but we can't - * exit the loop early, since we need to ensure that all copies - * of the old ASID are updated to reflect the mm. Failure to do - * so could result in us missing the reserved ASID in a future - * generation. - */ + if (per_cpu(reserved_asids, cur_cpu) == newasid) + return true; + for_each_possible_cpu(cpu) { - if (per_cpu(reserved_asids, cpu) == asid) { + if (cpu == cur_cpu) + continue; + + if (per_cpu(reserved_asids, cpu) == newasid) { hit = true; - per_cpu(reserved_asids, cpu) = newasid; + break; } } @@ -168,7 +167,7 @@ static u64 new_context(struct mm_struct *mm) * If our current ASID was active during a rollover, we * can continue to use it and this was just a false alarm. */ - if (check_update_reserved_asid(asid, newasid)) + if (check_update_reserved_asid(newasid)) return newasid; /* @@ -201,7 +200,7 @@ static u64 new_context(struct mm_struct *mm) /* We're out of ASIDs, so increment the global generation count */ generation = atomic64_add_return_relaxed(ASID_FIRST_VERSION, &asid_generation); - flush_context(); + flush_context(generation); /* We have more ASIDs than CPUs, so this will always succeed */ asid = find_next_zero_bit(asid_map, NUM_USER_ASIDS, 1); -- 2.20.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel