From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E17BC352A1 for ; Tue, 6 Dec 2022 17:07:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9NJMmvHE689/rmyj4vVnLapQVJJuyDs4EK0EhvjAx/U=; b=T+OWdtqQvrxhYY y2cJnnBzvLf8q6ox2LKNVJ0qJgYWY+sWN6VWEiUYrZTzMayr190NldASKD5gvV91OKNJ6WyJFSZCW /Ao735YXsPEksc/yt58k7IJ4FJzoIfKDVmTpo+/TnwFsdCV3tIQxpoaEc+CJWES5LySB2W04zQX8k g3dVAwNfoy27OAf05xWhDdsiN9Onw2AiYZlITRUe7zTiLQ8lbYDPRw83MfEttz2ngCDen33SjSDwD uaClgLg6LU0OOYNZhMK3CPi79WHKVd3exwS3lRMAynbFo6W/wgvQx0ty7Y5CutrB3I82Y+cdKxLlk IBI0MtlXbRMva6oeXZ+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p2bOR-00FdV2-Oa; Tue, 06 Dec 2022 17:06:07 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p2bOO-00FdHJ-0I for linux-arm-kernel@lists.infradead.org; Tue, 06 Dec 2022 17:06:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0BD0223A; Tue, 6 Dec 2022 09:06:06 -0800 (PST) Received: from [10.32.36.24] (e121896.Emea.Arm.com [10.32.36.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D45D63F73B; Tue, 6 Dec 2022 09:05:56 -0800 (PST) Message-ID: Date: Tue, 6 Dec 2022 17:05:54 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Subject: Re: [PATCH V5 6/7] arm64/perf: Add BRBE driver To: Anshuman Khandual Cc: Mark Brown , Rob Herring , Marc Zyngier , Suzuki Poulose , Ingo Molnar , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-arm-kernel@lists.infradead.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, will@kernel.org, catalin.marinas@arm.com References: <20221107062514.2851047-1-anshuman.khandual@arm.com> <20221107062514.2851047-7-anshuman.khandual@arm.com> <25658a70-0b37-966d-e46c-f86be2a76a8e@arm.com> <54d532cc-089c-0c6c-6bf4-be840bc27826@arm.com> Content-Language: en-US From: James Clark In-Reply-To: <54d532cc-089c-0c6c-6bf4-be840bc27826@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221206_090604_188388_8C5D0F67 X-CRM114-Status: GOOD ( 20.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 30/11/2022 04:49, Anshuman Khandual wrote: > > > On 11/29/22 21:23, James Clark wrote: [...] > > The latest code (not posted), disables TRBE completely while reading the > branch records during PMU interrupt. Could you please apply those changes > as well, or rather just use the branch instead. > > https://gitlab.arm.com/linux-arm/linux-anshuman/-/commit/ab17879711f0e61c280ed52400ccde172b67e04a > >> >> armv8pmu_brbe_enable+0xc/arm64_pmu_brbe_enable+0x0/P/-/-/0/CALL >> armpmu_start+0xe0/armv8pmu_brbe_enable+0x0/P/-/-/0/IND_CALL >> armv8pmu_brbe_reset+0x18/armpmu_start+0xd0/P/-/-/0/RET >> arm64_pmu_brbe_reset+0x18/armv8pmu_brbe_reset+0x10/P/-/-/0/RET > > I am wondering how the different privilege branch record leak is possible. > Because arm64_pmu_brbe_enable() should start BRBE from a clean state with > respect to privilege level filters. > > void arm64_pmu_brbe_enable(struct pmu_hw_events *cpuc) > { > u64 brbfcr, brbcr; > > if (brbe_disabled(cpuc)) > return; > > brbfcr = read_sysreg_s(SYS_BRBFCR_EL1); > brbfcr &= ~BRBFCR_EL1_BANK_MASK; > brbfcr &= ~(BRBFCR_EL1_EnI | BRBFCR_EL1_PAUSED | BRBE_FCR_MASK); > brbfcr |= (cpuc->brbfcr & BRBE_FCR_MASK); > write_sysreg_s(brbfcr, SYS_BRBFCR_EL1); > isb(); > > brbcr = read_sysreg_s(SYS_BRBCR_EL1); > brbcr &= ~BRBE_CR_MASK; --> Contains BRBCR_EL1_E1BRE and BRBCR_EL1_E0BRE > brbcr |= BRBCR_EL1_FZP; > brbcr |= (BRBCR_EL1_TS_PHYSICAL << BRBCR_EL1_TS_SHIFT); > brbcr |= (cpuc->brbcr & BRBE_CR_MASK); > write_sysreg_s(brbcr, SYS_BRBCR_EL1); > isb(); Yes I tracked down the problem to here as well. I added a arm64_pmu_brbe_reset(cpuc) to the end of arm64_pmu_brbe_enable() and it fixes the issue. The problem is, without ensuring that BRBFCR_EL1_PAUSED is set, there is no way to write to both BRBFCR and BRBCR in either order without new records being produced based on a partial configuration. BRBFCR_EL1_PAUSED isn't set from the previous session, and there is no obvious place to add a paused at the end of the session with the current callbacks. So the easiest fix is to flush the old kernel samples after configuring both registers. I'm not sure what the BRBFCR_EL1_PAUSED value is at power on either, so the issue might also be present with the very first brbe session, but less obvious than a userspace one following a kernel one. But the flush solves that problem too. > } > > Could these samples are from a previous session ? But they should have been > flushed in armpmu_start(). > > static void armpmu_start(struct perf_event *event, int flags) > { > ......... > if (has_branch_stack(event)) { > if (event->ctx->task && hw_events->brbe_context != event->ctx) { > armpmu->brbe_reset(hw_events); > hw_events->brbe_context = event->ctx; > } > armpmu->brbe_enable(hw_events); > hw_events->brbe_users++; > } > ......... > } > >> >> >> [1]: >> https://gitlab.arm.com/linux-arm/linux-jc/-/commit/7260b7bef06ac161eac88d05266e8c5c303d9881 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel