From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20A7FC433EF for ; Sun, 17 Oct 2021 12:07:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB04360F9E for ; Sun, 17 Oct 2021 12:07:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DB04360F9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xZ3mofmavOVTndgDO8yYFKs6tgodAM1981nGVoCyERY=; b=XggcTxZJEHCsST INfZGhTGhw2GR9UrlTL1sWlgUPITvRwSaTEU6Pfp/2LVFd3r8KWrc/EYamSXM/UDfeWroRAI/x+3j hfBwT/xeR7C+otGTsLThMJ7UbcMb0HibPCOQDtK63UtSMOTO86rGBNAdetUxtttaPnoCrX89fWM2h O23lOwJaAQu/YMEZ3HFcXjhkm2ipEZNRfTTPDVRRvg2S9BD7EHvmI/edHY00lGvS1uoVMKHtQzfY6 FRE1s7hLErXKl+xG0rskX1Easv0YvikmFdTFuZ0PnPZEDLDitHyJ+fjYEauJK+ySSR4maLP1tuMuN 67EgJwsctv3URm4QgZ2g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc4vN-00CM1o-Gb; Sun, 17 Oct 2021 12:05:57 +0000 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc4vJ-00CM0p-Pm for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 12:05:55 +0000 Received: by mail-pg1-x530.google.com with SMTP id e7so13192348pgk.2 for ; Sun, 17 Oct 2021 05:05:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=YMV+GzdlocvpvMdKO0g2ndC3o57dFYn2DVsmYDcVJw4=; b=pRU8RRtKsWk0WpcImZG4W3fXbjiJY2nHrWDjCYI8HhsDdTrPrvrkJd/BJHBpq4BQ7H JXiDTmWAyA6+qKctCWn5LNlXz514PgqAgbquzygefoTA/AokpGZMgm7jAIJ3TQzzaquH lUFGa82/5DxGaysD0hNEUeFcWmrheYKoymwxhf7BHt7yX8c2oWxv+9mLV4mN/TSmWrMr ZQWpmaoPyqR85jYP87XvbiIan2sy0jk7XP6cRlQcZe4Jg0UNiLqRmBACrzQHJRkGd7eq Qxiq1Pvlf/8RTrMsmuCFnCDKzllp0NlGufHoYMewkYT9+BPoc0uEr0mTqDg+iNkZKE7J ojmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=YMV+GzdlocvpvMdKO0g2ndC3o57dFYn2DVsmYDcVJw4=; b=GcFkrkYGOHQuUM60yHmnUxlTherlz2k1s+7bw7Jr055V4hQ+EBRN6v5Z5erL8S+ZYH 6Q6yovPhkU8+korNWVgGEhM75efmbFjXzQdb9JP1eo2Uv1pe7MI8UxEYyA3kCXPHtige cIjDwyhfPZFueRZVVv6DBH/5XjU+749r9F2Ig0HAB+ekydhq6uNBdmZmH38HqxuOmJ8g 8uOQE9SXJIl5XKhj4b6O4d3qsaKfyuMqspcqmWo1NVdba4eYCtTVBkrWFd3MpBwvg6WL VmGDGeZIx6axBmJc4x88v375oYYcbbt6YvZmbhCw8sNUr67CTbG5MRA4lLL1AkZwRZSc 7WvA== X-Gm-Message-State: AOAM5318WLlkWQVFAQRX9VyG1XDa03xrxzL+vuWYdBi5949+SY1p9VPx zemfU8o8kX8lSqfZzQAoBmJzyg== X-Google-Smtp-Source: ABdhPJy5Ok8KRBDEMCPZbGvK466Q9d4+Iq5MOISjnLu/GA6jIg6D/l1BrGO5u726FO00wt29zBv6kw== X-Received: by 2002:a62:17d3:0:b0:44c:6022:9428 with SMTP id 202-20020a6217d3000000b0044c60229428mr22669719pfx.65.1634472352677; Sun, 17 Oct 2021 05:05:52 -0700 (PDT) Received: from leoy-ThinkPad-X240s ([148.163.172.147]) by smtp.gmail.com with ESMTPSA id u3sm9998398pfl.155.2021.10.17.05.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Oct 2021 05:05:52 -0700 (PDT) Date: Sun, 17 Oct 2021 20:05:46 +0800 From: Leo Yan To: German Gomez Cc: linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, John Garry , Will Deacon , Mathieu Poirier , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Mike Leach , linux-arm-kernel@lists.infradead.org, coresight@lists.linaro.org Subject: Re: [PATCH 4/5] perf arm-spe: Implement find_snapshot callback Message-ID: <20211017120546.GB130233@leoy-ThinkPad-X240s> References: <20210916154635.1525-1-german.gomez@arm.com> <20210916154635.1525-4-german.gomez@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210916154635.1525-4-german.gomez@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_050553_881785_61A0C630 X-CRM114-Status: GOOD ( 27.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Sep 16, 2021 at 04:46:34PM +0100, German Gomez wrote: [...] > +static int arm_spe_find_snapshot(struct auxtrace_record *itr, int idx, > + struct auxtrace_mmap *mm, unsigned char *data, > + u64 *head, u64 *old) > +{ > + int err; > + bool wrapped; > + struct arm_spe_recording *ptr = > + container_of(itr, struct arm_spe_recording, itr); > + > + /* > + * Allocate memory to keep track of wrapping if this is the first > + * time we deal with this *mm. > + */ > + if (idx >= ptr->wrapped_cnt) { > + err = arm_spe_alloc_wrapped_array(ptr, idx); > + if (err) > + return err; > + } > + > + /* > + * Check to see if *head has wrapped around. If it hasn't only the > + * amount of data between *head and *old is snapshot'ed to avoid > + * bloating the perf.data file with zeros. But as soon as *head has > + * wrapped around the entire size of the AUX ring buffer it taken. > + */ > + wrapped = ptr->wrapped[idx]; > + if (!wrapped && arm_spe_buffer_has_wrapped(data, mm->len, *head)) { > + wrapped = true; > + ptr->wrapped[idx] = true; > + } > + > + pr_debug3("%s: mmap index %d old head %zu new head %zu size %zu\n", > + __func__, idx, (size_t)*old, (size_t)*head, mm->len); > + > + /* > + * No wrap has occurred, we can just use *head and *old. > + */ > + if (!wrapped) > + return 0; > + > + /* > + * *head has wrapped around - adjust *head and *old to pickup the > + * entire content of the AUX buffer. > + */ > + if (*head >= mm->len) { > + *old = *head - mm->len; > + } else { > + *head += mm->len; > + *old = *head - mm->len; > + } > + > + return 0; > +} If run a test case (the test is pasted at the end of the reply), I can get quite different AUX trace data with passing different wait period before sending the first USR2 signal. # sh test_arm_spe_snapshot.sh 2 Couldn't synthesize bpf events. stress: info: [5768] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd [ perf record: Woken up 3 times to write data ] [ perf record: Captured and wrote 2.833 MB perf.data ] # sh test_arm_spe_snapshot.sh 10 Couldn't synthesize bpf events. stress: info: [5776] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd [ perf record: Woken up 3 times to write data ] [ perf record: Captured and wrote 24.356 MB perf.data ] The first command passes argument '2' so the test will wait for 2 seconds before send USR2 signal for snapshot, and the perf data file is 2.833 MB (so this means the Arm SPE trace data is about 2MB) for three snapshots. In the second command, the argument '10' means it will wait for 10 seconds before sending the USR2 signals, and every time it records the trace data from the full AUX buffer (8MB), at the end it gets 24MB AUX trace data. The issue happens in the second command, waiting for 10 seconds leads to the *full* AUX ring buffer is filled by Arm SPE, so the function arm_spe_buffer_has_wrapped() always return back true for this case. Afterwards, arm_spe_find_snapshot() doesn't respect the passed old header (from '*old') and assumes the trace data size is 'mm->len'. To allow arm_spe_buffer_has_wrapped() to work properly, I think we need to clean up the top 8 bytes of the AUX buffer in Arm SPE driver when start the PMU event (please note, this change has an assumption that is meantioned in another email that suggests to remove redundant PERF_RECORD_AUX events so the function arm_spe_perf_aux_output_begin() is invoked only once when start PMU event, so we can use the top 8 bytes in AUX buffer to indicate trace is wrap around or not). diff --git a/drivers/perf/arm_spe_pmu.c b/drivers/perf/arm_spe_pmu.c index d44bcc29d99c..eb35f85d0efb 100644 --- a/drivers/perf/arm_spe_pmu.c +++ b/drivers/perf/arm_spe_pmu.c @@ -493,6 +493,16 @@ static void arm_spe_perf_aux_output_begin(struct perf_output_handle *handle, if (limit) limit |= BIT(SYS_PMBLIMITR_EL1_E_SHIFT); + /* + * Cleanup the top 8 bytes for snapshot mode; these 8 bytes are + * used to indicate if trace data is wrap around if they are not + * zero. + */ + if (buf->snapshot) { + void *tail = buf->base + (buf->nr_pages << PAGE_SHIFT) - 8; + memset(tail, 0x0, 8); + } + limit += (u64)buf->base; base = (u64)buf->base + PERF_IDX2OFF(handle->head, buf); write_sysreg_s(base, SYS_PMBPTR_EL1); Thanks, Leo ---8<--- #!/bin/sh ./perf record -e arm_spe/period=148576/u -C 0 -S -m8M,8M -- taskset --cpu-list 0 stress --cpu 1 & PERFPID=$! echo "sleep $1 seconds" > /sys/kernel/debug/tracing/trace_marker # Wait for perf program sleep $1 # Send signal to snapshot trace data kill -USR2 $PERFPID sleep .03 kill -USR2 $PERFPID sleep .03 kill -USR2 $PERFPID echo "Stop snapshot" > /sys/kernel/debug/tracing/trace_marker kill $PERFPID wait $PERFPID _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel