From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B98FE36AE0; Mon, 15 Apr 2024 14:34:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713191667; cv=none; b=CUqF8HUJynpFW+3RZA84tyEV23HdtSp5e3fLb0pp0wUtBRlfi2q1Xp89EFXrCbeG0N5ukz2Bq90JSKEKc7a3fVeBV5j9dsA0dNHAKR/HVKDfZ8TLVV4s+04dNk671+d2vzABBmJ3GVoq2h+fs5W0wtdBBMiPXaKQHrjkdsNF3Us= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713191667; c=relaxed/simple; bh=ola6rwMPm3y8qLlDmhCNtT1nyjw4B0aP+hCuJYsZJSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UYFh3dbtzuRrqWfkUr8rRtnz+8ao8cvKvnWXbZjID6uejU1pdT6cCeDAUq128nQzhOoxT2cPef0yD3MZOrNAfnc0gaELmEZIcN4xB6iYmxe5K+8wumbSmMsY77vnV3FG4Qo/9eW7CBscM7HYKX6Bs9M1ok2LGcCnWMzSn8EHqV0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=KwBfcqTV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="KwBfcqTV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1C1EC2BD10; Mon, 15 Apr 2024 14:34:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1713191667; bh=ola6rwMPm3y8qLlDmhCNtT1nyjw4B0aP+hCuJYsZJSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KwBfcqTVQ4SbIjf1l5LmOGOzOaLNw9HD/BEPy7w+bSKeXdyVUgYoj6VjJPRvrjlmv 4msGxafozN9j8E4LliRtzHXt6khEbnVddS5dEHlgngI5eryBnrhuZV9tCAqb5QnZU6 EtUeF3nxMrb2aOUneK7kRFaVGClWOJLTGt4W4GhE= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Mathieu Desnoyers , "Masami Hiramatsu (Google)" , "Steven Rostedt (Google)" Subject: [PATCH 6.6 008/122] ring-buffer: Only update pages_touched when a new page is touched Date: Mon, 15 Apr 2024 16:19:33 +0200 Message-ID: <20240415141953.626926938@linuxfoundation.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240415141953.365222063@linuxfoundation.org> References: <20240415141953.365222063@linuxfoundation.org> User-Agent: quilt/0.67 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: stable@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.6-stable review patch. If anyone has any objections, please let me know. ------------------ From: Steven Rostedt (Google) commit ffe3986fece696cf65e0ef99e74c75f848be8e30 upstream. The "buffer_percent" logic that is used by the ring buffer splice code to only wake up the tasks when there's no data after the buffer is filled to the percentage of the "buffer_percent" file is dependent on three variables that determine the amount of data that is in the ring buffer: 1) pages_read - incremented whenever a new sub-buffer is consumed 2) pages_lost - incremented every time a writer overwrites a sub-buffer 3) pages_touched - incremented when a write goes to a new sub-buffer The percentage is the calculation of: (pages_touched - (pages_lost + pages_read)) / nr_pages Basically, the amount of data is the total number of sub-bufs that have been touched, minus the number of sub-bufs lost and sub-bufs consumed. This is divided by the total count to give the buffer percentage. When the percentage is greater than the value in the "buffer_percent" file, it wakes up splice readers waiting for that amount. It was observed that over time, the amount read from the splice was constantly decreasing the longer the trace was running. That is, if one asked for 60%, it would read over 60% when it first starts tracing, but then it would be woken up at under 60% and would slowly decrease the amount of data read after being woken up, where the amount becomes much less than the buffer percent. This was due to an accounting of the pages_touched incrementation. This value is incremented whenever a writer transfers to a new sub-buffer. But the place where it was incremented was incorrect. If a writer overflowed the current sub-buffer it would go to the next one. If it gets preempted by an interrupt at that time, and the interrupt performs a trace, it too will end up going to the next sub-buffer. But only one should increment the counter. Unfortunately, that was not the case. Change the cmpxchg() that does the real switch of the tail-page into a try_cmpxchg(), and on success, perform the increment of pages_touched. This will only increment the counter once for when the writer moves to a new sub-buffer, and not when there's a race and is incremented for when a writer and its preempting writer both move to the same new sub-buffer. Link: https://lore.kernel.org/linux-trace-kernel/20240409151309.0d0e5056@gandalf.local.home Cc: stable@vger.kernel.org Cc: Mathieu Desnoyers Fixes: 2c2b0a78b3739 ("ring-buffer: Add percentage of ring buffer full to wake up reader") Acked-by: Masami Hiramatsu (Google) Signed-off-by: Steven Rostedt (Google) Signed-off-by: Greg Kroah-Hartman --- kernel/trace/ring_buffer.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -1536,7 +1536,6 @@ static void rb_tail_page_update(struct r old_write = local_add_return(RB_WRITE_INTCNT, &next_page->write); old_entries = local_add_return(RB_WRITE_INTCNT, &next_page->entries); - local_inc(&cpu_buffer->pages_touched); /* * Just make sure we have seen our old_write and synchronize * with any interrupts that come in. @@ -1573,8 +1572,9 @@ static void rb_tail_page_update(struct r */ local_set(&next_page->page->commit, 0); - /* Again, either we update tail_page or an interrupt does */ - (void)cmpxchg(&cpu_buffer->tail_page, tail_page, next_page); + /* Either we update tail_page or an interrupt does */ + if (try_cmpxchg(&cpu_buffer->tail_page, &tail_page, next_page)) + local_inc(&cpu_buffer->pages_touched); } }