Re: [PATCH 4/4] ring-buffer: Fix uninitialized read_stamp

Re: [PATCH 4/4] ring-buffer: Fix uninitialized read_stamp

[Steven Rostedt]

On Thu, 2012-06-28 at 19:16 -0400, Steven Rostedt wrote:
> From: Steven Rostedt <srostedt@redhat.com>
> 
> The ring buffer reader page is used to swap a page from the writable
> ring buffer. If the writer happens to be on that page, it ends up on the
> reader page, but will simply move off of it, back into the writable ring
> buffer as writes are added.
> 
> The time stamp passed back to the readers is stored in the cpu_buffer per
> CPU descriptor. This stamp is updated when a swap of the reader page takes
> place, and it reads the current stamp from the page taken from the writable
> ring buffer. Everytime a writer goes to a new page, it updates the time stamp
> of that page.
> 
> The problem happens if a reader reads a page from an empty per CPU ring buffer.
> If the buffer is empty, the swap still takes place, placing the writer at the
> start of the reader page. If at a later time, a write happens, it updates the
> page's time stamp and continues. But the problem is that the read_stamp does
> not get updated, because the page was already swapped.
> 
> The solution to this was to not swap the page if the ring buffer happens to
> be empty. This also removes the side effect that the writes on the reader
> page will not get updated because the writer never gets back on the reader
> page without a swap. That is, if a read happens on an empty buffer, but then
> no reads happen for a while. If a swap took place, and the writer were to start
> writing a lot of data (function tracer), it will start overflowing the ring buffer
> and overwrite the older data. But because the writer never goes back onto the
> reader page, the data left on the reader page never gets overwritten. This
> causes the reader to see really old data, followed by a jump to newer data.
> 
> Link: http://lkml.kernel.org/r/1340060577-9112-1-git-send-email-dhsharp@google.com
> Google-Bug-Id: 6410455
> Reported-by: David Sharp <dhsharp@google.com>
> tested-by: David Sharp <dhsharp@google.com>
> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

I'm starting to consider that this patch should be in stable.

Ingo, should I push this to urgent?

-- Steve

> ---
>  kernel/trace/ring_buffer.c |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
> index 1d0f6a8..82a3e0c 100644
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@ -3239,6 +3239,10 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer)
>  	if (cpu_buffer->commit_page == cpu_buffer->reader_page)
>  		goto out;
>  
> +	/* Don't bother swapping if the ring buffer is empty */
> +	if (rb_num_of_entries(cpu_buffer) == 0)
> +		goto out;
> +
>  	/*
>  	 * Reset the reader page to size zero.
>  	 */

Re: [PATCH 4/4] ring-buffer: Fix uninitialized read_stamp

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> On Thu, 2012-06-28 at 19:16 -0400, Steven Rostedt wrote:
> > From: Steven Rostedt <srostedt@redhat.com>
> > 
> > The ring buffer reader page is used to swap a page from the writable
> > ring buffer. If the writer happens to be on that page, it ends up on the
> > reader page, but will simply move off of it, back into the writable ring
> > buffer as writes are added.
> > 
> > The time stamp passed back to the readers is stored in the cpu_buffer per
> > CPU descriptor. This stamp is updated when a swap of the reader page takes
> > place, and it reads the current stamp from the page taken from the writable
> > ring buffer. Everytime a writer goes to a new page, it updates the time stamp
> > of that page.
> > 
> > The problem happens if a reader reads a page from an empty per CPU ring buffer.
> > If the buffer is empty, the swap still takes place, placing the writer at the
> > start of the reader page. If at a later time, a write happens, it updates the
> > page's time stamp and continues. But the problem is that the read_stamp does
> > not get updated, because the page was already swapped.
> > 
> > The solution to this was to not swap the page if the ring buffer happens to
> > be empty. This also removes the side effect that the writes on the reader
> > page will not get updated because the writer never gets back on the reader
> > page without a swap. That is, if a read happens on an empty buffer, but then
> > no reads happen for a while. If a swap took place, and the writer were to start
> > writing a lot of data (function tracer), it will start overflowing the ring buffer
> > and overwrite the older data. But because the writer never goes back onto the
> > reader page, the data left on the reader page never gets overwritten. This
> > causes the reader to see really old data, followed by a jump to newer data.
> > 
> > Link: http://lkml.kernel.org/r/1340060577-9112-1-git-send-email-dhsharp@google.com
> > Google-Bug-Id: 6410455
> > Reported-by: David Sharp <dhsharp@google.com>
> > tested-by: David Sharp <dhsharp@google.com>
> > Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
> 
> I'm starting to consider that this patch should be in stable.
> 
> Ingo, should I push this to urgent?

Yeah, probably makes sense to do so, especially as it's rather 
small.

Thanks,

	Ingo

Re: [PATCH 4/4] ring-buffer: Fix uninitialized read_stamp

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > On Thu, 2012-06-28 at 19:16 -0400, Steven Rostedt wrote:
> > > From: Steven Rostedt <srostedt@redhat.com>
> > > 
> > > The ring buffer reader page is used to swap a page from the writable
> > > ring buffer. If the writer happens to be on that page, it ends up on the
> > > reader page, but will simply move off of it, back into the writable ring
> > > buffer as writes are added.
> > > 
> > > The time stamp passed back to the readers is stored in the cpu_buffer per
> > > CPU descriptor. This stamp is updated when a swap of the reader page takes
> > > place, and it reads the current stamp from the page taken from the writable
> > > ring buffer. Everytime a writer goes to a new page, it updates the time stamp
> > > of that page.
> > > 
> > > The problem happens if a reader reads a page from an empty per CPU ring buffer.
> > > If the buffer is empty, the swap still takes place, placing the writer at the
> > > start of the reader page. If at a later time, a write happens, it updates the
> > > page's time stamp and continues. But the problem is that the read_stamp does
> > > not get updated, because the page was already swapped.
> > > 
> > > The solution to this was to not swap the page if the ring buffer happens to
> > > be empty. This also removes the side effect that the writes on the reader
> > > page will not get updated because the writer never gets back on the reader
> > > page without a swap. That is, if a read happens on an empty buffer, but then
> > > no reads happen for a while. If a swap took place, and the writer were to start
> > > writing a lot of data (function tracer), it will start overflowing the ring buffer
> > > and overwrite the older data. But because the writer never goes back onto the
> > > reader page, the data left on the reader page never gets overwritten. This
> > > causes the reader to see really old data, followed by a jump to newer data.
> > > 
> > > Link: http://lkml.kernel.org/r/1340060577-9112-1-git-send-email-dhsharp@google.com
> > > Google-Bug-Id: 6410455
> > > Reported-by: David Sharp <dhsharp@google.com>
> > > tested-by: David Sharp <dhsharp@google.com>
> > > Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
> > 
> > I'm starting to consider that this patch should be in stable.
> > 
> > Ingo, should I push this to urgent?
> 
> Yeah, probably makes sense to do so, especially as it's rather 
> small.

FYI, I have cherry picked it over into perf/urgent:

01c4359c155e ring-buffer: Fix uninitialized read_stamp


Thanks,

	Ingo