All of lore.kernel.org
 help / color / mirror / Atom feed
From: Wen Gong <wgong@codeaurora.org>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Brian Norris <briannorris@chromium.org>,
	linux-wireless <linux-wireless@vger.kernel.org>,
	ath10k <ath10k@lists.infradead.org>
Subject: Re: [PATCH] ath10k: change len of trace_ath10k_log_dbg_dump for large buffer size
Date: Wed, 10 Feb 2021 10:01:57 +0800	[thread overview]
Message-ID: <9b479a88331dbf969f07708eabe53d14@codeaurora.org> (raw)
In-Reply-To: <20210209163431.11133472@gandalf.local.home>

On 2021-02-10 05:34, Steven Rostedt wrote:
> On Tue, 9 Feb 2021 14:55:31 -0500
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
>> > [for-next][PATCH 2/2] tracing: Use temp buffer when filtering events
>> > https://lore.kernel.org/lkml/f16b14066317f6a926b6636df6974966@codeaurora.org/
>> 
>> Note, that is only used when filtering happens, which doesn't appear 
>> to be
>> the case here.
> 
> I was basing this off of the original commands, but the stack dump says
> otherwise. But it should still work.
> 
>> 
>> >
>> > It seems like we should still try to get that fixed somehow, even if
>> > the below change is fine on its own (it probably doesn't make sense to
>> > such a large amount of data via tracepoints). It would be unfortunate
>> > for next poor soul to hit the same issues, just because they wanted to
>> > dump a few KB.
>> 
>> Yeah, it was a design decision to cap the max size of events to just 
>> under
>> PAGE_SIZE. The ring buffer is broken up into pages (for zero copy
>> transfers to file systems and the network). Thus, no event is allowed 
>> to be
>> bigger than a page (and actually a bit smaller)
>> 
>> That said, it shouldn't have crashed, it should have just failed to 
>> record.
>> 
>> I'll test it out and see why it crashed.
> 
> Looking at the original report, I see:
> 
>  CPU: 1 PID: 141 Comm: kworker/u16:1 Not tainted 4.19.139 #162
> 
> Does this still crash on the latest kernel?
> 
> -- Steve
Not tested with latest kernel.
The reason is below which I said in 
https://lore.kernel.org/lkml/b504b3d7e989cae108669a0cd3072454@codeaurora.org/

the per cpu buffer seems it is initilized in 
trace_buffered_event_enable,
it is only 1 page size as below:
void trace_buffered_event_enable(void)
{
...
	for_each_tracing_cpu(cpu) {
		page = alloc_pages_node(cpu_to_node(cpu),
					GFP_KERNEL | __GFP_NORETRY, 0);
If the size of buffer to trace is more than 1 page, such as 46680, then
it trigger kernel crash/panic in my case while run trace-cmd.
After debugging, the trace_file->flags in
trace_event_buffer_lock_reserve is 0x40b while run trace-cmd, and it is
0x403 while collecting ftrace log.

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

WARNING: multiple messages have this Message-ID (diff)
From: Wen Gong <wgong@codeaurora.org>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Brian Norris <briannorris@chromium.org>,
	ath10k <ath10k@lists.infradead.org>,
	linux-wireless <linux-wireless@vger.kernel.org>
Subject: Re: [PATCH] ath10k: change len of trace_ath10k_log_dbg_dump for large buffer size
Date: Wed, 10 Feb 2021 10:01:57 +0800	[thread overview]
Message-ID: <9b479a88331dbf969f07708eabe53d14@codeaurora.org> (raw)
In-Reply-To: <20210209163431.11133472@gandalf.local.home>

On 2021-02-10 05:34, Steven Rostedt wrote:
> On Tue, 9 Feb 2021 14:55:31 -0500
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
>> > [for-next][PATCH 2/2] tracing: Use temp buffer when filtering events
>> > https://lore.kernel.org/lkml/f16b14066317f6a926b6636df6974966@codeaurora.org/
>> 
>> Note, that is only used when filtering happens, which doesn't appear 
>> to be
>> the case here.
> 
> I was basing this off of the original commands, but the stack dump says
> otherwise. But it should still work.
> 
>> 
>> >
>> > It seems like we should still try to get that fixed somehow, even if
>> > the below change is fine on its own (it probably doesn't make sense to
>> > such a large amount of data via tracepoints). It would be unfortunate
>> > for next poor soul to hit the same issues, just because they wanted to
>> > dump a few KB.
>> 
>> Yeah, it was a design decision to cap the max size of events to just 
>> under
>> PAGE_SIZE. The ring buffer is broken up into pages (for zero copy
>> transfers to file systems and the network). Thus, no event is allowed 
>> to be
>> bigger than a page (and actually a bit smaller)
>> 
>> That said, it shouldn't have crashed, it should have just failed to 
>> record.
>> 
>> I'll test it out and see why it crashed.
> 
> Looking at the original report, I see:
> 
>  CPU: 1 PID: 141 Comm: kworker/u16:1 Not tainted 4.19.139 #162
> 
> Does this still crash on the latest kernel?
> 
> -- Steve
Not tested with latest kernel.
The reason is below which I said in 
https://lore.kernel.org/lkml/b504b3d7e989cae108669a0cd3072454@codeaurora.org/

the per cpu buffer seems it is initilized in 
trace_buffered_event_enable,
it is only 1 page size as below:
void trace_buffered_event_enable(void)
{
...
	for_each_tracing_cpu(cpu) {
		page = alloc_pages_node(cpu_to_node(cpu),
					GFP_KERNEL | __GFP_NORETRY, 0);
If the size of buffer to trace is more than 1 page, such as 46680, then
it trigger kernel crash/panic in my case while run trace-cmd.
After debugging, the trace_file->flags in
trace_event_buffer_lock_reserve is 0x40b while run trace-cmd, and it is
0x403 while collecting ftrace log.

  reply	other threads:[~2021-02-10  2:02 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-09  2:59 [PATCH] ath10k: change len of trace_ath10k_log_dbg_dump for large buffer size Wen Gong
2021-02-09  2:59 ` Wen Gong
2021-02-09  7:34 ` Kalle Valo
2021-02-09  7:34 ` Kalle Valo
2021-02-09 19:35 ` Brian Norris
2021-02-09 19:35   ` Brian Norris
2021-02-09 19:55   ` Steven Rostedt
2021-02-09 19:55     ` Steven Rostedt
2021-02-09 21:34     ` Steven Rostedt
2021-02-09 21:34       ` Steven Rostedt
2021-02-10  2:01       ` Wen Gong [this message]
2021-02-10  2:01         ` Wen Gong
2021-02-10 16:30         ` Steven Rostedt
2021-02-10 16:30           ` Steven Rostedt
2021-02-10 16:48           ` Steven Rostedt
2021-02-10 16:48             ` Steven Rostedt
2021-02-10 16:53             ` Steven Rostedt
2021-02-10 16:53               ` Steven Rostedt
2021-03-08  7:22               ` Wen Gong
2021-03-08  7:22                 ` Wen Gong
2021-02-10  2:11   ` Wen Gong
2021-02-10  2:11     ` Wen Gong
2021-02-10  2:14     ` Brian Norris
2021-02-10  2:14       ` Brian Norris
2021-02-10  2:24       ` Wen Gong
2021-02-10  2:24         ` Wen Gong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9b479a88331dbf969f07708eabe53d14@codeaurora.org \
    --to=wgong@codeaurora.org \
    --cc=ath10k@lists.infradead.org \
    --cc=briannorris@chromium.org \
    --cc=linux-wireless@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.