From: Steven Rostedt <rostedt@goodmis.org>
To: Jeongho Choi <jh1012.choi@samsung.com>
Cc: linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org,
mhiramat@kernel.org, ji2yoon.jo@samsung.com,
minki.jang@samsung.com, hajun.sung@samsung.com
Subject: Re: [BUG] tracing: Too many tries to read user space
Date: Wed, 8 Jul 2026 09:18:04 -0400 [thread overview]
Message-ID: <20260708091804.584975f9@gandalf.local.home> (raw)
In-Reply-To: <20260708123753.GB1386@KORCO121415.samsungds.net>
On Wed, 8 Jul 2026 21:37:53 +0900
Jeongho Choi <jh1012.choi@samsung.com> wrote:
> The code at the WARN location mentioned in the log above is as follows.
>
> 7374 if (WARN_ONCE(trys++ > 100, "Error: Too many
> tries to read user space"))
> 7375 return NULL;
>
This happens when something forces a schedule.
>
> Our current analysis is as follows:
>
> In the Gmail process, during a low memory situation, LMKD writes strings
> to /sys/kernel/tracing/trace_marker for systrace recording. At the same
> time, it broadcasts a sigkill due to low memory, which is causing the
> LMKD trace marker operation to stall.
>
Can you see what is being scheduled in? Perhaps use the persistent ring
buffer (if you can) and enable sched_switch tracepoint in it.
The loop is this:
do {
/*
* It is possible that something is trying to migrate this
* task. What happens then, is when preemption is enabled,
* the migration thread will preempt this task, try to
* migrate it, fail, then let it run again. That will
* cause this to loop again and never succeed.
* On failures, enabled and disable preemption with
* migration enabled, to allow the migration thread to
* migrate this task.
*/
if (trys) {
preempt_enable_notrace();
preempt_disable_notrace();
cpu = smp_processor_id();
buffer = per_cpu_ptr(tinfo->tbuf, cpu)->buf;
}
/*
* If for some reason, copy_from_user() always causes a context
* switch, this would then cause an infinite loop.
* If this task is preempted by another user space task, it
* will cause this task to try again. But just in case something
* changes where the copying from user space causes another task
* to run, prevent this from going into an infinite loop.
* 100 tries should be plenty.
*/
if (WARN_ONCE(trys++ > 100, "Error: Too many tries to read user space"))
return NULL;
/* Read the current CPU context switch counter */
cnt = nr_context_switches_cpu(cpu);
/*
* Preemption is going to be enabled, but this task must
* remain on this CPU.
*/
migrate_disable();
/*
* Now preemption is being enabled and another task can come in
* and use the same buffer and corrupt our data.
*/
preempt_enable_notrace();
/* Make sure preemption is enabled here */
lockdep_assert_preemption_enabled();
if (copy_func) {
ret = copy_func(buffer, ptr, size, data);
} else {
ret = __copy_from_user(buffer, ptr, size);
}
preempt_disable_notrace();
migrate_enable();
/* if it faulted, no need to test if the buffer was corrupted */
if (ret)
return NULL;
/*
* Preemption is disabled again, now check the per CPU context
* switch counter. If it doesn't match, then another user space
* process may have schedule in and corrupted our buffer. In that
* case the copying must be retried.
*/
} while (nr_context_switches_cpu(cpu) != cnt);
What it does is to write into a per CPU buffer from user space. If it
schedules out while trying this, it will try again as another task could
have come in and corrupted the buffer.
For some reason, when preemption is enabled to copy from user, something
is forcing a schedule of the process. If we can figure out what is doing
that, perhaps we can fix this. A similar thing happened with the migration
thread that commit edca33a56297d ("tracing: Fix failure to read user space
from system call trace events") fixed.
-- Steve
next prev parent reply other threads:[~2026-07-08 13:18 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20260708123754epcas2p1f15cc305ddb09f97164491d750769ef7@epcas2p1.samsung.com>
2026-07-08 12:37 ` [BUG] tracing: Too many tries to read user space Jeongho Choi
2026-07-08 13:18 ` Steven Rostedt [this message]
2026-07-09 0:04 ` Masami Hiramatsu
2026-07-10 3:22 ` Masami Hiramatsu
2026-07-10 11:46 ` Steven Rostedt
2026-07-10 12:33 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260708091804.584975f9@gandalf.local.home \
--to=rostedt@goodmis.org \
--cc=hajun.sung@samsung.com \
--cc=jh1012.choi@samsung.com \
--cc=ji2yoon.jo@samsung.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=minki.jang@samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox