Re: [RFC patch 21/21] LTTng Kernel Trace Thread Flag API

[Frederic Weisbecker <fweisbec@gmail.com>]

On Sun, Mar 15, 2009 at 03:04:01PM -0400, Mathieu Desnoyers wrote:
> Add an API to set/clear the kernel wide tracing thread flags. Implemented in
> kernel/sched.c. Updates thread flags *asynchronously* while holding the tasklist
> lock.
> 
> Upon fork, the flag must be re-copied while the tasklist lock is held.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> ---
>  include/linux/sched.h |    3 ++
>  kernel/fork.c         |    9 ++++++++
>  kernel/sched.c        |   55 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 67 insertions(+)
> 
> Index: linux-2.6-lttng/include/linux/sched.h
> ===================================================================
> --- linux-2.6-lttng.orig/include/linux/sched.h	2009-01-09 18:15:54.000000000 -0500
> +++ linux-2.6-lttng/include/linux/sched.h	2009-01-09 18:17:50.000000000 -0500
> @@ -2300,6 +2300,9 @@ static inline void mm_init_owner(struct 
>  
>  #define TASK_STATE_TO_CHAR_STR "RSDTtZX"
>  
> +extern void clear_kernel_trace_flag_all_tasks(void);
> +extern void set_kernel_trace_flag_all_tasks(void);
> +
>  #endif /* __KERNEL__ */
>  
>  #endif
> Index: linux-2.6-lttng/kernel/fork.c
> ===================================================================
> --- linux-2.6-lttng.orig/kernel/fork.c	2009-01-09 18:17:38.000000000 -0500
> +++ linux-2.6-lttng/kernel/fork.c	2009-01-09 18:17:50.000000000 -0500
> @@ -1218,6 +1218,15 @@ static struct task_struct *copy_process(
>  			!cpu_online(task_cpu(p))))
>  		set_task_cpu(p, smp_processor_id());
>  
> +	/*
> +	 * The state of the parent's TIF_KTRACE flag may have changed
> +	 * since it was copied in dup_task_struct() so we re-copy it here.
> +	 */
> +	if (test_thread_flag(TIF_KERNEL_TRACE))
> +		set_tsk_thread_flag(p, TIF_KERNEL_TRACE);
> +	else
> +		clear_tsk_thread_flag(p, TIF_KERNEL_TRACE);
> +


I thought about that too. But the thread info is already
copied on a fork right? Which includes the thread flags.

But perhaps I'm wrong and miss one detail about this copy...


>  	/* CLONE_PARENT re-uses the old parent */
>  	if (clone_flags & (CLONE_PARENT|CLONE_THREAD))
>  		p->real_parent = current->real_parent;
> Index: linux-2.6-lttng/kernel/sched.c
> ===================================================================
> --- linux-2.6-lttng.orig/kernel/sched.c	2009-01-09 18:17:38.000000000 -0500
> +++ linux-2.6-lttng/kernel/sched.c	2009-01-09 18:17:50.000000000 -0500
> @@ -9395,3 +9395,58 @@ struct cgroup_subsys cpuacct_subsys = {
>  	.subsys_id = cpuacct_subsys_id,
>  };
>  #endif	/* CONFIG_CGROUP_CPUACCT */
> +
> +static DEFINE_MUTEX(kernel_trace_mutex);
> +static int kernel_trace_refcount;
> +
> +/**
> + * clear_kernel_trace_flag_all_tasks - clears all TIF_KERNEL_TRACE thread flags.
> + *
> + * This function iterates on all threads in the system to clear their
> + * TIF_KERNEL_TRACE flag. Setting the TIF_KERNEL_TRACE flag with the
> + * tasklist_lock held in copy_process() makes sure that once we finish clearing
> + * the thread flags, all threads have their flags cleared.
> + */
> +void clear_kernel_trace_flag_all_tasks(void)
> +{
> +	struct task_struct *p;
> +	struct task_struct *t;
> +
> +	mutex_lock(&kernel_trace_mutex);
> +	if (--kernel_trace_refcount)
> +		goto end;
> +	read_lock(&tasklist_lock);
> +	do_each_thread(p, t) {
> +		clear_tsk_thread_flag(t, TIF_KERNEL_TRACE);
> +	} while_each_thread(p, t);
> +	read_unlock(&tasklist_lock);
> +end:
> +	mutex_unlock(&kernel_trace_mutex);
> +}
> +EXPORT_SYMBOL_GPL(clear_kernel_trace_flag_all_tasks);
> +
> +/**
> + * set_kernel_trace_flag_all_tasks - sets all TIF_KERNEL_TRACE thread flags.
> + *
> + * This function iterates on all threads in the system to set their
> + * TIF_KERNEL_TRACE flag. Setting the TIF_KERNEL_TRACE flag with the
> + * tasklist_lock held in copy_process() makes sure that once we finish setting
> + * the thread flags, all threads have their flags set.
> + */
> +void set_kernel_trace_flag_all_tasks(void)
> +{
> +	struct task_struct *p;
> +	struct task_struct *t;
> +
> +	mutex_lock(&kernel_trace_mutex);
> +	if (kernel_trace_refcount++)
> +		goto end;
> +	read_lock(&tasklist_lock);
> +	do_each_thread(p, t) {
> +		set_tsk_thread_flag(t, TIF_KERNEL_TRACE);
> +	} while_each_thread(p, t);
> +	read_unlock(&tasklist_lock);
> +end:
> +	mutex_unlock(&kernel_trace_mutex);
> +}
> +EXPORT_SYMBOL_GPL(set_kernel_trace_flag_all_tasks);


Speaking about optimizations and latency. If you want to avoid
the overhead of a whole tasklist iteration making any writer spinning on
you, may be you can iterate it using rcu.

But it can be slightly racy in that you could miss a freshly inserted
task while playing with a nano delayed version of the list. So this is not
necessarily a good idea.


> 
> -- 
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68