Re: [PATCH] sched: don't clear PF_THREAD_BOUND in select_fallback_rq

[Sebastian Andrzej Siewior <bigeasy@linutronix.de>]

On 06/22/2013 05:27 AM, Qiang Huang wrote:

>> The only way I lose this flag is by starting a workqueue on a CPU which
>> if offline. Now that is wrong. I am not sure if the workqueue re-uses
> 
> How do you judge a thread lost PF_THREAD_BOUND flag or not?

I have a printk in the code path you want to kill.
> 
> My way is:
> # for pid in `ps -e -o pid`; do taskset -p -c 0-15 $pid; done
…snip…

> As you can see, many kernel threads' affinity can be changed, which were
> supposed to be attached to only one cpu, most of them are kworkers, as
> well as some other kernel threads.

Yes, I see your output but I can't reproduce it here.

> After applying my patch, turns out:
…snip…

> These changing all failed. We can't change the cpu affinity of those
> threads which are attached to only one cpu.
> 
> I don't kown if that's enought to say many threads' PF_THREAD_BOUND flag
> is cleared which should not be. But your patches definitely not resolve
> this problem, the taskset result is similar to my first one. I don't know
> if this is the direct reason for the hung problem, but this is truly a
> problem, right?

So it seems that you lose your flag but I don't know how. I only lose
it for threads which are woken up shortly before exit.

> And my test shows the same thing, after applying your patches, my
> cgroup_fj test will still cause system hung, after reboot, log message
> shows the same as I sent before:
> ...
…snip…

> ...
> 
> So right now, my patch is still my only solution.
> 
> One thing need to be clear, my test is on 3.4.45-rt60, but I think all 3.4-rt
> versions have this problem.

Oh boy. We have too many RT trees. Can you retest you problem on latest
v3.8 and see if it there a problem, too?
I just booted
"Linux sq 3.4.41-rt55 #5 SMP PREEMPT RT Fri Jun 28 13:44:38 CEST 2013
x86_64 GNU/Linux"

and I have a Script doing:

|for pid in $(ps -e -o pid);
|do
|        taskset -p -c 6,7 $pid >/dev/null 2>&1
|        if [ $? -eq 0 ]
|        then
|                C="$(cat /proc/$pid/cmdline)"
|                if [ -z "$C" ]
|                then
|                        echo "Okay: $(cat /proc/$pid/comm)"
|                fi
|        fi
|done


which is basically your test. It will print each kernel thread on which
the affinity change went well. There are no kworker, no migration
threads, no softirqs. Just kthreadd, irq/* and so on.
So I still have no idea how and why you lose this flag and I still
can't reproduce it.
The interesting part is, why do your threads lose their flag and mine
don't.

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html