Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity setting

[Frederic Weisbecker <fweisbec@gmail.com>]

On Tue, Jul 07, 2015 at 06:30:30PM +0200, Oleg Nesterov wrote:
> On 07/06, Frederic Weisbecker wrote:
> >
> > The call_usermodehelper_exec_[a]sync() kernel threads are created by
> > khelper precisely because
> 
> I think khelper should simply die. It doesn't make any sense today,
> kmod.c should use some system wq instead. But see below.
> 
> > Not only useless it even breaks nohz full. The housekeeping work
> > (general kernel internal code that user doesn't care much about) is
> > handled by a reduced set of CPUs in nohz full, precisely those that are
> > not included by nohz_full= kernel parameters. For example unbound
> > workqueues are handled by housekeeping CPUs.
> 
> Confused... I do not see how workqueue_attrs->cpumask can depend on
> tick_nohz_full_mask or housekeeping_mask. Could you explain?

People who want CPU isolation will likely write
/sys/devices/virtual/workqueue/cpumask to a reduced set of CPUs, typically
CPU 0 that is used for housekeeping in nohz full.

In fact we should add the code which initialize wq_unbound_cpumask
to housekeeping_mask automatically.

So this cpumask is inherited to khelper because it is a singlethread
workqueue.

> 
> > @@ -223,9 +223,6 @@ static int call_usermodehelper_exec_async(void *data)
> >  	flush_signal_handlers(current, 1);
> >  	spin_unlock_irq(&current->sighand->siglock);
> >
> > -	/* We can run anywhere, unlike our parent keventd(). */
> > -	set_cpus_allowed_ptr(current, cpu_all_mask);
> > -
> 
> I think this is fine, ->no_numa is true for khelper.
> 
> But this means that after this change kmod.c can't use a system wq,
> ->no_numa is false by default. And khelper is no_numa only because
> it is __WQ_ORDERED, but kmod.c doesn't need really need__WQ_ORDERED,
> except, again, this implies ->no_numa == T.

I'm not sure what means no_numa in the context of workqueues, I guess
it's about having system workqueues bound to one CPU or several in the
same nodes. But indeed we can't use system workqueues because they are
per-cpu and we inherit that. And it's ridiculous to call set_cpus_allowed_ptr()
to fix that.

Hence why we use a singlethread, even though we don't care about ordering.
Ok I guess that's more or less what you just said :o)

> So perhaps init_workqueues() should create another global
> WQ_UNBOUND/no_numa workqueue_struct so that we could kill khelper_wq?
> Or kmod.c can use system_unbound_wq, but then we need to keep this
> set_cpus_allowed_ptr().
> 
> To me khelper_wq looks just annoying.

That's a good idea. I can do that!

Perhaps queuing there would be done through schedule_work_unbound() ?
Or schedule_work_no_numa()?