From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752872AbcCKMmu (ORCPT <rfc822;w@1wt.eu>);
	Fri, 11 Mar 2016 07:42:50 -0500
Received: from casper.infradead.org ([85.118.1.10]:60068 "EHLO
	casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1752261AbcCKMmn (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Fri, 11 Mar 2016 07:42:43 -0500
Date: Fri, 11 Mar 2016 13:42:38 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Juergen Gross <jgross@suse.com>
Cc: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org,
        konrad.wilk@oracle.com, boris.ostrovsky@oracle.com,
        david.vrabel@citrix.com, mingo@redhat.com, Douglas_Warzecha@dell.com,
        pali.rohar@gmail.com, jdelvare@suse.com, linux@roeck-us.net,
        tglx@linutronix.de, hpa@zytor.com, x86@kernel.org
Subject: Re: [PATCH 2/6] sched: add function to execute a function
 synchronously on a physical cpu
Message-ID: <20160311124238.GN6375@twins.programming.kicks-ass.net>
References: <1457697574-6710-1-git-send-email-jgross@suse.com>
 <1457697574-6710-3-git-send-email-jgross@suse.com>
 <20160311121950.GZ6344@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20160311121950.GZ6344@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Mar 11, 2016 at 01:19:50PM +0100, Peter Zijlstra wrote:
> On Fri, Mar 11, 2016 at 12:59:30PM +0100, Juergen Gross wrote:
> > +int call_sync_on_phys_cpu(unsigned cpu, int (*func)(void *), void *par)
> > +{
> > +	cpumask_var_t old_mask;
> > +	int ret;
> > +
> > +	if (cpu >= nr_cpu_ids)
> > +		return -EINVAL;
> > +
> > +	if (!alloc_cpumask_var(&old_mask, GFP_KERNEL))
> > +		return -ENOMEM;
> > +
> > +	cpumask_copy(old_mask, &current->cpus_allowed);
> > +	ret = set_cpus_allowed_ptr(current, cpumask_of(cpu));
> > +	if (ret)
> > +		goto out;
> 
> So what happens if someone does sched_setaffinity() right about here?
> 
> > +
> > +	ret = func(par);
> > +
> > +	set_cpus_allowed_ptr(current, old_mask);
> > +
> > +out:
> > +	free_cpumask_var(old_mask);
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(call_sync_on_phys_cpu);
> 
> This is disgusting, and you're adding this to !Xen kernels too.

how about something like:

struct xen_callback_struct {
	struct work_struct	work;
	struct completion	done;
	void *			data;
	int			ret;
};

static void xen_callback_f(struct work_struct *work)
{
	struct xen_callback_struct *xcs = container_of(work, struct xen_callback_struct, work);

	xcs->ret = xcs->func(xcs->data);

	complete(&xcs->done);
}

xen_call_on_cpu_sync(int cpu, int (*func)(void *), void *data)
{
	struct xen_callback_state xcs = {
		.work = __WORK_INITIALIZER(xcs.work, xen_callback_f);
		.done = COMPLETION_INITIALIZER_ONSTACK(xcs.done),
		.data = data,
	};

	queue_work_on(&work, cpu);
	wait_for_completion(&xcs.done);

	return xcs.ret;
}

No mucking about with the scheduler state, no new exported functions
etc..