Re: [PATCH -v4] generic-ipi: remove kmalloc()

Peter Zijlstra
2009-02-17T21:38:38Z

On Tue, 2009-02-17 at 13:30 -0800, Paul E. McKenney wrote:

> > +static void csd_complete(struct call_single_data *data)
> > +{
> > + /*
> > + * Serialize stores to data with the flag clear and wakeup.
> > + */
> > + smp_wmb();
>
> Shouldn't the above be an smp_mb()? There are reads preceding the calls
> to csd_complete() that look to me like they need to remain ordered
> before the flag-clearing below -- just in case of a quick reuse of this
> call_single_data structure.

Good point, however I just did a patch that made CSD_FLAG_WAIT go
away :-)

> > + data->flags &= ~CSD_FLAG_WAIT;
> > +}
> > +
> > +static void csd_wait(struct call_single_data *data)
> > +{
> > + while (data->flags & CSD_FLAG_WAIT)
> > + cpu_relax();
> > +}
> > +
> > +/*
> > + * csd_lock/csd_unlock used to serialize access to per-cpu csd resources
> > + *
> > + * For non-synchronous ipi calls the csd can still be in use by the previous
> > + * function call. For multi-cpu calls its even more interesting as we'll have
> > + * to ensure no other cpu is observing our csd.
> > + */
> > +static void csd_lock(struct call_single_data *data)
> > {
> > - /* Wait for response */
> > - do {
> > - if (!(data->flags & CSD_FLAG_WAIT))
> > - break;
> > + while (data->flags & CSD_FLAG_LOCK)
> > cpu_relax();
> > - } while (1);
> > + data->flags = CSD_FLAG_LOCK;
>
> OK, I'll bite... Why don't we need a memory barrier here?

cpu_relax() is a compiler barrier, missing a memory barrier will just
make us spin this little while extra until the cacheline does hit us.

> > +}
> > +
> > +static void csd_unlock(struct call_single_data *data)
> > +{
> > + WARN_ON(!(data->flags & CSD_FLAG_LOCK));
> > + /*
> > + * Serialize stores to data with the flags clear.
> > + */
> > + smp_wmb();
>
> I am a bit worried about this being smp_wmb() rather than smp_mb(),
> but don't have a smoking gun.

data->func(data->info);

/*
* Unlocked CSDs are valid through generic_exec_single()
*/
if (data_flags & CSD_FLAG_LOCK)
csd_unlock(data);

could the data->info read be delayed until after csd_unlock() ?

I'll make it an mb().

> And about here I get lost -- trying to find what the heck this patch
> applies to... :-/

Right, I was in the process of sending out a full patch-set again.