Re: [PATCH 1/3] smp/ipi: Remove redundant cfd->cpumask_ipi mask

[Wang YanQing <udknight@gmail.com>]

On Sat, Jul 06, 2013 at 10:59:39AM +0530, Preeti U Murthy wrote:
> Hi Wang,
> 
> On 07/06/2013 08:43 AM, Wang YanQing wrote:
> > On Fri, Jul 05, 2013 at 09:57:01PM +0530, Preeti U Murthy wrote:
> >> cfd->cpumask_ipi is used only in smp_call_function_many().The existing
> >> comment around it says that this additional mask is used because
> >> cfd->cpumask can get overwritten.
> >>
> >> There is no reason why the cfd->cpumask can be overwritten, since this
> >> is a per_cpu mask; nobody can change it but us and we are
> >> called with preemption disabled.
> > 
> > The ChangeLog for f44310b98ddb7f0d06550d73ed67df5865e3eda5
> > which import cfd->cpumask_ipi saied the reason why we need
> > it:
> > 
> > "    As explained by Linus as well:
> >     
> >      |
> >      | Once we've done the "list_add_rcu()" to add it to the
> >      | queue, we can have (another) IPI to the target CPU that can
> >      | now see it and clear the mask.
> >      |
> >      | So by the time we get to actually send the IPI, the mask might
> >      | have been cleared by another IPI.
> 
> I am unable to understand where the cfd->cpumask of the source cpu is
> getting cleared. Surely not by itself, since it is preempt disabled.
> Also why should it get cleared?

Assume we have three CPUs: A,B,C

A call smp_call_function_many to notify C do something,
and current it execute on finished below codes:

"for_each_cpu(cpu, cfd->cpumask) {
                struct call_single_data *csd = per_cpu_ptr(cfd->csd, cpu);
                struct call_single_queue *dst =
                                        &per_cpu(call_single_queue, cpu);
                unsigned long flags;

                csd_lock(csd);
                csd->func = func;
                csd->info = info;

                raw_spin_lock_irqsave(&dst->lock, flags);
                list_add_tail(&csd->list, &dst->list);
                raw_spin_unlock_irqrestore(&dst->lock, flags);
        }
"
You see "list_add_tail(&csd->list, &dst->list);", it pass the address of csd,
and A stop before call arch_send_call_function_ipi_mask due interrupt.

At this time B send ipi to C also, then C will see the csd passed by A,
then C will clear itself in the cfd->cpumask.

Thanks.