Re: smp.c && barriers (Was: [PATCH 1/4] generic-smp: remove single ipi fallback for smp_call_function_many())

Ingo Molnar
2009-02-20T10:57:29Z

* Suresh Siddha <suresh.b.siddha@intel.com> wrote:

> On Thu, 2009-02-19 at 04:20 -0800, Ingo Molnar wrote:
> > Could you please refresh this patch to latest tip:master? The
> > APIC drivers moved to arch/x86/kernel/apic/.
>
> Appended the refreshed patch. Thanks.

thanks. Two details i noticed:

Firstly:

> +++ b/arch/x86/kernel/apic/x2apic_cluster.c
> +++ b/arch/x86/kernel/apic/x2apic_phys.c

how about x2apic_uv.c? It uses uv_write_global_mmr64() in its
IPI sending method, which uses:

static inline void uv_write_global_mmr64(int pnode, unsigned long offset,
unsigned long val)
{
*uv_global_mmr64_address(pnode, offset) = val;
}

which uses ->mmr_base, which is mapped via:

init_extra_mapping_uc(UV_LOCAL_MMR_BASE, UV_LOCAL_MMR_SIZE);

so it should be fine because uncached - but at minimum we should
put a comment into x2apic_uv.c that the generic IPI code relies
on the lowlevel code serializing - i.e. relies on the UC PAT
attribute.

Secondly, you added smp_mb(), which will translate to an MFENCE.

But in theory it should be enough to have a wmb() here. [Note,
not an smp_wmb() that i suggested before.] That will translate
to an SFENCE - which will serialize writes but still allows
reads/prefetches to pass.

So the question is, is an SFENCE there enough to serialize the
WRMSR with previous memory-writes? It's not specified in the
x2apic docs as far as i could see.

Ingo