Re: Fam10+ extended config space enabling

[Robert Richter <robert.richter@amd.com>]

On 06.01.11 05:38:48, Keir Fraser wrote:
> On 06/01/2011 10:15, "Jan Beulich" <JBeulich@novell.com> wrote:

> > starting with 2.6.27 Linux is doing this, using MSR accesses (these
> > days in CPU hotplug notification callbacks). Is there a reason this
> > cannot be done through PCI config space writes (as the register is
> > aliased), and then only once at boot time (perhaps utilizing some
> > of the code in arch/x86/kernel/k8.c)?

>From reading the documentation this should work too. The msr controls
a Northbridge function and thus is per-node. NB function msrs are
shared between all cores on the node and there is one instance per
node.

IIRC there were several reasons to choose msr access. During early cpu
bringup there are only some basic early pci access functions
available, requiring a little more complex code to scan NB devices. I
think in between there is code available that could be shared. Second,
if we implement the setup using pci access, we must adopt the code
with every new cpu family with new nb devices to support pci ecs with
new cpus. So, new cpus couldn't use pci ecs with an old kernel then.

But the msr 0xC001001F is not architectural. Actually this means the
assumption in the code is wrong, that all cpus from fam10h have IO
ECS. There might be future cpus not supporting it. But in practice all
current cpu families from family 10h support the msr (10h, 11h, 12h,
14h, 15h).

> > I'm particularly asking from the Xen perspective, where the MSR
> > based mechanism won't reliably work (since there's no guarantee
> > all of Dom0's virtual CPUs would, at initialization time, actually
> > cover all physical CPUs that Dom0 may ever run on).
> 
> Just thinking, could we simply do this in Xen, either unconditionally, or as
> soon as we detect dom0 doing it for at least one CPU?

I don't know if xen is capable to emulate per-node msrs, presumable
not. I don't have an overview which other per-node msrs might be worth
to be supported by xen. If so, it might be better to implement support
for per-node msrs.

But I am fine with a setup with pci access for some nb devices, but
would rather leave the setup via msrs as a fallback.

-Robert

-- 
Advanced Micro Devices, Inc.
Operating System Research Center