exposing FSB clock speed in /sys

exposing FSB clock speed in /sys

[Stephane Eranian]

Hello,

It seems that the kernel does not expose the Front-Side Bus (FSN) Clock
speed to user applications. I found code in the kernel dealing with
frequency scaling that extracts the information for x86 processors but
the value is never exposed.

Knowledge the the FSB speed is very useful to monitoring tools. It is used
to compute certain bus-related metrics.

Looking at the code, it seems that there is no standard way of extracting
the FSB speed. For each processor model, you have different MSRs. I would
think that the routines in the cpufreq code could be moved out and used
as the basis to expose the information somewhere in /sys.

Any comments?

-- 
-Stephane

Re: exposing FSB clock speed in /sys

[Arjan van de Ven]

On Fri, 2007-03-30 at 07:39 -0800, Stephane Eranian wrote:
> Hello,
> 
> It seems that the kernel does not expose the Front-Side Bus (FSN) Clock
> speed to user applications

and that is a good thing ;)



> Knowledge the the FSB speed is very useful to monitoring tools. It is used
> to compute certain bus-related metrics.

perhaps. 
> 
> Looking at the code, it seems that there is no standard way of extracting
> the FSB speed. For each processor model, you have different MSRs. I would
> think that the routines in the cpufreq code could be moved out and used
> as the basis to expose the information somewhere in /sys.

... yet not all CPU's *have* an FSB. Notably AMD ones do not, and I'm
sure you've read on a lot of online tech magazines that other vendors
may also not have one in the future  :-)

Exposing a concept that we KNOW does not make sense via a kernel
interface (which means we need to keep it around forever) is in my
opinion quite a bad mistake to make.... Please lets not do this.

Re: exposing FSB clock speed in /sys

[Alan Cox]

> Looking at the code, it seems that there is no standard way of extracting
> the FSB speed. For each processor model, you have different MSRs. I would
> think that the routines in the cpufreq code could be moved out and used
> as the basis to expose the information somewhere in /sys.

It's well un-documented, but one of the problems is that for many
processor lines the information isn't in public documents (eg the older
processors like the PII/PIII)

Alan

Re: [perfmon] exposing FSB clock speed in /sys

[Martin Cracauer]

Stephane Eranian wrote on Fri, Mar 30, 2007 at 07:39:37AM -0800: 
> Hello,
> 
> It seems that the kernel does not expose the Front-Side Bus (FSN) Clock
> speed to user applications. I found code in the kernel dealing with
> frequency scaling that extracts the information for x86 processors but
> the value is never exposed.
> 
> Knowledge the the FSB speed is very useful to monitoring tools. It is used
> to compute certain bus-related metrics.
> 
> Looking at the code, it seems that there is no standard way of extracting
> the FSB speed. For each processor model, you have different MSRs. I would
> think that the routines in the cpufreq code could be moved out and used
> as the basis to expose the information somewhere in /sys.

That is still problematic as finding out the FSB might not be
trivial.  For example, for the NForce4 chipsets we have a too to
manipulate FSB and multipliers, but it's not in the kernel and never
will be.  I don't think that K8 has a way to find the FSB from the CPU
only, so you are bound to tangle with "nice" chipsets from NVidia et al.

I like the idea, but I expect there will be reservations against
presenting a /proc file that cannot be supported for almost all
machines. 

In general, I would like the cpufreq code split up, as I need to reuse
parts of it for other clock manipulation projects.  Currently, cpufreq
is a blob of code to manipulate frequencies, and to deal with the
fallout.  These two should be split up.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/
FreeBSD - where you want to go, today.      http://www.freebsd.org/

Re: exposing FSB clock speed in /sys

[Andi Kleen]

Stephane Eranian <eranian-sDzT885Ts8HQT0dZR+AlfA@public.gmane.org> writes:

> It seems that the kernel does not expose the Front-Side Bus (FSN) Clock
> speed to user applications. 

You mean the APIC timer frequency which happens to match the FSB 
on some CPUs? 
 
> Knowledge the the FSB speed is very useful to monitoring tools. It is used
> to compute certain bus-related metrics.

Can you describe those metrics in detail? 

-Andi

Re: exposing FSB clock speed in /sys

[Stephane Eranian]

Andi,

On Sat, Mar 31, 2007 at 04:31:29AM +0200, Andi Kleen wrote:
> Stephane Eranian <eranian-sDzT885Ts8HQT0dZR+AlfA@public.gmane.org> writes:
> 
> > It seems that the kernel does not expose the Front-Side Bus (FSN) Clock
> > speed to user applications. 
> 
> You mean the APIC timer frequency which happens to match the FSB 
> on some CPUs? 

Well, I am talking about the bus that connects the processor socket to the
chipset on Intel machines. On Intel Core 2 Duo (aka Woodcrest), you have 
2 sockets, thus two buses connecting to the chipset which then connects
to the memory (among other things).

The Woodcrest (Intel Core PMU) is capable of measuring the number of bus
transactions to memory (look at the BUS_* events). You can count per core
or for the entire socket (both cores). In order to know if you saturate
the bus (from socket to chipset), you need to know the theoretical peak
number of transactions the bus can sustain. For that you need to bus
frequency. 

I am not interested in older processors, but I think for all recent Intel
processors, there is a fairly simple algorithm to get the frequency using
a couple of MSRs (including MSR_IA32_EBL_CR_POWERON or MSR_FSB_FREQ).

Don't we already have /sys entries that exits only for certain processors
or platforms?

I think the Opteron have HYPERTRANSPORT-related events which could be used
to obtain similar metrics.

Knowledge of bus saturation is important for multi-core programming.

>  
> > Knowledge the the FSB speed is very useful to monitoring tools. It is used
> > to compute certain bus-related metrics.
> 
> Can you describe those metrics in detail? 
> 
> -Andi

-- 

-Stephane

Re: exposing FSB clock speed in /sys

[Andi Kleen]

> I am not interested in older processors, but I think for all recent Intel
> processors, there is a fairly simple algorithm to get the frequency using
> a couple of MSRs (including MSR_IA32_EBL_CR_POWERON or MSR_FSB_FREQ).

Hmm, maybe make it an own driver that creates the sysfs entries somewhere.
I don't think it's useful enough for the main kernel.

But what happens during power saving that may lower the frequency? 
Wouldn't you need update events for that? That might complicate it.

> 
> Don't we already have /sys entries that exits only for certain processors
> or platforms?

Yes.

> I think the Opteron have HYPERTRANSPORT-related events which could be used
> to obtain similar metrics.

Yes, but you usually don't need the frequency.

-Andi

Re: exposing FSB clock speed in /sys

[Lennart Sorensen]

On Sat, Mar 31, 2007 at 12:18:32AM -0800, Stephane Eranian wrote:
> Well, I am talking about the bus that connects the processor socket to the
> chipset on Intel machines. On Intel Core 2 Duo (aka Woodcrest), you have 
> 2 sockets, thus two buses connecting to the chipset which then connects
> to the memory (among other things).
> 
> The Woodcrest (Intel Core PMU) is capable of measuring the number of bus
> transactions to memory (look at the BUS_* events). You can count per core
> or for the entire socket (both cores). In order to know if you saturate
> the bus (from socket to chipset), you need to know the theoretical peak
> number of transactions the bus can sustain. For that you need to bus
> frequency. 
> 
> I am not interested in older processors, but I think for all recent Intel
> processors, there is a fairly simple algorithm to get the frequency using
> a couple of MSRs (including MSR_IA32_EBL_CR_POWERON or MSR_FSB_FREQ).
> 
> Don't we already have /sys entries that exits only for certain processors
> or platforms?
> 
> I think the Opteron have HYPERTRANSPORT-related events which could be used
> to obtain similar metrics.
> 
> Knowledge of bus saturation is important for multi-core programming.

How about the speed between ram and the chipset?  Is it single or dual
channel?  Are there PCI devices currently doing DMA transfers to ram
taking up bandwidth?  What are the latencies of the ram, Etc, etc, etc.

And what could your software do differently if it knew this information?

--
Len Sorensen

Re: exposing FSB clock speed in /sys

[Stephane Eranian]

Len,

On Mon, Apr 02, 2007 at 12:14:43PM -0400, Lennart Sorensen wrote:
> On Sat, Mar 31, 2007 at 12:18:32AM -0800, Stephane Eranian wrote:
> > Well, I am talking about the bus that connects the processor socket to the
> > chipset on Intel machines. On Intel Core 2 Duo (aka Woodcrest), you have 
> > 2 sockets, thus two buses connecting to the chipset which then connects
> > to the memory (among other things).
> > 
> > The Woodcrest (Intel Core PMU) is capable of measuring the number of bus
> > transactions to memory (look at the BUS_* events). You can count per core
> > or for the entire socket (both cores). In order to know if you saturate
> > the bus (from socket to chipset), you need to know the theoretical peak
> > number of transactions the bus can sustain. For that you need to bus
> > frequency. 
> > 
> > I am not interested in older processors, but I think for all recent Intel
> > processors, there is a fairly simple algorithm to get the frequency using
> > a couple of MSRs (including MSR_IA32_EBL_CR_POWERON or MSR_FSB_FREQ).
> > 
> > Don't we already have /sys entries that exits only for certain processors
> > or platforms?
> > 
> > I think the Opteron have HYPERTRANSPORT-related events which could be used
> > to obtain similar metrics.
> > 
> > Knowledge of bus saturation is important for multi-core programming.
> 
> How about the speed between ram and the chipset?  Is it single or dual
> channel?  Are there PCI devices currently doing DMA transfers to ram
> taking up bandwidth?  What are the latencies of the ram, Etc, etc, etc.

This is information you can normally collect form chipset performance
counters. Unfortunately, those are rarely public.

I am focusing on what you can measure using the CPU performance counters.

> 
> And what could your software do differently if it knew this information?
> 
That is a good question! Thanks for asking.

Assuming you are using a dual core machine, if you would know the bus utilization,
you could determine whether or not it is worth using the 2nd core for your workload.
If one thread already saturates the bus, you can either use the 2nd core for a
non memory-bound workload, or just leave it unused. As you can guess, this information
could be used for thread placement.

Similarly, if you see a low bus utilization for a memory-bound program, then this
could be a sign that the program is not well optimized, e.g., not issuing software
prefetch. Here this is about code tuning.

There a lot of information to be gleaned for using the performance counters especially
for multi-core platforms.

-- 
-Stephane