Calibration issues with USB disc present.

Calibration issues with USB disc present.

[George Anzinger]

John,

Have you run into this.  One of the USB disc controllers has the ability to boot the system, 
however, it needs SMM code to do this.  This SMM code, somehow, causes SMI interrupts (which are 
higher priority than NMI interrutps and not maskable) which it needs to do its thing.

Problem is that if one of these occurs while calibrating the TSC or the delay code, it can cause a 
wrong result.  We have seen both a too long and a too short result (depending on where the interrut 
happens).

They have found the root cause of TSC calibration problem.
Now they ask for the fix or workaround.

That is the BIOS is periodically interrupted by USB controller and the CPU
waits during the processing of these interrupts.
Their experiments say the interrupt interval is 260mSec and the BIOS needs
150uSec - 200uSec for processing.
It is proved that the problem doesn't reproduce by masking such SMI in BIOS.
They say SMI is for BIOS emulation for connecting legacy devices to USB.
Without such an emulation it's impossible to boot from USB-FD for instance,
they say too.


-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: Calibration issues with USB disc present.

[john stultz]

On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
> John,
> 
> Have you run into this.  One of the USB disc controllers has the ability to boot the system, 
> however, it needs SMM code to do this.  This SMM code, somehow, causes SMI interrupts (which are 
> higher priority than NMI interrutps and not maskable) which it needs to do its thing.
> 
> Problem is that if one of these occurs while calibrating the TSC or the delay code, it can cause a 
> wrong result.  We have seen both a too long and a too short result (depending on where the interrut 
> happens).
> 
> They have found the root cause of TSC calibration problem.
> Now they ask for the fix or workaround.
> 
> That is the BIOS is periodically interrupted by USB controller and the CPU
> waits during the processing of these interrupts.
> Their experiments say the interrupt interval is 260mSec and the BIOS needs
> 150uSec - 200uSec for processing.
> It is proved that the problem doesn't reproduce by masking such SMI in BIOS.
> They say SMI is for BIOS emulation for connecting legacy devices to USB.
> Without such an emulation it's impossible to boot from USB-FD for instance,
> they say too.

Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
familiar with the pain BIOS SMIs can cause and I'm not surprised that it
would affect the TSC/delay calibration code.

Is this still an issue w/ 2.6.14? I know the new TSC based delay
calibration code is supposed to be SMI resilient, but I haven't really
played with it closely.

Not sure what the best method to move forward would be. I suspect
disabling the SMI code early in boot (I thought the usb legacy handoff
stuff already did this?) would help. Then the actual Linux USB drivers
can take over before we switch from the initrd to the root filesystem.

Greg, do you have a suggestion?

thanks
-john

Re: Calibration issues with USB disc present.

[Greg KH]

On Fri, Nov 11, 2005 at 01:57:07PM -0800, john stultz wrote:
> On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
> > John,
> > 
> > Have you run into this.  One of the USB disc controllers has the ability to boot the system, 
> > however, it needs SMM code to do this.  This SMM code, somehow, causes SMI interrupts (which are 
> > higher priority than NMI interrutps and not maskable) which it needs to do its thing.
> > 
> > Problem is that if one of these occurs while calibrating the TSC or the delay code, it can cause a 
> > wrong result.  We have seen both a too long and a too short result (depending on where the interrut 
> > happens).
> > 
> > They have found the root cause of TSC calibration problem.
> > Now they ask for the fix or workaround.
> > 
> > That is the BIOS is periodically interrupted by USB controller and the CPU
> > waits during the processing of these interrupts.
> > Their experiments say the interrupt interval is 260mSec and the BIOS needs
> > 150uSec - 200uSec for processing.
> > It is proved that the problem doesn't reproduce by masking such SMI in BIOS.
> > They say SMI is for BIOS emulation for connecting legacy devices to USB.
> > Without such an emulation it's impossible to boot from USB-FD for instance,
> > they say too.
> 
> Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
> familiar with the pain BIOS SMIs can cause and I'm not surprised that it
> would affect the TSC/delay calibration code.
> 
> Is this still an issue w/ 2.6.14? I know the new TSC based delay
> calibration code is supposed to be SMI resilient, but I haven't really
> played with it closely.
> 
> Not sure what the best method to move forward would be. I suspect
> disabling the SMI code early in boot (I thought the usb legacy handoff
> stuff already did this?) would help. Then the actual Linux USB drivers
> can take over before we switch from the initrd to the root filesystem.
> 
> Greg, do you have a suggestion?

I only ever saw this when people forgot to load the USB drivers.  Once
the kernel took over USB support, there was no problem (if there was,
that's a BIOS bug.)  The handoff code in 2.6.14 should help a lot with
this too.

thanks,

greg k-h

Re: Calibration issues with USB disc present.

[George Anzinger]

Greg KH wrote:
> On Fri, Nov 11, 2005 at 01:57:07PM -0800, john stultz wrote:
> 
>>On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
>>
>>>John,
>>>
>>>Have you run into this.  One of the USB disc controllers has the ability to boot the system, 
>>>however, it needs SMM code to do this.  This SMM code, somehow, causes SMI interrupts (which are 
>>>higher priority than NMI interrutps and not maskable) which it needs to do its thing.
>>>
>>>Problem is that if one of these occurs while calibrating the TSC or the delay code, it can cause a 
>>>wrong result.  We have seen both a too long and a too short result (depending on where the interrut 
>>>happens).
>>>
>>>They have found the root cause of TSC calibration problem.
>>>Now they ask for the fix or workaround.
>>>
>>>That is the BIOS is periodically interrupted by USB controller and the CPU
>>>waits during the processing of these interrupts.
>>>Their experiments say the interrupt interval is 260mSec and the BIOS needs
>>>150uSec - 200uSec for processing.
>>>It is proved that the problem doesn't reproduce by masking such SMI in BIOS.
>>>They say SMI is for BIOS emulation for connecting legacy devices to USB.
>>>Without such an emulation it's impossible to boot from USB-FD for instance,
>>>they say too.
>>
>>Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
>>familiar with the pain BIOS SMIs can cause and I'm not surprised that it
>>would affect the TSC/delay calibration code.
>>
>>Is this still an issue w/ 2.6.14? I know the new TSC based delay
>>calibration code is supposed to be SMI resilient, but I haven't really
>>played with it closely.
>>
>>Not sure what the best method to move forward would be. I suspect
>>disabling the SMI code early in boot (I thought the usb legacy handoff
>>stuff already did this?) would help. Then the actual Linux USB drivers
>>can take over before we switch from the initrd to the root filesystem.
>>
>>Greg, do you have a suggestion?
> 
> 
> I only ever saw this when people forgot to load the USB drivers.  Once
> the kernel took over USB support, there was no problem (if there was,
> that's a BIOS bug.)  The handoff code in 2.6.14 should help a lot with
> this too.
> 
Ah... are you saying that the USB support code stops the SMM/SMI prior to the TSC & delay 
calibration?  Also, this problem was noted in a 2.4.20 kernel.  Any help there?

-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: Calibration issues with USB disc present.

[Greg KH]

On Sat, Nov 12, 2005 at 08:06:37AM -0800, George Anzinger wrote:
> Greg KH wrote:
> >On Fri, Nov 11, 2005 at 01:57:07PM -0800, john stultz wrote:
> >
> >>On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
> >>
> >>>John,
> >>>
> >>>Have you run into this.  One of the USB disc controllers has the ability 
> >>>to boot the system, however, it needs SMM code to do this.  This SMM 
> >>>code, somehow, causes SMI interrupts (which are higher priority than NMI 
> >>>interrutps and not maskable) which it needs to do its thing.
> >>>
> >>>Problem is that if one of these occurs while calibrating the TSC or the 
> >>>delay code, it can cause a wrong result.  We have seen both a too long 
> >>>and a too short result (depending on where the interrut happens).
> >>>
> >>>They have found the root cause of TSC calibration problem.
> >>>Now they ask for the fix or workaround.
> >>>
> >>>That is the BIOS is periodically interrupted by USB controller and the 
> >>>CPU
> >>>waits during the processing of these interrupts.
> >>>Their experiments say the interrupt interval is 260mSec and the BIOS 
> >>>needs
> >>>150uSec - 200uSec for processing.
> >>>It is proved that the problem doesn't reproduce by masking such SMI in 
> >>>BIOS.
> >>>They say SMI is for BIOS emulation for connecting legacy devices to USB.
> >>>Without such an emulation it's impossible to boot from USB-FD for 
> >>>instance,
> >>>they say too.
> >>
> >>Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
> >>familiar with the pain BIOS SMIs can cause and I'm not surprised that it
> >>would affect the TSC/delay calibration code.
> >>
> >>Is this still an issue w/ 2.6.14? I know the new TSC based delay
> >>calibration code is supposed to be SMI resilient, but I haven't really
> >>played with it closely.
> >>
> >>Not sure what the best method to move forward would be. I suspect
> >>disabling the SMI code early in boot (I thought the usb legacy handoff
> >>stuff already did this?) would help. Then the actual Linux USB drivers
> >>can take over before we switch from the initrd to the root filesystem.
> >>
> >>Greg, do you have a suggestion?
> >
> >
> >I only ever saw this when people forgot to load the USB drivers.  Once
> >the kernel took over USB support, there was no problem (if there was,
> >that's a BIOS bug.)  The handoff code in 2.6.14 should help a lot with
> >this too.
> >
> Ah... are you saying that the USB support code stops the SMM/SMI prior to 
> the TSC & delay calibration?  Also, this problem was noted in a 2.4.20 
> kernel.  Any help there?

I do not know where in the boot process the TSC and delay calibration is
done.  If it happens before the PCI bus is probed, then no, it does not
happen before this.

On these boxes, I'd just recommend disabling USB legacy support
completly, if possible.  And then complain loudly to the vendor to fix
their BIOS.

thanks,

greg k-h

Re: Calibration issues with USB disc present.

[George Anzinger]

Greg KH wrote:
> On Sat, Nov 12, 2005 at 08:06:37AM -0800, George Anzinger wrote:
> 
>>Greg KH wrote:
>>
>>>On Fri, Nov 11, 2005 at 01:57:07PM -0800, john stultz wrote:
>>>
>>>
>>>>On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
>>>>
>>>>
>>>>>John,
>>>>>
>>>>>Have you run into this.  One of the USB disc controllers has the ability 
>>>>>to boot the system, however, it needs SMM code to do this.  This SMM 
>>>>>code, somehow, causes SMI interrupts (which are higher priority than NMI 
>>>>>interrutps and not maskable) which it needs to do its thing.
>>>>>
>>>>>Problem is that if one of these occurs while calibrating the TSC or the 
>>>>>delay code, it can cause a wrong result.  We have seen both a too long 
>>>>>and a too short result (depending on where the interrut happens).
>>>>>
>>>>>They have found the root cause of TSC calibration problem.
>>>>>Now they ask for the fix or workaround.
>>>>>
>>>>>That is the BIOS is periodically interrupted by USB controller and the 
>>>>>CPU
>>>>>waits during the processing of these interrupts.
>>>>>Their experiments say the interrupt interval is 260mSec and the BIOS 
>>>>>needs
>>>>>150uSec - 200uSec for processing.
>>>>>It is proved that the problem doesn't reproduce by masking such SMI in 
>>>>>BIOS.
>>>>>They say SMI is for BIOS emulation for connecting legacy devices to USB.
>>>>>Without such an emulation it's impossible to boot from USB-FD for 
>>>>>instance,
>>>>>they say too.
>>>>
>>>>Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
>>>>familiar with the pain BIOS SMIs can cause and I'm not surprised that it
>>>>would affect the TSC/delay calibration code.
>>>>
>>>>Is this still an issue w/ 2.6.14? I know the new TSC based delay
>>>>calibration code is supposed to be SMI resilient, but I haven't really
>>>>played with it closely.
>>>>
>>>>Not sure what the best method to move forward would be. I suspect
>>>>disabling the SMI code early in boot (I thought the usb legacy handoff
>>>>stuff already did this?) would help. Then the actual Linux USB drivers
>>>>can take over before we switch from the initrd to the root filesystem.
>>>>
>>>>Greg, do you have a suggestion?
>>>
>>>
>>>I only ever saw this when people forgot to load the USB drivers.  Once
>>>the kernel took over USB support, there was no problem (if there was,
>>>that's a BIOS bug.)  The handoff code in 2.6.14 should help a lot with
>>>this too.
>>>
>>
>>Ah... are you saying that the USB support code stops the SMM/SMI prior to 
>>the TSC & delay calibration?  Also, this problem was noted in a 2.4.20 
>>kernel.  Any help there?
> 
> 
> I do not know where in the boot process the TSC and delay calibration is
> done.  If it happens before the PCI bus is probed, then no, it does not
> happen before this.

My guess is that the PCI bus code would like to use delay() so I rather think it is calibrated prior 
to this :(
> 
> On these boxes, I'd just recommend disabling USB legacy support
> completly, if possible.  And then complain loudly to the vendor to fix
> their BIOS.

But if one is booting from that device...

-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: Calibration issues with USB disc present.

[Greg KH]

On Mon, Nov 14, 2005 at 10:56:05AM -0800, George Anzinger wrote:
> Greg KH wrote:
> >On Sat, Nov 12, 2005 at 08:06:37AM -0800, George Anzinger wrote:
> >
> >>Greg KH wrote:
> >>
> >>>On Fri, Nov 11, 2005 at 01:57:07PM -0800, john stultz wrote:
> >>>
> >>>
> >>>>On Fri, 2005-11-11 at 13:37 -0800, George Anzinger wrote:
> >>>>
> >>>>
> >>>>>John,
> >>>>>
> >>>>>Have you run into this.  One of the USB disc controllers has the 
> >>>>>ability to boot the system, however, it needs SMM code to do this.  
> >>>>>This SMM code, somehow, causes SMI interrupts (which are higher 
> >>>>>priority than NMI interrutps and not maskable) which it needs to do 
> >>>>>its thing.
> >>>>>
> >>>>>Problem is that if one of these occurs while calibrating the TSC or 
> >>>>>the delay code, it can cause a wrong result.  We have seen both a too 
> >>>>>long and a too short result (depending on where the interrut happens).
> >>>>>
> >>>>>They have found the root cause of TSC calibration problem.
> >>>>>Now they ask for the fix or workaround.
> >>>>>
> >>>>>That is the BIOS is periodically interrupted by USB controller and the 
> >>>>>CPU
> >>>>>waits during the processing of these interrupts.
> >>>>>Their experiments say the interrupt interval is 260mSec and the BIOS 
> >>>>>needs
> >>>>>150uSec - 200uSec for processing.
> >>>>>It is proved that the problem doesn't reproduce by masking such SMI in 
> >>>>>BIOS.
> >>>>>They say SMI is for BIOS emulation for connecting legacy devices to 
> >>>>>USB.
> >>>>>Without such an emulation it's impossible to boot from USB-FD for 
> >>>>>instance,
> >>>>>they say too.
> >>>>
> >>>>Hmmm. I haven't heard of this issue specifically, but yes, I'm quite
> >>>>familiar with the pain BIOS SMIs can cause and I'm not surprised that it
> >>>>would affect the TSC/delay calibration code.
> >>>>
> >>>>Is this still an issue w/ 2.6.14? I know the new TSC based delay
> >>>>calibration code is supposed to be SMI resilient, but I haven't really
> >>>>played with it closely.
> >>>>
> >>>>Not sure what the best method to move forward would be. I suspect
> >>>>disabling the SMI code early in boot (I thought the usb legacy handoff
> >>>>stuff already did this?) would help. Then the actual Linux USB drivers
> >>>>can take over before we switch from the initrd to the root filesystem.
> >>>>
> >>>>Greg, do you have a suggestion?
> >>>
> >>>
> >>>I only ever saw this when people forgot to load the USB drivers.  Once
> >>>the kernel took over USB support, there was no problem (if there was,
> >>>that's a BIOS bug.)  The handoff code in 2.6.14 should help a lot with
> >>>this too.
> >>>
> >>
> >>Ah... are you saying that the USB support code stops the SMM/SMI prior to 
> >>the TSC & delay calibration?  Also, this problem was noted in a 2.4.20 
> >>kernel.  Any help there?
> >
> >
> >I do not know where in the boot process the TSC and delay calibration is
> >done.  If it happens before the PCI bus is probed, then no, it does not
> >happen before this.
> 
> My guess is that the PCI bus code would like to use delay() so I rather 
> think it is calibrated prior to this :(
> >
> >On these boxes, I'd just recommend disabling USB legacy support
> >completly, if possible.  And then complain loudly to the vendor to fix
> >their BIOS.
> 
> But if one is booting from that device...

Booting from a USB device?  I can see this happening when installing a
distro, and you boot from the USB cdrom, but not for "normal"
operations.

Oh well, publicly mock the manufacturer for doing horrible things in
their BIOS and then no one will buy the boxes, and we will not have
problems :)

thanks,

greg k-h

Re: Calibration issues with USB disc present.

[Brad Campbell]

Greg KH wrote:
>>> completly, if possible.  And then complain loudly to the vendor to fix
>>> their BIOS.
>> But if one is booting from that device...
> 
> Booting from a USB device?  I can see this happening when installing a
> distro, and you boot from the USB cdrom, but not for "normal"
> operations.
> 

Just as a point of reference I have a machine here that boots from a USB keystick and runs from an 
NFS Root. It's a diskless machine that has no net-boot ability. I have used this on a number of 
machines over recent history.

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: Calibration issues with USB disc present.

[Greg KH]

On Mon, Nov 14, 2005 at 11:46:48PM +0400, Brad Campbell wrote:
> Greg KH wrote:
> >>>completly, if possible.  And then complain loudly to the vendor to fix
> >>>their BIOS.
> >>But if one is booting from that device...
> >
> >Booting from a USB device?  I can see this happening when installing a
> >distro, and you boot from the USB cdrom, but not for "normal"
> >operations.
> >
> 
> Just as a point of reference I have a machine here that boots from a USB 
> keystick and runs from an NFS Root. It's a diskless machine that has no 
> net-boot ability. I have used this on a number of machines over recent 
> history.

Then I guess your BIOS didn't do the horrible things that the original
poster was reporting, which is good :)

thanks,

greg k-h

Re: Calibration issues with USB disc present.

[john stultz]

On Mon, 2005-11-14 at 10:49 -0800, Greg KH wrote:
> On Mon, Nov 14, 2005 at 10:56:05AM -0800, George Anzinger wrote:
> > Greg KH wrote:
> > >On Sat, Nov 12, 2005 at 08:06:37AM -0800, George Anzinger wrote:
> > >>Greg KH wrote:
> > >On these boxes, I'd just recommend disabling USB legacy support
> > >completly, if possible.  And then complain loudly to the vendor to fix
> > >their BIOS.
> > 
> > But if one is booting from that device...
> 
> Booting from a USB device?  I can see this happening when installing a
> distro, and you boot from the USB cdrom, but not for "normal"
> operations.
> 
> Oh well, publicly mock the manufacturer for doing horrible things in
> their BIOS and then no one will buy the boxes, and we will not have
> problems :)

I suspect the right fix is in-between. We should try to push hardware
makers away from using SMIs recklessly, but we should also do our best
to work around those that don't. The same problems crop up w/
virtualization where time-based calibration may be interrupted.

George, again, there has been some SMI resistant delay calibration code
added recently. You mentioned this problem was seen on 2.4 kernel, so
you could verify that the new code in 2.6.14 works and if so, try
backporting it.

If not we need to see what else we can do about improving delay
calibration (its a similar tick-based problem to what I'm addressing
with the timeofday rework) or reducing the use of delay by using
something else.

thanks
-john

Re: Calibration issues with USB disc present.

[George Anzinger]

john stultz wrote:
> On Mon, 2005-11-14 at 10:49 -0800, Greg KH wrote:
> 
>>On Mon, Nov 14, 2005 at 10:56:05AM -0800, George Anzinger wrote:
>>
>>>Greg KH wrote:
>>>
>>>>On Sat, Nov 12, 2005 at 08:06:37AM -0800, George Anzinger wrote:
>>>>
>>>>>Greg KH wrote:
>>>>
>>>>On these boxes, I'd just recommend disabling USB legacy support
>>>>completly, if possible.  And then complain loudly to the vendor to fix
>>>>their BIOS.
>>>
>>>But if one is booting from that device...
>>
>>Booting from a USB device?  I can see this happening when installing a
>>distro, and you boot from the USB cdrom, but not for "normal"
>>operations.

I think it was a hard drive they were trying to support.
>>
>>Oh well, publicly mock the manufacturer for doing horrible things in
>>their BIOS and then no one will buy the boxes, and we will not have
>>problems :)

Long term, maybe, but it will not close the bug report I have in hand...
> 
> 
> I suspect the right fix is in-between. We should try to push hardware
> makers away from using SMIs recklessly, but we should also do our best
> to work around those that don't. The same problems crop up w/
> virtualization where time-based calibration may be interrupted.
> 
> George, again, there has been some SMI resistant delay calibration code
> added recently. You mentioned this problem was seen on 2.4 kernel, so
> you could verify that the new code in 2.6.14 works and if so, try
> backporting it.
> 
> If not we need to see what else we can do about improving delay
> calibration (its a similar tick-based problem to what I'm addressing
> with the timeofday rework) or reducing the use of delay by using
> something else.
> 
I will look at that code, but we also need to address the same problem in the TSC calibration area.
-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: Calibration issues with USB disc present.

[Max Krasnyansky]

George Anzinger wrote:

>>> Oh well, publicly mock the manufacturer for doing horrible things in
>>> their BIOS and then no one will buy the boxes, and we will not have
>>> problems :)
> 
> Long term, maybe, but it will not close the bug report I have in hand...
>>
>>
>> I suspect the right fix is in-between. We should try to push hardware
>> makers away from using SMIs recklessly, but we should also do our best
>> to work around those that don't. The same problems crop up w/
>> virtualization where time-based calibration may be interrupted.
>>
>> George, again, there has been some SMI resistant delay calibration code
>> added recently. You mentioned this problem was seen on 2.4 kernel, so
>> you could verify that the new code in 2.6.14 works and if so, try
>> backporting it.
>>
>> If not we need to see what else we can do about improving delay
>> calibration (its a similar tick-based problem to what I'm addressing
>> with the timeofday rework) or reducing the use of delay by using
>> something else.
>>
> I will look at that code, but we also need to address the same problem 
> in the TSC calibration area.

I think in the short term your best bet is to globally disable SMI at early
boot stage (ie before TSC calibration).
Some people might argue that it's not the most graceful solution because it might
brake some BIOS features but it's a very common trick that is used by RT folks
(for example RTAI has configurable option to enable SMI workaround) because
on some chipset/BIOS combinations SMIs introduce horrible latencies. And you
cannot do much about that other than disabling SMI.
I have not seen any reports of negative side effects of disabling SMI yet. But
if you're worried about that you could re-enable it later when you're done with
TSC calibration and stuff.

Max

Re: Calibration issues with USB disc present.

[Lee Revell]

On Wed, 2005-11-16 at 16:03 -0800, Max Krasnyansky wrote:
> I think in the short term your best bet is to globally disable SMI at
> early
> boot stage (ie before TSC calibration).
> Some people might argue that it's not the most graceful solution
> because it might
> brake some BIOS features but it's a very common trick that is used by
> RT folks
> (for example RTAI has configurable option to enable SMI workaround)
> because
> on some chipset/BIOS combinations SMIs introduce horrible latencies.
> And you
> cannot do much about that other than disabling SMI.
> I have not seen any reports of negative side effects of disabling SMI
> yet. But
> if you're worried about that you could re-enable it later when you're
> done with
> TSC calibration and stuff.
> 

Hmm, interesting, I had no idea this was possible.  Is there a generic
way to disable SMI?  It would be useful for the -rt tree as lots of low
latency audio users have problems with SMI induced underruns.

Lee

Re: Calibration issues with USB disc present.

[Max Krasnyansky]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

Lee Revell wrote:
> On Wed, 2005-11-16 at 16:03 -0800, Max Krasnyansky wrote:
>> I think in the short term your best bet is to globally disable SMI at early
>> boot stage (ie before TSC calibration).
> 
> Hmm, interesting, I had no idea this was possible.  Is there a generic
> way to disable SMI?  It would be useful for the -rt tree as lots of low
> latency audio users have problems with SMI induced underruns.
There is no generic way it's chipset specific.
Take a look at the attached smi.c that handles several Intel chipsets.
It's from Xenomai (ex Fusion branch of RTAI) source tree.

Max


[-- Attachment #2: smi.c --]
[-- Type: text/x-csrc, Size: 8593 bytes --]

/**
 *   @ingroup hal
 *   @file
 *
 *   SMI workaround for x86.
 *
 *   Cut/Pasted from Vitor Angelo "smi" module.
 *   Adapted by Gilles Chanteperdrix <gilles.chanteperdrix@laposte.net>.
 *
 *   This program is free software; you can redistribute it and/or modify
 *   it under the terms of the GNU General Public License as published by
 *   the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139,
 *   USA; either version 2 of the License, or (at your option) any later
 *   version.
 *
 *   This program is distributed in the hope that it will be useful,
 *   but WITHOUT ANY WARRANTY; without even the implied warranty of
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *   GNU General Public License for more details.
 *
 *   You should have received a copy of the GNU General Public License
 *   along with this program; if not, write to the Free Software
 *   Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
 */


#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
#include <linux/reboot.h>

#include <smi.h>

static struct pci_device_id rthal_smi_pci_tbl[] __initdata = {
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801AA_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801AB_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801BA_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801BA_10) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801E_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801CA_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801CA_12) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801DB_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801DB_12) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801EB_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_ICH6_0) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_ICH6_1) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_ICH6_2) },
{ 0, },
};

/* FIXME: Probably crippled too, need to be checked :

0x24dc 82801EB (ICH5) LPC Interface Bridge (not a real ID, but exists in the
pci.ids database, ICH5-M ?)
{ PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82801EB_12, PCI_ANY_ID, PCI_ANY_ID, },

*/

#define DEVFN        0xf8 /* device 31, function 0 */

#ifdef CONFIG_XENO_HW_SMI_WORKAROUND
    
#define PMBASE_B0    0x40
#define PMBASE_B1    0x41

#define SMI_CTRL_ADDR    0x30
#define SMI_STATUS_ADDR  0x34
#define SMI_MON_ADDR     0x40

/* SMI_EN register: ICH[0](16 bits), ICH[2-5](32 bits) */
#define INTEL_USB2_EN_BIT   (0x01 << 18) /* ICH4, ... */
#define LEGACY_USB2_EN_BIT  (0x01 << 17) /* ICH4, ... */
#define PERIODIC_EN_BIT     (0x01 << 14) /* called 1MIN_ in ICH0 */
#define TCO_EN_BIT          (0x01 << 13)
#define MCSMI_EN_BIT        (0x01 << 11)
#define SWSMI_TMR_EN_BIT    (0x01 << 6)
#define APMC_EN_BIT         (0x01 << 5)
#define SLP_EN_BIT          (0x01 << 4)
#define LEGACY_USB_EN_BIT   (0x01 << 3)
#define BIOS_EN_BIT         (0x01 << 2)
#define GBL_SMI_EN_BIT      (0x01) /* This is reset by a PCI reset event! */

static const unsigned rthal_smi_masked_bits =
#ifdef CONFIG_XENO_HW_SMI_ALL
    GBL_SMI_EN_BIT;
#else /* !defined(CONFIG_XENO_HW_SMI_ALL) */

    0
#ifndef CONFIG_XENO_HW_SMI_INTEL_USB2
    | INTEL_USB2_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_INTEL_USB2) */

#ifndef CONFIG_XENO_HW_SMI_LEGACY_USB2
    | LEGACY_USB2_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_LEGACY_USB2) */

#ifndef CONFIG_XENO_HW_SMI_PERIODIC
    | PERIODIC_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_PERIODC) */

#ifndef CONFIG_XENO_HW_SMI_TCO
    | TCO_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_TCO) */

#ifndef CONFIG_XENO_HW_SMI_MC
    | MCSMI_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_MCSMI) */

#ifndef CONFIG_XENO_HW_SMI_APMC
    | APMC_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_APMC) */

#ifndef CONFIG_XENO_HW_SMI_LEGACY_USB
    | LEGACY_USB_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_LEGACY_USB) */

#ifndef CONFIG_XENO_HW_SMI_BIOS
    | BIOS_EN_BIT
#endif /* !defined(CONFIG_XENO_HW_SMI_BIOS) */
    ;

#endif /* !defined(CONFIG_XENO_HW_SMI_ALL) */

static unsigned rthal_smi_saved_bits;
static unsigned short rthal_smi_en_addr;
static struct pci_dev *smi_dev;

#define mask_bits(v, p) outl(inl(p)&~(v),(p))
#define set_bits(v, p)  outl(inl(p)|(v), (p))

static int rthal_smi_reboot(struct notifier_block *nb, ulong event, void *buf);

static struct notifier_block rthal_smi_notifier = {
    .notifier_call = rthal_smi_reboot
};

static int rthal_smi_reboot(struct notifier_block *nb, ulong event, void *buf)
{
    if ((event != SYS_RESTART) && (event != SYS_HALT) &&
        (event != SYS_POWER_OFF))
        return NOTIFY_DONE;

    rthal_smi_restore();
    return NOTIFY_OK;
}

void rthal_smi_disable(void)
{
    if(!rthal_smi_en_addr)
        return;

    rthal_smi_saved_bits = inl(rthal_smi_en_addr) & rthal_smi_masked_bits;
    mask_bits(rthal_smi_masked_bits, rthal_smi_en_addr);

    register_reboot_notifier(&rthal_smi_notifier);
}

void rthal_smi_restore(void)
{
    if(!rthal_smi_en_addr)
        return;

    set_bits(rthal_smi_saved_bits, rthal_smi_en_addr);

    unregister_reboot_notifier(&rthal_smi_notifier);

    pci_dev_put(smi_dev);
}

static unsigned short __devinit get_smi_en_addr(struct pci_dev *dev)
{
    u_int8_t byte0, byte1;

    pci_read_config_byte (dev, PMBASE_B0, &byte0);
    pci_read_config_byte (dev, PMBASE_B1, &byte1);
    return SMI_CTRL_ADDR + (((byte1 << 1) | (byte0 >> 7)) << 7); // bits 7-15
}

#endif /* CONFIG_XENO_HW_SMI_WORKAROUND */

void __devinit rthal_smi_init(void)
{
    struct pci_dev *dev = NULL;
    struct pci_device_id *id;

    /*
     * Do not use pci_register_driver, pci_enable_device, ...
     * Just register the used ports.
     */
    for(id = &rthal_smi_pci_tbl[0]; dev == NULL && id->vendor != 0; id++)
        dev = pci_get_device(id->vendor, id->device, NULL);

    if(dev == NULL || dev->bus->number || dev->devfn != DEVFN)
        {
        pci_dev_put(dev);
        return ;
        }

#ifdef CONFIG_XENO_HW_SMI_WORKAROUND

    printk("Xenomai: Intel chipset found, enabling SMI workaround.\n");
    rthal_smi_en_addr = get_smi_en_addr(dev);
    smi_dev = dev;

#else /* ! CONFIG_XENO_HW_SMI_WORKAROUND */

    printk("Xenomai: Intel chipset found and SMI workaround not enabled,\n"
           "       you may encounter high interrupt latencies.\n");
    pci_dev_put(dev);

#endif /* ! CONFIG_XENO_HW_SMI_WORKAROUND */
}

/*

   FIXME: there are many more SMI sources than those of the SMI_EN
   register. From http://www.intel.com/design/chipsets/datashts/252516.htm
   there are at least the following other sources :

   pages 377, 386, 388, 389; Power management
       register GEN_PMCON1, bit SMI_LOCK, locks GLB_SMI_EN
       bits PER_SMI_SEL, allow selection of the periodic SMI
       registers PM1_STS, PM1_EN, PM1_CNT bit SCI_EN, if cleared generates SMI
       for power management events.

   pages 173, 381, 400; GPIOs
       register GPI[0-15]_ROUT allow routing each GPIO to SMI or SCI
       register ALT_GP_SMI_EN, ALT_GP_SMI_STS, allow masking SMIs for GPIOs

   pages 184, 188, 402; legacy devices emulation (ATA, floppy, parallel, UARTs,
       keyboard). I/O to specified ports may cause events, which can generate an
       SMI, depending on registers configuration :
       register DEVTRAP_EN, DEVTRAP_STS
       BIG FAT WARNING : globally disabling SMI on a box with SATA disks and
           SATA controller in "legacy" mode, probably prevents disks from
           working.

   pages 382, 383, 400; Monitors ?
       seem to be a generic legacy device emulation (like previous), registers
       MON[4-7]_FWD_EN, enables forwarding of I/O to LPC
       MON[4-7]_TRP_RNG, address of the emulated devices
       MON[4-7]_TRP_MSK and MON_SMI (registers MON[4-7]_TRAP_EN and
                                     MON[4-7]_TRAP_STS)

   page 407: TCO
       register TCO1_CNT, bit NMI2SMI_EN, enables TCO to use SMI instead of NMI,
       bit TCO_TMR_HLT, should be cleared to avoid being rebooted when the TCO
       timer expires. Dangerous bit: TCO_LOCK locks the TCO timer until reboot.
       register used by Linux drivers/char/watchdog/i8xx_tco.c

   page 492, 493: USB EHCI legacy support and SPECIAL SMI, i.e Intel Specific
       USB 2.0 SMI register.
       
   page 520, SMBus
       may be disabled by clearing register HOSTC, bit SMB_SMI_EN
       register used by Linux driver drivers/i2c/busses/i2c-i801.c

*/