Re: SMP spin-locks

Richard B. Johnson
2001-06-15T03:22:13Z

On Thu, 14 Jun 2001, Roger Larsson wrote:

> On Thursday 14 June 2001 23:05, you wrote:
> > On Thu, 14 Jun 2001, Roger Larsson wrote:
> > > Hi,
> > >
> > > Wait a minute...
> > >
> > > Spinlocks on a embedded system? Is it _really_ SMP?
> >
> > The embedded system is not SMP. However, there is definite
> > advantage to using an unmodified kernel that may/may-not
> > have been compiled for SMP. Of course spin-locks are used
> > to prevent interrupts from screwing up buffer pointers, etc.
> >
>
> Not really - it prevents another processor entering the same code
> segment (spin_lock_irqsave prevents both another processor and
> local interrupts).
>
> An interrupt on UP can not wait on a spin lock - it will never be released
> since no other code than the interrupt spinning will be able to execute)

An interrupt on a UP system will never spin, nor will the IP from
another CPU because there isn't another CPU. A spin-lock, compiled
for UP is:

pushf
popl some_register, currently EBX
cli ; Clear the interrupts on the only CPU you have

do_some_code_that_must_not_be_interrupted();

pushl same_register_as_above
popf ; Restore interrupts if they were enabled

For SMP is:

pushf
popl some_register
cli ; Clear interrupts
modify_a_memory_variable
x: see_if_it_is_what_you_expect
if_not_loop_to x

do_some_code_that_must_not_be_interrupted();

modify_the_memory_variable_back
pushl same_register_as_above
popf

Since `cli` will only stop interrupts on the CPU that actually
fetches the instruction, another CPU can enter the code unless
it is forced to spin until the lock is released.

If this code is executed on a UP machine, the memory variable
will always become exactly as expected so it will never spin.
Therefore SMP code should be perfectly safe on a UP machine,
in fact must be perfectly safe, or it's broken.

The current spinlock code does work perfectly on a UP machine.
However, the large difference in performance shows that something
is quite less than optimum in the coding.

Spinlocks are machine dependent. A simple increment of a byte
memory variable, spinning if it's not 1 will do fine. Decrementing
this variable will release the lock. A `lock` prefix is not necessary
because all Intel byte operations are atomic anyway. This assumes
that the lock was initialized to 0. It doesn't have to be. It
could be initialized to 0xaa (anything) and spin if it's not
0xab (or anything + 1).

>
> SMP compiled kernel, but running on UP hardware - right?
> Then this _should not_ happen!
>
> see linux/Documentation/spinlocks.txt
>

This, in fact, will happen. Machines booted from the network should
have SMP code so a SMP machine can use all its CPUs. This same
code, booted from the network, should have no measurable performance
penalty in UP machines.

Also, when you develop drivers on a workstation, test them on
a workstation, then upload everything to an embedded system, you
had better be executing the same code, kernel, drivers, et all,
or you are in a world of hurt. Many embedded systems don't have
any 'standard I/O' so you can't prove that it meets its specs
(exception handling, etc) on the target. You have to test that
logic elsewhere.

This workstation has two CPUs. All drivers are modules. It uses
initrd to install the ones for my SCSI disks, network, etc.

Script started on Thu Jun 14 23:13:10 2001
lsmod
Module Size Used by
ramdisk 4448 0
loop 8212 0 (autoclean)
ipx 19248 0 (unused)
3c59x 25020 1 (autoclean)
nls_cp437 4408 4 (autoclean)
BusLogic 38320 6
sd_mod 10932 6
scsi_mod 59460 2 [BusLogic sd_mod]
# exit
exit

Script done on Thu Jun 14 23:13:45 2001

The same kernel, uploaded to an embedded system, also uses
initrd to load the machine-specific drivers. In this way, only
the drivers that are actually used, are loaded. The kernel remains
small. There is a slight performance penality for using modules,
but none other.

# telnet platinum
Trying 10.106.100.166...
Connected to platinum.analogic.com.
Escape character is '^]'.

Enter "help" for commands

PLATINUM> sho modules

pcilynx 13468 1
raw1394 7984 1
ieee1394 22984 0 [pcilynx raw1394]
rtc_drvr 2372 0
vxibus 10660 6
gpib_drvr 19200 2
ramdisk 4428 0
pcnet32se 15640 1

PLATINUM> exit
Exit

Connection closed by foreign host.
# exit
exit

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.