Re: + spinlock_debug-dont-recompute-jiffies_per_loop.patch added to -mm tree

Re: + spinlock_debug-dont-recompute-jiffies_per_loop.patch added to -mm tree

[Dave Jones]

On Tue, Jul 25, 2006 at 12:10:28PM -0700, Andrew Morton wrote:
 > 
 > The patch titled
 > 
 >      spinlock_debug: don't recompute (jiffies_per_loop applied-patches arch backup bin block config-x COPYING CREDITS CREDITS~git-gfs2 crypto CVS Documentation drivers fs include init ipc Kbuild kernel lib linus-series log mach MAINTAINERS MAINTAINERS~avr32-arch MAINTAINERS~edac-new-opteron-athlon64-memory-controller-driver MAINTAINERS~git-gfs2 MAINTAINERS~git-klibc MAINTAINERS.orig MAINTAINERS~origin MAINTAINERS~qla3xxx-NIC-driver MAINTAINERS~x86_64-mm-add-a-maintainers-entry-for-calgary Makefile Makefile~git-kbuild Makefile~git-klibc Makefile~mm Makefile.orig Makefile.rej mm net notes patches pc README REPORTING-BUGS scripts security series 'series sound Tags test.img txt usr HZ) in spinloop
 > 
 > has been added to the -mm tree.  Its filename is

Wow, that's umm, spectacular ;-)

 > ------------------------------------------------------
 > Subject: spinlock_debug: don't recompute (jiffies_per_loop applied-patches arch backup bin block config-x COPYING CREDITS CREDITS~git-gfs2 crypto CVS Documentation drivers fs include init ipc Kbuild kernel lib linus-series log mach MAINTAINERS MAINTAINERS~avr32-arch MAINTAINERS~edac-new-opteron-athlon64-memory-controller-driver MAINTAINERS~git-gfs2 MAINTAINERS~git-klibc MAINTAINERS.orig MAINTAINERS~origin MAINTAINERS~qla3xxx-NIC-driver MAINTAINERS~x86_64-mm-add-a-maintainers-entry-for-calgary Makefile Makefile~git-kbuild Makefile~git-klibc Makefile~mm Makefile.orig Makefile.rej mm net notes patches pc README REPORTING-BUGS scripts security series 'series sound Tags test.img txt usr HZ) in spinloop
 > From: Chuck Ebbert <76306.1226@compuserve.com>
 > 
 > In spinlock_debug.c, the spinloops call __delay() on every iteration. 
 > Because that is an external function, (jiffies_per_loop * HZ), the loop's
 > iteration limit, gets recomputed every time.  Caching it explicitly
 > prevents that.

What is the purpose of those __delays being there at all ?
Seems odd to be waiting that long when the spinlock could become available
a lot sooner.  (These also make spinlock debug really painful
on boxes with huge numbers of CPUs).

		Dave

-- 
http://www.codemonkey.org.uk

Re: + spinlock_debug-dont-recompute-jiffies_per_loop.patch added to -mm tree

[Ingo Molnar]

* Dave Jones <davej@redhat.com> wrote:

>  > iteration limit, gets recomputed every time.  Caching it explicitly 
>  > prevents that.
> 
> What is the purpose of those __delays being there at all ? Seems odd 
> to be waiting that long when the spinlock could become available a lot 
> sooner.  (These also make spinlock debug really painful on boxes with 
> huge numbers of CPUs).

the debug code has to figure out when to trigger a deadlock warning 
message. If we are looping in a deadlock with irqs disabled on all CPUs, 
there's nothing that advances jiffies. The TSC is not reliable. The 
thing that remains is to use __delay(1). We could calibrate the loop 
separately perhaps?

	Ingo

Re: + spinlock_debug-dont-recompute-jiffies_per_loop.patch added to -mm tree

[Chuck Ebbert]

In-Reply-To: <20060725204306.GA22547@elte.hu>

On Tue, 25 Jul 2006 22:43:06 +0200, Ingo Molnar wrote:
> 
> * Dave Jones <davej@redhat.com> wrote:
> 
> >  > iteration limit, gets recomputed every time.  Caching it explicitly 
> >  > prevents that.
> > 
> > What is the purpose of those __delays being there at all ? Seems odd 
> > to be waiting that long when the spinlock could become available a lot 
> > sooner.  (These also make spinlock debug really painful on boxes with 
> > huge numbers of CPUs).
> 
> the debug code has to figure out when to trigger a deadlock warning 
> message. If we are looping in a deadlock with irqs disabled on all CPUs, 
> there's nothing that advances jiffies. The TSC is not reliable. The 
> thing that remains is to use __delay(1). We could calibrate the loop 
> separately perhaps?

Is there some reason this code:

                for (i = 0; i < loops_per_jiffy * HZ; i++) {
                        if (__raw_spin_trylock(&lock->raw_lock))
                                return;
                        __delay(1);
                }

needs to continuously try to update the spinlock?  Shouldn't it just
read it first, like this, to avoid the bus update traffic?

                        if (spin_can_lock(&lock->raw_lock) &&
                            __raw_spin_trylock(&lock->raw_lock))
                                return;


Also, looking at __delay(), I foresee problems on i386 with the HPET timer.
Every call to __delay() causes at least two HPET timer reads and it looks
like they're slow (using readl() on ioremapped memory, anyway.)

-- 
Chuck