From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 26 Oct 2006 17:50:17 +0100 (BST) Received: from localhost.localdomain ([127.0.0.1]:37781 "EHLO dl5rb.ham-radio-op.net") by ftp.linux-mips.org with ESMTP id S20038624AbWJZQuP (ORCPT ); Thu, 26 Oct 2006 17:50:15 +0100 Received: from denk.linux-mips.net (denk.linux-mips.net [127.0.0.1]) by dl5rb.ham-radio-op.net (8.13.8/8.13.7) with ESMTP id k9QGodbm029086; Thu, 26 Oct 2006 17:50:40 +0100 Received: (from ralf@localhost) by denk.linux-mips.net (8.13.8/8.13.8/Submit) id k9QGoS3x029056; Thu, 26 Oct 2006 17:50:37 +0100 Date: Thu, 26 Oct 2006 17:50:28 +0100 From: Ralf Baechle To: "Kevin D. Kissell" Cc: Atsushi Nemoto , creideiki+linux-mips@ferretporn.se, linux-mips@linux-mips.org Subject: Re: Extreme system overhead on large IP27 Message-ID: <20061026165023.GA24373@linux-mips.org> References: <20061024140614.GB27800@linux-mips.org> <6285.136.163.203.3.1161704681.squirrel@www.ferretporn.se> <20061025.174504.71086461.nemoto@toshiba-tops.co.jp> <20061026.130552.11963152.nemoto@toshiba-tops.co.jp> <20061026125624.GA14122@linux-mips.org> <003001c6f905$db2781f0$10eca8c0@grendel> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <003001c6f905$db2781f0$10eca8c0@grendel> User-Agent: Mutt/1.4.2.2i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 13104 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Thu, Oct 26, 2006 at 03:51:35PM +0200, Kevin D. Kissell wrote: > I don't see what's different here than in any other SMP case. It just happened to be a coonfiguration which happened to trigger the issue. But the underlying problem could exist on any other SMP system using per-processor timers. > Is it really > true that the MIPS SMP support *requires* that all CPUs in the system > come out of reset on the same clock, with the same value in Count? There isn't even an requirement to use the cp0 counter at all. It just happens to be that the VSMP kernel is using that timer. It also happens to be quite a logic choice on the Malta where the alternative would be specific to one of the several system controllers. SGI systems are infamous for potencially using mixed spec CPUs from the same family. That includes different clock speeds; something like having 180MHz R10000 and 500MHz R14000 would be possible. The only sane cure for the time code in such cases is avoiding c0_count and relying on some other system-wide time source. The same is may be needed in case of variable CPU clock. That said, Linux doesn't care just need a little bit of glue code to deal with arbitrary timers. > I find that very surprising (and a little disappointing). Is this a general > limitation of Linux? MIPS32/MIPS64 PRAs call out the reset value > of Count as being undefined, and chip specs for pre-MIPS32 CPUs > like the R10000 and the R4400 do not call out any reset value for > Count either. The count / compare code is very much did originate on uniprocessor systems and the sole thing it cares about is the speed the counter is incrementing at, not the absolute value. > If there's going to be skew between CPU clocks, all it really means > is that one cannot directly compare timestamps generated by different > CPUs. At a given point in time, "How long will it be until you hit an > absolute Count value X?" will have a slightly different answer on each CPU > if there is skew, but "What will the local Count value be N jiffies from now?" > should be something that can be correctly calculated independently on each > node. Where are we depending on the former, and can that usage be converted > into something more like the later? > Kevin K. Ralf