From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 13 Dec 2007 03:38:02 +0000 (GMT) Received: from mail.gnss.com ([209.47.22.10]:38916 "EHLO tormf01.gmi.domain") by ftp.linux-mips.org with ESMTP id S20035727AbXLMDhx (ORCPT ); Thu, 13 Dec 2007 03:37:53 +0000 Received: from tormf01.gmi.domain (127.0.0.1) by tormf01.gmi.domain (MlfMTA v3.2r9) id hc2kh20171s3 for ; Wed, 12 Dec 2007 22:37:22 -0500 (envelope-from ) Received: from INDEXCH2003.gmi.domain ([10.41.1.181]) by tormf01.gmi.domain (SonicWALL 6.0.1.9157) with ESMTP; Wed, 12 Dec 2007 22:37:22 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C83D39.7724DEF1" Subject: Inter processor synchronization Date: Thu, 13 Dec 2007 09:07:20 +0530 Message-ID: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Inter processor synchronization Thread-Index: Acg9OXc4eZ2or01CQpavhNXQtClAVQ== From: "Nilanjan Roychowdhury" To: X-Mlf-Version: 6.0.1.9157 X-Mlf-UniqueId: o200712130337200048605 Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 17805 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: Nilanjan.Roychowdhury@gnss.com Precedence: bulk X-list: linux-mips This is a multi-part message in MIME format. ------_=_NextPart_001_01C83D39.7724DEF1 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Hi, I have a scenario where two images of the same Linux kernel are running on two MIPS cores. One is 24K and another is 4KEC. What is the best way to achieve inter processor synchronization between them? I guess the locks for LL/SC are local to a particular core and can not be extended across a multi core system.=20 =20 Will it be easier for me if both of them becomes same core ( like both 24k) and I run the SMP version of Linux. Please throw some light. =20 Thanks, Nilanjan =20 =20 ------_=_NextPart_001_01C83D39.7724DEF1 Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable

Hi,

I have a scenario where two images of the same Linux = kernel are running on two MIPS cores. One is 24K and another is 4KEC. What is = the best way to achieve inter processor synchronization between = them?

I guess the locks for LL/SC are local to a particular = core and can not be extended across a multi core system. =

 

Will it be easier for me if both of them becomes same = core ( like both 24k) and I run the SMP version of = Linux.

Please throw some light.

 

Thanks,

Nilanjan

 

 

------_=_NextPart_001_01C83D39.7724DEF1-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 13 Dec 2007 12:58:52 +0000 (GMT) Received: from localhost.localdomain ([127.0.0.1]:61149 "EHLO dl5rb.ham-radio-op.net") by ftp.linux-mips.org with ESMTP id S28575685AbXLMM6t (ORCPT ); Thu, 13 Dec 2007 12:58:49 +0000 Received: from denk.linux-mips.net (denk.linux-mips.net [127.0.0.1]) by dl5rb.ham-radio-op.net (8.14.1/8.13.8) with ESMTP id lBDCwm14001441; Thu, 13 Dec 2007 12:58:49 GMT Received: (from ralf@localhost) by denk.linux-mips.net (8.14.1/8.14.1/Submit) id lBDCwlx5001440; Thu, 13 Dec 2007 12:58:47 GMT Date: Thu, 13 Dec 2007 12:58:47 +0000 From: Ralf Baechle To: Nilanjan Roychowdhury Cc: linux-mips@linux-mips.org Subject: Re: Inter processor synchronization Message-ID: <20071213125847.GA1352@linux-mips.org> References: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> User-Agent: Mutt/1.5.17 (2007-11-01) Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 17806 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Thu, Dec 13, 2007 at 09:07:20AM +0530, Nilanjan Roychowdhury wrote: > I have a scenario where two images of the same Linux kernel are running > on two MIPS cores. One is 24K and another is 4KEC. What is the best way > to achieve inter processor synchronization between them? > > I guess the locks for LL/SC are local to a particular core and can not > be extended across a multi core system. 4K and 24K cores don't support cache coherency. So SMP is out of question. This is a _total_ showstopper for SMP, don't waste your time thinking on possible workarounds. The you could do is some sort of clusting, running two OS images, one on the 4K and one on the 24K which would communicate through a carefully cache managed or even uncached shared memory region. > Will it be easier for me if both of them becomes same core ( like both > 24k) and I run the SMP version of Linux. Within limits Linux supports mixing different CPU types such as R4000MC / R4400MC and R10000 / R12000 / R14000 mixes because those processors are similar enough Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 14 Dec 2007 04:21:51 +0000 (GMT) Received: from mail.gnss.com ([209.47.22.10]:10256 "EHLO tormf01.gmi.domain") by ftp.linux-mips.org with ESMTP id S20021610AbXLNEVl convert rfc822-to-8bit (ORCPT ); Fri, 14 Dec 2007 04:21:41 +0000 Received: from tormf01.gmi.domain (127.0.0.1) by tormf01.gmi.domain (MlfMTA v3.2r9) id hc82da0171sb; Thu, 13 Dec 2007 23:21:27 -0500 (envelope-from ) Received: from INDEXCH2003.gmi.domain ([10.41.1.181]) by tormf01.gmi.domain (SonicWALL 6.0.1.9157) with ESMTP; Thu, 13 Dec 2007 23:21:26 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 8BIT Subject: RE: Inter processor synchronization Date: Fri, 14 Dec 2007 09:51:23 +0530 Message-ID: <9D98C51005D80D43A19A3DF329A61D690106A297@INDEXCH2003.gmi.domain> In-Reply-To: <20071213125847.GA1352@linux-mips.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Inter processor synchronization Thread-Index: Acg9h+v6X6nQlzAoSFKIiGz9oKlz7AAgHufA References: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> <20071213125847.GA1352@linux-mips.org> From: "Nilanjan Roychowdhury" To: "Ralf Baechle" Cc: X-Mlf-Version: 6.0.1.9157 X-Mlf-UniqueId: o200712140421240079106 Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 17810 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: Nilanjan.Roychowdhury@gnss.com Precedence: bulk X-list: linux-mips on Thursday, December 13, 2007 6:29 PM:, Ralf Baechle wrote: > On Thu, Dec 13, 2007 at 09:07:20AM +0530, Nilanjan Roychowdhury wrote: > >> I have a scenario where two images of the same Linux kernel are >> running on two MIPS cores. One is 24K and another is 4KEC. What is >> the best way to achieve inter processor synchronization between them? >> >> I guess the locks for LL/SC are local to a particular core and can >> not be extended across a multi core system. > 4K and 24K cores don't support cache coherency. So SMP is out of > question. > This is a _total_ showstopper for SMP, don't waste your time thinking > on possible workarounds. > > The you could do is some sort of clusting, running two OS images, one > on the 4K and one on the 24K which would communicate through a > carefully cache managed or even uncached shared memory region. I guess I am left with only this option. Can you please throw some more lights On the IPC you are mentioning? >> Will it be easier for me if both of them becomes same core ( like >> both 24k) and I run the SMP version of Linux. > > > Within limits Linux supports mixing different CPU types such as > R4000MC / R4400MC and R10000 / R12000 / R14000 mixes because those > processors are similar enough > > Ralf Thanks, Nilanjan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 14 Dec 2007 09:34:41 +0000 (GMT) Received: from dns0.mips.com ([63.167.95.198]:46579 "EHLO dns0.mips.com") by ftp.linux-mips.org with ESMTP id S20023038AbXLNJec (ORCPT ); Fri, 14 Dec 2007 09:34:32 +0000 Received: from mercury.mips.com (mercury [192.168.64.101]) by dns0.mips.com (8.12.11/8.12.11) with ESMTP id lBE9QLMf005921; Fri, 14 Dec 2007 01:26:21 -0800 (PST) Received: from grendel (grendel [192.168.236.16]) by mercury.mips.com (8.13.5/8.13.5) with SMTP id lBE9QqPJ003446; Fri, 14 Dec 2007 01:26:53 -0800 (PST) Message-ID: <00c801c83e33$75572a00$10eca8c0@grendel> From: "Kevin D. Kissell" To: "Nilanjan Roychowdhury" , "Ralf Baechle" Cc: References: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> <20071213125847.GA1352@linux-mips.org> <9D98C51005D80D43A19A3DF329A61D690106A297@INDEXCH2003.gmi.domain> Subject: Re: Inter processor synchronization Date: Fri, 14 Dec 2007 10:26:49 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1914 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1914 Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 17811 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: kevink@mips.com Precedence: bulk X-list: linux-mips > >> I have a scenario where two images of the same Linux kernel are > >> running on two MIPS cores. One is 24K and another is 4KEC. What is > >> the best way to achieve inter processor synchronization between them? > >> > >> I guess the locks for LL/SC are local to a particular core and can > >> not be extended across a multi core system. Just to be clear, LL/SC are indeed local to a particular core *but*, in a cache coherent multiprocessor system, they provide multiprocessor synchronization - the fact that another core has referenced the coherent location will clear the link bit so that the SC will fail locally. > > 4K and 24K cores don't support cache coherency. So SMP is out of > > question. > > This is a _total_ showstopper for SMP, don't waste your time thinking > > on possible workarounds. > > > > The you could do is some sort of clusting, running two OS images, one > > on the 4K and one on the 24K which would communicate through a > > carefully cache managed or even uncached shared memory region. > > I guess I am left with only this option. Can you please throw some more > lights On the IPC you are mentioning? Unless one has special-purpose hardware that implements atomic operations (e.g. a hardware semaphore device), one must use algorithms that do not require atomic read-modify-write. Most classically, one uses mailboxes where each memory location has a single reader and a single writer. There are other, more general but less efficient algorithms (e.g. Decker's algorithm) out there as well. If one is doing this in cacheable memory, one needs to take care that (a) an explicit forced cache writeback operation is done to complete each update to the shared memory array, and (b) the "ownership" is at a granularity of a cache line, and not a memory word. If the memory is mapped uncached, and one has a message queue "next" pointer that is written by CPU A and a "last-read" pointer that is written by B, those two pointers can be in consecutive memory locations. But if the memory is cached, they must be in separate cache lines to avoid the writebacks of one CPU destroying the writebacks of another. Regards, Kevin K. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.gnss.com ([209.47.22.10]:38916 "EHLO tormf01.gmi.domain") by ftp.linux-mips.org with ESMTP id S20035727AbXLMDhx (ORCPT ); Thu, 13 Dec 2007 03:37:53 +0000 Received: from tormf01.gmi.domain (127.0.0.1) by tormf01.gmi.domain (MlfMTA v3.2r9) id hc2kh20171s3 for ; Wed, 12 Dec 2007 22:37:22 -0500 (envelope-from ) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C83D39.7724DEF1" Subject: Inter processor synchronization Date: Thu, 13 Dec 2007 09:07:20 +0530 Message-ID: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> From: "Nilanjan Roychowdhury" Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: linux-mips@linux-mips.org Message-ID: <20071213033720.I2T41Jl20slSJHdrDmwO1xNpldZfgpQzIUJPZJLRbA4@z> This is a multi-part message in MIME format. ------_=_NextPart_001_01C83D39.7724DEF1 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Hi, I have a scenario where two images of the same Linux kernel are running on two MIPS cores. One is 24K and another is 4KEC. What is the best way to achieve inter processor synchronization between them? I guess the locks for LL/SC are local to a particular core and can not be extended across a multi core system.=20 =20 Will it be easier for me if both of them becomes same core ( like both 24k) and I run the SMP version of Linux. Please throw some light. =20 Thanks, Nilanjan =20 =20 ------_=_NextPart_001_01C83D39.7724DEF1 Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable

Hi,

I have a scenario where two images of the same Linux = kernel are running on two MIPS cores. One is 24K and another is 4KEC. What is = the best way to achieve inter processor synchronization between = them?

I guess the locks for LL/SC are local to a particular = core and can not be extended across a multi core system. =

 

Will it be easier for me if both of them becomes same = core ( like both 24k) and I run the SMP version of = Linux.

Please throw some light.

 

Thanks,

Nilanjan

 

 

------_=_NextPart_001_01C83D39.7724DEF1-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.gnss.com ([209.47.22.10]:10256 "EHLO tormf01.gmi.domain") by ftp.linux-mips.org with ESMTP id S20021610AbXLNEVl convert rfc822-to-8bit (ORCPT ); Fri, 14 Dec 2007 04:21:41 +0000 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 8BIT Subject: RE: Inter processor synchronization Date: Fri, 14 Dec 2007 09:51:23 +0530 Message-ID: <9D98C51005D80D43A19A3DF329A61D690106A297@INDEXCH2003.gmi.domain> In-Reply-To: <20071213125847.GA1352@linux-mips.org> References: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> <20071213125847.GA1352@linux-mips.org> From: "Nilanjan Roychowdhury" Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: Ralf Baechle Cc: linux-mips@linux-mips.org Message-ID: <20071214042123.qf7bGojifio1_fIXQuLIIv2kLSrccfXjC_aNwDlLRb8@z> on Thursday, December 13, 2007 6:29 PM:, Ralf Baechle wrote: > On Thu, Dec 13, 2007 at 09:07:20AM +0530, Nilanjan Roychowdhury wrote: > >> I have a scenario where two images of the same Linux kernel are >> running on two MIPS cores. One is 24K and another is 4KEC. What is >> the best way to achieve inter processor synchronization between them? >> >> I guess the locks for LL/SC are local to a particular core and can >> not be extended across a multi core system. > 4K and 24K cores don't support cache coherency. So SMP is out of > question. > This is a _total_ showstopper for SMP, don't waste your time thinking > on possible workarounds. > > The you could do is some sort of clusting, running two OS images, one > on the 4K and one on the 24K which would communicate through a > carefully cache managed or even uncached shared memory region. I guess I am left with only this option. Can you please throw some more lights On the IPC you are mentioning? >> Will it be easier for me if both of them becomes same core ( like >> both 24k) and I run the SMP version of Linux. > > > Within limits Linux supports mixing different CPU types such as > R4000MC / R4400MC and R10000 / R12000 / R14000 mixes because those > processors are similar enough > > Ralf Thanks, Nilanjan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from dns0.mips.com ([63.167.95.198]:46579 "EHLO dns0.mips.com") by ftp.linux-mips.org with ESMTP id S20023038AbXLNJec (ORCPT ); Fri, 14 Dec 2007 09:34:32 +0000 Message-ID: <00c801c83e33$75572a00$10eca8c0@grendel> From: "Kevin D. Kissell" References: <9D98C51005D80D43A19A3DF329A61D690106A282@INDEXCH2003.gmi.domain> <20071213125847.GA1352@linux-mips.org> <9D98C51005D80D43A19A3DF329A61D690106A297@INDEXCH2003.gmi.domain> Subject: Re: Inter processor synchronization Date: Fri, 14 Dec 2007 10:26:49 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: Nilanjan Roychowdhury , Ralf Baechle Cc: linux-mips@linux-mips.org Message-ID: <20071214092649.ZI079K5D62RCIgUqr21Ew6MTKlayF3vB5dvATXqWN2c@z> > >> I have a scenario where two images of the same Linux kernel are > >> running on two MIPS cores. One is 24K and another is 4KEC. What is > >> the best way to achieve inter processor synchronization between them? > >> > >> I guess the locks for LL/SC are local to a particular core and can > >> not be extended across a multi core system. Just to be clear, LL/SC are indeed local to a particular core *but*, in a cache coherent multiprocessor system, they provide multiprocessor synchronization - the fact that another core has referenced the coherent location will clear the link bit so that the SC will fail locally. > > 4K and 24K cores don't support cache coherency. So SMP is out of > > question. > > This is a _total_ showstopper for SMP, don't waste your time thinking > > on possible workarounds. > > > > The you could do is some sort of clusting, running two OS images, one > > on the 4K and one on the 24K which would communicate through a > > carefully cache managed or even uncached shared memory region. > > I guess I am left with only this option. Can you please throw some more > lights On the IPC you are mentioning? Unless one has special-purpose hardware that implements atomic operations (e.g. a hardware semaphore device), one must use algorithms that do not require atomic read-modify-write. Most classically, one uses mailboxes where each memory location has a single reader and a single writer. There are other, more general but less efficient algorithms (e.g. Decker's algorithm) out there as well. If one is doing this in cacheable memory, one needs to take care that (a) an explicit forced cache writeback operation is done to complete each update to the shared memory array, and (b) the "ownership" is at a granularity of a cache line, and not a memory word. If the memory is mapped uncached, and one has a message queue "next" pointer that is written by CPU A and a "last-read" pointer that is written by B, those two pointers can be in consecutive memory locations. But if the memory is cached, they must be in separate cache lines to avoid the writebacks of one CPU destroying the writebacks of another. Regards, Kevin K.