* Setting I&D cache enable in the same mtspr instruction
From: Assaf Hoffman @ 2006-05-08 10:39 UTC (permalink / raw)
To: linuxppc-embedded; +Cc: Rita Shtern, Ronen Shitrit
Hi,
I think the implementation of setup_common_caches() in file
cpu_setup_6xx.S; not according to the spec as far as MPC74xx concerns.
Looking in the spec (MPC7450 RISC Microprocessor Family Reference
Manual, MPC7450UM Rev. 5 1/2005) section 3.4.1.5 L1 Instruction and Data
Cache Flash Invalidation it says:=20
"Note that HID0[ICFI] and HID0[DCFI] must not both be set with the same
mtspr instruction, due to the synchronization requirements described in
Section 2.4.2.4.1, "Context Synchronization."
But in the code those two do set together.
Also, the same section says:=20
"An isync must precede the setting of the HID0[ICFI] in order for the
setting to take effect."
But in the code, only 'sync' can be found.
/* Enable caches for 603's, 604, 750 & 7400 */
setup_common_caches:
mfspr r11,SPRN_HID0
andi. r0,r11,HID0_DCE
ori r11,r11,HID0_ICE|HID0_DCE
ori r8,r11,HID0_ICFI
bne 1f /* don't invalidate the D-cache
*/
ori r8,r8,HID0_DCI /* unless it wasn't enabled */
1: sync
mtspr SPRN_HID0,r8 /* enable and invalidate caches
*/=20
^^^^^^^^^^^^^^^^^^^ Here we set both ICFI and DCFI in the same
mtspr instruction. Also, no isync before setting ICFI.
sync
mtspr SPRN_HID0,r11 /* enable caches */
sync
isync
blr
Please advice.
Thanks.
^ permalink raw reply
* Low latency Patch
From: Prabhat_Singh @ 2006-05-08 6:35 UTC (permalink / raw)
To: linuxppc-dev
[-- Attachment #1: Type: text/plain, Size: 698 bytes --]
Hi ,
Can any one please tell me from where I can get Low-latency and
rtc-debug patches for linux-2.4.24 kernel source and for MPC8248
processor platform.
Thanks
Prabhat Singh
DISCLAIMER:
This email (including any attachments) is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you are not the intended recipient, please contact the sender by email and delete all copies; your cooperation in this regard is appreciated.
[-- Attachment #2: Type: text/html, Size: 2884 bytes --]
^ permalink raw reply
* Low latecy patch
From: Prabhat_Singh @ 2006-05-08 6:02 UTC (permalink / raw)
To: linuxppc-embedded
[-- Attachment #1: Type: text/plain, Size: 689 bytes --]
Hi ,
Can any one please tell me from where I can get Low-latency and
rtc-debug patches for linux-2.4.24 kernel source and MPC8248 processor
platform.
Thanks
Prabhat Singh
DISCLAIMER:
This email (including any attachments) is intended for the sole use of the intended recipient/s and may contain material that is CONFIDENTIAL AND PRIVATE COMPANY INFORMATION. Any review or reliance by others or copying or distribution or forwarding of any or all of the contents in this message is STRICTLY PROHIBITED. If you are not the intended recipient, please contact the sender by email and delete all copies; your cooperation in this regard is appreciated.
[-- Attachment #2: Type: text/html, Size: 2748 bytes --]
^ permalink raw reply
* Re: [patch] powerpc: remove do-nothing cpu setup routines
From: Segher Boessenkool @ 2006-05-08 0:12 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Arnd Bergmann, linuxppc-dev, Paul Mackerras, cbe-oss-dev
In-Reply-To: <1147041095.4777.170.camel@localhost.localdomain>
>> Removes the processor specific do-nothing routines
>> __setup_cpu_power3 and
>> __setup_cpu_power4 with the generic routine __setup_cpu_null.
>
> Why not just change the caller to test for NULL ?
Yes, please do (as Paul suggested already).
Segher
^ permalink raw reply
* Re: [patch] powerpc: remove do-nothing cpu setup routines
From: Benjamin Herrenschmidt @ 2006-05-07 22:31 UTC (permalink / raw)
To: Geoff Levand; +Cc: linuxppc-dev, Paul Mackerras, cbe-oss-dev, Arnd Bergmann
In-Reply-To: <445BE72D.3050205@am.sony.com>
On Fri, 2006-05-05 at 17:00 -0700, Geoff Levand wrote:
> Removes the processor specific do-nothing routines __setup_cpu_power3 and
> __setup_cpu_power4 with the generic routine __setup_cpu_null.
Why not just change the caller to test for NULL ?
Cheers,
Ben.
^ permalink raw reply
* [PATCH] ppc32 8xx: Fix r3 thrashing due to 8MB TLB page instantiation (!CONFIG_8xx_CPU6)
From: Marcelo Tosatti @ 2006-05-05 20:09 UTC (permalink / raw)
To: Paul Mackerras; +Cc: David Jander, linux-ppc-embedded
Instantiation of 8MB pages on the TLB cache for the kernel static
mapping thrashes r3 register on !CONFIG_8xx_CPU6 configurations.
Signed-off-by: Marcelo Tosatti <marcelo@kvack.org>
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index ec53c7d..7a2f205 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -355,9 +355,7 @@ #endif
. = 0x1200
DataStoreTLBMiss:
-#ifdef CONFIG_8xx_CPU6
stw r3, 8(r0)
-#endif
DO_8xx_CPU6(0x3f80, r3)
mtspr SPRN_M_TW, r10 /* Save a couple of working registers */
mfcr r10
@@ -417,9 +415,7 @@ #endif
lwz r11, 0(r0)
mtcr r11
lwz r11, 4(r0)
-#ifdef CONFIG_8xx_CPU6
lwz r3, 8(r0)
-#endif
rfi
/* This is an instruction TLB error on the MPC8xx. This could be due
^ permalink raw reply related
* Re: Calculating virtual address from physical address
From: Grant Likely @ 2006-05-06 19:38 UTC (permalink / raw)
To: Chris Dumoulin; +Cc: dhlii, linuxppc-embedded
In-Reply-To: <200605061843.k46Ih5Vd032048@www-webmail1.magma.ca>
On 5/6/06, Chris Dumoulin <cdumoulin@ics-ltd.com> wrote:
> You said that the temporary TLB entries setup in head_4xx.S will eventual=
ly be replaced.
> Where is the code that creates these new TLB entries later on? Are the 'r=
eal' TLB entries
> only created once, and persist for as long as the system is running, or d=
o TLB entries
> change often while the system is running?
The kernel maintains a list of mappings between virtual and physical
space. When the processor takes a TLB miss exception, then exception
handler loads the needed mapping into the TLB and returns from
exception. (Look in head_4xx.S; specifically at finish_tlb_load).=20
TLB entries are loaded in a round-robin fashion as needed. Since your
early TLB mappings aren't in the kernel page tables; they get
overwritten and can't be reloaded by the TLB miss exception handler.
> Can you point me to some information about Grant's platform bus changes t=
hat you were
> talking about? I am using a custom V2Pro board, and I'd be interested to =
see if this code
> is something I should be looking at.
The platform bus changes moves the devices to use the "Platform Bus"
infrastructure. It's kind of a move away from multiple
processor-specific bus management schemes for "simple" busses. ie.=20
If the processor can access it without special setup code; the device
can go on the platform bus.
Check out http://patchwork.ozlabs.org/linuxppc/ and search for my
name. There's about 9 patches grouped together.
Cheers,
g.
--
Grant Likely, B.Sc. P.Eng.
Secret Lab Technologies Ltd.
(403) 399-0195
^ permalink raw reply
* Re: Calculating virtual address from physical address
From: Chris Dumoulin @ 2006-05-06 18:43 UTC (permalink / raw)
To: dhlii, Chris Dumoulin; +Cc: linuxppc-embedded
Thanks for your reply; I found it very useful and interesting. Now, I have a whole bunch
of questions.
You said that the temporary TLB entries setup in head_4xx.S will eventually be replaced.
Where is the code that creates these new TLB entries later on? Are the 'real' TLB entries
only created once, and persist for as long as the system is running, or do TLB entries
change often while the system is running?
Do you know what state the MSR will be in at this point in the code? I know what the
power-on reset state is, but I'm wondering if it'll be in a different state by the time we
get to this point in head_4xx.S.
When you suggest disabling instruction or data address translation, is that just so I
could access my hardware directly, or is there some other reason?
You were enabling the MSR bits, one at a time, and found that the machine check was
causing the hang (I'm assuming that's what you meant by 'sent me to space'). Was the idea
there to just isolate what type of exception was causing the hang, or were you looking to
make some permanent changes to the MSR? Is a machine check interrupt caused by trying to
access an address that doesn't have a TLB entry?
Can you point me to some information about Grant's platform bus changes that you were
talking about? I am using a custom V2Pro board, and I'd be interested to see if this code
is something I should be looking at.
Thanks alot,
Chris
On May 05, "David H. Lynch Jr." <dhlii@dlasys.net> wrote:
>
> Chris Dumoulin wrote:
> > My LEDs are at address 0x4F600000 and my CONFIG_KERNEL_START is
> > 0xC0000000. If this address were low enough, I would just add 0xC0000000
> > to the address to get the virtual address, but since my LED address is
> > so high, the sum will be well past the 32-bit maximum address value. How
> > is a virtual address calculated for a high address like 0x4F600000?
> >
> There are macros tophys and tovirt that convert addresses between
> physical and virtual. There are use example in the head_4xx.S file you
> are already in.
>
> If you are going to use a port for debugging you need to create a
> tlb entry for it.
> Same file in initial_mmu the code inside the if
> defined(CONFIG_SERIAL_TEXT_DEBUG) should provide an example how to do that.
>
> Be forwarned that any entries you create now will eventually
> disappear (took 2 weeks to figure that out once), but that may not
> happen intil after /init starts.
>
> Also with a little of jiggering arround the bits in MSR_KERNEL you
> can enable Data address translation independently of instruction address
> translation as well as disable or enable a variety of
> checks. It took me three weeks to get a new Xilinx V4 board through
> the rfi and to start_here in the same turn_on_mmu code you are working on.
>
> Eventually I ended up enabling the MSR bits one at a time until I
> discovered that enabling the Machine Check sent me to space.
>
> Regardless, once I relialized I could test the code with the MSR
> bits enabled one at a time isolating the problem became easier.
>
>
> The two issues I addressed above which relate specifically to your
> efforts with the ml300, constituted more than 80% of my effort to get a
> Xilinx Virtex 4 running.
>
> Finally, I started prior to grants platform bus changes. I have been
> adapting my V4 code to fit with Grants changes (the client has what they
> want so they do not care)
> I have not put alot of effort into this, but I currently get
> waylayed much later in new platform bus specific initialization code.
> I had no problem with the older board specific initialization code.
>
> If you are running on a real ml300 I am surprised you are having any
> problems though I do not have an ml300 to check that.
> But if you are running on a custom V2Pro board you have to get the
> board specific initalization right and therefore could trip over the
> issue I am currently having migrating from old to new.
>
>
>
>
>
>
>
> > BTW, he is the assembly code that I'm working with (from
> > arch/ppc/kernel/head_4xx.S):
> >
> > .text
> > _GLOBAL(_stext)
> > _GLOBAL(_start)
> >
> > /* Save parameters we are passed.
> > */
> > mr r31,r3
> > mr r30,r4
> > mr r29,r5
> > mr r28,r6
> > mr r27,r7
> >
> > /* CRD: set LED state here */
> > lis r26,0x4F600000@h
> > ori r26,r26,0x4F600000@l
> > li r25,LED_STATE_0
> > stw r25,0(r26)
> >
> > /* We have to turn on the MMU right away so we get cache modes
> > * set correctly.
> > */
> > bl initial_mmu
> >
> > /* CRD: set LED state here */
> > lis r26,0x4F600000@h
> > ori r26,r26,0x4F600000@l
> > li r25,LED_STATE_1
> > stw r25,0(r26)
> >
> > /* We now have the lower 16 Meg mapped into TLB entries, and the caches
> > * ready to work.
> > */
> > turn_on_mmu:
> > lis r0,MSR_KERNEL@h
> > ori r0,r0,MSR_KERNEL@l
> > mtspr SPRN_SRR1,r0
> > lis r0,start_here@h
> > ori r0,r0,start_here@l
> > mtspr SPRN_SRR0,r0
> > SYNC
> >
> > /* CRD: set LED state here */
> > lis r26,0x4F600000@h
> > ori r26,r26,0x4F600000@l
> > li r25,LED_STATE_2
> > stw r25,0(r26)
> >
> > rfi /* enables MMU */
> >
> > /* CRD: set LED state here */
> > /* This address should be a virtual address */
> > lis r26,0x4F600000@h
> > ori r26,r26,0x4F600000@l
> > li r25,LED_STATE_3
> > stw r25,0(r26)
> >
> > b . /* prevent prefetch past rfi */
> >
> > Regards,
> > Chris Dumoulin
> >
>
>
> --
> Dave Lynch DLA Systems
> Software Development: Embedded Linux
> 717.627.3770 dhlii@dlasys.net <a
href='http://www.dlasys.net'>http://www.dlasys.net</a>
> fax: 1.253.369.9244 Cell: 1.717.587.7774
> Over 25 years' experience in platforms, languages, and technologies too numerous to list.
>
> "Any intelligent fool can make things bigger and more complex... It takes a touch of
genius - and a lot of courage to move in the opposite direction."
> Albert Einstein
>
>
^ permalink raw reply
* RE: Calculating virtual address from physical address
From: Alexander Szekely @ 2006-05-06 8:43 UTC (permalink / raw)
To: linuxppc-embedded
> How is a virtual address=20
> calculated for a high address like 0x4F600000?
Add this to the MMU initialzation code (initial_mmu):
/* map leds - 1:1 phys=3Dvirt */
lis r3,0x4F600000@h
ori r3,r3,0x4F600000@l
mr r4,r3
clrrwi r4,r4,12
ori r4,r4,(TLB_WR|TLB_I|TLB_M|TLB_G)
clrrwi r3,r3,12
ori r3,r3,(TLB_VALID | TLB_PAGESZ(PAGESZ_4K))
li r0,0 /* TLB slot 0 */
tlbwe r4,r0,TLB_DATA
tlbwe r3,r0,TLB_TAG
Then you have a one to one mapping between physical and virtual memory,=20
and you can access the LEDs at the same address after turning one the
MMU.
BTW, we had a similar problem, but the reason was not turning on the
MMU. The MMU is enabled by writing MSR_KERNEL the status register. This
also enables the machine state exception interrupt.=20
Our hardware design produced PLB bus exceptions, which was unnoticed on
standalone applications. But this bus exceptions also trigger machine
state exceptions. So instead of jumping to start_here the processor
branched to 0x200 - unfortunatly the trap table is not set up at this
point in the boot process...
Alex=20
^ permalink raw reply
* [patch] powerpc: remove do-nothing cpu setup routines
From: Geoff Levand @ 2006-05-06 0:00 UTC (permalink / raw)
To: Paul Mackerras; +Cc: linuxppc-dev, cbe-oss-dev, Arnd Bergmann
Removes the processor specific do-nothing routines __setup_cpu_power3 and
__setup_cpu_power4 with the generic routine __setup_cpu_null.
Signed-off-by: Geoff Levand <geoffrey.levand@am.sony.com>
Index: cell--alp--3/arch/powerpc/kernel/cpu_setup_power4.S
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/cpu_setup_power4.S 2006-05-05 15:59:58.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/cpu_setup_power4.S 2006-05-05 16:40:44.000000000 -0700
@@ -73,9 +73,6 @@
isync
blr
-_GLOBAL(__setup_cpu_power4)
- blr
-
_GLOBAL(__setup_cpu_ppc970)
mfspr r0,SPRN_HID0
li r11,5 /* clear DOZE and SLEEP */
Index: cell--alp--3/arch/powerpc/kernel/cputable.c
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/cputable.c 2006-05-05 16:29:06.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/cputable.c 2006-05-05 16:40:41.000000000 -0700
@@ -32,8 +32,6 @@
*/
#ifdef CONFIG_PPC64
extern void __setup_cpu_null(unsigned long offset, struct cpu_spec* spec);
-extern void __setup_cpu_power3(unsigned long offset, struct cpu_spec* spec);
-extern void __setup_cpu_power4(unsigned long offset, struct cpu_spec* spec);
#else
extern void __setup_cpu_603(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_604(unsigned long offset, struct cpu_spec* spec);
@@ -80,7 +78,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power3",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "power3",
@@ -94,7 +92,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power3",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "power3",
@@ -108,7 +106,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/rs64",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "rs64",
@@ -122,7 +120,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/rs64",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "rs64",
@@ -136,7 +134,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/rs64",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "rs64",
@@ -150,7 +148,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power3,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/rs64",
.oprofile_type = PPC_OPROFILE_RS64,
.platform = "rs64",
@@ -164,7 +162,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power4,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power4",
.oprofile_type = PPC_OPROFILE_POWER4,
.platform = "power4",
@@ -178,7 +176,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 8,
- .cpu_setup = __setup_cpu_power4,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power4",
.oprofile_type = PPC_OPROFILE_POWER4,
.platform = "power4",
@@ -244,7 +242,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 6,
- .cpu_setup = __setup_cpu_power4,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power5",
.oprofile_type = PPC_OPROFILE_POWER4,
.platform = "power5",
@@ -258,7 +256,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 6,
- .cpu_setup = __setup_cpu_power4,
+ .cpu_setup = __setup_cpu_null,
.oprofile_cpu_type = "ppc64/power5+",
.oprofile_type = PPC_OPROFILE_POWER4,
.platform = "power5+",
@@ -285,7 +283,7 @@
.icache_bsize = 128,
.dcache_bsize = 128,
.num_pmcs = 6,
- .cpu_setup = __setup_cpu_power4,
+ .cpu_setup = __setup_cpu_null,
.platform = "power4",
}
#endif /* CONFIG_PPC64 */
Index: cell--alp--3/arch/powerpc/kernel/misc_64.S
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/misc_64.S 2006-05-05 16:04:59.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/misc_64.S 2006-05-05 16:40:47.000000000 -0700
@@ -771,9 +771,6 @@
_GLOBAL(__setup_cpu_null)
blr
-_GLOBAL(__setup_cpu_power3)
- blr
-
_GLOBAL(execve)
li r0,__NR_execve
sc
^ permalink raw reply
* Re: [PATCH 04/13] cell: remove broken __setup_cpu_be function
From: Geoff Levand @ 2006-05-06 0:00 UTC (permalink / raw)
To: Paul Mackerras
Cc: Arnd Bergmann, Levand, Geoffrey, linux-kernel, linuxppc-dev,
Arnd Bergmann, cbe-oss-dev
In-Reply-To: <17498.60066.92373.6527@cargo.ozlabs.ibm.com>
Paul Mackerras wrote:
> Arnd Bergmann writes:
>
>> From: Geoff Levand <geoffrey.levand@am.sony.com>
>>
>> This patch removes the incorrect Cell processor setup routine
>> __setup_cpu_be. This routine improperly accesses the hypervisor
>> page size configuration at SPR HID6. The correct behavior is for
>> firmware, or if needed, platform setup code, to set the correct
>> page size.
>
>> - .cpu_setup = __setup_cpu_be,
>> + .cpu_setup = __setup_cpu_power4,
>
> That looks a bit dodgy. Either just remove the contents of
> __setup_cpu_be (leaving only the blr), or define a __setup_cpu_null
> that does nothing, or make the identify_cpu not call the cpu setup
> function if the pointer is NULL.
OK, I set it up with __setup_cpu_null. An updated patch follows.
It falls out from this that we can replace the do-nothing routines
__setup_cpu_power3 and __setup_cpu_power4 with __setup_cpu_null also.
I'll post a separate patch for consideration.
-Geoff
Replaced the Cell processor specific routine __setup_cpu_be with
a new generic routine __setup_cpu_null. __setup_cpu_be improperly
accessed the hypervisor page size configuration at SPR HID6. Correct
behavior is for firmware, or if needed, platform setup code, to set
the correct page size.
Signed-off-by: Geoff Levand <geoffrey.levand@am.sony.com>
Index: cell--alp--3/arch/powerpc/kernel/cpu_setup_power4.S
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/cpu_setup_power4.S 2006-04-26 19:19:25.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/cpu_setup_power4.S 2006-05-05 15:59:58.000000000 -0700
@@ -76,20 +76,6 @@
_GLOBAL(__setup_cpu_power4)
blr
-_GLOBAL(__setup_cpu_be)
- /* Set large page sizes LP=0: 16MB, LP=1: 64KB */
- addi r3, 0, 0
- ori r3, r3, HID6_LB
- sldi r3, r3, 32
- nor r3, r3, r3
- mfspr r4, SPRN_HID6
- and r4, r4, r3
- addi r3, 0, 0x02000
- sldi r3, r3, 32
- or r4, r4, r3
- mtspr SPRN_HID6, r4
- blr
-
_GLOBAL(__setup_cpu_ppc970)
mfspr r0,SPRN_HID0
li r11,5 /* clear DOZE and SLEEP */
Index: cell--alp--3/arch/powerpc/kernel/cputable.c
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/cputable.c 2006-04-26 19:19:25.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/cputable.c 2006-05-05 16:29:06.000000000 -0700
@@ -31,9 +31,9 @@
* and ppc64
*/
#ifdef CONFIG_PPC64
+extern void __setup_cpu_null(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_power3(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_power4(unsigned long offset, struct cpu_spec* spec);
-extern void __setup_cpu_be(unsigned long offset, struct cpu_spec* spec);
#else
extern void __setup_cpu_603(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_604(unsigned long offset, struct cpu_spec* spec);
@@ -273,7 +273,7 @@
PPC_FEATURE_SMT,
.icache_bsize = 128,
.dcache_bsize = 128,
- .cpu_setup = __setup_cpu_be,
+ .cpu_setup = __setup_cpu_null,
.platform = "ppc-cell-be",
},
{ /* default match */
Index: cell--alp--3/arch/powerpc/kernel/misc_64.S
===================================================================
--- cell--alp--3.orig/arch/powerpc/kernel/misc_64.S 2006-04-26 19:19:25.000000000 -0700
+++ cell--alp--3/arch/powerpc/kernel/misc_64.S 2006-05-05 16:04:59.000000000 -0700
@@ -768,6 +768,9 @@
#endif /* CONFIG_ALTIVEC */
+_GLOBAL(__setup_cpu_null)
+ blr
+
_GLOBAL(__setup_cpu_power3)
blr
^ permalink raw reply
* Re: Calculating virtual address from physical address
From: David H. Lynch Jr. @ 2006-05-05 23:35 UTC (permalink / raw)
To: Chris Dumoulin; +Cc: linuxppc-embedded
In-Reply-To: <445B6A80.2090202@ics-ltd.com>
Chris Dumoulin wrote:
> My LEDs are at address 0x4F600000 and my CONFIG_KERNEL_START is
> 0xC0000000. If this address were low enough, I would just add 0xC0000000
> to the address to get the virtual address, but since my LED address is
> so high, the sum will be well past the 32-bit maximum address value. How
> is a virtual address calculated for a high address like 0x4F600000?
>
There are macros tophys and tovirt that convert addresses between
physical and virtual. There are use example in the head_4xx.S file you
are already in.
If you are going to use a port for debugging you need to create a
tlb entry for it.
Same file in initial_mmu the code inside the if
defined(CONFIG_SERIAL_TEXT_DEBUG) should provide an example how to do that.
Be forwarned that any entries you create now will eventually
disappear (took 2 weeks to figure that out once), but that may not
happen intil after /init starts.
Also with a little of jiggering arround the bits in MSR_KERNEL you
can enable Data address translation independently of instruction address
translation as well as disable or enable a variety of
checks. It took me three weeks to get a new Xilinx V4 board through
the rfi and to start_here in the same turn_on_mmu code you are working on.
Eventually I ended up enabling the MSR bits one at a time until I
discovered that enabling the Machine Check sent me to space.
Regardless, once I relialized I could test the code with the MSR
bits enabled one at a time isolating the problem became easier.
The two issues I addressed above which relate specifically to your
efforts with the ml300, constituted more than 80% of my effort to get a
Xilinx Virtex 4 running.
Finally, I started prior to grants platform bus changes. I have been
adapting my V4 code to fit with Grants changes (the client has what they
want so they do not care)
I have not put alot of effort into this, but I currently get
waylayed much later in new platform bus specific initialization code.
I had no problem with the older board specific initialization code.
If you are running on a real ml300 I am surprised you are having any
problems though I do not have an ml300 to check that.
But if you are running on a custom V2Pro board you have to get the
board specific initalization right and therefore could trip over the
issue I am currently having migrating from old to new.
> BTW, he is the assembly code that I'm working with (from
> arch/ppc/kernel/head_4xx.S):
>
> .text
> _GLOBAL(_stext)
> _GLOBAL(_start)
>
> /* Save parameters we are passed.
> */
> mr r31,r3
> mr r30,r4
> mr r29,r5
> mr r28,r6
> mr r27,r7
>
> /* CRD: set LED state here */
> lis r26,0x4F600000@h
> ori r26,r26,0x4F600000@l
> li r25,LED_STATE_0
> stw r25,0(r26)
>
> /* We have to turn on the MMU right away so we get cache modes
> * set correctly.
> */
> bl initial_mmu
>
> /* CRD: set LED state here */
> lis r26,0x4F600000@h
> ori r26,r26,0x4F600000@l
> li r25,LED_STATE_1
> stw r25,0(r26)
>
> /* We now have the lower 16 Meg mapped into TLB entries, and the caches
> * ready to work.
> */
> turn_on_mmu:
> lis r0,MSR_KERNEL@h
> ori r0,r0,MSR_KERNEL@l
> mtspr SPRN_SRR1,r0
> lis r0,start_here@h
> ori r0,r0,start_here@l
> mtspr SPRN_SRR0,r0
> SYNC
>
> /* CRD: set LED state here */
> lis r26,0x4F600000@h
> ori r26,r26,0x4F600000@l
> li r25,LED_STATE_2
> stw r25,0(r26)
>
> rfi /* enables MMU */
>
> /* CRD: set LED state here */
> /* This address should be a virtual address */
> lis r26,0x4F600000@h
> ori r26,r26,0x4F600000@l
> li r25,LED_STATE_3
> stw r25,0(r26)
>
> b . /* prevent prefetch past rfi */
>
> Regards,
> Chris Dumoulin
>
--
Dave Lynch DLA Systems
Software Development: Embedded Linux
717.627.3770 dhlii@dlasys.net http://www.dlasys.net
fax: 1.253.369.9244 Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.
"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein
^ permalink raw reply
* Re: Calculating virtual address from physical address
From: Josh Boyer @ 2006-05-05 20:58 UTC (permalink / raw)
To: Matt Porter; +Cc: Sylvain Munaut, Chris Dumoulin, linuxppc-embedded
In-Reply-To: <20060505132441.B30563@cox.net>
On Fri, 2006-05-05 at 13:24 -0700, Matt Porter wrote:
>
> There are no BATs on 4xx. However, the same conceptual thing can be
> done by wiring a fixed TLB entry to cover those LEDs temporarily
> during bringup debug. The temporary TLB entry will be wiped out by
> normal tlb misses after things are running whenever the fixed entry
> is clobbered by the round robin replacement.
At which point you should be able to call ioremap to generate a virtual
address inside a device driver or board file.
josh
^ permalink raw reply
* Re: Calculating virtual address from physical address
From: Matt Porter @ 2006-05-05 20:24 UTC (permalink / raw)
To: Sylvain Munaut; +Cc: Chris Dumoulin, linuxppc-embedded
In-Reply-To: <445BA736.30208@246tNt.com>
On Fri, May 05, 2006 at 09:27:50PM +0200, Sylvain Munaut wrote:
> Chris Dumoulin wrote:
> > I'm using a Virtex II Pro-based board with a PPC405. The board is
> > hanging somewhere very early in the kernel boot process. I believe it
> > may be dying at the point where the MMU is enabled. In order to
> > determine the exact point at which my board hangs, I'm blinking two LEDs
> > in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I
> > am only able to successfully access the LEDs before the MMU is turned
> > on, but I can't be sure that I'm calculating the virtual address
> > properly when I try to access the LED after the MMU is turned on.
>
> Typical when trying to bring up board ...
>
> Once the MMU is turned on, you leds register are most likely ... nowhere
> ... i.e.
> if you don't create a mapping your self there is just no virtual address
> that will
> access your leds physical address.
>
> What I did on some ppc work was tu use a quick BAT mapping to map some leds.
> It's pretty easy to setup. Be aware though that this mapping will get
> wiped out when
> the kernel sets up the BAT for itself.
There are no BATs on 4xx. However, the same conceptual thing can be
done by wiring a fixed TLB entry to cover those LEDs temporarily
during bringup debug. The temporary TLB entry will be wiped out by
normal tlb misses after things are running whenever the fixed entry
is clobbered by the round robin replacement.
-Matt
^ permalink raw reply
* [PATCH] ppc32 8xx: Fix r3 thrashing due to 8MB TLB page instantiation (!CONFIG_8xx_CPU6)
From: Marcelo Tosatti @ 2006-05-05 20:22 UTC (permalink / raw)
To: Paul Mackerras; +Cc: David Jander, linux-ppc-embedded
(please ignore last patch, its incomplete)
Instantiation of 8MB pages on the TLB cache for the kernel static
mapping thrashes r3 register on !CONFIG_8xx_CPU6 configurations.
Signed-off-by: Marcelo Tosatti <marcelo@kvack.org>
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index ec53c7d..09b3adc 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -355,9 +355,7 @@ #endif
. = 0x1200
DataStoreTLBMiss:
-#ifdef CONFIG_8xx_CPU6
stw r3, 8(r0)
-#endif
DO_8xx_CPU6(0x3f80, r3)
mtspr SPRN_M_TW, r10 /* Save a couple of working registers */
mfcr r10
@@ -417,9 +415,7 @@ #endif
lwz r11, 0(r0)
mtcr r11
lwz r11, 4(r0)
-#ifdef CONFIG_8xx_CPU6
lwz r3, 8(r0)
-#endif
rfi
/* This is an instruction TLB error on the MPC8xx. This could be due
@@ -500,9 +496,7 @@ LoadLargeDTLB:
lwz r11, 4(r0)
lwz r12, 16(r0)
-#ifdef CONFIG_8xx_CPU6
lwz r3, 8(r0)
-#endif
rfi
/* This is the data TLB error on the MPC8xx. This could be due to
^ permalink raw reply related
* Re: Calculating virtual address from physical address
From: Sylvain Munaut @ 2006-05-05 19:27 UTC (permalink / raw)
To: Chris Dumoulin, linuxppc-embedded
In-Reply-To: <445B6A80.2090202@ics-ltd.com>
Chris Dumoulin wrote:
> I'm using a Virtex II Pro-based board with a PPC405. The board is
> hanging somewhere very early in the kernel boot process. I believe it
> may be dying at the point where the MMU is enabled. In order to
> determine the exact point at which my board hangs, I'm blinking two LEDs
> in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I
> am only able to successfully access the LEDs before the MMU is turned
> on, but I can't be sure that I'm calculating the virtual address
> properly when I try to access the LED after the MMU is turned on.
Typical when trying to bring up board ...
Once the MMU is turned on, you leds register are most likely ... nowhere
... i.e.
if you don't create a mapping your self there is just no virtual address
that will
access your leds physical address.
What I did on some ppc work was tu use a quick BAT mapping to map some leds.
It's pretty easy to setup. Be aware though that this mapping will get
wiped out when
the kernel sets up the BAT for itself.
Sylvain
^ permalink raw reply
* Re: [PATCH] powermac: U4 DART improvements
From: Olof Johansson @ 2006-05-05 15:46 UTC (permalink / raw)
To: Johannes Berg; +Cc: linuxppc-dev, paulus
In-Reply-To: <1146840992.16487.0.camel@localhost>
On Fri, May 05, 2006 at 04:56:32PM +0200, Johannes Berg wrote:
>
> > This boots happily on my quad at home,
>
> Boots fine on my quad and seems to survive a bunch of normal tasks.
> Haven't gotten to do any benchmarking yet.
Thanks for testing. I've been running with it all week here as well. I'll
try to do some benchmarking this weekend; there are a couple of other
things that should be changed in the patch as well.
-Olof
^ permalink raw reply
* Re: Moving from 2.4 to 2.6 kernel
From: Grant Likely @ 2006-05-05 15:31 UTC (permalink / raw)
To: Chris Dumoulin; +Cc: linuxppc-embedded
In-Reply-To: <445B5597.9010801@ics-ltd.com>
On 5/5/06, Chris Dumoulin <cdumoulin@ics-ltd.com> wrote:
> I generated the xparameters_ml300.h file and built my kernel configured
> for the ML300 board, but I still had the same problem. I'm wondering if
> there's something in the hardware that's not being setup by u-boot as
> the linux kernel expects it. I'm looking through the ML300 code in
> u-boot 1.1.4 to see if can spot anything that's being done in there
> that's not being done in my u-boot code. Do you have any ideas about
> some hardware (or other) initialization that I could be neglecting to
> do in u-boot that might cause things to hang?
Probably not; The nice thing about the V2Pro is that there is pretty
much no setup that needs to be done. The bitstream takes care of
configuring SDRAM. :). For my daily work; I don't even use u-boot.=20
I'm downloading a zImage directly through my BDI 2000. (But u-boot
does work for me too)
Can you post your xparameters file? (Or email it to me directly)
Are you using a full UART or UART lite?
> On a similar note, is there any guide to porting u-boot that I could
> read that would enumerate all the steps I should be taking in u-boot? I
> know the README in the u-boot source basically says to browse the
> mailing list (which I'm doing), read the DULG (which I've done), and
> read the source (which I'm also doing), but some kind of succinct guide
> would be nice, if it exists.
Not that I know of; I had to pile throught the same stuff to learn it.
> Interactive Circuits and Systems Ltd.
> 5430 Canotek Road
> Ottawa, ON
Hey; you going to OLS this year?
Cheers,
g.
--
Grant Likely, B.Sc. P.Eng.
Secret Lab Technologies Ltd.
(403) 399-0195
^ permalink raw reply
* Calculating virtual address from physical address
From: Chris Dumoulin @ 2006-05-05 15:08 UTC (permalink / raw)
To: linuxppc-embedded
I'm using a Virtex II Pro-based board with a PPC405. The board is
hanging somewhere very early in the kernel boot process. I believe it
may be dying at the point where the MMU is enabled. In order to
determine the exact point at which my board hangs, I'm blinking two LEDs
in the assembly code found in arch/ppc/kernel/head_4xx.S, . Currently I
am only able to successfully access the LEDs before the MMU is turned
on, but I can't be sure that I'm calculating the virtual address
properly when I try to access the LED after the MMU is turned on.
My LEDs are at address 0x4F600000 and my CONFIG_KERNEL_START is
0xC0000000. If this address were low enough, I would just add 0xC0000000
to the address to get the virtual address, but since my LED address is
so high, the sum will be well past the 32-bit maximum address value. How
is a virtual address calculated for a high address like 0x4F600000?
BTW, he is the assembly code that I'm working with (from
arch/ppc/kernel/head_4xx.S):
.text
_GLOBAL(_stext)
_GLOBAL(_start)
/* Save parameters we are passed.
*/
mr r31,r3
mr r30,r4
mr r29,r5
mr r28,r6
mr r27,r7
/* CRD: set LED state here */
lis r26,0x4F600000@h
ori r26,r26,0x4F600000@l
li r25,LED_STATE_0
stw r25,0(r26)
/* We have to turn on the MMU right away so we get cache modes
* set correctly.
*/
bl initial_mmu
/* CRD: set LED state here */
lis r26,0x4F600000@h
ori r26,r26,0x4F600000@l
li r25,LED_STATE_1
stw r25,0(r26)
/* We now have the lower 16 Meg mapped into TLB entries, and the caches
* ready to work.
*/
turn_on_mmu:
lis r0,MSR_KERNEL@h
ori r0,r0,MSR_KERNEL@l
mtspr SPRN_SRR1,r0
lis r0,start_here@h
ori r0,r0,start_here@l
mtspr SPRN_SRR0,r0
SYNC
/* CRD: set LED state here */
lis r26,0x4F600000@h
ori r26,r26,0x4F600000@l
li r25,LED_STATE_2
stw r25,0(r26)
rfi /* enables MMU */
/* CRD: set LED state here */
/* This address should be a virtual address */
lis r26,0x4F600000@h
ori r26,r26,0x4F600000@l
li r25,LED_STATE_3
stw r25,0(r26)
b . /* prevent prefetch past rfi */
Regards,
Chris Dumoulin
--
*--Christopher Dumoulin--*
Software Team Leader
<http://ics-ltd.com/>
<http://ics-ltd.com/>
Interactive Circuits and Systems Ltd.
5430 Canotek Road
Ottawa, ON
K1J 9G2
(613)749-9241
1-800-267-9794 (USA only)
------------------------------------------------------------------------
This e-mail is private and confidential and is for the addressee only.
If misdirected, please notify us by telephone and confirm that it has
been deleted from your system and any hard copies destroyed. You are
strictly prohibited from using, printing, distributing or disseminating
it or any information contained in it save to the intended recipient.
^ permalink raw reply
* Re: [PATCH] powermac: U4 DART improvements
From: Johannes Berg @ 2006-05-05 14:56 UTC (permalink / raw)
To: Olof Johansson; +Cc: linuxppc-dev, paulus
In-Reply-To: <20060430191430.GU5518@pb15.lixom.net>
[-- Attachment #1: Type: text/plain, Size: 171 bytes --]
> This boots happily on my quad at home,
Boots fine on my quad and seems to survive a bunch of normal tasks.
Haven't gotten to do any benchmarking yet.
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 793 bytes --]
^ permalink raw reply
* Re: [openib-general] [PATCH 07/16] ehca: interrupt handling routines
From: Roland Dreier @ 2006-05-05 14:49 UTC (permalink / raw)
To: Heiko J Schick
Cc: linux-kernel, openib-general, linuxppc-dev, Christoph Raisch,
Hoang-Nam Nguyen, Marcus Eder
In-Reply-To: <445B4DA9.9040601@de.ibm.com>
Heiko> Originaly, we had the same idea as you mentioned, that it
Heiko> would be better to do this in the higher levels. The point
Heiko> is that we can't see so far any simple posibility how this
Heiko> can done in the OpenIB stack, the TCP/IP network layer or
Heiko> somewhere in the Linux kernel.
Heiko> For example: For IPoIB we get the best throughput when we
Heiko> do the CQ callbacks on different CPUs and not to stay on
Heiko> the same CPU.
So why not do it in IPoIB then? This approach is not optimal
globally. For example, uverbs event dispatch is just going to queue
an event and wake up the process waiting for events, and doing this on
some random CPU not related to the where the process will run is
clearly the worst possible way to dispatch the event.
Heiko> In other papers and slides (see [1]) you can see similar
Heiko> approaches.
Heiko> [1]: Speeding up Networking, Van Jacobson and Bob
Heiko> Felderman,
Heiko> http://www.lemis.com/grog/Documentation/vj/lca06vj.pdf
I think you've misunderstood this paper. It's about maximizing CPU
locality and pushing processing directly into the consumer. In the
context of slide 9, what you've done is sort of like adding another
control loop inside the kernel, since you dispatch from interrupt
handler to driver thread to final consumer. So I would argue that
your approach is exactly the opposite of what VJ is advocating.
- R.
^ permalink raw reply
* Re: frequent sig 11 with malloc() on mpc8xx
From: Mark Chambers @ 2006-05-05 14:17 UTC (permalink / raw)
To: Gautam Borad, linuxppc-embedded
In-Reply-To: <445B5A98.8020004@eisodus.com>
> We are aware that its a FAQ and we rechecked the SDRAM configuration,
> everything seems fine.
> We disabled burst mode and tried but that didnt help. Now we want to
> disable cache and check.
Another thing you can try on the 852 is to change the processor frequency
via the PLPRCR register. If slowing down the clock helps, don't assume
that you have a timing problem though - these PLL circuits are notoriously
twitchy, and it could be noise (or, more properly, resonant frequencies in
your
layout)
Mark Chambers
P.S. Hopefully you have a hardware debugger - you can halt the processor
and change this register on the fly. Same with cache - no sense compiling
these changes, just do them manually.
^ permalink raw reply
* Re: Moving from 2.4 to 2.6 kernel
From: Chris Dumoulin @ 2006-05-05 13:39 UTC (permalink / raw)
To: Grant Likely; +Cc: linuxppc-embedded
In-Reply-To: <528646bc0605041304q695a183ao68a4720e974ad4f@mail.gmail.com>
Grant Likely wrote:
> On 5/4/06, Chris Dumoulin <cdumoulin@ics-ltd.com> wrote:
>
>> I'm looking into getting a BDI 2000 so I can start stepping through and
>> see what is going on.
>>
>> I looked at the ML300 and V2Pro code in the arch/ppc/platforms/4xx
>> folder, but I did not use any of that in my code. It looks like this is
>> intended to be used with the BSP that is generated by Xilinx Platform
>> Studio. I've tried generating the BSP this way, but the generated code
>> is obviously not a complete patch to port Linux to your hardware, and I
>> figure that by the time I figure out what I do/don't have to add, I
>> might as well write the whole thing by myself. Have you had success
>> integrating the automatically generated BSP from Xilinx Platform Studio?
>
>
> No; the stuff in 2.6 is not integrated w/ platform studio. (only 2.4
> is). However, you do need to extract the xparameters.h file from the
> platform studio BSP. You can generate a Linux BSP w/o actually
> telling it where your Linux tree is. Once its generated; pull out
> xparameters.h and drop it into arch/ppc/platforms/4xx/xparameters/ in
> your source tree. Note: it's important that you generate a LINUX BSP;
> not a 'standalone' bsp. If you don't, then you'll be missing a bunch
> of #defines.
>
> Let me say that once more for clarity: The only file you need from
> platform studio is the generated xparameters.h
>
> This will give you at the very least a serial port driver. Once your
> booting with that, you can focus on other device drivers.
>
> Trust me; this is the path of far less pain.
>
> Cheers,
> g.
>
>
> --
> Grant Likely, B.Sc. P.Eng.
> Secret Lab Technologies Ltd.
> (403) 399-0195
>
>
I generated the xparameters_ml300.h file and built my kernel configured
for the ML300 board, but I still had the same problem. I'm wondering if
there's something in the hardware that's not being setup by u-boot as
the linux kernel expects it. I'm looking through the ML300 code in
u-boot 1.1.4 to see if can spot anything that's being done in there
that's not being done in my u-boot code. Do you have any ideas about
some hardware (or other) initialization that I could be neglecting to
do in u-boot that might cause things to hang?
On a similar note, is there any guide to porting u-boot that I could
read that would enumerate all the steps I should be taking in u-boot? I
know the README in the u-boot source basically says to browse the
mailing list (which I'm doing), read the DULG (which I've done), and
read the source (which I'm also doing), but some kind of succinct guide
would be nice, if it exists.
Cheers,
Chris
--
*--Christopher Dumoulin--*
Software Team Leader
<http://ics-ltd.com/>
<http://ics-ltd.com/>
Interactive Circuits and Systems Ltd.
5430 Canotek Road
Ottawa, ON
K1J 9G2
(613)749-9241
1-800-267-9794 (USA only)
------------------------------------------------------------------------
This e-mail is private and confidential and is for the addressee only.
If misdirected, please notify us by telephone and confirm that it has
been deleted from your system and any hard copies destroyed. You are
strictly prohibited from using, printing, distributing or disseminating
it or any information contained in it save to the intended recipient.
^ permalink raw reply
* Re: frequent sig 11 with malloc() on mpc8xx
From: Gautam Borad @ 2006-05-05 14:00 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <20060505094007.0F169353AD3@atlas.denx.de>
Wolfgang Denk wrote:
>No, of course not. Please read the FAQ to understand why standard RAM
>tests will never detect this type of problem.
>
>Best regards,
>
>Wolfgang Denk
>
>
>
Thanks for the reply.
We are aware that its a FAQ and we rechecked the SDRAM configuration,
everything seems fine.
We disabled burst mode and tried but that didnt help. Now we want to
disable cache and check.
Is the cache disabled from u-boot or linux? Where do we have to modify
the code to disable
the cache completely. Basically we want to run the linux without using
cache.
regards,
gautam.
^ permalink raw reply
* Re: [openib-general] [PATCH 07/16] ehca: interrupt handling routines
From: Heiko J Schick @ 2006-05-05 13:05 UTC (permalink / raw)
To: Roland Dreier
Cc: linux-kernel, openib-general, linuxppc-dev, Christoph Raisch,
Hoang-Nam Nguyen, Marcus Eder
In-Reply-To: <adaejz9o4vh.fsf@cisco.com>
Hello Roland,
Roland Dreier wrote:
> It seems that you are deferring completion event dispatch into threads
> spread across all the CPUs. This seems like a very strange thing to
> me -- you are adding latency and possibly causing cacheline pingpong.
>
> It may help throughput in some cases to spread the work across
> multiple CPUs but it seems strange to me to do this in the low-level
> driver. My intuition would be that it would be better to do this in
> the higher levels, and leave open the possibility for protocols that
> want the lowest possible latency to be called directly from the
> interrupt handler.
We've implemented this "spread CQ callbacks across multiple CPUs"
functionality to get better throughput on a SMP system, as you have
seen.
Originaly, we had the same idea as you mentioned, that it would be better
to do this in the higher levels. The point is that we can't see so far
any simple posibility how this can done in the OpenIB stack, the TCP/IP
network layer or somewhere in the Linux kernel.
For example:
For IPoIB we get the best throughput when we do the CQ callbacks on
different CPUs and not to stay on the same CPU.
In other papers and slides (see [1]) you can see similar approaches.
I think such one implementation or functionality could require more
or less a non-trivial changes. This could be also releated to other
I/O traffic.
[1]: Speeding up Networking, Van Jacobson and Bob Felderman,
http://www.lemis.com/grog/Documentation/vj/lca06vj.pdf
Regards,
Heiko
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox