LinuxPPC-Dev Archive on lore.kernel.org
 help / color / mirror / Atom feed
* Re: which driver for uartlite with 2.6.17?
From: David Bolcsfoldi @ 2006-10-13 16:47 UTC (permalink / raw)
  To: Robert Corley; +Cc: linux linuxppc-embedded
In-Reply-To: <20061013155908.4944.qmail@web56302.mail.re3.yahoo.com>

Hi :-)

Good question. Well I was unaware of the fact that there were other
uartlite drivers so you should take mine off that list. Peter's driver
is slated for inclusion in the mainline and I am currently in the
process of adding a few things to it.

For 2.6.17 support you should probably just remove the IRQF_DISABLED
flag and change IRQF_SAMPLE_RANDOM to SA_SAMPLE_RANDOM in the
request_irq call and it should work. I am unaware of any other
important differences between the two releases affecting this driver.

Cheers,
David

On 10/13/06, Robert Corley <rcorley@aegis-inc.net> wrote:
> All,
>
> Based on what was posted recently, it appears that there is more than one uartlite driver
> floating about.
>
> Can someone please inform me of which driver works for which kernel?
>
> Here is what I see
>
> D. Lynch    2.6.15    polled, (Pico E12 board specific?)
> P. Korsgaard    2.6.18    interrupt driven, official lanana.org nodes
> D. Bolcsfoldi    2.6.18    interrupt driven
>
> patch locations:
> D. Lynch    http://ozlabs.org/pipermail/linuxppc-embedded/2006-January/021592.html
> P. Korsgaard    http://lkml.org/lkml/2006/9/13/71
> D. Bolcsfoldi    http://ozlabs.org/pipermail/linuxppc-embedded/2006-October/024697.html
>
> Comments?
>
> And, lastly, I need a driver for 2.6.17.  Can someone point me in the right direction?
> I have tried Peter's driver but it fails to compile, showing this:
>
> CC      drivers/serial/serial_core.o
> CC      drivers/serial/uartlite.o
> drivers/serial/uartlite.c: In function `ulite_startup':
> drivers/serial/uartlite.c:199: error: `IRQF_DISABLED' undeclared (first use in this function)
> drivers/serial/uartlite.c:199: error: (Each undeclared identifier is reported only once
> drivers/serial/uartlite.c:199: error: for each function it appears in.)
> drivers/serial/uartlite.c:199: error: `IRQF_SAMPLE_RANDOM' undeclared (first use in this function)
> make[2]: *** [drivers/serial/uartlite.o] Error 1
> make[1]: *** [drivers/serial] Error 2
> make: *** [drivers] Error 2
>
>
>
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

^ permalink raw reply

* Re: [PATCH] powerpc: Add mktree utility to arch/powerpc/boot
From: Geoff Levand @ 2006-10-13 16:14 UTC (permalink / raw)
  To: Josh Boyer; +Cc: linuxppc-dev, paulus
In-Reply-To: <1160752023.3351.3.camel@zod.rchland.ibm.com>

Josh Boyer wrote:
> + * Makes a tree bootable image for IBM Evaluation boards.
> + * Basically, just take a zImage, skip the ELF header, and stuff
> + * a 32 byte header on the front.

I guess you mean on some standard bootloader that comes with
IBM Evaluation boards?  Should this comment reflect that this
operation is required by that bootloader, not the hardware?

-Geoff

^ permalink raw reply

* which driver for uartlite with 2.6.17?
From: Robert Corley @ 2006-10-13 15:59 UTC (permalink / raw)
  To: linux linuxppc-embedded

All,=0A=0ABased on what was posted recently, it appears that there is more =
than one uartlite driver =0Afloating about.=0A=0ACan someone please inform =
me of which driver works for which kernel?=0A=0AHere is what I see=0A=0AD. =
Lynch    2.6.15    polled, (Pico E12 board specific?)=0AP. Korsgaard    2.6=
.18    interrupt driven, official lanana.org nodes=0AD. Bolcsfoldi    2.6.1=
8    interrupt driven=0A=0Apatch locations:=0AD. Lynch    http://ozlabs.org=
/pipermail/linuxppc-embedded/2006-January/021592.html=0AP. Korsgaard    htt=
p://lkml.org/lkml/2006/9/13/71=0AD. Bolcsfoldi    http://ozlabs.org/piperma=
il/linuxppc-embedded/2006-October/024697.html=0A=0AComments?=0A=0AAnd, last=
ly, I need a driver for 2.6.17.  Can someone point me in the right directio=
n?  =0AI have tried Peter's driver but it fails to compile, showing this:=
=0A=0ACC      drivers/serial/serial_core.o=0ACC      drivers/serial/uartlit=
e.o=0Adrivers/serial/uartlite.c: In function `ulite_startup':=0Adrivers/ser=
ial/uartlite.c:199: error: `IRQF_DISABLED' undeclared (first use in this fu=
nction)=0Adrivers/serial/uartlite.c:199: error: (Each undeclared identifier=
 is reported only once=0Adrivers/serial/uartlite.c:199: error: for each fun=
ction it appears in.)=0Adrivers/serial/uartlite.c:199: error: `IRQF_SAMPLE_=
RANDOM' undeclared (first use in this function)=0Amake[2]: *** [drivers/ser=
ial/uartlite.o] Error 1=0Amake[1]: *** [drivers/serial] Error 2=0Amake: ***=
 [drivers] Error 2=0A=0A=0A=0A

^ permalink raw reply

* Re: GCC version to kernel version
From: Kumar Gala @ 2006-10-13 15:54 UTC (permalink / raw)
  To: Clint Thomas; +Cc: linuxppc-embedded
In-Reply-To: <3C02138692C13C4BB675FE7EA24095291F5A15@bluefin.Soneticom.local>


On Oct 13, 2006, at 10:39 AM, Clint Thomas wrote:

> I tried building some different cross-tools earlier on today that used
> GCC 3.4.3 and GLIBC 2.3.3 (I believe Kumar, you provided links to  
> these
> in another mailing list) They were from some e500 patches to
> crosstool-0.28, but I decided to just patch crosstool-0.42 since  
> that is
> what is available. They were not able to compile my linux kernel  
> source
> without a build error, so I am running out of ideas. I have been  
> trying
> to build GCC 3.4.1 / GLIBC 2.3.3 by passing it the necessary
> TARGET_CFLAGS="-msoft-float -mno-string -Wa,-me500 -O2" and
> GLIBC_EXTRA_CONFIG="--without-fp", but this is not building correctly
> either. Any advice or recommendations are appreciated.

I'd suggest taking a look at the ltib project as it has prebuilt  
compilers and support for 85xx:

http://savannah.nongnu.org/projects/ltib/
http://www.bitshrine.org/

- k

^ permalink raw reply

* RE: Problems with DMA from user space on MPC834x (Cache coherency?)
From: Liu Dave-r63238 @ 2006-10-13 15:49 UTC (permalink / raw)
  To: Kumar Gala, Lauri Ehrenpreis; +Cc: linuxppc-embedded
In-Reply-To: <CDCE6FAC-2DA9-4D94-8BB6-C26622DF6685@kernel.crashing.org>

You can make use of the hardware cache coherency. Setup the DMA
attribute as global.
If you use the hardware way, you don't need these flushing operation.

-Dave

> > I added following before starting DMA:
> >
> > kaddr =3D (unsigned long)kmap(pages[i]);=20
> flush_dcache_range(kaddr, kaddr=20
> > + PAGE_SIZE); kunmap(pages[i]);
> >
> > ...and everything works now. As I understand this now,=20
> dma_map_page is=20
> > not flushing caches on MPS834x because the processor should=20
> snoop the=20
> > memory transactions and flush cache automatically if needed=20
> > (__dma_sync_page is defined as do { } while (0) in kernel).=20
> But there=20
> > exists an errata
> > (PCI5) which states that there may be some problems with this=20
> > automatic flushing. So the caches must be flushed with=20
> > flush_dcache_range.
>=20
> What silicon version are you using.  PCI5 should be fixed in=20
> rev1.1 and newer.  You can be taking a big perf hit having to=20
> flush all the time.
>=20
> - k

^ permalink raw reply

* RE: GCC version to kernel version
From: Clint Thomas @ 2006-10-13 15:39 UTC (permalink / raw)
  To: linuxppc-embedded
In-Reply-To: <EE6BAB8C-DD74-412B-BDD1-1758426B62F7@kernel.crashing.org>

I tried building some different cross-tools earlier on today that used
GCC 3.4.3 and GLIBC 2.3.3 (I believe Kumar, you provided links to these
in another mailing list) They were from some e500 patches to
crosstool-0.28, but I decided to just patch crosstool-0.42 since that is
what is available. They were not able to compile my linux kernel source
without a build error, so I am running out of ideas. I have been trying
to build GCC 3.4.1 / GLIBC 2.3.3 by passing it the necessary
TARGET_CFLAGS=3D"-msoft-float -mno-string -Wa,-me500 -O2" and
GLIBC_EXTRA_CONFIG=3D"--without-fp", but this is not building correctly
either. Any advice or recommendations are appreciated.

Clint

-----Original Message-----
From: Kumar Gala [mailto:galak@kernel.crashing.org]=20
Sent: Friday, October 13, 2006 11:34 AM
To: Clint Thomas
Cc: linuxppc-embedded@ozlabs.org
Subject: Re: GCC version to kernel version


On Oct 13, 2006, at 9:55 AM, Clint Thomas wrote:

> Hey guys,
>
> Generic question here. To each linux kernel version, is there a=20
> specific version of GCC and GLIBC I should be using to build the=20
> kernel, RFS, etc? I cannot seem to find any concrete proof that this=20
> is the case, but I am having issues building a new RFS with my older=20
> GCC and GLIBC, so I'm looking at updating them. I'm currently using=20
> GCC 3.3.2 and GLIBC 2.3.2 to build for the Linux 2.4.27 kernel on an=20
> e500 (MPC8541) core. Thanks!
>

It depends on what user space functionality you want/need form e500.  =20
If you plan on using SPE or fp I'd say to make sure to get tools chains
that support e500.  If you are just running normal integer code than any
toolchain should be fine.

- k

^ permalink raw reply

* Re: GCC version to kernel version
From: Kumar Gala @ 2006-10-13 15:34 UTC (permalink / raw)
  To: Clint Thomas; +Cc: linuxppc-embedded
In-Reply-To: <3C02138692C13C4BB675FE7EA24095291F5A08@bluefin.Soneticom.local>


On Oct 13, 2006, at 9:55 AM, Clint Thomas wrote:

> Hey guys,
>
> Generic question here. To each linux kernel version, is there a  
> specific version of GCC and GLIBC I should be using to build the  
> kernel, RFS, etc? I cannot seem to find any concrete proof that  
> this is the case, but I am having issues building a new RFS with my  
> older GCC and GLIBC, so I'm looking at updating them. I'm currently  
> using GCC 3.3.2 and GLIBC 2.3.2 to build for the Linux 2.4.27  
> kernel on an e500 (MPC8541) core. Thanks!
>

It depends on what user space functionality you want/need form e500.   
If you plan on using SPE or fp I'd say to make sure to get tools  
chains that support e500.  If you are just running normal integer  
code than any toolchain should be fine.

- k

^ permalink raw reply

* Re: unresolved symbol on target ... while insmod
From: Kumar Gala @ 2006-10-13 15:28 UTC (permalink / raw)
  To: kwkang; +Cc: linuxppc-embedded
In-Reply-To: <200610130653.k9D6r0Jq028482@bLinux.pontusys.com>


On Oct 13, 2006, at 1:49 AM, kwkang wrote:

> Hi Srideep,
>
>
>
> Can anyone show me how I can overcome below problem?
>
> I have searched mailing list but I can not find any answer for that  
> problem.
>
>
>
> [test ~]#insmod linux-kernel-bde.o
>
> Using linux-kernel-bde.o
>
> Using linux-kernel-bde.o
>
> insmod: Warning: kernel-module version mismatch
>
>         linux-kernel-bde.o was compiled for kernel version 2.4.20_dev
>
>         while this kernel is version 2.4.21-pre5
>
> insmod: unresolved symbol devfs_register
>
> insmod: unresolved symbol devfs_unregister

This most likely means you dont have devfs built into your kernel or  
the module for devfs isn't loaded.

- k

^ permalink raw reply

* Re: Problems with DMA from user space on MPC834x (Cache coherency?)
From: Kumar Gala @ 2006-10-13 15:27 UTC (permalink / raw)
  To: Lauri Ehrenpreis; +Cc: linuxppc-embedded
In-Reply-To: <op.thcr4fbjes1gi5@laurie.liewenthal.ee>


On Oct 13, 2006, at 4:18 AM, Lauri Ehrenpreis wrote:

>
> Thank you!
>
> I added following before starting DMA:
>
> kaddr = (unsigned long)kmap(pages[i]);
> flush_dcache_range(kaddr, kaddr + PAGE_SIZE);
> kunmap(pages[i]);
>
> ...and everything works now. As I understand this now, dma_map_page is
> not flushing caches on MPS834x because the processor should snoop the
> memory transactions and flush cache automatically if needed
> (__dma_sync_page
> is defined as do { } while (0) in kernel). But there exists an errata
> (PCI5) which states that there may be some problems with this  
> automatic
> flushing. So the caches must be flushed with flush_dcache_range.

What silicon version are you using.  PCI5 should be fixed in rev1.1  
and newer.  You can be taking a big perf hit having to flush all the  
time.

- k

^ permalink raw reply

* [PATCH] powerpc: Add mktree utility to arch/powerpc/boot
From: Josh Boyer @ 2006-10-13 15:07 UTC (permalink / raw)
  To: paulus; +Cc: linuxppc-dev

This patch adds the mktree program that is needed to post process zImage
wrappers for various PowerPC 4xx boards

Signed-off-by: Josh Boyer <jwboyer@linux.vnet.ibm.com>

---
 arch/powerpc/boot/Makefile |    4 -
 arch/powerpc/boot/mktree.c |  152 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 154 insertions(+), 2 deletions(-)

--- linux-2.6.orig/arch/powerpc/boot/Makefile
+++ linux-2.6/arch/powerpc/boot/Makefile
@@ -94,13 +94,13 @@ $(patsubst %.S,%.o, $(filter %.S, $(src-
 $(obj)/wrapper.a: $(obj-wlib)
 	$(call cmd,bootar)
 
-hostprogs-y	:= addnote addRamDisk hack-coff
+hostprogs-y	:= addnote addRamDisk hack-coff mktree
 
 extra-y		:= $(obj)/crt0.o $(obj)/wrapper.a $(obj-plat) $(obj)/empty.o \
 		   $(obj)/zImage.lds $(obj)/zImage.coff.lds
 
 wrapper		:=$(srctree)/$(src)/wrapper
-wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff)
+wrapperbits	:= $(extra-y) $(addprefix $(obj)/,addnote hack-coff mktree)
 
 #############
 # Bits for building various flavours of zImage
--- /dev/null
+++ linux-2.6/arch/powerpc/boot/mktree.c
@@ -0,0 +1,152 @@
+/*
+ * Makes a tree bootable image for IBM Evaluation boards.
+ * Basically, just take a zImage, skip the ELF header, and stuff
+ * a 32 byte header on the front.
+ *
+ * We use htonl, which is a network macro, to make sure we're doing
+ * The Right Thing on an LE machine.  It's non-obvious, but it should
+ * work on anything BSD'ish.
+ */
+
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <netinet/in.h>
+#ifdef __sun__
+#include <inttypes.h>
+#else
+#include <stdint.h>
+#endif
+
+/* This gets tacked on the front of the image.  There are also a few
+ * bytes allocated after the _start label used by the boot rom (see
+ * head.S for details).
+ */
+typedef struct boot_block {
+	uint32_t bb_magic;		/* 0x0052504F */
+	uint32_t bb_dest;		/* Target address of the image */
+	uint32_t bb_num_512blocks;	/* Size, rounded-up, in 512 byte blks */
+	uint32_t bb_debug_flag;	/* Run debugger or image after load */
+	uint32_t bb_entry_point;	/* The image address to start */
+	uint32_t bb_checksum;	/* 32 bit checksum including header */
+	uint32_t reserved[2];
+} boot_block_t;
+
+#define IMGBLK	512
+char	tmpbuf[IMGBLK];
+
+int main(int argc, char *argv[])
+{
+	int	in_fd, out_fd;
+	int	nblks, i;
+	uint	cksum, *cp;
+	struct	stat	st;
+	boot_block_t	bt;
+
+	if (argc < 3) {
+		fprintf(stderr, "usage: %s <zImage-file> <boot-image> [entry-point]\n",argv[0]);
+		exit(1);
+	}
+
+	if (stat(argv[1], &st) < 0) {
+		perror("stat");
+		exit(2);
+	}
+
+	nblks = (st.st_size + IMGBLK) / IMGBLK;
+
+	bt.bb_magic = htonl(0x0052504F);
+
+	/* If we have the optional entry point parameter, use it */
+	if (argc == 4)
+		bt.bb_dest = bt.bb_entry_point = htonl(strtoul(argv[3], NULL, 0));
+	else
+		bt.bb_dest = bt.bb_entry_point = htonl(0x500000);
+
+	/* We know these from the linker command.
+	 * ...and then move it up into memory a little more so the
+	 * relocation can happen.
+	 */
+	bt.bb_num_512blocks = htonl(nblks);
+	bt.bb_debug_flag = 0;
+
+	bt.bb_checksum = 0;
+
+	/* To be neat and tidy :-).
+	*/
+	bt.reserved[0] = 0;
+	bt.reserved[1] = 0;
+
+	if ((in_fd = open(argv[1], O_RDONLY)) < 0) {
+		perror("zImage open");
+		exit(3);
+	}
+
+	if ((out_fd = open(argv[2], (O_RDWR | O_CREAT | O_TRUNC), 0666)) < 0) {
+		perror("bootfile open");
+		exit(3);
+	}
+
+	cksum = 0;
+	cp = (void *)&bt;
+	for (i=0; i<sizeof(bt)/sizeof(uint); i++)
+		cksum += *cp++;
+
+	/* Assume zImage is an ELF file, and skip the 64K header.
+	*/
+	if (read(in_fd, tmpbuf, IMGBLK) != IMGBLK) {
+		fprintf(stderr, "%s is too small to be an ELF image\n",
+				argv[1]);
+		exit(4);
+	}
+
+	if ((*(uint *)tmpbuf) != htonl(0x7f454c46)) {
+		fprintf(stderr, "%s is not an ELF image\n", argv[1]);
+		exit(4);
+	}
+
+	if (lseek(in_fd, (64 * 1024), SEEK_SET) < 0) {
+		fprintf(stderr, "%s failed to seek in ELF image\n", argv[1]);
+		exit(4);
+	}
+
+	nblks -= (64 * 1024) / IMGBLK;
+
+	/* And away we go......
+	*/
+	if (write(out_fd, &bt, sizeof(bt)) != sizeof(bt)) {
+		perror("boot-image write");
+		exit(5);
+	}
+
+	while (nblks-- > 0) {
+		if (read(in_fd, tmpbuf, IMGBLK) < 0) {
+			perror("zImage read");
+			exit(5);
+		}
+		cp = (uint *)tmpbuf;
+		for (i=0; i<sizeof(tmpbuf)/sizeof(uint); i++)
+			cksum += *cp++;
+		if (write(out_fd, tmpbuf, sizeof(tmpbuf)) != sizeof(tmpbuf)) {
+			perror("boot-image write");
+			exit(5);
+		}
+	}
+
+	/* rewrite the header with the computed checksum.
+	*/
+	bt.bb_checksum = htonl(cksum);
+	if (lseek(out_fd, 0, SEEK_SET) < 0) {
+		perror("rewrite seek");
+		exit(1);
+	}
+	if (write(out_fd, &bt, sizeof(bt)) != sizeof(bt)) {
+		perror("boot-image rewrite");
+		exit(1);
+	}
+
+	exit(0);
+}

^ permalink raw reply

* GCC version to kernel version
From: Clint Thomas @ 2006-10-13 14:55 UTC (permalink / raw)
  To: linuxppc-embedded

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

Hey guys,

Generic question here. To each linux kernel version, is there a specific
version of GCC and GLIBC I should be using to build the kernel, RFS,
etc? I cannot seem to find any concrete proof that this is the case, but
I am having issues building a new RFS with my older GCC and GLIBC, so
I'm looking at updating them. I'm currently using GCC 3.3.2 and GLIBC
2.3.2 to build for the Linux 2.4.27 kernel on an e500 (MPC8541) core.
Thanks!
 
Clinton Thomas
cthomas@soneticom.com
 

[-- Attachment #2: Type: text/html, Size: 909 bytes --]

^ permalink raw reply

* preempt crash in 2.6.18
From: Rune Torgersen @ 2006-10-13 14:16 UTC (permalink / raw)
  To: linuxppc-dev

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]


This is from a Freescale 8265 running 2.6.18
Anyone have any idea what this is and how to fix it?

TIPC: Blocking bearer <eth:eth0>
huh, entered a0232b3c with preempt_count 00000102, exited with 00000203?
kernel BUG in __run_timers at kernel/timer.c:453!
Oops: Exception in kernel mode, sig: 5 [#1]
PREEMPT
NIP: A0024574 LR: A0024574 CTR: A0032104
REGS: a02cfd90 TRAP: 0700   Tainted: P       (2.6.18)
MSR: 00029032 <EE,ME,IR,DR>  CR: 44024024  XER: 00000000
TASK = a02900a8[0] 'swapper' THREAD: a02ce000
GPR00: A0024574 A02CFE40 A02900A8 0000004C A02F0E34 00004481 F4915203 00000000
GPR08: 00000008 00000203 DE53FE9C A02CE000 44024084 414404C0 0FFCE000 FFFFFFFF
GPR16: 00000000 0FFC8D68 007FFF00 0FF77600 00000002 00200040 00800000 007FFEA0
GPR24: 00000000 A02F0000 A0290000 A02F61C8 A02CFE50 00000102 A0232B3C A02CE000
NIP [A0024574] run_timer_softirq+0x12c/0x214
LR [A0024574] run_timer_softirq+0x12c/0x214
Call Trace:
[A02CFE40] [A0024574] run_timer_softirq+0x12c/0x214 (unreliable)
[A02CFE70] [A001F38C] __do_softirq+0x80/0xe4
[A02CFEA0] [A0009ACC] do_softirq+0x58/0x60
[A02CFEB0] [A001F450] irq_exit+0x60/0x80
[A02CFEC0] [A0009A28] do_IRQ+0x70/0xbc
[A02CFED0] [A00047B4] ret_from_except+0x0/0x14
[A02CFF90] [A000C9BC] cpu_idle+0xf8/0x108
[A02CFFB0] [A0003790] rest_init+0x4c/0x60
[A02CFFD0] [A02D06A4] start_kernel+0x1b8/0x1f4
[A02CFFF0] [000032C0] 0x32c0
Instruction dump:
7fa3eb78 7fc803a6 83bf000c 4e800021 80df000c 3c60a025 7f86e800 3863eaf4
7fc4f378 7fa5eb78 419e000c 4bff4dfd <0fe00000> 7c0000a6 5400045e 7c000124
Kernel panic - not syncing: Aiee, killing interrupt handler!

[-- Attachment #2: Type: text/html, Size: 2192 bytes --]

^ permalink raw reply

* Re: Recently removed io accessors
From: Peter Korsgaard @ 2006-10-13 12:38 UTC (permalink / raw)
  To: Benjamin Herrenschmidt; +Cc: sfr, linuxppc-dev
In-Reply-To: <1160741900.4792.255.camel@localhost.localdomain>

>>>>> "BH" == Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

Hi,

>> Unfortunately they didn't so I need to enable big endian mode and
>> NOT swap on normal register access and swap on access to the FIFO.

BH> That's where it bothers me... you need to enable BE register
BH> mode...

Yes, because the hw people didn't swap the byte lanes. To go back to
my previous mail:

Endian register set to little endian (default at powerup):
RedBoot> x -b 0x8e000064 -2 -l 4
8E000064: 4321 8765
RedBoot> x -b 0x8e000064 -4 -l 4
8E000064: 43218765

With the bytelanes swapped the 32bit read would return 21436587 which
is simply the little endian form of what we need.

But because they didn't swap the lanes I need to enable the BE mode
and not do bswap.

BH> On normal HW, you don't do that... you get LE registers and
BH> everybody is happy with that and the fifo just works. Unless what
BH> you really do is enable BE register mode (and it looks like BE) or
BH> something around those lines.

Sorry, I cannot parse that. I AM enabling BE mode.

Again:
- Bytelanes directly connected (as on our boards): You have to enable BE
  mode in the chip, NOT swap normal register accesses and SWAP accesses
  to/from the FIFOs.

- Bytelanes swapped: You should NOT enable BE mode, SWAP normal
  register accesses and NOT swap accesses from/to the FIFOs.

BH> Have you actually verified that it works if you endian swap the
BH> whole buffer ? Maybe it's just some headers added by the chip that
BH> need swapping in which case it's all fine, just add the proper
BH> leXX_to_cpu() to the driver when reading those...

Yes, we've be using it since January (I worked together with Dustin
before smc911x.c got submitted to mainline) - We need to swap all data
to/from the FIFOs.

>> The question is what to do about it. Adding another special case to
>> smc911x.h for my board is not a big deal, but I would like to get
>> the _insl / _outsl implementations back in misc.S instead of adding
>> them to my platform file.

BH> Maybe with a different name but that's possible. I just wnat to be
BH> 150% sure that this is what you need and the problem isn't lurking
BH> somewhere else ;)

Not as far as I can see.

BH> Overkill ? well, not that much.. you get free byteswap from the
BH> hardware with just a bit set in the PTEs :) Just a matter of
BH> making sure the TLB miss handlers does proerly forward that bit
BH> from the linux PTE to the TLB.

It's still a lot more work than simply doing a s/_outsl_ns/_outsl/.
Isn't the stwbrx as fast as a regular stw?

-- 
Bye, Peter Korsgaard

^ permalink raw reply

* Re: Recently removed io accessors
From: Benjamin Herrenschmidt @ 2006-10-13 12:18 UTC (permalink / raw)
  To: Peter Korsgaard; +Cc: sfr, linuxppc-dev
In-Reply-To: <87ejtctu9s.fsf@sleipner.barco.com>

> Exactly. As I wrote in my previous mail: If the hw people had swapped
> the 2 byte lanes I would need to byteswap on the normal register
> accesses, but not on the FIFO. That would have been the preferable
> setup.
> 
> Unfortunately they didn't so I need to enable big endian mode and NOT
> swap on normal register access and swap on access to the FIFO.

That's where it bothers me... you need to enable BE register mode...

On normal HW, you don't do that... you get LE registers and everybody is
happy with that and the fifo just works. Unless what you really do is
enable BE register mode (and it looks like BE) or something around those
lines.

Have you actually verified that it works if you endian swap the whole
buffer ? Maybe it's just some headers added by the chip that need
swapping in which case it's all fine, just add the proper leXX_to_cpu()
to the driver when reading those...

> The question is what to do about it. Adding another special case to
> smc911x.h for my board is not a big deal, but I would like to get the
> _insl / _outsl implementations back in misc.S instead of adding them
> to my platform file.

Maybe with a different name but that's possible. I just wnat to be 150%
sure that this is what you need and the problem isn't lurking somewhere
else ;)

> BH> I'll read the chip spec and try to figure out what can be
> BH> done. Maybe an option is to use the per-page little endian flag
> BH> available on 4xx parts, I think you may have that in your xilinx,
> BH> and thus have automatic byteswapping. We don't support that bit
> BH> but it shouldn't be too hard to add it so that you can pass it to
> BH> __ioremap. But again, I'm surprised that's necessary.
> 
> Sorry, but that sounds overkill to me. Everything works fine as long
> as I don't byteswap on normal registers and use something like _insl /
> _outsl for the FIFOs.

Overkill ? well, not that much.. you get free byteswap from the hardware
with just a bit set in the PTEs :) Just a matter of making sure the TLB
miss handlers does proerly forward that bit from the linux PTE to the
TLB.

Ben.

^ permalink raw reply

* Re: Recently removed io accessors
From: Peter Korsgaard @ 2006-10-13  9:55 UTC (permalink / raw)
  To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Peter Korsgaard, sfr
In-Reply-To: <1160731859.4792.249.camel@localhost.localdomain>

>>>>> "BH" == Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

Hi,

BH> That's what bothers me... usually, if the registers are little
BH> endian and you need a byteswap to read them, then you don't need a
BH> byteswap to pump the fifo. For example, things like IDE, sound
BH> cards, wireless chips etc... have LE registers (we use byteswap to
BH> access them) and we are fine not swapping on the fifo read/writes.

Exactly. As I wrote in my previous mail: If the hw people had swapped
the 2 byte lanes I would need to byteswap on the normal register
accesses, but not on the FIFO. That would have been the preferable
setup.

Unfortunately they didn't so I need to enable big endian mode and NOT
swap on normal register access and swap on access to the FIFO.

The question is what to do about it. Adding another special case to
smc911x.h for my board is not a big deal, but I would like to get the
_insl / _outsl implementations back in misc.S instead of adding them
to my platform file.

BH> I'll read the chip spec and try to figure out what can be
BH> done. Maybe an option is to use the per-page little endian flag
BH> available on 4xx parts, I think you may have that in your xilinx,
BH> and thus have automatic byteswapping. We don't support that bit
BH> but it shouldn't be too hard to add it so that you can pass it to
BH> __ioremap. But again, I'm surprised that's necessary.

Sorry, but that sounds overkill to me. Everything works fine as long
as I don't byteswap on normal registers and use something like _insl /
_outsl for the FIFOs.

-- 
Bye, Peter Korsgaard

^ permalink raw reply

* Re: unresolved symbol on target ... while insmod
From: Kyungwan Kang @ 2006-10-13  9:41 UTC (permalink / raw)
  To: Linuxppc-embedded
In-Reply-To: <200610130653.k9D6r0Jq028482@bLinux.pontusys.com>

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

I can find the option for /dev filesystem in 2.4.x kernel.
Thanks.


2006/10/13, kwkang <kwkang@pontusys.com>:
>
>  Hi Srideep,
>
>
>
> Can anyone show me how I can overcome below problem?
>
> I have searched mailing list but I can not find any answer for that
> problem.
>
>
>
> [test ~]#insmod linux-kernel-bde.o
>
> Using linux-kernel-bde.o
>
> Using linux-kernel-bde.o
>
> insmod: Warning: kernel-module version mismatch
>
>         linux-kernel-bde.o was compiled for kernel version 2.4.20_dev
>
>         while this kernel is version 2.4.21-pre5
>
> insmod: unresolved symbol devfs_register
>
> insmod: unresolved symbol devfs_unregister
>
>
>
> Thanks.
>
> William.
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
>

[-- Attachment #2: Type: text/html, Size: 3148 bytes --]

^ permalink raw reply

* Re: Recently removed io accessors
From: Benjamin Herrenschmidt @ 2006-10-13  9:30 UTC (permalink / raw)
  To: Peter Korsgaard; +Cc: sfr, linuxppc-dev
In-Reply-To: <87slhstxlk.fsf@sleipner.barco.com>


> Endian register set to little endian (default at powerup):
> RedBoot> x -b 0x8e000064 -2 -l 4
> 8E000064: 4321 8765
> RedBoot> x -b 0x8e000064 -4 -l 4
> 8E000064: 43218765
> 
> And with it set to big endian:
> RedBoot> x -b 0x8e000064 -2 -l 4
> 8E000064: 8765 4321
> RedBoot> x -b 0x8e000064 -4 -l 4
> 8E000064: 87654321
> 
> So with this setup I need to enable the big endian mode to read
> registers without byteswapping and use byteswapping for the packet
> fifos.

That's what bothers me... usually, if the registers are little endian
and you need a byteswap to read them, then you don't need a byteswap to
pump the fifo. For example, things like IDE, sound cards, wireless chips
etc... have LE registers (we use byteswap to access them) and we are
fine not swapping on the fifo read/writes.

I'll read the chip spec and try to figure out what can be done. Maybe an
option is to use the per-page little endian flag available on 4xx parts,
I think you may have that in your xilinx, and thus have automatic
byteswapping. We don't support that bit but it shouldn't be too hard to
add it so that you can pass it to __ioremap. But again, I'm surprised
that's necessary.

Ben.

^ permalink raw reply

* Re: Problems with DMA from user space on MPC834x (Cache coherency?)
From: Lauri Ehrenpreis @ 2006-10-13  9:18 UTC (permalink / raw)
  To: linuxppc-embedded
In-Reply-To: <20061012173231.56756.qmail@web34703.mail.mud.yahoo.com>


Thank you!

I added following before starting DMA:

kaddr = (unsigned long)kmap(pages[i]);
flush_dcache_range(kaddr, kaddr + PAGE_SIZE);
kunmap(pages[i]);

...and everything works now. As I understand this now, dma_map_page is
not flushing caches on MPS834x because the processor should snoop the
memory transactions and flush cache automatically if needed
(__dma_sync_page
is defined as do { } while (0) in kernel). But there exists an errata
(PCI5) which states that there may be some problems with this automatic
flushing. So the caches must be flushed with flush_dcache_range.

--
Lauri


On Thu, 12 Oct 2006 20:32:31 +0300, Manish Joshi <mjoshi_blr@yahoo.com>
wrote:

> Yeah it looks like a cache coherency problem from the first look.
> You may try enabling snooping on the DMA channel if MPC824X supports it.  
> Check in Mode register setting.
> Else you may want to use flush_dcache_range(start, last) functions and  
> see if it helps.
> ----- Original Message ----
> From: Lauri Ehrenpreis <lauri.ehrenpreis@liewenthal.ee>
> To: linuxppc-embedded@ozlabs.org
> Sent: Wednesday, October 11, 2006 7:20:03 AM
> Subject: Problems with DMA from user space on MPC834x (Cache coherency?)
>
>
> Hi!
>
> I have a problem with DMA from user space on a platform powered by  
> MPC834x
> processor (which has powerpc e300 core inside). Our linux kernel version  
> is
> 2.6.17.
>
> My user space program does something like this:
>
> file_fd = open("/disk/file", O_RDONLY);
> result = read(file_fd, page_aligned_buf, len);
> call_driver_ioctl_which_performs_dma();
>
> while /disk is mounted to a partition residing on USB memory stick or SD
> card.
> page_aligned_buf starts from page boundary and contains enough full pages
> for
> the data (so I can allways map full page with dma_map_page in kernel).
>
> The buffer address and data length will then be passed to a driver, which
> calls
> get_user_pages, then maps each page with dma_map_page and tells a PCI
> device
> to start reading from that page:
>
> ...
> down_read(&current->mm->mmap_sem);
> result = get_user_pages(current, current->mm, page_aligned_buf, nr_pages,
> 0, 0, pages, NULL);
> up_read(&current->mm->mmap_sem);
>
> for (i = 0; i < nr_pages; i++) {
>     find_data_checksum(pages[i]);
>     dma_addr = dma_map_page(&fpga.pci_dev->dev, pages[i], 0, PAGE_SIZE,
> DMA_TO_DEVICE);
>
>     start_dma();
>
>     wait_until_dma_ready();
>
>     dma_unmap_page(&fpga.pci_dev->dev, dma_addr, dma_len, DMA_TO_DEVICE);
>
>     if(!checksum_ok_in_fpga())
>         print_page_data();
> }
>
> for (i = 0; i < nr_pages; i++)
>     page_cache_release(pages[i]);
>
> The problem is that sometimes the PCI device receives wrong bytes at  
> random
> locations. I find the data checksum inside the driver and inside the  
> FPGA.
> Inside the driver I use kmap(page) and kunmap(page) to access data on the
> user page. Sometimes the checksums do not match and I can see with the  
> FPGA
> debugging tool, that the FPGA receives different data than the driver is
> printing out.
>
> I have noticed 3 things regarding this error:
> 1) When I copy the data I read from the block device to another buffer in
> userspace and pass this new buffer to the driver, then this error will
> never occur:
>
> file_fd = open("/disk/file", O_RDONLY);
> result = read(file_fd, page_aligned_buf, len);
> memcpy(new_page_aligned_buf, page_aligned_buf, len);
> call_driver_ioctl_which_performs_dma(new_page_aligned_buf, len);
>
> 2) this error does not occur when I use copy_from_user instead of
> get_user_pages
> and dma_map_page inside driver.
>
> 3) this error does not occur on our previuos device, which has x86
> platform.
>
> I am currently out of ideas what to do next.. It seems to me like a cache
> coherency problem. Can anyone suggest what might be wrong?
>
> --
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
>

^ permalink raw reply

* Re: Recently removed io accessors
From: Peter Korsgaard @ 2006-10-13  8:43 UTC (permalink / raw)
  To: Benjamin Herrenschmidt; +Cc: sfr, linuxppc-dev
In-Reply-To: <1160724670.4792.195.camel@localhost.localdomain>

>>>>> "BH" == Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

Hi,

BH> Well, I still don't see it. It all depends how the HW has been
BH> wired I suppose but you should not need byteswap regardless of the
BH> 32 bits being broken in packs of 2x16 bits or whatever... If the
BH> chip has a HW byteswap for registers and not for the packet
BH> buffer, that makes it even more clear that such swapping should
BH> not be necessary.

Ok, a bit of details:

We're using the smsc9117
(http://www.smsc.com/main/datasheets/9117.pdf) connected over a 16bit
EMC bus (together with a spansion 29PL127 flash) to a Xilinx V4FX FPGA
with a 405 core in it.

LSB of the processor is connected to LSB of of the 9117 (and same for
the flash) and so on.

Reading the byte test register of the 9117 (supposed to contain
0x87654321) gives:

Endian register set to little endian (default at powerup):
RedBoot> x -b 0x8e000064 -2 -l 4
8E000064: 4321 8765
RedBoot> x -b 0x8e000064 -4 -l 4
8E000064: 43218765

And with it set to big endian:
RedBoot> x -b 0x8e000064 -2 -l 4
8E000064: 8765 4321
RedBoot> x -b 0x8e000064 -4 -l 4
8E000064: 87654321

So with this setup I need to enable the big endian mode to read
registers without byteswapping and use byteswapping for the packet
fifos.

If on the other hand the hw people had inverted the 2 byte lanes
(connected b0..b7 on the CPU to b8..b15, and b8..15 to b0..b7) I would
need to use byteswapping on the normal register accesses and no byte
swapping on the packet fifos.

They didn't unfortunately :/

BH> Unless the chip has been wired backward on the processor bus (in
BH> which case, btw, DMA will not work either unless you have one of
BH> those magically swapping dma controllers)..

True. We are not using DMA though.

BH> So I still claim that you should not need them and if you do, then
BH> the chip has probably been incorrect wired to your CPU bus. In
BH> which case, you can either grab an old copy of the functions and
BH> put them in your driver for your platform or add a cpu_to_leXX()
BH> loop to byteswap the data in/out, but it's probably not the right
BH> thing to do in the generic driver since it would be a problem
BH> specific to your board.

I could add another #ifdef CONFIG_<my board> to the smc911x.h to
select the right I/O accessors, but sticking the implementation of
_insl/_outsl in my platform file isn't that nice - Couldn't we put
them back in misc.S?

BH> Unless I'm missing something ... It would be useful to have more
BH> details about your setup.

I hope this makes it a bit more clear.

-- 
Bye, Peter Korsgaard

^ permalink raw reply

* [PATCH 5/5] powerpc: Cell timebase bug workaround
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

The Cell CPU timebase has an errata. When reading the entire 64 bits of
the timebase with one mftb instruction, there is a handful of cycles
window during which one might read a value with the low order 32 bits
already reset to 0x00000000 but the high order bits not yet incremeted
by one. Simply reading the timebase a second time if the low order bits
are 0 is enough (the second read will always miss that window).

Note that there is still a potential issue if the process gets
interrupted between the 2 reads for long enough that the second reads
pricely hits that same window on the next 32 bits wrap. However, this is
so extremely unlikely to happen that we choose not to do anything
against it (the 32 bits wrap happens every few seconds or so, in fact
every few minutes on IBM cell blades).

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---

This is the "old" patch adapted to apply on top of the previous changes.
I'll post a new version of the patch using a different technique for the
workaround asap. (should have been today but testing & debugging the
previous patches of this serie took more time than expected).


Index: linux-cell/arch/powerpc/kernel/vdso64/gettimeofday.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/vdso64/gettimeofday.S	2006-10-13 16:00:25.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/vdso64/gettimeofday.S	2006-10-13 16:59:41.000000000 +1000
@@ -229,8 +229,10 @@ V_FUNCTION_BEGIN(__do_get_xsec)
 	xor	r0,r8,r8		/* create dependency */
 	add	r3,r3,r0
 
-	/* Get TB & offset it */
-	mftb	r7
+	/* Get TB & offset it. We use the MFTB macro which will generate
+	 * workaround code for Cell.
+	 */
+	MFTB(r7)
 	ld	r9,CFG_TB_ORIG_STAMP(r3)
 	subf	r7,r9,r7
 
Index: linux-cell/include/asm-powerpc/cputable.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/cputable.h	2006-10-13 16:58:13.000000000 +1000
+++ linux-cell/include/asm-powerpc/cputable.h	2006-10-13 16:59:41.000000000 +1000
@@ -143,6 +143,7 @@ extern struct cpu_spec *identify_cpu(uns
 #define CPU_FTR_CI_LARGE_PAGE		LONG_ASM_CONST(0x0000100000000000)
 #define CPU_FTR_PAUSE_ZERO		LONG_ASM_CONST(0x0000200000000000)
 #define CPU_FTR_PURR			LONG_ASM_CONST(0x0000400000000000)
+#define CPU_FTR_CELL_TB_BUG		LONG_ASM_CONST(0x0000800000000000)
 
 #ifndef __ASSEMBLY__
 
@@ -331,7 +332,7 @@ extern struct cpu_spec *identify_cpu(uns
 #define CPU_FTRS_CELL	(CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_USE_TB | \
 	    CPU_FTR_HPTE_TABLE | CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
-	    CPU_FTR_PAUSE_ZERO | CPU_FTR_CI_LARGE_PAGE)
+	    CPU_FTR_PAUSE_ZERO | CPU_FTR_CI_LARGE_PAGE | CPU_FTR_CELL_TB_BUG)
 #define CPU_FTRS_PA6T (CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_USE_TB | \
 	    CPU_FTR_HPTE_TABLE | CPU_FTR_PPCAS_ARCH_V2 | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_CI_LARGE_PAGE | \
Index: linux-cell/include/asm-powerpc/ppc_asm.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/ppc_asm.h	2006-10-13 16:00:25.000000000 +1000
+++ linux-cell/include/asm-powerpc/ppc_asm.h	2006-10-13 16:59:41.000000000 +1000
@@ -30,9 +30,9 @@ BEGIN_FTR_SECTION;							\
 	mfspr	ra,SPRN_PURR;		/* get processor util. reg */	\
 END_FTR_SECTION_IFSET(CPU_FTR_PURR);					\
 BEGIN_FTR_SECTION;							\
-	mftb	ra;			/* or get TB if no PURR */	\
+	MFTB(ra);			/* or get TB if no PURR */	\
 END_FTR_SECTION_IFCLR(CPU_FTR_PURR);					\
-	ld	rb,PACA_STARTPURR(r13);				\
+	ld	rb,PACA_STARTPURR(r13);					\
 	std	ra,PACA_STARTPURR(r13);					\
 	subf	rb,rb,ra;		/* subtract start value */	\
 	ld	ra,PACA_USER_TIME(r13);					\
@@ -45,9 +45,9 @@ BEGIN_FTR_SECTION;							\
 	mfspr	ra,SPRN_PURR;		/* get processor util. reg */	\
 END_FTR_SECTION_IFSET(CPU_FTR_PURR);					\
 BEGIN_FTR_SECTION;							\
-	mftb	ra;			/* or get TB if no PURR */	\
+	MFTB(ra);			/* or get TB if no PURR */	\
 END_FTR_SECTION_IFCLR(CPU_FTR_PURR);					\
-	ld	rb,PACA_STARTPURR(r13);				\
+	ld	rb,PACA_STARTPURR(r13);					\
 	std	ra,PACA_STARTPURR(r13);					\
 	subf	rb,rb,ra;		/* subtract start value */	\
 	ld	ra,PACA_SYSTEM_TIME(r13);				\
@@ -274,6 +274,18 @@ END_FTR_SECTION_IFSET(CPU_FTR_601)
 #define ISYNC_601
 #endif
 
+#ifdef CONFIG_PPC_CELL
+#define MFTB(dest)			\
+	mftb  dest;			\
+BEGIN_FTR_SECTION_NESTED(96);		\
+	cmpwi dest,0;			\
+	bne+  90f;			\
+	mftb  dest;			\
+90:					\
+END_FTR_SECTION_NESTED(CPU_FTR_CELL_TB_BUG, CPU_FTR_CELL_TB_BUG, 96)
+#else
+#define MFTB(dest)			mftb dest
+#endif
 
 #ifndef CONFIG_SMP
 #define TLBSYNC
Index: linux-cell/include/asm-powerpc/reg.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/reg.h	2006-10-13 16:00:25.000000000 +1000
+++ linux-cell/include/asm-powerpc/reg.h	2006-10-13 16:59:41.000000000 +1000
@@ -617,11 +617,34 @@
 			asm volatile("mfspr %0," __stringify(rn) \
 				: "=r" (rval)); rval;})
 #define mtspr(rn, v)	asm volatile("mtspr " __stringify(rn) ",%0" : : "r" (v))
-
+#ifdef CONFIG_PPC_CELL
+#define mftb()		({unsigned long rval;				\
+			asm volatile(					\
+				"	mftb %0;\n"			\
+				"97:	cmpwi %0,0;\n"			\
+				"	bne+ 99f;\n"			\
+				"	mftb %0;\n"			\
+				"99:\n"					\
+				".section __ftr_fixup,\"a\"\n"		\
+				".align 3\n"				\
+				"98:\n"					\
+				"	.llong %1\n"			\
+				"	.llong %1\n"			\
+				"	.llong 97b-98b\n"		\
+				"	.llong 99b-98b\n"		\
+				".previous"				\
+			: "=r" (rval) : "i" (CPU_FTR_CELL_TB_BUG)); rval;})
+#else
 #define mftb()		({unsigned long rval;	\
 			asm volatile("mftb %0" : "=r" (rval)); rval;})
+#endif
+
+#ifndef __powerpc64__
 #define mftbl()		({unsigned long rval;	\
 			asm volatile("mftbl %0" : "=r" (rval)); rval;})
+#define mftbu()		({unsigned long rval;	\
+			asm volatile("mftbu %0" : "=r" (rval)); rval;})
+#endif /* __powerpc64__ */
 
 #define mttbl(v)	asm volatile("mttbl %0":: "r"(v))
 #define mttbu(v)	asm volatile("mttbu %0":: "r"(v))
Index: linux-cell/include/asm-powerpc/time.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/time.h	2006-10-13 16:00:25.000000000 +1000
+++ linux-cell/include/asm-powerpc/time.h	2006-10-13 16:59:41.000000000 +1000
@@ -85,26 +85,28 @@ struct div_result {
 /* On ppc64 this gets us the whole timebase; on ppc32 just the lower half */
 static inline unsigned long get_tbl(void)
 {
-	unsigned long tbl;
-
 #if defined(CONFIG_403GCX)
+	unsigned long tbl;
 	asm volatile("mfspr %0, 0x3dd" : "=r" (tbl));
+	return tbl;
+#elif defined(CONFIG_PPC32)
+	return mftbl();
 #else
-	asm volatile("mftb %0" : "=r" (tbl));
+	return mftb();
 #endif
-	return tbl;
 }
 
 static inline unsigned int get_tbu(void)
 {
+#ifdef CONFIG_403GCX
 	unsigned int tbu;
-
-#if defined(CONFIG_403GCX)
 	asm volatile("mfspr %0, 0x3dc" : "=r" (tbu));
+	return tbu;
+#elif defined(CONFIG_PPC32)
+	return mftbu();
 #else
-	asm volatile("mftbu %0" : "=r" (tbu));
+	return mftb();
 #endif
-	return tbu;
 }
 
 static inline unsigned int get_rtcl(void)
Index: linux-cell/include/asm-powerpc/timex.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/timex.h	2006-10-13 16:58:13.000000000 +1000
+++ linux-cell/include/asm-powerpc/timex.h	2006-10-13 16:59:41.000000000 +1000
@@ -8,6 +8,7 @@
  */
 
 #include <asm/cputable.h>
+#include <asm/reg.h>
 
 #define CLOCK_TICK_RATE	1024000 /* Underlying HZ */
 
@@ -15,13 +16,11 @@ typedef unsigned long cycles_t;
 
 static inline cycles_t get_cycles(void)
 {
-	cycles_t ret;
-
 #ifdef __powerpc64__
-
-	__asm__ __volatile__("mftb %0" : "=r" (ret) : );
-
+	return mftb();
 #else
+	cycles_t ret;
+
 	/*
 	 * For the "cycle" counter we use the timebase lower half.
 	 * Currently only used on SMP.
@@ -41,9 +40,8 @@ static inline cycles_t get_cycles(void)
 		"	.long 99b-98b\n"
 		".previous"
 		: "=r" (ret) : "i" (CPU_FTR_601));
-#endif
-
 	return ret;
+#endif
 }
 
 #endif	/* __KERNEL__ */

^ permalink raw reply

* [PATCH 4/5] powerpc: Support feature fixups in modules
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

This patch adds support for feature fixups in modules. This involves
adding support for R_PPC64_REL64 relocs to the 64 bits module loader.
It also modifies modpost.c to ignore the powerpc fixup sections (or it
would warn when used in .init.text).

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

Index: linux-cell/arch/powerpc/kernel/module_32.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/module_32.c	2006-10-13 16:00:26.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/module_32.c	2006-10-13 16:59:36.000000000 +1000
@@ -24,6 +24,8 @@
 #include <linux/kernel.h>
 #include <linux/cache.h>
 
+#include "setup.h"
+
 #if 0
 #define DEBUGP printk
 #else
@@ -269,33 +271,50 @@ int apply_relocate_add(Elf32_Shdr *sechd
 	return 0;
 }
 
+static const Elf_Shdr *find_section(const Elf_Ehdr *hdr,
+				    const Elf_Shdr *sechdrs,
+				    const char *name)
+{
+	char *secstrings;
+	unsigned int i;
+
+	secstrings = (char *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
+	for (i = 1; i < hdr->e_shnum; i++)
+		if (strcmp(secstrings+sechdrs[i].sh_name, name) == 0)
+			return &sechdrs[i];
+	return NULL;
+}
+
 int module_finalize(const Elf_Ehdr *hdr,
 		    const Elf_Shdr *sechdrs,
 		    struct module *me)
 {
-	char *secstrings;
-	unsigned int i;
+	const Elf_Shdr *sect;
 
 	me->arch.bug_table = NULL;
 	me->arch.num_bugs = 0;
 
 	/* Find the __bug_table section, if present */
-	secstrings = (char *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
-	for (i = 1; i < hdr->e_shnum; i++) {
-		if (strcmp(secstrings+sechdrs[i].sh_name, "__bug_table"))
-			continue;
-		me->arch.bug_table = (void *) sechdrs[i].sh_addr;
-		me->arch.num_bugs = sechdrs[i].sh_size / sizeof(struct bug_entry);
-		break;
+	sect = find_section(hdr, sechdrs, "__bug_table");
+	if (sect != NULL) {
+		me->arch.bug_table = (void *) sect->sh_addr;
+		me->arch.num_bugs = sect->sh_size / sizeof(struct bug_entry);
 	}
 
-	/*
+ 	/*
 	 * Strictly speaking this should have a spinlock to protect against
 	 * traversals, but since we only traverse on BUG()s, a spinlock
 	 * could potentially lead to deadlock and thus be counter-productive.
 	 */
 	list_add(&me->arch.bug_list, &module_bug_list);
 
+	/* Apply feature fixups */
+	sect = find_section(hdr, sechdrs, "__ftr_fixup");
+	if (sect != NULL)
+		do_feature_fixups(cur_cpu_spec->cpu_features,
+				  (void *)sect->sh_addr,
+				  (void *)sect->sh_addr + sect->sh_size);
+
 	return 0;
 }
 
Index: linux-cell/arch/powerpc/kernel/module_64.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/module_64.c	2006-10-13 16:00:26.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/module_64.c	2006-10-13 16:59:36.000000000 +1000
@@ -22,6 +22,9 @@
 #include <linux/vmalloc.h>
 #include <asm/module.h>
 #include <asm/uaccess.h>
+#include <asm/firmware.h>
+
+#include "setup.h"
 
 /* FIXME: We don't do .init separately.  To do this, we'd need to have
    a separate r2 value in the init and core section, and stub between
@@ -400,6 +403,11 @@ int apply_relocate_add(Elf64_Shdr *sechd
 				| (value & 0x03fffffc);
 			break;
 
+		case R_PPC64_REL64:
+			/* 64 bits relative (used by features fixups) */
+			*location = value - (unsigned long)location;
+			break;
+
 		default:
 			printk("%s: Unknown ADD relocation: %lu\n",
 			       me->name,
@@ -413,23 +421,33 @@ int apply_relocate_add(Elf64_Shdr *sechd
 
 LIST_HEAD(module_bug_list);
 
-int module_finalize(const Elf_Ehdr *hdr,
-		const Elf_Shdr *sechdrs, struct module *me)
+static const Elf_Shdr *find_section(const Elf_Ehdr *hdr,
+				    const Elf_Shdr *sechdrs,
+				    const char *name)
 {
 	char *secstrings;
 	unsigned int i;
 
+	secstrings = (char *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
+	for (i = 1; i < hdr->e_shnum; i++)
+		if (strcmp(secstrings+sechdrs[i].sh_name, name) == 0)
+			return &sechdrs[i];
+	return NULL;
+}
+
+int module_finalize(const Elf_Ehdr *hdr,
+		const Elf_Shdr *sechdrs, struct module *me)
+{
+	const Elf_Shdr *sect;
+
 	me->arch.bug_table = NULL;
 	me->arch.num_bugs = 0;
 
 	/* Find the __bug_table section, if present */
-	secstrings = (char *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
-	for (i = 1; i < hdr->e_shnum; i++) {
-		if (strcmp(secstrings+sechdrs[i].sh_name, "__bug_table"))
-			continue;
-		me->arch.bug_table = (void *) sechdrs[i].sh_addr;
-		me->arch.num_bugs = sechdrs[i].sh_size / sizeof(struct bug_entry);
-		break;
+	sect = find_section(hdr, sechdrs, "__bug_table");
+	if (sect != NULL) {
+		me->arch.bug_table = (void *) sect->sh_addr;
+		me->arch.num_bugs = sect->sh_size / sizeof(struct bug_entry);
 	}
 
 	/*
@@ -439,6 +457,19 @@ int module_finalize(const Elf_Ehdr *hdr,
 	 */
 	list_add(&me->arch.bug_list, &module_bug_list);
 
+	/* Apply feature fixups */
+	sect = find_section(hdr, sechdrs, "__ftr_fixup");
+	if (sect != NULL)
+		do_feature_fixups(cur_cpu_spec->cpu_features,
+				  (void *)sect->sh_addr,
+				  (void *)sect->sh_addr + sect->sh_size);
+
+	sect = find_section(hdr, sechdrs, "__fw_ftr_fixup");
+	if (sect != NULL)
+		do_feature_fixups(powerpc_firmware_features,
+				  (void *)sect->sh_addr,
+				  (void *)sect->sh_addr + sect->sh_size);
+
 	return 0;
 }
 
Index: linux-cell/scripts/mod/modpost.c
===================================================================
--- linux-cell.orig/scripts/mod/modpost.c	2006-10-06 13:48:29.000000000 +1000
+++ linux-cell/scripts/mod/modpost.c	2006-10-13 17:19:55.000000000 +1000
@@ -921,6 +921,8 @@ static int init_section_ref_ok(const cha
 		".fixup",
 		".smp_locks",
 		".plt",  /* seen on ARCH=um build on x86_64. Harmless */
+		"__ftr_fixup",		/* powerpc cpu feature fixup */
+		"__fw_ftr_fixup",	/* powerpc firmware feature fixup */
 		NULL
 	};
 	/* Start of section names */

^ permalink raw reply

* [PATCH 3/5] powerpc: Support feature fixups in vdso's
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

This patch reworks the feature fixup mecanism so vdso's can be fixed up.
The main issue was that the construct:

        .long   label  (or .llong on 64 bits)

will not work in the case of a shared library like the vdso. It will
generate an empty placeholder in the fixup table along with a reloc,
which is not something we can deal with in the vdso.

The idea here (thanks Alan Modra !) is to instead use something like:

1:
        .long   label - 1b

That is, the feature fixup tables no longer contain addresses of bits of
code to patch, but offsets of such code from the fixup table entry
itself. That is properly resolved by ld when building the .so's. I've
modified the fixup mecanism generically to use that method for the rest
of the kernel as well.

Another trick is that the 32 bits vDSO included in the 64 bits kernel
need to have a table in the 64 bits format. However, gas does not
support 32 bits code with a statement of the form:

        .llong  label - 1b  (Or even just .llong label)

That is, it cannot emit the right fixup/relocation for the linker to use
to assign a 32 bits address to an .llong field. Thus, in the specific
case of the 32 bits vdso built as part of the 64 bits kernel, we are
using a modified macro that generates:

        .long   0xffffffff
        .llong  label - 1b

Note that is assumes that the value is negative which is enforced by
the .lds (those offsets are always negative as the .text is always
before the fixup table and gas doesn't support emiting the reloc the
other way around).

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

Index: linux-cell/arch/powerpc/kernel/vdso.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/vdso.c	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/vdso.c	2006-10-13 16:58:13.000000000 +1000
@@ -36,6 +36,8 @@
 #include <asm/vdso.h>
 #include <asm/vdso_datapage.h>
 
+#include "setup.h"
+
 #undef DEBUG
 
 #ifdef DEBUG
@@ -586,6 +588,43 @@ static __init int vdso_fixup_datapage(st
 	return 0;
 }
 
+
+static __init int vdso_fixup_features(struct lib32_elfinfo *v32,
+				      struct lib64_elfinfo *v64)
+{
+	void *start32;
+	unsigned long size32;
+
+#ifdef CONFIG_PPC64
+	void *start64;
+	unsigned long size64;
+
+	start64 = find_section64(v64->hdr, "__ftr_fixup", &size64);
+	if (start64)
+		do_feature_fixups(cur_cpu_spec->cpu_features,
+				  start64, start64 + size64);
+
+	start64 = find_section64(v64->hdr, "__fw_ftr_fixup", &size64);
+	if (start64)
+		do_feature_fixups(powerpc_firmware_features,
+				  start64, start64 + size64);
+#endif /* CONFIG_PPC64 */
+
+	start32 = find_section32(v32->hdr, "__ftr_fixup", &size32);
+	if (start32)
+		do_feature_fixups(cur_cpu_spec->cpu_features,
+				  start32, start32 + size32);
+
+#ifdef CONFIG_PPC64
+	start32 = find_section32(v32->hdr, "__fw_ftr_fixup", &size32);
+	if (start32)
+		do_feature_fixups(powerpc_firmware_features,
+				  start32, start32 + size32);
+#endif /* CONFIG_PPC64 */
+
+	return 0;
+}
+
 static __init int vdso_fixup_alt_funcs(struct lib32_elfinfo *v32,
 				       struct lib64_elfinfo *v64)
 {
@@ -634,6 +673,9 @@ static __init int vdso_setup(void)
 	if (vdso_fixup_datapage(&v32, &v64))
 		return -1;
 
+	if (vdso_fixup_features(&v32, &v64))
+		return -1;
+
 	if (vdso_fixup_alt_funcs(&v32, &v64))
 		return -1;
 
@@ -714,6 +756,7 @@ void __init vdso_init(void)
 	 * Setup the syscall map in the vDOS
 	 */
 	vdso_setup_syscall_map();
+
 	/*
 	 * Initialize the vDSO images in memory, that is do necessary
 	 * fixups of vDSO symbols, locate trampolines, etc...
Index: linux-cell/arch/powerpc/kernel/vdso32/vdso32.lds.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/vdso32/vdso32.lds.S	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/vdso32/vdso32.lds.S	2006-10-13 16:58:13.000000000 +1000
@@ -32,6 +32,18 @@ SECTIONS
   PROVIDE (_etext = .);
   PROVIDE (etext = .);
 
+  . = ALIGN(8);
+  __ftr_fixup : {
+    *(__ftr_fixup)
+  }
+
+#ifdef CONFIG_PPC64
+  . = ALIGN(8);
+  __fw_ftr_fixup : {
+    *(__fw_ftr_fixup)
+  }
+#endif
+
   /* Other stuff is appended to the text segment: */
   .rodata		: { *(.rodata .rodata.* .gnu.linkonce.r.*) }
   .rodata1		: { *(.rodata1) }
Index: linux-cell/arch/powerpc/kernel/vdso64/vdso64.lds.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/vdso64/vdso64.lds.S	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/vdso64/vdso64.lds.S	2006-10-13 16:58:13.000000000 +1000
@@ -31,6 +31,16 @@ SECTIONS
   PROVIDE (_etext = .);
   PROVIDE (etext = .);
 
+  . = ALIGN(8);
+  __ftr_fixup : {
+    *(__ftr_fixup)
+  }
+
+  . = ALIGN(8);
+  __fw_ftr_fixup : {
+    *(__fw_ftr_fixup)
+  }
+
   /* Other stuff is appended to the text segment: */
   .rodata         : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
   .rodata1        : { *(.rodata1) }
Index: linux-cell/arch/powerpc/kernel/setup-common.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup-common.c	2006-10-13 16:39:03.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup-common.c	2006-10-13 16:58:55.000000000 +1000
@@ -521,14 +521,13 @@ void __init setup_panic(void)
 	atomic_notifier_chain_register(&panic_notifier_list, &ppc_panic_block);
 }
 
-void do_feature_fixups(unsigned long offset, unsigned long value,
-		       void *fixup_start, void *fixup_end)
+void do_feature_fixups(unsigned long value, void *fixup_start, void *fixup_end)
 {
 	struct fixup_entry {
 		unsigned long	mask;
 		unsigned long	value;
-		unsigned int	*start;
-		unsigned int	*end;
+		long		start_off;
+		long		end_off;
 	} *fcur, *fend;
 
 	fcur = fixup_start;
@@ -543,8 +542,8 @@ void do_feature_fixups(unsigned long off
 		/* These PTRRELOCs will disappear once the new scheme for
 		 * modules and vdso is implemented
 		 */
-		pstart = PTRRELOC(fcur->start);
-		pend = PTRRELOC(fcur->end);
+		pstart = ((unsigned int *)fcur) + (fcur->start_off / 4);
+		pend = ((unsigned int *)fcur) + (fcur->end_off / 4);
 
 		for (p = pstart; p < pend; p++) {
 			*p = 0x60000000u;
Index: linux-cell/arch/powerpc/kernel/setup.h
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup.h	2006-10-13 16:06:24.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup.h	2006-10-13 16:58:13.000000000 +1000
@@ -4,8 +4,8 @@
 extern void check_for_initrd(void);
 extern void do_init_bootmem(void);
 extern void setup_panic(void);
-extern void do_feature_fixups(unsigned long offset, unsigned long value,
-			      void *fixup_start, void *fixup_end);
+extern void do_feature_fixups(unsigned long value, void *fixup_start,
+			      void *fixup_end);
 
 extern int do_early_xmon;
 
Index: linux-cell/arch/powerpc/kernel/setup_32.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup_32.c	2006-10-13 16:38:21.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup_32.c	2006-10-13 16:59:28.000000000 +1000
@@ -104,7 +104,7 @@ unsigned long __init early_init(unsigned
 	 */
 	spec = identify_cpu(offset);
 
-	do_feature_fixups(offset, spec->cpu_features,
+	do_feature_fixups(spec->cpu_features,
 			  PTRRELOC(&__start___ftr_fixup),
 			  PTRRELOC(&__stop___ftr_fixup));
 
Index: linux-cell/arch/powerpc/kernel/setup_64.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup_64.c	2006-10-13 16:42:44.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup_64.c	2006-10-13 16:58:13.000000000 +1000
@@ -357,9 +357,9 @@ void __init setup_system(void)
 	/* Apply the CPUs-specific and firmware specific fixups to kernel
 	 * text (nop out sections not relevant to this CPU or this firmware)
 	 */
-	do_feature_fixups(0, cur_cpu_spec->cpu_features,
+	do_feature_fixups(cur_cpu_spec->cpu_features,
 			  &__start___ftr_fixup, &__stop___ftr_fixup);
-	do_feature_fixups(0, powerpc_firmware_features,
+	do_feature_fixups(powerpc_firmware_features,
 			  &__start___fw_ftr_fixup, &__stop___fw_ftr_fixup);
 
 	/*
Index: linux-cell/include/asm-powerpc/asm-compat.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/asm-compat.h	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/include/asm-powerpc/asm-compat.h	2006-10-13 16:58:13.000000000 +1000
@@ -14,6 +14,58 @@
 #  define ASM_CONST(x)		__ASM_CONST(x)
 #endif
 
+
+/*
+ * Feature section common macros
+ *
+ * Note that the entries now contain offsets between the table entry
+ * and the code rather than absolute code pointers in order to be
+ * useable with the vdso shared library. There is also an assumption
+ * that values will be negative, that is, the fixup table has to be
+ * located after the code it fixes up.
+ */
+#ifdef CONFIG_PPC64
+#ifdef __powerpc64__
+/* 64 bits kernel, 64 bits code */
+#define MAKE_FTR_SECTION_ENTRY(msk, val, label, sect)	\
+99:							\
+	.section sect,"a";				\
+	.align 3;					\
+98:						       	\
+	.llong msk;					\
+	.llong val;					\
+	.llong label##b-98b;				\
+	.llong 99b-98b;		 			\
+	.previous
+#else /* __powerpc64__ */
+/* 64 bits kernel, 32 bits code (ie. vdso32) */
+#define MAKE_FTR_SECTION_ENTRY(msk, val, label, sect)	\
+99:							\
+	.section sect,"a";				\
+	.align 3;					\
+98:						       	\
+	.llong msk;					\
+	.llong val;					\
+	.long 0xffffffff;      				\
+	.long label##b-98b;				\
+	.long 0xffffffff;	       			\
+	.long 99b-98b;		 			\
+	.previous
+#endif /* !__powerpc64__ */
+#else /* CONFIG_PPC64 */
+/* 32 bits kernel, 32 bits code */
+#define MAKE_FTR_SECTION_ENTRY(msk, val, label, sect)	\
+99:						       	\
+	.section sect,"a";			       	\
+	.align 2;				       	\
+98:						       	\
+	.long msk;				       	\
+	.long val;				       	\
+	.long label##b-98b;			       	\
+	.long 99b-98b;				       	\
+	.previous
+#endif /* !CONFIG_PPC64 */
+
 #ifdef __powerpc64__
 
 /* operations for longs and pointers */
Index: linux-cell/include/asm-powerpc/cputable.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/cputable.h	2006-10-13 16:58:07.000000000 +1000
+++ linux-cell/include/asm-powerpc/cputable.h	2006-10-13 16:58:13.000000000 +1000
@@ -431,32 +431,11 @@ static inline int cpu_has_feature(unsign
 #ifdef __ASSEMBLY__
 
 #define BEGIN_FTR_SECTION_NESTED(label)	label:
-#define BEGIN_FTR_SECTION		BEGIN_FTR_SECTION_NESTED(98)
-
-#ifndef __powerpc64__
-#define END_FTR_SECTION_NESTED(msk, val, label) \
-99:						\
-	.section __ftr_fixup,"a";		\
-	.align 2;				\
-	.long msk;				\
-	.long val;				\
-	.long label##b;				\
-	.long 99b;				\
-	.previous
-#else /* __powerpc64__ */
+#define BEGIN_FTR_SECTION		BEGIN_FTR_SECTION_NESTED(97)
 #define END_FTR_SECTION_NESTED(msk, val, label) \
-99:						\
-	.section __ftr_fixup,"a";		\
-	.align 3;				\
-	.llong msk;				\
-	.llong val;				\
-	.llong label##b;				\
-	.llong 99b;	 			\
-	.previous
-#endif /* __powerpc64__ */
-
+	MAKE_FTR_SECTION_ENTRY(msk, val, label, __ftr_fixup)
 #define END_FTR_SECTION(msk, val)		\
-	END_FTR_SECTION_NESTED(msk, val, 98)
+	END_FTR_SECTION_NESTED(msk, val, 97)
 
 #define END_FTR_SECTION_IFSET(msk)	END_FTR_SECTION((msk), (msk))
 #define END_FTR_SECTION_IFCLR(msk)	END_FTR_SECTION((msk), 0)
Index: linux-cell/include/asm-powerpc/firmware.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/firmware.h	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/include/asm-powerpc/firmware.h	2006-10-13 16:58:13.000000000 +1000
@@ -98,17 +98,12 @@ extern int fwnmi_active;
 
 #else /* __ASSEMBLY__ */
 
-#define BEGIN_FW_FTR_SECTION		96:
-
+#define BEGIN_FW_FTR_SECTION_NESTED(label)	label:
+#define BEGIN_FW_FTR_SECTION			BEGIN_FW_FTR_SECTION_NESTED(97)
+#define END_FW_FTR_SECTION_NESTED(msk, val, label) \
+	MAKE_FTR_SECTION_ENTRY(msk, val, label, __fw_ftr_fixup)
 #define END_FW_FTR_SECTION(msk, val)		\
-97:						\
-	.section __fw_ftr_fixup,"a";		\
-	.align 3;				\
-	.llong msk;				\
-	.llong val;				\
-	.llong 96b;				\
-	.llong 97b;				\
-	.previous
+	END_FW_FTR_SECTION_NESTED(msk, val, 97)
 
 #define END_FW_FTR_SECTION_IFSET(msk)	END_FW_FTR_SECTION((msk), (msk))
 #define END_FW_FTR_SECTION_IFCLR(msk)	END_FW_FTR_SECTION((msk), 0)
Index: linux-cell/include/asm-powerpc/timex.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/timex.h	2006-10-13 16:00:27.000000000 +1000
+++ linux-cell/include/asm-powerpc/timex.h	2006-10-13 16:58:13.000000000 +1000
@@ -30,13 +30,15 @@ static inline cycles_t get_cycles(void)
 	ret = 0;
 
 	__asm__ __volatile__(
-		"98:	mftb %0\n"
+		"97:	mftb %0\n"
 		"99:\n"
 		".section __ftr_fixup,\"a\"\n"
+		".align 2\n"
+		"98:\n"
 		"	.long %1\n"
 		"	.long 0\n"
-		"	.long 98b\n"
-		"	.long 99b\n"
+		"	.long 97b-98b\n"
+		"	.long 99b-98b\n"
 		".previous"
 		: "=r" (ret) : "i" (CPU_FTR_601));
 #endif

^ permalink raw reply

* [PATCH 2/5] powerpc: support nested cpu features
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

This patch adds some macros that can be used with an explicit label in
order to nest cpu features. This should be used very careful but is
necessary for the upcoming cell TB fixup.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

Index: linux-cell/include/asm-powerpc/cputable.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/cputable.h	2006-10-13 16:19:51.000000000 +1000
+++ linux-cell/include/asm-powerpc/cputable.h	2006-10-13 16:58:07.000000000 +1000
@@ -430,30 +430,34 @@ static inline int cpu_has_feature(unsign
 
 #ifdef __ASSEMBLY__
 
-#define BEGIN_FTR_SECTION		98:
+#define BEGIN_FTR_SECTION_NESTED(label)	label:
+#define BEGIN_FTR_SECTION		BEGIN_FTR_SECTION_NESTED(98)
 
 #ifndef __powerpc64__
-#define END_FTR_SECTION(msk, val)		\
+#define END_FTR_SECTION_NESTED(msk, val, label) \
 99:						\
 	.section __ftr_fixup,"a";		\
 	.align 2;				\
 	.long msk;				\
 	.long val;				\
-	.long 98b;				\
+	.long label##b;				\
 	.long 99b;				\
 	.previous
 #else /* __powerpc64__ */
-#define END_FTR_SECTION(msk, val)		\
+#define END_FTR_SECTION_NESTED(msk, val, label) \
 99:						\
 	.section __ftr_fixup,"a";		\
 	.align 3;				\
 	.llong msk;				\
 	.llong val;				\
-	.llong 98b;				\
+	.llong label##b;				\
 	.llong 99b;	 			\
 	.previous
 #endif /* __powerpc64__ */
 
+#define END_FTR_SECTION(msk, val)		\
+	END_FTR_SECTION_NESTED(msk, val, 98)
+
 #define END_FTR_SECTION_IFSET(msk)	END_FTR_SECTION((msk), (msk))
 #define END_FTR_SECTION_IFCLR(msk)	END_FTR_SECTION((msk), 0)
 #endif /* __ASSEMBLY__ */

^ permalink raw reply

* [PATCH 1/5] powerpc: consolidate feature fixup code
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

There are currently two versions of the functions for applying the
feature fixups, one for CPU features and one for firmware features. In
addition, they are both in assembly and with separate implementations
for 32 and 64 bits. identify_cpu() is also implemented in assembly and
separately for 32 and 64 bits.

This patch replaces them with a pair of C functions. The call sites are
slightly moved on ppc64 as well to be called from C instead of from
assembly, though it's a very small change, and thus shouldn't cause any
problem.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>


Index: linux-cell/arch/powerpc/kernel/head_64.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/head_64.S	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/head_64.S	2006-10-13 16:43:05.000000000 +1000
@@ -1580,11 +1580,6 @@ _STATIC(__start_initialization_iSeries)
 	li	r0,0
 	stdu	r0,-STACK_FRAME_OVERHEAD(r1)
 
-	LOAD_REG_IMMEDIATE(r3,cpu_specs)
-	LOAD_REG_IMMEDIATE(r4,cur_cpu_spec)
-	li	r5,0
-	bl	.identify_cpu
-
 	LOAD_REG_IMMEDIATE(r2,__toc_start)
 	addi	r2,r2,0x4000
 	addi	r2,r2,0x4000
@@ -1964,13 +1959,6 @@ _STATIC(start_here_multiplatform)
 	addi	r2,r2,0x4000
 	add	r2,r2,r26
 
-	LOAD_REG_IMMEDIATE(r3, cpu_specs)
-	add	r3,r3,r26
-	LOAD_REG_IMMEDIATE(r4,cur_cpu_spec)
-	add	r4,r4,r26
-	mr	r5,r26
-	bl	.identify_cpu
-
 	/* Do very early kernel initializations, including initial hash table,
 	 * stab and slb setup before we turn on relocation.	*/
 
@@ -2000,13 +1988,6 @@ _STATIC(start_here_common)
 	li	r0,0
 	stdu	r0,-STACK_FRAME_OVERHEAD(r1)
 
-	/* Apply the CPUs-specific fixups (nop out sections not relevant
-	 * to this CPU
-	 */
-	li	r3,0
-	bl	.do_cpu_ftr_fixups
-	bl	.do_fw_ftr_fixups
-
 	/* ptr to current */
 	LOAD_REG_IMMEDIATE(r4, init_task)
 	std	r4,PACACURRENT(r13)
Index: linux-cell/arch/powerpc/kernel/misc_64.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/misc_64.S	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/misc_64.S	2006-10-13 16:40:14.000000000 +1000
@@ -246,130 +246,6 @@ _GLOBAL(__flush_dcache_icache)
 	isync
 	blr
 
-/*
- * identify_cpu and calls setup_cpu
- * In:	r3 = base of the cpu_specs array
- *	r4 = address of cur_cpu_spec
- *	r5 = relocation offset
- */
-_GLOBAL(identify_cpu)
-	mfpvr	r7
-1:
-	lwz	r8,CPU_SPEC_PVR_MASK(r3)
-	and	r8,r8,r7
-	lwz	r9,CPU_SPEC_PVR_VALUE(r3)
-	cmplw	0,r9,r8
-	beq	1f
-	addi	r3,r3,CPU_SPEC_ENTRY_SIZE
-	b	1b
-1:
-	sub	r0,r3,r5
-	std	r0,0(r4)
-	ld	r4,CPU_SPEC_SETUP(r3)
-	cmpdi	0,r4,0
-	add	r4,r4,r5
-	beqlr
-	ld	r4,0(r4)
-	add	r4,r4,r5
-	mtctr	r4
-	/* Calling convention for cpu setup is r3=offset, r4=cur_cpu_spec */
-	mr	r4,r3
-	mr	r3,r5
-	bctr
-
-/*
- * do_cpu_ftr_fixups - goes through the list of CPU feature fixups
- * and writes nop's over sections of code that don't apply for this cpu.
- * r3 = data offset (not changed)
- */
-_GLOBAL(do_cpu_ftr_fixups)
-	/* Get CPU 0 features */
-	LOAD_REG_IMMEDIATE(r6,cur_cpu_spec)
-	sub	r6,r6,r3
-	ld	r4,0(r6)
-	sub	r4,r4,r3
-	ld	r4,CPU_SPEC_FEATURES(r4)
-	/* Get the fixup table */
-	LOAD_REG_IMMEDIATE(r6,__start___ftr_fixup)
-	sub	r6,r6,r3
-	LOAD_REG_IMMEDIATE(r7,__stop___ftr_fixup)
-	sub	r7,r7,r3
-	/* Do the fixup */
-1:	cmpld	r6,r7
-	bgelr
-	addi	r6,r6,32
-	ld	r8,-32(r6)	/* mask */
-	and	r8,r8,r4
-	ld	r9,-24(r6)	/* value */
-	cmpld	r8,r9
-	beq	1b
-	ld	r8,-16(r6)	/* section begin */
-	ld	r9,-8(r6)	/* section end */
-	subf.	r9,r8,r9
-	beq	1b
-	/* write nops over the section of code */
-	/* todo: if large section, add a branch at the start of it */
-	srwi	r9,r9,2
-	mtctr	r9
-	sub	r8,r8,r3
-	lis	r0,0x60000000@h	/* nop */
-3:	stw	r0,0(r8)
-	andi.	r10,r4,CPU_FTR_SPLIT_ID_CACHE@l
-	beq	2f
-	dcbst	0,r8		/* suboptimal, but simpler */
-	sync
-	icbi	0,r8
-2:	addi	r8,r8,4
-	bdnz	3b
-	sync			/* additional sync needed on g4 */
-	isync
-	b	1b
-
-/*
- * do_fw_ftr_fixups - goes through the list of firmware feature fixups
- * and writes nop's over sections of code that don't apply for this firmware.
- * r3 = data offset (not changed)
- */
-_GLOBAL(do_fw_ftr_fixups)
-	/* Get firmware features */
-	LOAD_REG_IMMEDIATE(r6,powerpc_firmware_features)
-	sub	r6,r6,r3
-	ld	r4,0(r6)
-	/* Get the fixup table */
-	LOAD_REG_IMMEDIATE(r6,__start___fw_ftr_fixup)
-	sub	r6,r6,r3
-	LOAD_REG_IMMEDIATE(r7,__stop___fw_ftr_fixup)
-	sub	r7,r7,r3
-	/* Do the fixup */
-1:	cmpld	r6,r7
-	bgelr
-	addi	r6,r6,32
-	ld	r8,-32(r6)	/* mask */
-	and	r8,r8,r4
-	ld	r9,-24(r6)	/* value */
-	cmpld	r8,r9
-	beq	1b
-	ld	r8,-16(r6)	/* section begin */
-	ld	r9,-8(r6)	/* section end */
-	subf.	r9,r8,r9
-	beq	1b
-	/* write nops over the section of code */
-	/* todo: if large section, add a branch at the start of it */
-	srwi	r9,r9,2
-	mtctr	r9
-	sub	r8,r8,r3
-	lis	r0,0x60000000@h	/* nop */
-3:	stw	r0,0(r8)
-BEGIN_FTR_SECTION
-	dcbst	0,r8		/* suboptimal, but simpler */
-	sync
-	icbi	0,r8
-END_FTR_SECTION_IFSET(CPU_FTR_SPLIT_ID_CACHE)
-	addi	r8,r8,4
-	bdnz	3b
-	sync			/* additional sync needed on g4 */
-	isync
-	b	1b
 
 #if defined(CONFIG_PPC_PMAC) || defined(CONFIG_PPC_MAPLE)
 /*
Index: linux-cell/arch/powerpc/kernel/setup-common.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup-common.c	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup-common.c	2006-10-13 16:39:03.000000000 +1000
@@ -520,3 +520,39 @@ void __init setup_panic(void)
 {
 	atomic_notifier_chain_register(&panic_notifier_list, &ppc_panic_block);
 }
+
+void do_feature_fixups(unsigned long offset, unsigned long value,
+		       void *fixup_start, void *fixup_end)
+{
+	struct fixup_entry {
+		unsigned long	mask;
+		unsigned long	value;
+		unsigned int	*start;
+		unsigned int	*end;
+	} *fcur, *fend;
+
+	fcur = fixup_start;
+	fend = fixup_end;
+
+	for (; fcur < fend; fcur++) {
+		unsigned int *pstart, *pend, *p;
+
+		if ((value & fcur->mask) == fcur->value)
+			continue;
+
+		/* These PTRRELOCs will disappear once the new scheme for
+		 * modules and vdso is implemented
+		 */
+		pstart = PTRRELOC(fcur->start);
+		pend = PTRRELOC(fcur->end);
+
+		for (p = pstart; p < pend; p++) {
+			*p = 0x60000000u;
+			asm volatile ("dcbst 0, %0" : : "r" (p));
+		}
+		asm volatile ("sync" : : : "memory");
+		for (p = pstart; p < pend; p++)
+			asm volatile ("icbi 0,%0" : : "r" (p));
+		asm volatile ("sync; isync" : : : "memory");
+	}
+}
Index: linux-cell/arch/powerpc/kernel/setup.h
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup.h	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup.h	2006-10-13 16:06:24.000000000 +1000
@@ -1,9 +1,12 @@
 #ifndef _POWERPC_KERNEL_SETUP_H
 #define _POWERPC_KERNEL_SETUP_H
 
-void check_for_initrd(void);
-void do_init_bootmem(void);
-void setup_panic(void);
+extern void check_for_initrd(void);
+extern void do_init_bootmem(void);
+extern void setup_panic(void);
+extern void do_feature_fixups(unsigned long offset, unsigned long value,
+			      void *fixup_start, void *fixup_end);
+
 extern int do_early_xmon;
 
 #endif /* _POWERPC_KERNEL_SETUP_H */
Index: linux-cell/arch/powerpc/kernel/setup_32.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup_32.c	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup_32.c	2006-10-13 16:38:21.000000000 +1000
@@ -90,7 +90,9 @@ int ucache_bsize;
  */
 unsigned long __init early_init(unsigned long dt_ptr)
 {
+	extern unsigned int __start___ftr_fixup, __stop___ftr_fixup;
 	unsigned long offset = reloc_offset();
+	struct cpu_spec *spec;
 
 	/* First zero the BSS -- use memset_io, some platforms don't have
 	 * caches on yet */
@@ -100,8 +102,11 @@ unsigned long __init early_init(unsigned
 	 * Identify the CPU type and fix up code sections
 	 * that depend on which cpu we have.
 	 */
-	identify_cpu(offset, 0);
-	do_cpu_ftr_fixups(offset);
+	spec = identify_cpu(offset);
+
+	do_feature_fixups(offset, spec->cpu_features,
+			  PTRRELOC(&__start___ftr_fixup),
+			  PTRRELOC(&__stop___ftr_fixup));
 
 	return KERNELBASE + offset;
 }
Index: linux-cell/arch/powerpc/kernel/setup_64.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/setup_64.c	2006-10-13 16:04:10.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/setup_64.c	2006-10-13 16:42:44.000000000 +1000
@@ -170,6 +170,9 @@ void __init setup_paca(int cpu)
 
 void __init early_setup(unsigned long dt_ptr)
 {
+	/* Identify CPU type */
+	identify_cpu(0);
+
 	/* Assume we're on cpu 0 for now. Don't write to the paca yet! */
 	setup_paca(0);
 
@@ -346,8 +349,19 @@ static void __init initialize_cache_info
  */
 void __init setup_system(void)
 {
+	extern unsigned int __start___ftr_fixup, __stop___ftr_fixup;
+	extern unsigned int __start___fw_ftr_fixup, __stop___fw_ftr_fixup;
+
 	DBG(" -> setup_system()\n");
 
+	/* Apply the CPUs-specific and firmware specific fixups to kernel
+	 * text (nop out sections not relevant to this CPU or this firmware)
+	 */
+	do_feature_fixups(0, cur_cpu_spec->cpu_features,
+			  &__start___ftr_fixup, &__stop___ftr_fixup);
+	do_feature_fixups(0, powerpc_firmware_features,
+			  &__start___fw_ftr_fixup, &__stop___fw_ftr_fixup);
+
 	/*
 	 * Unflatten the device-tree passed by prom_init or kexec
 	 */
Index: linux-cell/arch/powerpc/kernel/cputable.c
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/cputable.c	2006-10-06 13:47:54.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/cputable.c	2006-10-13 16:47:31.000000000 +1000
@@ -73,7 +73,7 @@ extern void __restore_cpu_ppc970(void);
 #define PPC_FEATURE_SPE_COMP	0
 #endif
 
-struct cpu_spec	cpu_specs[] = {
+static struct cpu_spec cpu_specs[] = {
 #ifdef CONFIG_PPC64
 	{	/* Power3 */
 		.pvr_mask		= 0xffff0000,
@@ -1152,3 +1152,25 @@ struct cpu_spec	cpu_specs[] = {
 #endif /* !CLASSIC_PPC */
 #endif /* CONFIG_PPC32 */
 };
+
+struct cpu_spec *identify_cpu(unsigned long offset)
+{
+	struct cpu_spec *s = cpu_specs;
+	struct cpu_spec **cur = &cur_cpu_spec;
+	unsigned int pvr = mfspr(SPRN_PVR);
+	int i;
+
+	s = PTRRELOC(s);
+	cur = PTRRELOC(cur);
+
+	if (*cur != NULL)
+		return PTRRELOC(*cur);
+
+	for (i = 0; i < ARRAY_SIZE(cpu_specs); i++,s++)
+		if ((pvr & s->pvr_mask) == s->pvr_value) {
+			*cur = cpu_specs + i;
+			return s;
+		}
+	BUG();
+	return NULL;
+}
Index: linux-cell/arch/powerpc/kernel/misc_32.S
===================================================================
--- linux-cell.orig/arch/powerpc/kernel/misc_32.S	2006-10-06 13:47:54.000000000 +1000
+++ linux-cell/arch/powerpc/kernel/misc_32.S	2006-10-13 16:26:43.000000000 +1000
@@ -102,80 +102,6 @@ _GLOBAL(reloc_got2)
 	blr
 
 /*
- * identify_cpu,
- * called with r3 = data offset and r4 = CPU number
- * doesn't change r3
- */
-_GLOBAL(identify_cpu)
-	addis	r8,r3,cpu_specs@ha
-	addi	r8,r8,cpu_specs@l
-	mfpvr	r7
-1:
-	lwz	r5,CPU_SPEC_PVR_MASK(r8)
-	and	r5,r5,r7
-	lwz	r6,CPU_SPEC_PVR_VALUE(r8)
-	cmplw	0,r6,r5
-	beq	1f
-	addi	r8,r8,CPU_SPEC_ENTRY_SIZE
-	b	1b
-1:
-	addis	r6,r3,cur_cpu_spec@ha
-	addi	r6,r6,cur_cpu_spec@l
-	sub	r8,r8,r3
-	stw	r8,0(r6)
-	blr
-
-/*
- * do_cpu_ftr_fixups - goes through the list of CPU feature fixups
- * and writes nop's over sections of code that don't apply for this cpu.
- * r3 = data offset (not changed)
- */
-_GLOBAL(do_cpu_ftr_fixups)
-	/* Get CPU 0 features */
-	addis	r6,r3,cur_cpu_spec@ha
-	addi	r6,r6,cur_cpu_spec@l
-	lwz	r4,0(r6)
-	add	r4,r4,r3
-	lwz	r4,CPU_SPEC_FEATURES(r4)
-
-	/* Get the fixup table */
-	addis	r6,r3,__start___ftr_fixup@ha
-	addi	r6,r6,__start___ftr_fixup@l
-	addis	r7,r3,__stop___ftr_fixup@ha
-	addi	r7,r7,__stop___ftr_fixup@l
-
-	/* Do the fixup */
-1:	cmplw	0,r6,r7
-	bgelr
-	addi	r6,r6,16
-	lwz	r8,-16(r6)	/* mask */
-	and	r8,r8,r4
-	lwz	r9,-12(r6)	/* value */
-	cmplw	0,r8,r9
-	beq	1b
-	lwz	r8,-8(r6)	/* section begin */
-	lwz	r9,-4(r6)	/* section end */
-	subf.	r9,r8,r9
-	beq	1b
-	/* write nops over the section of code */
-	/* todo: if large section, add a branch at the start of it */
-	srwi	r9,r9,2
-	mtctr	r9
-	add	r8,r8,r3
-	lis	r0,0x60000000@h	/* nop */
-3:	stw	r0,0(r8)
-	andi.	r10,r4,CPU_FTR_SPLIT_ID_CACHE@l
-	beq	2f
-	dcbst	0,r8		/* suboptimal, but simpler */
-	sync
-	icbi	0,r8
-2:	addi	r8,r8,4
-	bdnz	3b
-	sync			/* additional sync needed on g4 */
-	isync
-	b	1b
-
-/*
  * call_setup_cpu - call the setup_cpu function for this cpu
  * r3 = data offset, r24 = cpu number
  *
Index: linux-cell/include/asm-powerpc/cputable.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/cputable.h	2006-10-13 16:00:28.000000000 +1000
+++ linux-cell/include/asm-powerpc/cputable.h	2006-10-13 16:19:51.000000000 +1000
@@ -89,8 +89,7 @@ struct cpu_spec {
 
 extern struct cpu_spec		*cur_cpu_spec;
 
-extern void identify_cpu(unsigned long offset, unsigned long cpu);
-extern void do_cpu_ftr_fixups(unsigned long offset);
+extern struct cpu_spec *identify_cpu(unsigned long offset);
 
 #endif /* __ASSEMBLY__ */
 
Index: linux-cell/include/asm-powerpc/system.h
===================================================================
--- linux-cell.orig/include/asm-powerpc/system.h	2006-10-06 13:48:24.000000000 +1000
+++ linux-cell/include/asm-powerpc/system.h	2006-10-13 16:24:45.000000000 +1000
@@ -378,7 +378,11 @@ extern unsigned long reloc_offset(void);
 extern unsigned long add_reloc_offset(unsigned long);
 extern void reloc_got2(unsigned long);
 
+#ifdef CONFIG_PPC64
+#define PTRRELOC(x)	x
+#else
 #define PTRRELOC(x)	((typeof(x)) add_reloc_offset((unsigned long)(x)))
+#endif
 
 static inline void create_instruction(unsigned long addr, unsigned int instr)
 {
Index: linux-cell/arch/powerpc/platforms/iseries/setup.c
===================================================================
--- linux-cell.orig/arch/powerpc/platforms/iseries/setup.c	2006-10-06 13:47:54.000000000 +1000
+++ linux-cell/arch/powerpc/platforms/iseries/setup.c	2006-10-13 16:49:14.000000000 +1000
@@ -694,6 +694,11 @@ void * __init iSeries_early_setup(void)
 {
 	unsigned long phys_mem_size;
 
+	/* Identify CPU type. This is done again by the common code later
+	 * on but calling this function multiple times is fine.
+	 */
+	identify_cpu(0);
+
 	powerpc_firmware_features |= FW_FEATURE_ISERIES;
 	powerpc_firmware_features |= FW_FEATURE_LPAR;
 

^ permalink raw reply

* [PATCH 0/5] powerpc: feature fixup updates
From: Benjamin Herrenschmidt @ 2006-10-13  8:04 UTC (permalink / raw)
  To: linuxppc-dev list

Here's a new batch of the feature fixup updates.

Changes since last time:

 - moves identify_cpu() to C code (removes more assembly gunk)
 - works on 32 bits :)
 - invert use of nested in MFTB for the cell fixup
 - adds a patch for fixup in modules support
 - the fixup code does 2 loops as suggested by Olof

The bulk could be merged if you want, except the cell TB fix which still
needs to change to the new workaround method. I haven't had time to
finish that today.

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox