Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table overflow

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table overflow

[Carlos O'Donell]

On Fri, Jul 31, 2009 at 5:17 AM, Frans Pop<elendil@planet.nl> wrote:
> Affects both stable and unstable!
>
> kernel: Linux version 2.6.26-2-parisc64-smp [...]
> kernel: nfs: Global Offset Table overflow (used 1075, allowed 1023)
>
> kernel: Linux version 2.6.30-1-parisc64 [...]
> kernel: nfs: Global Offset Table overflow (used 1164, allowed 1023)
>
> The error comes from arch/parisc/kernel/module.c.
> Looks like it is a known issue:
> http://lists.parisc-linux.org/pipermail/parisc-linux/2006-October/054826.html

CC'ing parisc-linux since this is a kernel issue.

Helge,

Did you ever work around the GOT limitations?

To give you a bit of background, position independent code (a module)
can't have any virtual addresses (they aren't known), therefore when
you need to compute the address of an object you do so using the
global offset table. After relocation processing the GOT allows you to
translate an object by name to a virtual address e.g. If you take the
address of a function, then relocations would cause a GOT entry to be
filled such that this entry contains the virtual address of the
function. The GOT stubs are pieces of code that load virtual addresses
from the GOT and call them. We use GOT stubs to call functions which
are not local to the module.

Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
which becomes a 13-bit limit when loading positive offsets e.g.
+0x1fff or 1023 GOT slots.

However, on 64-bit the long format of ldd has a 16-bit signed
immediate offset (0xffff), meaning it can reach +0x7fff e.g. 4095 GOT
slots.

Do you have the time to test something out?

* Make this conditional on 32-bit vs. 64-bit and allow for 4095 GOT
entries on 64-bit.
* Fix ELF_GOT_STUB for the 64-bit case. It needs to reassemble a
16-bit offset, the current code is IMO incorrect. i.e. it should be "&
0x7fff", and use a new reassemble_16 see the PA 2.0 book definition of
ldd.
* Build kernel.
* Test loading NFS moudle.

That should be it :-)

> I tried unloading other modules, but that made no difference
> ("used" value remained unchanged).

Unloading modules won't help, it's one GOT per module.

> Does this mean that using nfs on hppa is not possible at all?

No, I use nfs on my hppa system.

Cheers,
Carlos.

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

> On Fri, Jul 31, 2009 at 5:17 AM, Frans Pop<elendil@planet.nl> wrote:
> > Affects both stable and unstable!
> >
> > kernel: Linux version 2.6.26-2-parisc64-smp [...]
> > kernel: nfs: Global Offset Table overflow (used 1075, allowed 1023)
> >
> > kernel: Linux version 2.6.30-1-parisc64 [...]
> > kernel: nfs: Global Offset Table overflow (used 1164, allowed 1023)
> >
> > The error comes from arch/parisc/kernel/module.c.
> > Looks like it is a known issue:
> > http://lists.parisc-linux.org/pipermail/parisc-linux/2006-October/054826.html

I've seen the same problem.  Sent a message to the parisc-linux list
about this recently.

> Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
> which becomes a 13-bit limit when loading positive offsets e.g.
> +0x1fff or 1023 GOT slots.

Can't we offset the table and double the number of entries?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Helge Deller]

On 07/31/2009 09:03 PM, John David Anglin wrote:
>> Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
>> which becomes a 13-bit limit when loading positive offsets e.g.
>> +0x1fff or 1023 GOT slots.
>
> Can't we offset the table and double the number of entries?

Dave,
Can you explain this idea a little more?
Helge

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Fri, Jul 31, 2009 at 5:09 PM, Helge Deller<deller@gmx.de> wrote:
> On 07/31/2009 09:03 PM, John David Anglin wrote:
>>>
>>> Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
>>> which becomes a 13-bit limit when loading positive offsets e.g.
>>> +0x1fff or 1023 GOT slots.
>>
>> Can't we offset the table and double the number of entries?
>
> Dave,
> Can you explain this idea a little more?

I would also like a little more details.

However, this is similar to the DT_PLTGOT issue in dynamic libraries.
The value chosen for %dp is arbitrary, and if we made it point into
the middle of the GOT table, then you would reference the GOT using
both positive and negative offsets.

For example, this code:
fdesc->gp = (Elf_Addr)me->module_core + me->arch.got_offset;

Arbitrary chooses the module %dp to point at the start of got_offset,
why not make that got_offset + <half way>.

Cheers,
Carlos.

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Fri, Jul 31, 2009 at 5:13 PM, Carlos O'Donell<carlos@systemhalted.org> wrote:
> On Fri, Jul 31, 2009 at 5:09 PM, Helge Deller<deller@gmx.de> wrote:
>> On 07/31/2009 09:03 PM, John David Anglin wrote:
>>>>
>>>> Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
>>>> which becomes a 13-bit limit when loading positive offsets e.g.
>>>> +0x1fff or 1023 GOT slots.
>>>
>>> Can't we offset the table and double the number of entries?
>>
>> Dave,
>> Can you explain this idea a little more?
>
> I would also like a little more details.
>
> However, this is similar to the DT_PLTGOT issue in dynamic libraries.
> The value chosen for %dp is arbitrary, and if we made it point into
> the middle of the GOT table, then you would reference the GOT using
> both positive and negative offsets.
>
> For example, this code:
> fdesc->gp = (Elf_Addr)me->module_core + me->arch.got_offset;
>
> Arbitrary chooses the module %dp to point at the start of got_offset,
> why not make that got_offset + <half way>.

Let me be clearer, the value of "(Elf_Addr)me->module_core +
me->arch.got_offset" is the start of the GOT table for the module.

Cheers,
Carlos.

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

> On Fri, Jul 31, 2009 at 5:09 PM, Helge Deller<deller@gmx.de> wrote:
> > On 07/31/2009 09:03 PM, John David Anglin wrote:
> >>>
> >>> Only 32-bit targets have the 14-bit signed immediate offset (0x3fff),
> >>> which becomes a 13-bit limit when loading positive offsets e.g.
> >>> +0x1fff or 1023 GOT slots.
> >>
> >> Can't we offset the table and double the number of entries?
> >
> > Dave,
> > Can you explain this idea a little more?
> 
> I would also like a little more details.
> 
> However, this is similar to the DT_PLTGOT issue in dynamic libraries.
> The value chosen for %dp is arbitrary, and if we made it point into
> the middle of the GOT table, then you would reference the GOT using
> both positive and negative offsets.
> 
> For example, this code:
> fdesc->gp = (Elf_Addr)me->module_core + me->arch.got_offset;
> 
> Arbitrary chooses the module %dp to point at the start of got_offset,
> why not make that got_offset + <half way>.

I don't have more details...  The idea is as Carlos outlined.  There's
code in the binutils elf32-hppa.c and elf64-hppa.c files to implement
the above for dynamic libraries.  That's what made me think of it.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Fri, Jul 31, 2009 at 5:26 PM, John David
Anglin<dave@hiauly1.hia.nrc.ca> wrote:
> I don't have more details... =A0The idea is as Carlos outlined. =A0Th=
ere's
> code in the binutils elf32-hppa.c and elf64-hppa.c files to implement
> the above for dynamic libraries. =A0That's what made me think of it.

Binutils is not involved in the kernel module loader, instead
arch/parisc/kernel/module.c (get_fdesc) chooses where the gp will
point to.

If you set gp to the middle of the GOT table, *and* implement
long/short ldd access on 64-bit, then you would get a total of 8191
possible slots per module.

Personally I think the lower risk, quicker fix, is to implement a fix
for 64-bit kernels that uses ldd in format 3 for all offsets > 15
bytes, and thus allow you to set MAX_GOTS to 4095.

Note: ldd format 3 can't be used to load immediate values between 15
and -16 bytes.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Kyle McMartin]

On Fri, Jul 31, 2009 at 06:00:48PM -0400, Carlos O'Donell wrote:
> On Fri, Jul 31, 2009 at 5:26 PM, John David
> Anglin<dave@hiauly1.hia.nrc.ca> wrote:
> > I don't have more details... =A0The idea is as Carlos outlined. =A0=
There's
> > code in the binutils elf32-hppa.c and elf64-hppa.c files to impleme=
nt
> > the above for dynamic libraries. =A0That's what made me think of it=
=2E
>=20
> Binutils is not involved in the kernel module loader, instead
> arch/parisc/kernel/module.c (get_fdesc) chooses where the gp will
> point to.
>=20
> If you set gp to the middle of the GOT table, *and* implement
> long/short ldd access on 64-bit, then you would get a total of 8191
> possible slots per module.
>=20
> Personally I think the lower risk, quicker fix, is to implement a fix
> for 64-bit kernels that uses ldd in format 3 for all offsets > 15
> bytes, and thus allow you to set MAX_GOTS to 4095.
>=20
> Note: ldd format 3 can't be used to load immediate values between 15
> and -16 bytes.
>=20

Is it as simple as:

diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c
index ef5caf2..0502fab 100644
--- a/arch/parisc/kernel/module.c
+++ b/arch/parisc/kernel/module.c
@@ -82,13 +82,6 @@
 		return -ENOEXEC;			\
 	}
=20
-/* Maximum number of GOT entries. We use a long displacement ldd from
- * the bottom of the table, which has a maximum signed displacement of
- * 0x3fff; however, since we're only going forward, this becomes
- * 0x1fff, and thus, since each GOT entry is 8 bytes long we can have
- * at most 1023 entries */
-#define MAX_GOTS	1023
-
 /* three functions to determine where in the module core
  * or init pieces the location is */
 static inline int in_init(struct module *me, void *loc)
@@ -126,6 +119,14 @@ struct stub_entry {
 };
 #endif
=20
+/* Maximum number of GOT entries. We use a long displacement ldd from
+ * the bottom of the table, which has 16-bit signed displacement from
+ * %dp. Because we only use the forward direction, we're limited to
+ * 15-bits - 1, and because each GOT entry is 8-bytes wide, we're limi=
ted
+ * to 4095 entries.
+ */
+#define MAX_GOTS	(((1 << 15) - 1) / sizeof(struct got_entry))
+
 /* Field selection types defined by hppa */
 #define rnd(x)			(((x)+0x1000)&~0x1fff)
 /* fsel: full 32 bits */
@@ -151,6 +152,15 @@ static inline int reassemble_14(int as14)
 		((as14 & 0x2000) >> 13));
 }
=20
+/* Unusual 16-bit encoding, for wide mode only.  */
+static inline int reassemble_16a(int as16)
+{
+	int s, t;
+	t =3D (as16 << 1) & 0xffff;
+	s =3D (as16 & 0x8000);
+	return (t ^ s ^ (s >> 1)) | (s >> 15);
+}
+
 static inline int reassemble_17(int as17)
 {
 	return (((as17 & 0x10000) >> 16) |
@@ -460,12 +470,16 @@ static Elf_Addr get_stub(struct module *me, unsig=
ned long value, long addend,
  */
 	switch (stub_type) {
 	case ELF_STUB_GOT:
+		unsigned int d =3D get_got(me, value, addend) & 0x7fff;
+
 		stub->insns[0] =3D 0x537b0000;	/* ldd 0(%dp),%dp	*/
 		stub->insns[1] =3D 0x53610020;	/* ldd 10(%dp),%r1	*/
 		stub->insns[2] =3D 0xe820d000;	/* bve (%r1)		*/
 		stub->insns[3] =3D 0x537b0030;	/* ldd 18(%dp),%dp	*/
=20
-		stub->insns[0] |=3D reassemble_14(get_got(me, value, addend) & 0x3ff=
f);
+		if (d > 15)
+			stub->insns[0] |=3D reassemble_16a(d);
+
 		break;
 	case ELF_STUB_MILLI:
 		stub->insns[0] =3D 0x20200000;	/* ldil 0,%r1		*/

I don't think we need to worry about the initial 15-bytes displacement,
since they're all within the first got_entry? (The resulting assembly
looks alright from a 64-bit toolchain:

kyle@shortfin ~ $ cat foo.S
	.text
a:
	ldd 32760(%r27),%r27
	break	0,0

0000000000000000 <a>:
   0:	53 7b ff f0 	ldd 7ff8(dp),dp

int main(void) {
        unsigned int opcode =3D 0x537b0000;
        opcode |=3D re_assemble_16(32760);
        printf("0x%x\n", opcode);
        return 0;
}

kyle@shortfin ~ $ ./foo
0x537bfff0

Looks pretty happy?
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Helge Deller]

[-- Attachment #1: Type: text/plain, Size: 4583 bytes --]

On 08/01/2009 01:38 AM, Kyle McMartin wrote:
> On Fri, Jul 31, 2009 at 06:00:48PM -0400, Carlos O'Donell wrote:
>> On Fri, Jul 31, 2009 at 5:26 PM, John David
>> Anglin<dave@hiauly1.hia.nrc.ca>  wrote:
>>> I don't have more details...  The idea is as Carlos outlined.  There's
>>> code in the binutils elf32-hppa.c and elf64-hppa.c files to implement
>>> the above for dynamic libraries.  That's what made me think of it.
>> Binutils is not involved in the kernel module loader, instead
>> arch/parisc/kernel/module.c (get_fdesc) chooses where the gp will
>> point to.
>>
>> If you set gp to the middle of the GOT table, *and* implement
>> long/short ldd access on 64-bit, then you would get a total of 8191
>> possible slots per module.
>>
>> Personally I think the lower risk, quicker fix, is to implement a fix
>> for 64-bit kernels that uses ldd in format 3 for all offsets>  15
>> bytes, and thus allow you to set MAX_GOTS to 4095.
>>
>> Note: ldd format 3 can't be used to load immediate values between 15
>> and -16 bytes.
>>
>
> Is it as simple as:
>
> diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c
> index ef5caf2..0502fab 100644
> --- a/arch/parisc/kernel/module.c
> +++ b/arch/parisc/kernel/module.c
> @@ -82,13 +82,6 @@
>   		return -ENOEXEC;			\
>   	}
>
> -/* Maximum number of GOT entries. We use a long displacement ldd from
> - * the bottom of the table, which has a maximum signed displacement of
> - * 0x3fff; however, since we're only going forward, this becomes
> - * 0x1fff, and thus, since each GOT entry is 8 bytes long we can have
> - * at most 1023 entries */
> -#define MAX_GOTS	1023
> -
>   /* three functions to determine where in the module core
>    * or init pieces the location is */
>   static inline int in_init(struct module *me, void *loc)
> @@ -126,6 +119,14 @@ struct stub_entry {
>   };
>   #endif
>
> +/* Maximum number of GOT entries. We use a long displacement ldd from
> + * the bottom of the table, which has 16-bit signed displacement from
> + * %dp. Because we only use the forward direction, we're limited to
> + * 15-bits - 1, and because each GOT entry is 8-bytes wide, we're limited
> + * to 4095 entries.
> + */
> +#define MAX_GOTS	(((1<<  15) - 1) / sizeof(struct got_entry))
> +
>   /* Field selection types defined by hppa */
>   #define rnd(x)			(((x)+0x1000)&~0x1fff)
>   /* fsel: full 32 bits */
> @@ -151,6 +152,15 @@ static inline int reassemble_14(int as14)
>   		((as14&  0x2000)>>  13));
>   }
>
> +/* Unusual 16-bit encoding, for wide mode only.  */
> +static inline int reassemble_16a(int as16)
> +{
> +	int s, t;
> +	t = (as16<<  1)&  0xffff;
> +	s = (as16&  0x8000);
> +	return (t ^ s ^ (s>>  1)) | (s>>  15);
> +}
> +
>   static inline int reassemble_17(int as17)
>   {
>   	return (((as17&  0x10000)>>  16) |
> @@ -460,12 +470,16 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
>    */
>   	switch (stub_type) {
>   	case ELF_STUB_GOT:
> +		unsigned int d = get_got(me, value, addend)&  0x7fff;
> +
>   		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
>   		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
>   		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
>   		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
>
> -		stub->insns[0] |= reassemble_14(get_got(me, value, addend)&  0x3fff);
> +		if (d>  15)
> +			stub->insns[0] |= reassemble_16a(d);
> +
>   		break;
>   	case ELF_STUB_MILLI:
>   		stub->insns[0] = 0x20200000;	/* ldil 0,%r1		*/
>
> I don't think we need to worry about the initial 15-bytes displacement,
> since they're all within the first got_entry? (The resulting assembly
> looks alright from a 64-bit toolchain:
>
> kyle@shortfin ~ $ cat foo.S
> 	.text
> a:
> 	ldd 32760(%r27),%r27
> 	break	0,0
>
> 0000000000000000<a>:
>     0:	53 7b ff f0 	ldd 7ff8(dp),dp
>
> int main(void) {
>          unsigned int opcode = 0x537b0000;
>          opcode |= re_assemble_16(32760);
>          printf("0x%x\n", opcode);
>          return 0;
> }
>
> kyle@shortfin ~ $ ./foo
> 0x537bfff0
>
> Looks pretty happy?
>

Kyle, you beat me.
Attached is my patch ....

Tested and works.

root@c3000:~# uname -a
Linux c3000 2.6.31-rc4-64bit #42 SMP Sat Aug 1 01:37:29 CEST 2009 parisc64 GNU/Linux
root@c3000:~# lsmod
Module                  Size  Used by
ipv6                  493320  70
reiserfs              461624  0
nfs                   300704  0
lockd                 144456  1 nfs
nfs_acl                 5592  1 nfs
sunrpc                382312  3 nfs,lockd,nfs_acl
msdos                  15032  0
fat                    91248  1 msdos

Helge

[-- Attachment #2: got.patch --]
[-- Type: text/x-patch, Size: 2434 bytes --]

parisc: module.c - fix GOT table overflow with large kernel modules on 64 bit kernels

Signed-off-by: Helge Deller <deller@gmx.de>

diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c
index ef5caf2..d280219 100644
--- a/arch/parisc/kernel/module.c
+++ b/arch/parisc/kernel/module.c
@@ -86,8 +86,12 @@
  * the bottom of the table, which has a maximum signed displacement of
  * 0x3fff; however, since we're only going forward, this becomes
  * 0x1fff, and thus, since each GOT entry is 8 bytes long we can have
- * at most 1023 entries */
-#define MAX_GOTS	1023
+ * at most 1023 entries.
+ * To overcome this 14bit displacement with some kernel modules, we'll
+ * use instead the unusal 16bit displacement method (see reassemble_16a)
+ * which gives us a maximum positive displacement of 0x7fff, and as such 
+ * allows us to allocate up to 4095 GOT entries. */
+#define MAX_GOTS	4095
 
 /* three functions to determine where in the module core
  * or init pieces the location is */
@@ -151,6 +155,17 @@ static inline int reassemble_14(int as14)
 		((as14 & 0x2000) >> 13));
 }
 
+static inline int reassemble_16a(int as16)
+{
+	int s, t;
+
+	/* Unusual 16-bit encoding, for wide mode only.  */
+	t = (as16 << 1) & 0xffff;
+	s = (as16 & 0x8000);
+	return (t ^ s ^ (s >> 1)) | (s >> 15);
+}
+
+
 static inline int reassemble_17(int as17)
 {
 	return (((as17 & 0x10000) >> 16) |
@@ -407,6 +422,7 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
 	enum elf_stub_type stub_type, Elf_Addr loc0, unsigned int targetsec)
 {
 	struct stub_entry *stub;
+	int d;
 
 	/* initialize stub_offset to point in front of the section */
 	if (!me->arch.section[targetsec].stub_offset) {
@@ -460,12 +476,17 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
  */
 	switch (stub_type) {
 	case ELF_STUB_GOT:
-		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
+		stub->insns[0] = 0x537b0000;    /* ldd 0(%dp),%dp	*/
 		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
 		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
 		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
 
-		stub->insns[0] |= reassemble_14(get_got(me, value, addend) & 0x3fff);
+		d = get_got(me, value, addend);
+		if (d <= 15)
+			stub->insns[0] |= reassemble_14(d);
+		else
+			stub->insns[0] |= reassemble_16a(d);
+
 		break;
 	case ELF_STUB_MILLI:
 		stub->insns[0] = 0x20200000;	/* ldil 0,%r1		*/

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

>  	case ELF_STUB_GOT:
> -		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
> +		stub->insns[0] = 0x537b0000;    /* ldd 0(%dp),%dp	*/
>  		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
>  		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
>  		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
>  
> -		stub->insns[0] |= reassemble_14(get_got(me, value, addend) & 0x3fff);
> +		d = get_got(me, value, addend);
> +		if (d <= 15)
> +			stub->insns[0] |= reassemble_14(d);

reassemble_14 is wrong for ldd format 3.  Need format 5 and im5 insertion.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

> >  	case ELF_STUB_GOT:
> > -		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
> > +		stub->insns[0] = 0x537b0000;    /* ldd 0(%dp),%dp	*/
> >  		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
> >  		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
> >  		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
> >  
> > -		stub->insns[0] |= reassemble_14(get_got(me, value, addend) & 0x3fff);
> > +		d = get_got(me, value, addend);
> > +		if (d <= 15)
> > +			stub->insns[0] |= reassemble_14(d);
> 
> reassemble_14 is wrong for ldd format 3.  Need format 5 and im5 insertion.

The format 5 version of ldd 0(%dp),%dp is 0x0f6010db.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Thu, Jul 30, 2009 at 8:37 PM, John David
Anglin<dave@hiauly1.hia.nrc.ca> wrote:
>> =A0 =A0 =A0 case ELF_STUB_GOT:
>> - =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] =3D 0x537b0000; =A0 =A0/* l=
dd 0(%dp),%dp =A0 =A0 =A0 */
>> + =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] =3D 0x537b0000; =A0 =A0/* l=
dd 0(%dp),%dp =A0 =A0 =A0 */
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[1] =3D 0x53610020; =A0 =A0/*=
 ldd 10(%dp),%r1 =A0 =A0 =A0*/
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[2] =3D 0xe820d000; =A0 =A0/*=
 bve (%r1) =A0 =A0 =A0 =A0 =A0 =A0*/
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[3] =3D 0x537b0030; =A0 =A0/*=
 ldd 18(%dp),%dp =A0 =A0 =A0*/
>>
>> - =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] |=3D reassemble_14(get_got(=
me, value, addend) & 0x3fff);
>> + =A0 =A0 =A0 =A0 =A0 =A0 d =3D get_got(me, value, addend);
>> + =A0 =A0 =A0 =A0 =A0 =A0 if (d <=3D 15)
>> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] |=3D reasse=
mble_14(d);
>
> reassemble_14 is wrong for ldd format 3. =A0Need format 5 and im5 ins=
ertion.

This is using reassemble_14 for ldd format 5, which is correct.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

On Fri, 31 Jul 2009, Carlos O'Donell wrote:

> >> + =A0 =A0 =A0 =A0 =A0 =A0 if (d <=3D 15)
> >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] |=3D reas=
semble_14(d);
> >
> > reassemble_14 is wrong for ldd format 3. =A0Need format 5 and im5 i=
nsertion.
>=20
> This is using reassemble_14 for ldd format 5, which is correct.

Huh?  Format 5 has a five bit immediate and it's not compatible with
reassemble_14.  The value is actually being stuffed into a format 3 ldd
pattern (i.e., format 3 is being used for displacements 0 and 8).

If format 3 is going to be used for short displacements, then use
reassemble_16a as it is the inverse to the assemble_16a operation
described in the arch.  Using reassemble_14 with ldd is confusing.

As you pointed out, the arch shows using format 5 for short displacemen=
ts.
It's unclear whether there is a performance or functional difference as=
ide
from the behavior of space selection.  There may be no requirement for
hardware to implement short displacements using format 3.

Dave
--=20
J. David Anglin                                  dave.anglin@nrc-cnrc.g=
c.ca
National Research Council of Canada              (613) 990-0752 (FAX: 9=
52-6602)
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[John David Anglin]

> >  	case ELF_STUB_GOT:
> > -		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
> > +		stub->insns[0] = 0x537b0000;    /* ldd 0(%dp),%dp	*/
> >  		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
> >  		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
> >  		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
> >  
> > -		stub->insns[0] |= reassemble_14(get_got(me, value, addend) & 0x3fff);
> > +		d = get_got(me, value, addend);
> > +		if (d <= 15)
> > +			stub->insns[0] |= reassemble_14(d);
> 
> reassemble_14 is wrong for ldd format 3.  Need format 5 and im5 insertion.

Since I complained about not using format 5 for small displacements,
here's an updated patch for review.  Seems to work:

dave@mx3210:/usr/src/D$ lsmod
Module                  Size  Used by
dm_snapshot            45680  0 
dm_mirror              27480  0 
dm_region_hash         17408  1 dm_mirror
dm_log                 18968  2 dm_mirror,dm_region_hash
dm_mod                111200  3 dm_snapshot,dm_mirror,dm_log
ext2                   99648  2 
sd_mod                 63792  4 
crc_t10dif              2368  1 sd_mod
tg3                   196428  0 
sym53c8xx             127568  3 
libphy                 39280  1 tg3
scsi_transport_spi     43528  1 sym53c8xx
scsi_mod              261104  3 sd_mod,sym53c8xx,scsi_transport_spi

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Signed-off-by: John David Anglin <dave.anglin@nrc-cnrc.gc.ca>

diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c
index ecd1c50..88989cb 100644
--- a/arch/parisc/kernel/module.c
+++ b/arch/parisc/kernel/module.c
@@ -86,8 +86,12 @@
  * the bottom of the table, which has a maximum signed displacement of
  * 0x3fff; however, since we're only going forward, this becomes
  * 0x1fff, and thus, since each GOT entry is 8 bytes long we can have
- * at most 1023 entries */
-#define MAX_GOTS	1023
+ * at most 1023 entries.
+ * To overcome this 14bit displacement with some kernel modules, we'll
+ * use instead the unusal 16bit displacement method (see reassemble_16a)
+ * which gives us a maximum positive displacement of 0x7fff, and as such 
+ * allows us to allocate up to 4095 GOT entries. */
+#define MAX_GOTS	4095
 
 /* three functions to determine where in the module core
  * or init pieces the location is */
@@ -145,12 +149,40 @@ struct stub_entry {
 /* The reassemble_* functions prepare an immediate value for
    insertion into an opcode. pa-risc uses all sorts of weird bitfields
    in the instruction to hold the value.  */
+static inline int sign_unext (int x, int len)
+{
+	int len_ones;
+
+	len_ones = (1 << len) - 1;
+	return x & len_ones;
+}
+
+static inline int low_sign_unext(int x, int len)
+{
+	int sign, temp;
+
+	sign = (x >> (len-1)) & 1;
+	temp = sign_unext (x, len-1);
+	return (temp << 1) | sign;
+}
+
 static inline int reassemble_14(int as14)
 {
 	return (((as14 & 0x1fff) << 1) |
 		((as14 & 0x2000) >> 13));
 }
 
+static inline int reassemble_16a(int as16)
+{
+	int s, t;
+
+	/* Unusual 16-bit encoding, for wide mode only.  */
+	t = (as16 << 1) & 0xffff;
+	s = (as16 & 0x8000);
+	return (t ^ s ^ (s >> 1)) | (s >> 15);
+}
+
+
 static inline int reassemble_17(int as17)
 {
 	return (((as17 & 0x10000) >> 16) |
@@ -409,6 +441,7 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
 	enum elf_stub_type stub_type, Elf_Addr loc0, unsigned int targetsec)
 {
 	struct stub_entry *stub;
+	int d;
 
 	/* initialize stub_offset to point in front of the section */
 	if (!me->arch.section[targetsec].stub_offset) {
@@ -462,12 +495,19 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
  */
 	switch (stub_type) {
 	case ELF_STUB_GOT:
-		stub->insns[0] = 0x537b0000;	/* ldd 0(%dp),%dp	*/
+		d = get_got(me, value, addend);
+		if (d <= 15) {
+			/* Format 5 */
+			stub->insns[0] = 0x0f6010db; /* ldd 0(%dp),%dp	*/
+			stub->insns[0] |= low_sign_unext(d, 5) << 16;
+		} else {
+			/* Format 3 */
+			stub->insns[0] = 0x537b0000; /* ldd 0(%dp),%dp	*/
+			stub->insns[0] |= reassemble_16a(d);
+		}
 		stub->insns[1] = 0x53610020;	/* ldd 10(%dp),%r1	*/
 		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
 		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Sat, Aug 1, 2009 at 3:07 PM, John David
Anglin<dave@hiauly1.hia.nrc.ca> wrote:
> Signed-off-by: John David Anglin <dave.anglin@nrc-cnrc.gc.ca>
>

Frans,

I suggest you use Dave's patch please, it is IMO the most correct patch.

Helge, Kyle, thanks also for the initial patches!

Cheers,
Carlos.

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Frans Pop]

On Saturday 01 August 2009, Carlos O'Donell wrote:
> I suggest you use Dave's patch please, it is IMO the most correct
> patch.

Right. I think the original patch is probably responsible for endless
errors on shutdown/reboot:

Bad Address (null pointer deref?): Code=15 regs=00000000bea7cf70 (Addr=c00007ffbea7c)

     YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00001000000001001111111100001110 Not tainted
r00-03  000000ff0804ff0e 00000000405ead40 000000006fc00000 00000000bea7ca18
r04-07  0000000037de0000 ffffffffffe00008 00000000bea7ca18 00000000405ea540
r08-11  000000000fc212c1 000000006bc23fd9 000000000000000f 00000000faf59048
r12-15  00000000be784b28 0000000000000025 00000000faf5932d 0000000000000000
r16-19  00000000bea7bfa0 0000000000000014 000000004074b134 00000000bea7cda0
r20-23  00000000ffffe000 000000004011d2b4 0000000040479004 0000000000000010
r24-27  0000000000000000 000000004011d1e0 000000004011d838 00000000405dcd40
r28-31  ffffffffbea7cd90 0000000000000350 00000000bea7cf70 ffffffffbea7cda0
sr00-03  000000000607b000 0000000000000000 0000000000000000 000000000607b000
sr04-07  0000000000000000 0000000000000000 0000000000000000 0000000000000000

IASQ: 0000000000000000 0000000000000000 IAOQ: 0000000040127a10 0000000040127a14
 IIR: 0f8010dc    ISR: 000000003ffff800  IOR: c00007ffbea7cd90
 CPU:        1   CR30: 00000000bea60000 CR31: 0000000011111111
 ORIG_R28: 0000000000000000
 IAOQ[0]: unwind_once+0x370/0x3d0
 IAOQ[1]: unwind_once+0x374/0x3d0
 RP(r2): 0x6fc00000


John's version works too for me and the system now shuts down cleanly.

Cheers,
FJP

P.S. If anybody ever wants access to my box, just ask:
- model: 9000/785/J5600
- cpu: 2 x PA8600 (PCX-W+) at 552.000000 MHz
- memory: 2048 MB
- 3 x 9.1 GB SCSI harddisks

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Frans Pop]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

tags 539378 patch
thanks

On Saturday 01 August 2009, Helge Deller wrote:
> Kyle, you beat me.
> Attached is my patch ....
>
> Tested and works.

Works for me too. Cool.

Your patch contained a few whitespace errors and, because of that, one 
unnecessary change. Attached a version with those cleaned up.

Thanks,
FJP


[-- Attachment #2: got.patch --]
[-- Type: text/x-diff, Size: 2217 bytes --]

parisc: module.c - fix GOT table overflow with large kernel modules on 64 bit kernels

Signed-off-by: Helge Deller <deller@gmx.de>

diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c
index ef5caf2..d291bf9 100644
--- a/arch/parisc/kernel/module.c
+++ b/arch/parisc/kernel/module.c
@@ -86,8 +86,12 @@
  * the bottom of the table, which has a maximum signed displacement of
  * 0x3fff; however, since we're only going forward, this becomes
  * 0x1fff, and thus, since each GOT entry is 8 bytes long we can have
- * at most 1023 entries */
-#define MAX_GOTS	1023
+ * at most 1023 entries.
+ * To overcome this 14bit displacement with some kernel modules, we'll
+ * use instead the unusal 16bit displacement method (see reassemble_16a)
+ * which gives us a maximum positive displacement of 0x7fff, and as such
+ * allows us to allocate up to 4095 GOT entries. */
+#define MAX_GOTS	4095
 
 /* three functions to determine where in the module core
  * or init pieces the location is */
@@ -151,6 +155,16 @@ static inline int reassemble_14(int as14)
 		((as14 & 0x2000) >> 13));
 }
 
+static inline int reassemble_16a(int as16)
+{
+	int s, t;
+
+	/* Unusual 16-bit encoding, for wide mode only. */
+	t = (as16 << 1) & 0xffff;
+	s = (as16 & 0x8000);
+	return (t ^ s ^ (s >> 1)) | (s >> 15);
+}
+
 static inline int reassemble_17(int as17)
 {
 	return (((as17 & 0x10000) >> 16) |
@@ -407,6 +421,7 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
 	enum elf_stub_type stub_type, Elf_Addr loc0, unsigned int targetsec)
 {
 	struct stub_entry *stub;
+	int d;
 
 	/* initialize stub_offset to point in front of the section */
 	if (!me->arch.section[targetsec].stub_offset) {
@@ -465,7 +480,12 @@ static Elf_Addr get_stub(struct module *me, unsigned long value, long addend,
 		stub->insns[2] = 0xe820d000;	/* bve (%r1)		*/
 		stub->insns[3] = 0x537b0030;	/* ldd 18(%dp),%dp	*/
 
-		stub->insns[0] |= reassemble_14(get_got(me, value, addend) & 0x3fff);
+		d = get_got(me, value, addend);
+		if (d <= 15)
+			stub->insns[0] |= reassemble_14(d);
+		else
+			stub->insns[0] |= reassemble_16a(d);
+
 		break;
 	case ELF_STUB_MILLI:
 		stub->insns[0] = 0x20200000;	/* ldil 0,%r1		*/

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table

[Carlos O'Donell]

On Fri, Jul 31, 2009 at 7:38 PM, Kyle McMartin<kyle@mcmartin.ca> wrote:
> Is it as simple as:
>
> diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.=
c
> index ef5caf2..0502fab 100644
> --- a/arch/parisc/kernel/module.c
> +++ b/arch/parisc/kernel/module.c
> @@ -82,13 +82,6 @@
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return -ENOEXEC; =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0\
> =A0 =A0 =A0 =A0}
>
> -/* Maximum number of GOT entries. We use a long displacement ldd fro=
m
> - * the bottom of the table, which has a maximum signed displacement =
of
> - * 0x3fff; however, since we're only going forward, this becomes
> - * 0x1fff, and thus, since each GOT entry is 8 bytes long we can hav=
e
> - * at most 1023 entries */
> -#define MAX_GOTS =A0 =A0 =A0 1023
> -
> =A0/* three functions to determine where in the module core
> =A0* or init pieces the location is */
> =A0static inline int in_init(struct module *me, void *loc)
> @@ -126,6 +119,14 @@ struct stub_entry {
> =A0};
> =A0#endif
>
> +/* Maximum number of GOT entries. We use a long displacement ldd fro=
m
> + * the bottom of the table, which has 16-bit signed displacement fro=
m
> + * %dp. Because we only use the forward direction, we're limited to
> + * 15-bits - 1, and because each GOT entry is 8-bytes wide, we're li=
mited
> + * to 4095 entries.
> + */
> +#define MAX_GOTS =A0 =A0 =A0 (((1 << 15) - 1) / sizeof(struct got_en=
try))
> +

OK

> =A0/* Field selection types defined by hppa */
> =A0#define rnd(x) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (((x)+0x1000)&~0x1f=
ff)
> =A0/* fsel: full 32 bits */
> @@ -151,6 +152,15 @@ static inline int reassemble_14(int as14)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0((as14 & 0x2000) >> 13));
> =A0}
>
> +/* Unusual 16-bit encoding, for wide mode only. =A0*/
> +static inline int reassemble_16a(int as16)
> +{
> + =A0 =A0 =A0 int s, t;
> + =A0 =A0 =A0 t =3D (as16 << 1) & 0xffff;
> + =A0 =A0 =A0 s =3D (as16 & 0x8000);
> + =A0 =A0 =A0 return (t ^ s ^ (s >> 1)) | (s >> 15);
> +}
> +

OK

> =A0static inline int reassemble_17(int as17)
> =A0{
> =A0 =A0 =A0 =A0return (((as17 & 0x10000) >> 16) |
> @@ -460,12 +470,16 @@ static Elf_Addr get_stub(struct module *me, uns=
igned long value, long addend,
> =A0*/
> =A0 =A0 =A0 =A0switch (stub_type) {
> =A0 =A0 =A0 =A0case ELF_STUB_GOT:
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 unsigned int d =3D get_got(me, value, a=
ddend) & 0x7fff;
> +
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stub->insns[0] =3D 0x537b0000; =A0 =A0=
/* ldd 0(%dp),%dp =A0 =A0 =A0 */
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stub->insns[1] =3D 0x53610020; =A0 =A0=
/* ldd 10(%dp),%r1 =A0 =A0 =A0*/
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stub->insns[2] =3D 0xe820d000; =A0 =A0=
/* bve (%r1) =A0 =A0 =A0 =A0 =A0 =A0*/
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stub->insns[3] =3D 0x537b0030; =A0 =A0=
/* ldd 18(%dp),%dp =A0 =A0 =A0*/
>
> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] |=3D reassemble_14(get_g=
ot(me, value, addend) & 0x3fff);
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (d > 15)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 stub->insns[0] |=3D rea=
ssemble_16a(d);
> +

You need to rewrite stub->insn[0[, the long format 3 ldd is a
different opcode, see the PA 2.0 manual, it will no longer be
0x537b0000.

You also still need a <=3D 15 byte case which uses the old short format
5 ldd with a 14-bit immediate.

> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break;
> =A0 =A0 =A0 =A0case ELF_STUB_MILLI:
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0stub->insns[0] =3D 0x20200000; =A0 =A0=
/* ldil 0,%r1 =A0 =A0 =A0 =A0 =A0 */
>
> I don't think we need to worry about the initial 15-bytes displacemen=
t,
> since they're all within the first got_entry? (The resulting assembly
> looks alright from a 64-bit toolchain:

No, we still have to worry about the initial 15-bytes. Within the
first 15-bytes you have one GOT entry (%dp + 0) and thus you need to
add the case for the short format 3 ldd.

Thanks for hacking this up!

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#539378: [hppa]: fails to load nfs module: Global Offset Table overflow

[Helge Deller]

On 07/31/2009 08:49 PM, Carlos O'Donell wrote:
> [...]
> However, on 64-bit the long format of ldd has a 16-bit signed
> immediate offset (0xffff), meaning it can reach +0x7fff e.g. 4095 GOT
> slots.
>
> Do you have the time to test something out?
>
> * Make this conditional on 32-bit vs. 64-bit and allow for 4095 GOT
> entries on 64-bit.
> * Fix ELF_GOT_STUB for the 64-bit case. It needs to reassemble a
> 16-bit offset, the current code is IMO incorrect. i.e. it should be "&
> 0x7fff", and use a new reassemble_16 see the PA 2.0 book definition of
> ldd.
> * Build kernel.
> * Test loading NFS moudle.

Carlos, thanks a lot for those explanations (and keep up your work with NPTL :-)).
I'll know what you mean, and if it works it's a good idea.
I'll try to come up with a patch.

A few notes:
- the GOT table is only used for 64bit anyway, so no need to differentiate for 32/64bits
- Another possibility could be to sort the tables, so to reduce the number of needed entries.

Helge