2.4 kernels + >=binutils-2.14.90.0.8

2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Having an odd issue with booting 2.4 kernels built with 
binutils-2.14.90.0.8 and newer on an Indy R5000:

System Maintenance Menu

1) Start System
2) Install System Software
3) Run Diagnostics
4) Recover System
5) Enter Command Monitor

Option? 5
Command Monitor.  Type "exit" to return to the menu.
 >> ls
scsi(0)disk(4)rdisk(0)partition(8)/:
2422x0  2425p6x1  2603x0  arcboot  2425x0  2423x0  2422x1
/dev/sda3/: no such device
 >> boot -f 2425x0

Cannot load scsi(0)disk(4)rdisk(0)partition(8)/2425x0.
Text start 0x8000000, size 0x194400 doesn't fit in a FreeMemory area.
Unable to load 2425x0: ``2425x0'' is not a valid file to boot.
 >>

This issue started appearing with binutils-2.14.90.0.8, and still exists 
in binutils-2.15.90.0.1.1.  If I downgrade to binutils-2.14.90.0.7, the 
issue goes away (This is a cross-compiled kernel, btw).  So this seems 
to be a binutils-specific issue.  I'm not sure what the change was that 
led to this.  Any binutils people have an idea or need more test data run?

For reference, the compiler used was gcc-3.3.3, and has been tried on 
2.4.22, 2.4.23, and 2.4.25 kernels.  I haven't tried it with 2.6.x 
kernels yet.  Other binaries built with these binutils don't seem to 
show any outward signs of problems, just seems to be kernels.


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
[snip]
> >> boot -f 2425x0
> 
> Cannot load scsi(0)disk(4)rdisk(0)partition(8)/2425x0.
> Text start 0x8000000, size 0x194400 doesn't fit in a FreeMemory area.

What's the output of readelf -l for this kernel?


Thiemo

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Thiemo Seufer wrote:

> What's the output of readelf -l for this kernel?

# mips-unknown-linux-gnu-readelf -l vmlinux

Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 3 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 
0x10000
   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab .data.init_task 
.text.init .data.init .setup.init .initcall.init .data.cacheline_aligned 
.reginfo .data .bss
    02


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
> Thiemo Seufer wrote:
> 
> >What's the output of readelf -l for this kernel?
> 
> # mips-unknown-linux-gnu-readelf -l vmlinux
> 
> Elf file type is EXEC (Executable file)
> Entry point 0x88144040
> There are 3 program headers, starting at offset 52
> 
> Program Headers:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
>   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 0x10000

The REGINFO looks weird, pointing in the load segment. Maybe readelf -S
tells more (especially if compared wit an earlier working version).

>   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

PAX_FLAGS?


Thiemo

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Thiemo Seufer wrote:

> The REGINFO looks weird, pointing in the load segment. Maybe readelf -S
> tells more (especially if compared wit an earlier working version).

Hmm, well, The readelf -l and -S output from a 2.14.90.0.7-based 
cross-compiler is attached, along with -l & -S outout from the 
2.15.90.0.1.1 (--version reports 2.15.90.0.1) as well for comparison.

The PAX_FLAGS bit comes from a patch added in gentoo for PaX support in 
binaries.  More info on PaX is at http://pax.grsecurity.net.  I'm going 
to rebuild my kernel cross-compiler without that one patch and see what 
the results are.


--Kumba


-----------------------------------------------


# mips-unknown-linux-gnu-readelf --version
GNU readelf 2.14.90.0.7 20031029
Copyright 2003 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.



# mips-unknown-linux-gnu-readelf -l vmlinux

Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 3 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1563c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x001000 0x88002000 0x88002000 0x13ffc0 0x13ffc0 R E 
0x1000
   LOAD           0x141000 0x88142000 0x88142000 0x2b000 0x52400 RWE 0x1000

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab
    02     .data.init_task .text.init .data.init .setup.init 
.initcall.init .data.cacheline_aligned .reginfo .data .bss



# mips-unknown-linux-gnu-readelf -S vmlinux
There are 22 section headers, starting at offset 0x190884:

Section Headers:
   [Nr] Name              Type            Addr     Off    Size   ES Flg 
Lk Inf Al
   [ 0]                   NULL            00000000 000000 000000 00 
  0   0  0
   [ 1] .text             PROGBITS        88002000 001000 137790 00  AX 
  0   0 32
   [ 2] .fixup            PROGBITS        88139790 138790 00121c 00  AX 
  0   0  1
   [ 3] .kstrtab          PROGBITS        8813a9ac 1399ac 004094 00   A 
  0   0  4
   [ 4] __ex_table        PROGBITS        8813ea40 13da40 0016f8 00   A 
  0   0  4
   [ 5] __dbe_table       PROGBITS        88140138 13f138 000000 00   A 
  0   0  1
   [ 6] __ksymtab         PROGBITS        88140138 13f138 001e88 00   A 
  0   0  4
   [ 7] .data.init_task   PROGBITS        88142000 141000 002000 00  WA 
  0   0  8
   [ 8] .text.init        PROGBITS        88144000 143000 010bb4 00  AX 
  0   0  4
   [ 9] .data.init        PROGBITS        88154bb4 153bb4 000724 00  WA 
  0   0  4
   [10] .setup.init       PROGBITS        881552e0 1542e0 0000b8 00  WA 
  0   0  4
   [11] .initcall.init    PROGBITS        88155398 154398 00008c 00  WA 
  0   0  4
   [12] .data.cacheline_a PROGBITS        88156000 155000 0013c0 00  WA 
  0   0 32
   [13] .reginfo          MIPS_REGINFO    881573c0 1563c0 000018 18   A 
  0   0  4
   [14] .data             PROGBITS        88158000 157000 015000 00  WA 
  0   0 4096
   [15] .bss              NOBITS          8816d000 16c000 027400 00  WA 
  0   0 32
   [16] .comment          PROGBITS        88194400 16c000 0052c6 00 
  0   0  1
   [17] .pdr              PROGBITS        00000000 1712c8 01f4e0 00 
  0   0  4
   [18] .mdebug.abi32     PROGBITS        00000000 1907a8 000000 00 
  0   0  1
   [19] .shstrtab         STRTAB          00000000 1907a8 0000db 00 
  0   0  1
   [20] .symtab           SYMTAB          00000000 190bf4 020510 10 
21 da0  4
   [21] .strtab           STRTAB          00000000 1b1104 021f71 00 
  0   0  1
Key to Flags:
   W (write), A (alloc), X (execute), M (merge), S (strings)
   I (info), L (link order), G (group), x (unknown)
   O (extra OS processing required) o (OS specific), p (processor specific)




---------------------------------------------------------


# mips-unknown-linux-gnu-readelf --version
GNU readelf 2.15.90.0.1 20040303
Copyright 2004 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.



# mips-unknown-linux-gnu-readelf -l vmlinux

Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 3 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 
0x10000
   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab .data.init_task 
.text.init .data.init .setup.init .initcall.init .data.cacheline_aligned 
.reginfo .data .bss
    02



# mips-unknown-linux-gnu-readelf -S vmlinux
There are 23 section headers, starting at offset 0x19188c:

Section Headers:
   [Nr] Name              Type            Addr     Off    Size   ES Flg 
Lk Inf Al
   [ 0]                   NULL            00000000 000000 000000 00 
  0   0  0
   [ 1] .text             PROGBITS        88002000 002000 137790 00  AX 
  0   0 32
   [ 2] .fixup            PROGBITS        88139790 139790 00121c 00  AX 
  0   0  1
   [ 3] .kstrtab          PROGBITS        8813a9ac 13a9ac 004094 00   A 
  0   0  4
   [ 4] __ex_table        PROGBITS        8813ea40 13ea40 0016f8 00   A 
  0   0  4
   [ 5] __dbe_table       PROGBITS        88140138 140138 000000 00   A 
  0   0  1
   [ 6] __ksymtab         PROGBITS        88140138 140138 001e88 00   A 
  0   0  4
   [ 7] .data.init_task   PROGBITS        88142000 142000 002000 00  WA 
  0   0  8
   [ 8] .text.init        PROGBITS        88144000 144000 010bb4 00  AX 
  0   0  4
   [ 9] .data.init        PROGBITS        88154bb4 154bb4 000724 00  WA 
  0   0  4
   [10] .setup.init       PROGBITS        881552e0 1552e0 0000b8 00  WA 
  0   0  4
   [11] .initcall.init    PROGBITS        88155398 155398 00008c 00  WA 
  0   0  4
   [12] .data.cacheline_a PROGBITS        88156000 156000 0013c0 00  WA 
  0   0 32
   [13] .reginfo          MIPS_REGINFO    881573c0 1573c0 000018 18   A 
  0   0  4
   [14] .data             PROGBITS        88158000 158000 015000 00  WA 
  0   0 4096
   [15] .sbss             NOBITS          8816d000 16d000 000000 00 WAp 
  0   0  4
   [16] .bss              NOBITS          8816d000 16d000 027400 00  WA 
  0   0 32
   [17] .comment          PROGBITS        88194400 16d000 0052c6 00 
  0   0  1
   [18] .pdr              PROGBITS        00000000 1722c8 01f4e0 00 
  0   0  4
   [19] .mdebug.abi32     PROGBITS        00000000 1917a8 000000 00 
  0   0  1
   [20] .shstrtab         STRTAB          00000000 1917a8 0000e1 00 
  0   0  1
   [21] .symtab           SYMTAB          00000000 191c24 020520 10 
22 da1  4
   [22] .strtab           STRTAB          00000000 1b2144 021f71 00 
  0   0  1
Key to Flags:
   W (write), A (alloc), X (execute), M (merge), S (strings)
   I (info), L (link order), G (group), x (unknown)
   O (extra OS processing required) o (OS specific), p (processor specific)





-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
[snip]
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   REGINFO        0x1563c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
>   LOAD           0x001000 0x88002000 0x88002000 0x13ffc0 0x13ffc0 R E 0x1000
>   LOAD           0x141000 0x88142000 0x88142000 0x2b000 0x52400 RWE 0x1000
[snip]
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
>   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 0x10000
>   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

It looks like that pax patch makes the difference.

Speculation:
With this large alignment, the load would overwrite the exception
handlers at 0x80000000 if the firmware wouldn't recognize this area
as already in use. Reducing the alignment to 4k may improve things.

Btw, an empty segment with no section assigned looks like a bug to me.

Btw2, converting the non-writable segment into a writeable one
doesn't look like an improvement (but doesn't matter much).


Thiemo

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Thiemo Seufer wrote:

> It looks like that pax patch makes the difference.
> 
> Speculation:
> With this large alignment, the load would overwrite the exception
> handlers at 0x80000000 if the firmware wouldn't recognize this area
> as already in use. Reducing the alignment to 4k may improve things.
> 
> Btw, an empty segment with no section assigned looks like a bug to me.
> 
> Btw2, converting the non-writable segment into a writeable one
> doesn't look like an improvement (but doesn't matter much).

Well, I re-generated my cross-compiler w/o the patch, doesn't seem to 
have affected much.

I'll strip the remaining patches out, and see what the outcome is 
(although the remaining patches have been in gentoo's binutils since 
2.13.* and haven't had issues).


--Kumba

-------------------------------------------------

# mips-unknown-linux-gnu-readelf --version
GNU readelf 2.15.90.0.1.1 20040303
Copyright 2004 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.



# mips-unknown-linux-gnu-readelf -l vmlinux

Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 2 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 
0x10000

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab .data.init_task 
.text.init .data.init .setup.init .initcall.init .data.cacheline_aligned 
.reginfo .data .bss



# mips-unknown-linux-gnu-readelf -S vmlinux
There are 23 section headers, starting at offset 0x19188c:

Section Headers:
   [Nr] Name              Type            Addr     Off    Size   ES Flg 
Lk Inf Al
   [ 0]                   NULL            00000000 000000 000000 00 
  0   0  0
   [ 1] .text             PROGBITS        88002000 002000 137790 00  AX 
  0   0 32
   [ 2] .fixup            PROGBITS        88139790 139790 00121c 00  AX 
  0   0  1
   [ 3] .kstrtab          PROGBITS        8813a9ac 13a9ac 004094 00   A 
  0   0  4
   [ 4] __ex_table        PROGBITS        8813ea40 13ea40 0016f8 00   A 
  0   0  4
   [ 5] __dbe_table       PROGBITS        88140138 140138 000000 00   A 
  0   0  1
   [ 6] __ksymtab         PROGBITS        88140138 140138 001e88 00   A 
  0   0  4
   [ 7] .data.init_task   PROGBITS        88142000 142000 002000 00  WA 
  0   0  8
   [ 8] .text.init        PROGBITS        88144000 144000 010bb4 00  AX 
  0   0  4
   [ 9] .data.init        PROGBITS        88154bb4 154bb4 000724 00  WA 
  0   0  4
   [10] .setup.init       PROGBITS        881552e0 1552e0 0000b8 00  WA 
  0   0  4
   [11] .initcall.init    PROGBITS        88155398 155398 00008c 00  WA 
  0   0  4
   [12] .data.cacheline_a PROGBITS        88156000 156000 0013c0 00  WA 
  0   0 32
   [13] .reginfo          MIPS_REGINFO    881573c0 1573c0 000018 18   A 
  0   0  4
   [14] .data             PROGBITS        88158000 158000 015000 00  WA 
  0   0 4096
   [15] .sbss             NOBITS          8816d000 16d000 000000 00 WAp 
  0   0  4
   [16] .bss              NOBITS          8816d000 16d000 027400 00  WA 
  0   0 32
   [17] .comment          PROGBITS        88194400 16d000 0052c6 00 
  0   0  1
   [18] .pdr              PROGBITS        00000000 1722c8 01f4e0 00 
  0   0  4
   [19] .mdebug.abi32     PROGBITS        00000000 1917a8 000000 00 
  0   0  1
   [20] .shstrtab         STRTAB          00000000 1917a8 0000e1 00 
  0   0  1
   [21] .symtab           SYMTAB          00000000 191c24 020520 10 
22 da1  4
   [22] .strtab           STRTAB          00000000 1b2144 021f71 00 
  0   0  1
Key to Flags:
   W (write), A (alloc), X (execute), M (merge), S (strings)
   I (info), L (link order), G (group), x (unknown)
   O (extra OS processing required) o (OS specific), p (processor specific)


-----------------------------


 >> boot -f 2425x1

Cannot load scsi(0)disk(4)rdisk(0)partition(8)/2425x1.
Text start 0x8000000, size 0x194400 doesn't fit in a FreeMemory area.
Unable to load 2425x1: ``2425x1'' is not a valid file to boot.



-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
> Thiemo Seufer wrote:
> 
> >It looks like that pax patch makes the difference.
> >
> >Speculation:
> >With this large alignment, the load would overwrite the exception
> >handlers at 0x80000000 if the firmware wouldn't recognize this area
> >as already in use. Reducing the alignment to 4k may improve things.
> >
> >Btw, an empty segment with no section assigned looks like a bug to me.
> >
> >Btw2, converting the non-writable segment into a writeable one
> >doesn't look like an improvement (but doesn't matter much).
> 
> Well, I re-generated my cross-compiler w/o the patch, doesn't seem to 
> have affected much.
> 
> I'll strip the remaining patches out, and see what the outcome is 
> (although the remaining patches have been in gentoo's binutils since 
> 2.13.* and haven't had issues).
[snip]
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
>   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 0x10000

Well, then the effects I wrote about were not caused by that patch
but by a broken linker. Re-doing the final link with the old linker
should be enough to prove that.

From the different alignment, this _might_ be related to Maciej's
binutils patch for PAGE_SIZE != 4k.
http://sources.redhat.com/ml/binutils/2003-12/msg00380.html

[snip]
> >> boot -f 2425x1
> 
> Cannot load scsi(0)disk(4)rdisk(0)partition(8)/2425x1.
> Text start 0x8000000, size 0x194400 doesn't fit in a FreeMemory area.

The text start should be at 0x8002000 or higher, else it will fail.


Thiemo

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
> Thiemo Seufer wrote:
> 
> >It looks like that pax patch makes the difference.
> >
> >Speculation:
> >With this large alignment, the load would overwrite the exception
> >handlers at 0x80000000 if the firmware wouldn't recognize this area
> >as already in use. Reducing the alignment to 4k may improve things.
> >
> >Btw, an empty segment with no section assigned looks like a bug to me.
> >
> >Btw2, converting the non-writable segment into a writeable one
> >doesn't look like an improvement (but doesn't matter much).
> 
> Well, I re-generated my cross-compiler w/o the patch, doesn't seem to 
> have affected much.
> 
> I'll strip the remaining patches out, and see what the outcome is 
> (although the remaining patches have been in gentoo's binutils since 
> 2.13.* and haven't had issues).
[snip]
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
>   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 0x10000

Well, then the effects I wrote about were not caused by that patch
but by a broken linker. Re-doing the final link with the old linker
should be enough to prove that.

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Ralf Baechle]

On Mon, Mar 08, 2004 at 08:08:25PM -0500, Kumba wrote:

> Hmm, well, The readelf -l and -S output from a 2.14.90.0.7-based 
> cross-compiler is attached, along with -l & -S outout from the 
> 2.15.90.0.1.1 (--version reports 2.15.90.0.1) as well for comparison.
> 
> The PAX_FLAGS bit comes from a patch added in gentoo for PaX support in 
> binaries.  More info on PaX is at http://pax.grsecurity.net.  I'm going 
> to rebuild my kernel cross-compiler without that one patch and see what 
> the results are.

PAX can't be fully supported on MIPS anyway; the architecture doesn't
have a no-exec flag in it's pages.

PAX docs are bullshit btw.  execution proection doesn't require a split TLB
and anyway, the MIPS uTLBs are split.

  Ralf

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Thiemo Seufer wrote:

> Well, then the effects I wrote about were not caused by that patch
> but by a broken linker. Re-doing the final link with the old linker
> should be enough to prove that.
> 
>>From the different alignment, this _might_ be related to Maciej's
> binutils patch for PAGE_SIZE != 4k.
> http://sources.redhat.com/ml/binutils/2003-12/msg00380.html

This patch looks to be the culprit.  Removing it from 
binutils-2.15.90.0.1.1 source and rebuilding my cross-compiler creates a 
bootable kernel (2.4.25).  I also noticed it changed the output of 
'readelf -l vmlinux' so that there is a second 'LOAD' program header. 
The PaX patch doesn't make a bit of difference, and I've test booted 
kernels built without Maciej's patch, including and excluding the PaX patch.

In the readelf -l <target> snippets below, one was built with Maciej's 
patch, one without, and the one without is the one that booted on my Indy.


With:
Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 3 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1573c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x000000 0x88000000 0x88000000 0x16d000 0x194400 RWE 
0x10000
   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab .data.init_task 
.text.init .data.init .setup.init .initcall.init .data.cacheline_aligned 
.reginfo .data .bss
    02



Without:
Elf file type is EXEC (Executable file)
Entry point 0x88144040
There are 4 program headers, starting at offset 52

Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   REGINFO        0x1563c0 0x881573c0 0x881573c0 0x00018 0x00018 R   0x4
   LOAD           0x001000 0x88002000 0x88002000 0x13ffc0 0x13ffc0 R E 
0x1000
   LOAD           0x141000 0x88142000 0x88142000 0x2b000 0x52400 RWE 0x1000
   PAX_FLAGS      0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

  Section to Segment mapping:
   Segment Sections...
    00     .reginfo
    01     .text .fixup .kstrtab __ex_table __ksymtab
    02     .data.init_task .text.init .data.init .setup.init 
.initcall.init .data.cacheline_aligned .reginfo .data .bss
    03




--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Ralf Baechle wrote:

> PAX can't be fully supported on MIPS anyway; the architecture doesn't
> have a no-exec flag in it's pages.
> 
> PAX docs are bullshit btw.  execution proection doesn't require a split TLB
> and anyway, the MIPS uTLBs are split.
> 
>   Ralf

I'm aware of the inability to fully support PaX on mips.  It does give 
some support, mainly in the Address Space Layout Randomization bit, so 
it's better than nothing, imho.  The binutils patch for this support in 
gentoo isn't targetted at mips anyways, it's applied for all the 
supported architectures.


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Tiago Assumpção]

[-- Attachment #1: Type: text/plain, Size: 3605 bytes --]

Yes, MIPS has no execution control flag in page-level.
And agreed, yet I nor PaX team see a way to make MIPS fully supported by PaX
-- if I'm not wrong, at the moment MIPS boards are only supported by ASLR.

I see that MIPS has split TLB's, which can not be distinguished by software
level in another hand. Thus when a page-fault occours I don't see how a piece
of (non-microcoded) exception handler can get aware whether the I-Fetch is 
being
done in original ``code area'' or as an attempt to execute injected payload in
a memory area supposed to carry only readable/writeable data.
Plus JTLB's holding data and code together in the address translation cache.
Plus situations like kseg0 and kseg1 unmaped translations, which would occour
outside of any TLB (having virtual address subtracted by 0x80000000 and
0xA0000000 respectively to get physiscal locations) making, as you mentioned,
only split uTLB's (not counting kseg2 special case). But PaX wants to take 
care of
kernel level security too.
Even MIPS split cache unities (which can be probed separately by software) 
wouldn't
make the approach possible since if you have a piece of data previously 
cached in
D-Cache (load/store) the cache line would need to suffer an invalidation 
and the
context to be saved in the I-Cache before the I-Fetch pipe stage succeeds.

Indeed, execution protection (in a general way) does not require split TLB.
Other solutions designed and implemented by PaX are SEGMEXEC (using specific
segmentation features of x86 basead core's) and MPROTECT. The last one uses
vm_flags to control every memory mapping's state, ensuring that these never 
hold
VM_WRITE | VM_MAYWRITE together with VM_EXEC | VM_MAYEXEC. But as the
solution becomes more complex it also tends to gain more issues. First of 
all, this
wouldn't be as simple and ``automatic'' as per page control. Another point 
is that this
solution wouldn't prevent kernel level attacks so, among others, any 
compromise in
this level could lead to direct manipulation of a task's mappings flags. At 
the end
a known problem is an attacker who is able to write to the filesystem and 
to request
this file to be mapped in memory as PROT_EXEC. In other words: yes it is 
possible to
achieve execution protection in other ways, but not as precise as page-level.

If anybody has an idea of how to design and implement such solution on MIPS 
computers
I'd really like to hear it.


http://pax.grsecurity.net for further information.



PS.: Kumba, I'd like to have a copy of your kernel image to take a look,
could you please send it to me?



Best regards,


--
                    Tiago Assumpcao

                 module@whatever.org.ar
                7D9A A6BA 8275 964E EF47
                EE5A 7AFF C759 B578 ACAA
      http://whatever.org.ar/~module [/myself.asc]




At 01:09 9/3/2004, you wrote:
>On Mon, Mar 08, 2004 at 08:08:25PM -0500, Kumba wrote:
>
> > Hmm, well, The readelf -l and -S output from a 2.14.90.0.7-based
> > cross-compiler is attached, along with -l & -S outout from the
> > 2.15.90.0.1.1 (--version reports 2.15.90.0.1) as well for comparison.
> >
> > The PAX_FLAGS bit comes from a patch added in gentoo for PaX support in
> > binaries.  More info on PaX is at http://pax.grsecurity.net.  I'm going
> > to rebuild my kernel cross-compiler without that one patch and see what
> > the results are.
>
>PAX can't be fully supported on MIPS anyway; the architecture doesn't
>have a no-exec flag in it's pages.
>
>PAX docs are bullshit btw.  execution proection doesn't require a split TLB
>and anyway, the MIPS uTLBs are split.
>
>   Ralf



[-- Attachment #2: Type: text/html, Size: 4678 bytes --]

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Ralf Baechle]

On Tue, Mar 09, 2004 at 12:12:53PM -0300, Tiago Assump??o wrote:

> Yes, MIPS has no execution control flag in page-level.
> And agreed, yet I nor PaX team see a way to make MIPS fully supported by PaX
> -- if I'm not wrong, at the moment MIPS boards are only supported by ASLR.
> 
> I see that MIPS has split TLB's, which can not be distinguished by software
> level in another hand. Thus when a page-fault occours I don't see how a 
> piece of (non-microcoded) exception handler can get aware whether the
> I-Fetch is being done in original ``code area'' or as an attempt to execute
> injected payload in a memory area supposed to carry only readable/writeable
> data.

In a TLB reload handler you can distinguish betwen instruction and TLB
fault by comparing the fault address in badvaddr with the EPC value.  That
gives you non-exec protection for anything that doesn't already reside in
the TLB.  There is still one spare bit in the pagetables which you can use
as the exec permission bit.  So if I-fetch && !exec_bit -> SIGILL or
something like that.

The usual warning applies - any modification to the TLB code is going to
have extreme performance impact (a nop in the TLB reload handler will cost
about 0.5% of some benchmarks) and any attempts to execute code that is
already mapped will be missed.

Another better-than-nothing idea would be poisoning the I-cache by
pre-loading it.  Exploits probably don't flush the cache first so the
resulting I-cache non-coherence may crash the process instead.  Like
the previous idea this is performance intrusive and also a very dirty
solution.

> Plus situations like kseg0 and kseg1 unmaped translations, which would 
> occour
> outside of any TLB (having virtual address subtracted by 0x80000000 and
> 0xA0000000 respectively to get physiscal locations) making, as you 
> mentioned,
> only split uTLB's (not counting kseg2 special case). But PaX wants to take 
> care of
> kernel level security too.
> Even MIPS split cache unities (which can be probed separately by software) 
> wouldn't
> make the approach possible since if you have a piece of data previously 
> cached in
> D-Cache (load/store) the cache line would need to suffer an invalidation 
> and the
> context to be saved in the I-Cache before the I-Fetch pipe stage succeeds.

Okay, this paragraph was somewhat hard to understand so my comment may be
a bit off ...  All that's required is writing back data to memory or in
cache of Alchemy processors or uncached area not even that.  So chances
that exploit code actually works without having performed a cacheflush
are actually fairly good.

> Indeed, execution protection (in a general way) does not require split TLB.
> Other solutions designed and implemented by PaX are SEGMEXEC (using specific
> segmentation features of x86 basead core's) and MPROTECT. The last one uses
> vm_flags to control every memory mapping's state, ensuring that these never 
> hold
> VM_WRITE | VM_MAYWRITE together with VM_EXEC | VM_MAYEXEC. But as the
> solution becomes more complex it also tends to gain more issues. First of 
> all, this

> wouldn't be as simple and ``automatic'' as per page control. Another point 
> is that this

In the end anything less than per-page control is pretty inflexible.

> solution wouldn't prevent kernel level attacks so, among others, any 
> compromise in this level could lead to direct manipulation of a task's
> mappings flags. At the end a known problem is an attacker who is able to
> write to the filesystem and > to request this file to be mapped in
> memory as PROT_EXEC. In other words: yes it is  possible to achieve
> execution protection in other ways, but not as precise as page-level.

Okay, but that's outside the scope of what no-exec should attempt to do.

> If anybody has an idea of how to design and implement such solution on MIPS 
> computers

You could try to run the mapped kernel in supervisor mode.  Again lots
of performance implications.

  Ralf

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Tue, 9 Mar 2004, Thiemo Seufer wrote:

> >From the different alignment, this _might_ be related to Maciej's
> binutils patch for PAGE_SIZE != 4k.
> http://sources.redhat.com/ml/binutils/2003-12/msg00380.html
> 
> [snip]
> > >> boot -f 2425x1
> > 
> > Cannot load scsi(0)disk(4)rdisk(0)partition(8)/2425x1.
> > Text start 0x8000000, size 0x194400 doesn't fit in a FreeMemory area.
> 
> The text start should be at 0x8002000 or higher, else it will fail.

 It looks like a bug somewhere in binutils, probably BFD.  The segment's
start address should be rounded up to 0x8010000, not down to 0x8000000.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Maciej W. Rozycki wrote:

>  It looks like a bug somewhere in binutils, probably BFD.  The segment's
> start address should be rounded up to 0x8010000, not down to 0x8000000.

Well, I did test removing the patch Thiemo mentioned 
(http://sources.redhat.com/ml/binutils/2003-12/msg00380.html), and 
rebuilding a kernel, and now they boot.  I tested a 2.4.25 on an Indy, 
and 2.6.4 on an O2.  Perhaps a bug in this specific patch?


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Wed, 17 Mar 2004, Kumba wrote:

> >  It looks like a bug somewhere in binutils, probably BFD.  The segment's
> > start address should be rounded up to 0x8010000, not down to 0x8000000.
> 
> Well, I did test removing the patch Thiemo mentioned 
> (http://sources.redhat.com/ml/binutils/2003-12/msg00380.html), and 
> rebuilding a kernel, and now they boot.  I tested a 2.4.25 on an Indy, 
> and 2.6.4 on an O2.  Perhaps a bug in this specific patch?

 The patch just triggers it.  Previously, the segment's start address as
set by Linux in a linker script was already aligned to the page size as it
was defined then.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Maciej W. Rozycki wrote:

>  The patch just triggers it.  Previously, the segment's start address as
> set by Linux in a linker script was already aligned to the page size as it
> was defined then.

Hmm, so would removing the patch function as a temporary workaround 
until the real problem is fixed, or not recommended (meaning unbootable 
kernels till it's fixed)?


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Kumba wrote:
> Maciej W. Rozycki wrote:
> 
> > The patch just triggers it.  Previously, the segment's start address as
> >set by Linux in a linker script was already aligned to the page size as it
> >was defined then.
> 
> Hmm, so would removing the patch function as a temporary workaround 
> until the real problem is fixed, or not recommended (meaning unbootable 
> kernels till it's fixed)?

It works as a workaround, unless the kernel wants to take advantage
from pagesizes other than 4kB.


Thiemo

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Wed, 17 Mar 2004, Kumba wrote:

> Hmm, so would removing the patch function as a temporary workaround 
> until the real problem is fixed, or not recommended (meaning unbootable 
> kernels till it's fixed)?

 A simpler workaround (no need to rebuild binutils) might be setting:

LOADADDR := 0x88010000

for CONFIG_SGI_IP22 in arch/mips/Makefile.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Kumba]

Maciej W. Rozycki wrote:

>  A simpler workaround (no need to rebuild binutils) might be setting:
> 
> LOADADDR := 0x88010000
> 
> for CONFIG_SGI_IP22 in arch/mips/Makefile.

And/or CONFIG_SGI_IP32.  I've seen the issue appear there as well.


--Kumba

-- 
"Such is oft the course of deeds that move the wheels of the world: 
small hands do them because they must, while the eyes of the great are 
elsewhere."  --Elrond

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Wed, 17 Mar 2004, Kumba wrote:

> >  A simpler workaround (no need to rebuild binutils) might be setting:
> > 
> > LOADADDR := 0x88010000
> > 
> > for CONFIG_SGI_IP22 in arch/mips/Makefile.
> 
> And/or CONFIG_SGI_IP32.  I've seen the issue appear there as well.

 Essentially all platforms that currently set the address to something
that's not aligned to a 64kB boundry.  I'd like binutils to be fixed
instead, though -- I'll try to track the problem down and cook a patch
before 2.15.  I think the problem may be considered serious enough the
release may even be deferred for a few days if necessary (since I believe
it's quite close).

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Thu, 18 Mar 2004, Maciej W. Rozycki wrote:

>  Essentially all platforms that currently set the address to something
> that's not aligned to a 64kB boundry.  I'd like binutils to be fixed
> instead, though -- I'll try to track the problem down and cook a patch
> before 2.15.  I think the problem may be considered serious enough the
> release may even be deferred for a few days if necessary (since I believe
> it's quite close).

 After a study of the relevant BFD code, I'm now pretty sure it does its
job right -- the .text section which is placed at a fixed offset by the
linker script only imposes an alignment of 4 and the 64kB alignment is
required by the segment the section is placed in.  So BFD does the right 
job by lowering the segment's VMA so that the .text section is placed at 
the requested offset.

 What's important, segment alignment happens under the assumption a binary 
will be used in a paged environment.  This is not normally the case with a 
MIPS Linux kernel, so I think the right solution is to ask the linker not 
to do page aligning using the "-n" option.  Here's a patch that should do 
that.

 Ralf, OK to apply this?

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

patch-mips-2.4.25-20040322-nmagic-0
diff -up --recursive --new-file linux-mips-2.4.25-20040322.macro/arch/mips/Makefile linux-mips-2.4.25-20040322/arch/mips/Makefile
--- linux-mips-2.4.25-20040322.macro/arch/mips/Makefile	2004-03-11 03:57:07.000000000 +0000
+++ linux-mips-2.4.25-20040322/arch/mips/Makefile	2004-03-23 11:45:26.000000000 +0000
@@ -46,7 +46,7 @@ check_gcc = $(shell if $(CC) $(1) -S -o 
 GCCFLAGS	:= -I $(TOPDIR)/include/asm/gcc
 GCCFLAGS	+= -G 0 -mno-abicalls -fno-pic -pipe
 GCCFLAGS	+= $(call check_gcc, -finline-limit=100000,)
-LINKFLAGS	+= -G 0 -static # -N
+LINKFLAGS	+= -G 0 -static -n
 MODFLAGS	+= -mlong-calls
 
 ifdef CONFIG_DEBUG_INFO
diff -up --recursive --new-file linux-mips-2.4.25-20040322.macro/arch/mips64/Makefile linux-mips-2.4.25-20040322/arch/mips64/Makefile
--- linux-mips-2.4.25-20040322.macro/arch/mips64/Makefile	2004-01-03 03:56:46.000000000 +0000
+++ linux-mips-2.4.25-20040322/arch/mips64/Makefile	2004-03-23 11:45:39.000000000 +0000
@@ -39,7 +39,7 @@ endif
 GCCFLAGS	:= -I $(TOPDIR)/include/asm/gcc
 GCCFLAGS	+= -mabi=64 -G 0 -mno-abicalls -fno-pic -Wa,--trap -pipe
 GCCFLAGS	+= $(call check_gcc, -finline-limit=100000,)
-LINKFLAGS	+= -G 0 -static # -N
+LINKFLAGS	+= -G 0 -static -n
 MODFLAGS	+= -mlong-calls
 
 ifdef CONFIG_DEBUG_INFO

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Ralf Baechle]

On Tue, Mar 23, 2004 at 12:49:02PM +0100, Maciej W. Rozycki wrote:

> >  Essentially all platforms that currently set the address to something
> > that's not aligned to a 64kB boundry.  I'd like binutils to be fixed
> > instead, though -- I'll try to track the problem down and cook a patch
> > before 2.15.  I think the problem may be considered serious enough the
> > release may even be deferred for a few days if necessary (since I believe
> > it's quite close).
> 
>  After a study of the relevant BFD code, I'm now pretty sure it does its
> job right -- the .text section which is placed at a fixed offset by the
> linker script only imposes an alignment of 4 and the 64kB alignment is
> required by the segment the section is placed in.  So BFD does the right 
> job by lowering the segment's VMA so that the .text section is placed at 
> the requested offset.
> 
>  What's important, segment alignment happens under the assumption a binary 
> will be used in a paged environment.  This is not normally the case with a 
> MIPS Linux kernel, so I think the right solution is to ask the linker not 
> to do page aligning using the "-n" option.  Here's a patch that should do 
> that.
> 
>  Ralf, OK to apply this?

Sure, I don't see any possible drawback from this.

  Ralf

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Maciej W. Rozycki]

On Tue, 23 Mar 2004, Ralf Baechle wrote:

> >  What's important, segment alignment happens under the assumption a binary 
> > will be used in a paged environment.  This is not normally the case with a 
> > MIPS Linux kernel, so I think the right solution is to ask the linker not 
> > to do page aligning using the "-n" option.  Here's a patch that should do 
> > that.
> > 
> >  Ralf, OK to apply this?
> 
> Sure, I don't see any possible drawback from this.

 Some picky firmware may be unhappy about a bit different ELF layout it
yields.  Anyway, this is the right way to go and any problems with bad
firmware may be able to be compensated with updates to our linker scripts.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Ralf Baechle]

On Tue, Mar 23, 2004 at 01:50:27PM +0100, Maciej W. Rozycki wrote:

>  Some picky firmware may be unhappy about a bit different ELF layout it
> yields.  Anyway, this is the right way to go and any problems with bad
> firmware may be able to be compensated with updates to our linker scripts.

I've had lots of trouble with ECOFF implementations but not with ELF
which is a nicer design - in particular the obvious way of implementing
ELF loading is even likely to be the right one.

Oh well, we'll see - and I guess a binutils person will object to obove
paragraph the next five minutes ;-)

  Ralf

Re: 2.4 kernels + >=binutils-2.14.90.0.8

[Thiemo Seufer]

Ralf Baechle wrote:
> On Tue, Mar 23, 2004 at 01:50:27PM +0100, Maciej W. Rozycki wrote:
> 
> >  Some picky firmware may be unhappy about a bit different ELF layout it
> > yields.  Anyway, this is the right way to go and any problems with bad
> > firmware may be able to be compensated with updates to our linker scripts.
> 
> I've had lots of trouble with ECOFF implementations but not with ELF
> which is a nicer design - in particular the obvious way of implementing
> ELF loading is even likely to be the right one.

Well, some people chose to analyse ELF sections for their boot loader
as the "obvious" way...

> Oh well, we'll see - and I guess a binutils person will object to obove
> paragraph the next five minutes ;-)

Am I still in time? ;-)


Thiemo