SiByte 1480 & Branch Likely instructions?

SiByte 1480 & Branch Likely instructions?

[Kaz Kylheku]

Hi All,

Not really a kernel-related question. I've discovered that GCC 4.1.1
(which I'm not using for kernel compiling, but user space) generates
branch likely instructions by default, even though the documentation
says that their use is off by default for MIPS32 and MIPS64, because
they are considered deprecated. They are documented as obsolete for the
Broadcom chips I am working with.

I'm investigating a software anomaly which looks like might be caused by
failure to annul the delay slot of a branch-likely in the fall-through
case. 

In parallel with writing some tests, I thought I would ask whether
anyone happens know whether or not these instructions are known to
actually work correctly on the SB1480 silicon (and perhaps any
additional details, like what revisions, etc)?

Thanks

SiByte 1480 & Branch Likely instructions?

[Kaz Kylheku]

Hi All,

Not really a kernel-related question. I've discovered that GCC 4.1.1
(which I'm not using for kernel compiling, but user space) generates
branch likely instructions by default, even though the documentation
says that their use is off by default for MIPS32 and MIPS64, because
they are considered deprecated. They are documented as obsolete for the
Broadcom chips I am working with.

I'm investigating a software anomaly which looks like might be caused by
failure to annul the delay slot of a branch-likely in the fall-through
case. 

In parallel with writing some tests, I thought I would ask whether
anyone happens know whether or not these instructions are known to
actually work correctly on the SB1480 silicon (and perhaps any
additional details, like what revisions, etc)?

Thanks

RE: SiByte 1480 & Branch Likely instructions?

[Kaz Kylheku]

Kaz wrote:
> Hi All,
> 
> Not really a kernel-related question. I've discovered that GCC 4.1.1
> (which I'm not using for kernel compiling, but user space) generates
> branch likely instructions by default, even though the documentation
> says that their use is off by default for MIPS32 and MIPS64, because

That's because the compiler is not configured correctly. The default CPU
string "from-abi" ends up being used, and so the target ISA is MIPS III.

> In parallel with writing some tests, I thought I would ask whether
> anyone happens know whether or not these instructions are known to
> actually work correctly on the SB1480 silicon (and perhaps any
> additional details, like what revisions, etc)?

A basic sanity test does find bnezl working.

#include <stdio.h>
#include <stdlib.h>

static int branch_likely_works(void)
{
    int one = 1;
    int result;

    __asm__ __volatile__
    ("        .set push\n"
     "        .set noreorder\n"
     "1:      move %0, $0\n"
     "        bnezl %0, 1b\n"
     "        lw %0, %1\n"
     "        .set pop\n"
     : "=r" (result)
     : "m" (one));

     return result == 0;
}

int main(void)
{
    if (branch_likely_works()) {
        puts("branch-likely instruction bnezl correctly annuls delay
slot");
        return 0;
    } 
    puts("branch-likely instruction bnezl fails to annul delay slot");
    return EXIT_FAILURE;
}

RE: SiByte 1480 & Branch Likely instructions?

[Kaz Kylheku]

Kaz wrote:
> Hi All,
> 
> Not really a kernel-related question. I've discovered that GCC 4.1.1
> (which I'm not using for kernel compiling, but user space) generates
> branch likely instructions by default, even though the documentation
> says that their use is off by default for MIPS32 and MIPS64, because

That's because the compiler is not configured correctly. The default CPU
string "from-abi" ends up being used, and so the target ISA is MIPS III.

> In parallel with writing some tests, I thought I would ask whether
> anyone happens know whether or not these instructions are known to
> actually work correctly on the SB1480 silicon (and perhaps any
> additional details, like what revisions, etc)?

A basic sanity test does find bnezl working.

#include <stdio.h>
#include <stdlib.h>

static int branch_likely_works(void)
{
    int one = 1;
    int result;

    __asm__ __volatile__
    ("        .set push\n"
     "        .set noreorder\n"
     "1:      move %0, $0\n"
     "        bnezl %0, 1b\n"
     "        lw %0, %1\n"
     "        .set pop\n"
     : "=r" (result)
     : "m" (one));

     return result == 0;
}

int main(void)
{
    if (branch_likely_works()) {
        puts("branch-likely instruction bnezl correctly annuls delay
slot");
        return 0;
    } 
    puts("branch-likely instruction bnezl fails to annul delay slot");
    return EXIT_FAILURE;
}

Re: SiByte 1480 & Branch Likely instructions?

[Ralf Baechle]

On Fri, Dec 07, 2007 at 03:39:57PM -0800, Kaz Kylheku wrote:

> > Not really a kernel-related question. I've discovered that GCC 4.1.1
> > (which I'm not using for kernel compiling, but user space) generates
> > branch likely instructions by default, even though the documentation
> > says that their use is off by default for MIPS32 and MIPS64, because
> 
> That's because the compiler is not configured correctly. The default CPU
> string "from-abi" ends up being used, and so the target ISA is MIPS III.
> 
> > In parallel with writing some tests, I thought I would ask whether
> > anyone happens know whether or not these instructions are known to
> > actually work correctly on the SB1480 silicon (and perhaps any
> > additional details, like what revisions, etc)?
> 
> A basic sanity test does find bnezl working.
> 
> #include <stdio.h>
> #include <stdlib.h>
> 
> static int branch_likely_works(void)
> {
>     int one = 1;
>     int result;
> 
>     __asm__ __volatile__
>     ("        .set push\n"
>      "        .set noreorder\n"
>      "1:      move %0, $0\n"
>      "        bnezl %0, 1b\n"
>      "        lw %0, %1\n"
>      "        .set pop\n"
>      : "=r" (result)
>      : "m" (one));
> 
>      return result == 0;
> }
> 
> int main(void)
> {
>     if (branch_likely_works()) {
>         puts("branch-likely instruction bnezl correctly annuls delay
> slot");
>         return 0;
>     } 
>     puts("branch-likely instruction bnezl fails to annul delay slot");
>     return EXIT_FAILURE;
> }

That's a very basic test and it'd be very unlikely for the CPU to fail
such a simple test.

But think of scenarios like a load in the delay slot of a branch likely
where the load instruction is on a different page than the branch and a
tlb exception is getting caused.  There are many other special cases
which might be improperly implemented.

But honestly - branch likely instructions were introduced into the MIPS
architecture by the MIPS II R6000 in '89.  And the SB1 core is 2000
vintage so I'd assume by now have figured out how to get it right.  And
branch likely is used in existing binaries.  So I'd be surprised if it
was broken.

  Ralf

GCC bug affecting MIPS (was Re: SiByte 1480 & Branch Likely instructions?)

[Kaz Kylheku]

"Kaz Kylheku" <kaz@zeugmasystems.com> wrote on December 07, 2007:
> Kaz wrote:
>> Hi All,
>>
>> Not really a kernel-related question. I've discovered that GCC 4.1.1
>> (which I'm not using for kernel compiling, but user space) generates
>> branch likely instructions by default, even though the documentation
>> says that their use is off by default for MIPS32 and MIPS64, because
>
> That's because the compiler is not configured correctly. The default CPU
> string "from-abi" ends up being used, and so the target ISA is MIPS III.

I managed to root-cause the original problem, and moments ago filed this bug 
report:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34456

GCC can screw up when doing branch delay slot filling, because in computing 
register liveness, it makes an incorrectly computed assumption about what 
registers are clobbered by a CALL_INSN. By unfortunate coincidence, it's 
possible for an instruction which restores the caller's GP to be wrongly 
moved into a non-annulled delay slot, wreaking havoc on the fall-through 
path where GP is in fact used. Jumps and data accesses are then attempted 
using what is possibly the wrong global offset table.

GCC bug affecting MIPS (was Re: SiByte 1480 & Branch Likely instructions?)

[Kaz Kylheku]

"Kaz Kylheku" <kaz@zeugmasystems.com> wrote on December 07, 2007:
> Kaz wrote:
>> Hi All,
>>
>> Not really a kernel-related question. I've discovered that GCC 4.1.1
>> (which I'm not using for kernel compiling, but user space) generates
>> branch likely instructions by default, even though the documentation
>> says that their use is off by default for MIPS32 and MIPS64, because
>
> That's because the compiler is not configured correctly. The default CPU
> string "from-abi" ends up being used, and so the target ISA is MIPS III.

I managed to root-cause the original problem, and moments ago filed this bug 
report:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34456

GCC can screw up when doing branch delay slot filling, because in computing 
register liveness, it makes an incorrectly computed assumption about what 
registers are clobbered by a CALL_INSN. By unfortunate coincidence, it's 
possible for an instruction which restores the caller's GP to be wrongly 
moved into a non-annulled delay slot, wreaking havoc on the fall-through 
path where GP is in fact used. Jumps and data accesses are then attempted 
using what is possibly the wrong global offset table.

Re: SiByte 1480 & Branch Likely instructions?

[Ralf Baechle]

On Fri, Dec 07, 2007 at 01:54:30PM -0800, Kaz Kylheku wrote:

> Not really a kernel-related question. I've discovered that GCC 4.1.1
> (which I'm not using for kernel compiling, but user space) generates
> branch likely instructions by default, even though the documentation
> says that their use is off by default for MIPS32 and MIPS64, because
> they are considered deprecated. They are documented as obsolete for the
> Broadcom chips I am working with.

Microarchitecture guys love to hate branch likely.  But the deprecation is
a dream.  Binary compatibility will always require those instructions to
continue to exist so the genie is out of the bottle and so I feel very
certain to predict that a future MIPS 3 specification will contain branch
likely.

Afair the SB1 core has a full blown implementation of branch likely -
unlike the R10000 for example where implementors were lazy that is the
branch predictor predicts branch likely instructions as always taken.
So on the R10000 branch likely is only good as loop closure instruction
while on SB1 it should actually do a decent job wherever it can be
scheduled apropriately.

> I'm investigating a software anomaly which looks like might be caused by
> failure to annul the delay slot of a branch-likely in the fall-through
> case. 
> 
> In parallel with writing some tests, I thought I would ask whether
> anyone happens know whether or not these instructions are known to
> actually work correctly on the SB1480 silicon (and perhaps any
> additional details, like what revisions, etc)?

I have no indications of the contrary.

  Ralf

Re: SiByte 1480 & Branch Likely instructions?

[Maciej W. Rozycki]

On Sun, 9 Dec 2007, Ralf Baechle wrote:

> > Not really a kernel-related question. I've discovered that GCC 4.1.1
> > (which I'm not using for kernel compiling, but user space) generates
> > branch likely instructions by default, even though the documentation
> > says that their use is off by default for MIPS32 and MIPS64, because
> > they are considered deprecated. They are documented as obsolete for the
> > Broadcom chips I am working with.
> 
> Microarchitecture guys love to hate branch likely.  But the deprecation is
> a dream.  Binary compatibility will always require those instructions to
> continue to exist so the genie is out of the bottle and so I feel very
> certain to predict that a future MIPS 3 specification will contain branch
> likely.

 We have been there before -- binary compatibility does not preclude 
emulation.  And I do not mean keeping the MIPS I toys (as they might be 
seen these days) running, but serious products deployed commercially, like 
newer VAX implementations that kept full binary compatibility with their 
predecessors in the area of the some of the more arcane instructions only 
by means of emulating them in the OS.

  Maciej

Re: SiByte 1480 & Branch Likely instructions?

[Ralf Baechle]

On Mon, Dec 10, 2007 at 03:28:52PM +0000, Maciej W. Rozycki wrote:

> > > Not really a kernel-related question. I've discovered that GCC 4.1.1
> > > (which I'm not using for kernel compiling, but user space) generates
> > > branch likely instructions by default, even though the documentation
> > > says that their use is off by default for MIPS32 and MIPS64, because
> > > they are considered deprecated. They are documented as obsolete for the
> > > Broadcom chips I am working with.
> > 
> > Microarchitecture guys love to hate branch likely.  But the deprecation is
> > a dream.  Binary compatibility will always require those instructions to
> > continue to exist so the genie is out of the bottle and so I feel very
> > certain to predict that a future MIPS 3 specification will contain branch
> > likely.
> 
>  We have been there before -- binary compatibility does not preclude 
> emulation.  And I do not mean keeping the MIPS I toys (as they might be 
> seen these days) running, but serious products deployed commercially, like 
> newer VAX implementations that kept full binary compatibility with their 
> predecessors in the area of the some of the more arcane instructions only 
> by means of emulating them in the OS.

It would devastate the performance of some binaries.

As an intellectual challenge, how far can you strip down a MIPS
implementation and emulate removed instructions in the kernel ;-)

  Ralf

Re: SiByte 1480 & Branch Likely instructions?

[Maciej W. Rozycki]

On Mon, 10 Dec 2007, Ralf Baechle wrote:

> It would devastate the performance of some binaries.

 I think this is what the deprecation is about. ;-)

> As an intellectual challenge, how far can you strip down a MIPS
> implementation and emulate removed instructions in the kernel ;-)

 Well, going back to MIPS I is certainly achievable (OK, we could keep 
ll/sc for the sake of sanity) and then perhaps a little bit further.  
After all, all of the ALU ops can be done with the NOR op only. ;-)

  Maciej