Linux doesn't follow x86/x86-64 ABI wrt direction flag

Aurelien Jarno
2008-03-05T15:30:35Z

Hi all,

Since version 4.3, gcc changed its behaviour concerning the x86/x86-64
ABI and the direction flag, that is it now assumes that the direction
flag is cleared at the entry of a function and it doesn't clear once
more if needed.

This causes some problems with the Linux kernel which does not clear
the direction flag when entering a signal handler. The small code below
(for x86-64) demonstrates that.

If the signal handler is using code that need the direction flag cleared
(for example bzero() or memset()), the code is incorrectly executed.

I guess this has to be fixed on the kernel side, but also gcc-4.3 could
revert back to the old behaviour, that is clearing the direction flag
when entering a routine that touches it until most people are running a
fixed kernel.

Kind regards,
Aurelien

[1] http://gcc.gnu.org/gcc-4.3/changes.html

#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <signal.h>

void handler(int signal) {
uint64_t rflags;

asm volatile("pushfq ; popq %0" : "=g" (rflags));

if (rflags & (1 << 10))
printf("DF = 1\n");
else
printf("DF = 0\n");
}

int main() {
signal(SIGUSR1, handler);

while(1)
{
asm volatile("std\r\n");
}

return 0;
}

--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net