Disk I/O degraded performance

Ramon Chimeno
2007-09-19T20:09:22Z

Hi all

I migrated one of my server from kernel 2.6.18 to the latest 2.6.22
and I experienced lower disk performance for processes that open file
with the O_DIRECT flag.

I did a very simple test program that opens two files with O_DIRECT
flag and reads the files to end. I monitored the time spent to read
the files and I have ~ 40% of difference between 2.6.18 and 2.6.22.

For information, the files are stored on a XFS partition which is part
of a software raid-5 block device (the raid-5 is made with 3 SATA
drives).

Does that make sense to you gurus ? Of course, I didn't change the
configuration (task scheduler, HZ, default I/O scheduler, ...) between
the two kernels.

For information, the code of my test program is pasted below (I
snipped the #include part):
----------------------8<----------------------8<----------------------
#define BSIZE (512*512)
int main(int ac, char * av[])
{
int fd[2];
char* buffer;

if (ac != 3) {
printf("USAGE: %s <FILE-1> <FILE-2>\n", av[0]);
exit(EXIT_FAILURE);
}

if ((fd[0] = open(av[1], O_RDONLY|O_DIRECT)) == -1)
exit(EXIT_FAILURE);
if ((fd[1] = open(av[2], O_RDONLY|O_DIRECT)) == -1)
exit(EXIT_FAILURE);
if ((buffer = (char *)memalign(4096, BSIZE)) == 0)
exit(EXIT_FAILURE);

for (;;) {
if (read(fd[0], buffer, BSIZE) != BSIZE)
break;
if (read(fd[1], buffer, BSIZE) != BSIZE)
break;
}
close(fd[0]);
close(fd[1]);
free(buffer);
return EXIT_SUCCESS;
}
----------------------8<----------------------8<----------------------

Any help/hint would be appreciate.

-- R.C