[RFC] orangefs: non-POSIX I/O semantics

[martin@omnibond.com]

David Reynolds (david@omnibond.com) has brought an issue to my
attention.

Git does not work on OrangeFS.  Instead it fails as follows.

$ git clone git@github.com:waltligon/orangefs.git
Cloning into 'orangefs'...
remote: Counting objects: 116605, done.
remote: Compressing objects: 100% (5/5), done.
fatal: write error: Permission denied
fatal: index-pack failed

Some investigation has resulted in the following test.  On non-OrangeFS
systems, the following works.  On OrangeFS, the fact that the file has
mode 444 (r--r--r--) prevents the write.

#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

int main(void)
{
	ssize_t r;
	int fd;
	fd = open("foo", O_RDWR|O_CREAT|O_EXCL, 0444);
	printf("fd: %d\n", fd);
	if (fd == -1)
		return 1;
	r = write(fd, "hello\n", 6);
	printf("r: %d\n", (int)r);
	if (r == -1)
		perror("write");
	return 0;
}

As far as I can tell, the client-core does not have the right capability
to write to the file.  It is not generated because the file is mode 444.
Is it possible to obtain it (as the kernel does know the file is being
opened O_RDWR and can pass this to the client-core), or will the server
refuse it as long as the file is mode 444?

However, I doubt including some logic to obtain a write capability on a
O_WRONLY/O_RDWR file will fix all of our non-POSIX semantics concerning
I/O.  Nothing short of a major redesign will fix everything.  It may
still be worth fixing what we can.

Thoughts?

Martin