Re: dup2 return value mismatch

[Al Viro <viro@ZenIV.linux.org.uk>]

On Wed, Oct 31, 2012 at 02:18:12AM +0000, Linliangjie wrote:
> 
> Test:
> negative test for dup2 call with maxfd
> 
> 
> It has been noticed that new changes return directly -EMFILE when the newfd is larger than maxfd in 3.7-rc3 instead of EBADF in 3.6-rc6
> file changed : /fs/fcntl.c
> Kernel Commit id of the change : f33ff9927f42045116d738ee47ff7bc59f739bd7
> 
> 
> More details: "https://bugzilla.kernel.org/show_bug.cgi?id=49791"
> 
> 
> IS this an issue or is there a man page update required?

Umm...  After looking at what POSIX actually says...  There is an issue,
all right.  Thanks for spotting.  Fix follows:

Return the right error value when dup2() or dup3() newfd argument is too large

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
---
diff --git a/fs/file.c b/fs/file.c
index ec20de9..ddf57c5 100644
--- a/fs/file.c
+++ b/fs/file.c
@@ -894,7 +894,7 @@ int replace_fd(unsigned fd, struct file *file, unsigned flags)
 		return __close_fd(files, fd);
 
 	if (fd >= rlimit(RLIMIT_NOFILE))
-		return -EMFILE;
+		return -EBADF;
 
 	spin_lock(&files->file_lock);
 	err = expand_files(files, fd);