Re: [PATCH] dup2: Fix return value with oldfd == newfd and invalid fd

[Al Viro <viro@ZenIV.linux.org.uk>]

On Mon, May 11, 2009 at 12:26:59PM -0700, Linus Torvalds wrote:

> Hmm. They've been "unsigned int" for as long as our history goes back 
> (including BK), but yes, making them "int" would have hidden this issue as 
> well. 
> 
> That said, I think we had reasons to do our fd's as unsigned, ie the whole 
> "compare against MAX" thing that doesn't take negative values into 
> account.
> 
> In fact, I think we should do more of those. Right now we literally depend 
> on things like "max_fds" being "unsigned int", and that the compiler then 
> turns all the
> 
> 	if (fd < fdt->max_fds)
> 
> tests silently into unsigned tests even when 'fd' is 'int'.
> 
> So I suspect we should probably make fs/file.c use _more_ "unsigned int" 
> rather than having less of them.

What we should do is a careful review of the propagation paths of file
descriptors ;-/  As it is, we have an interesting mix of int/unsigned/long
used to carry those around, and quite a few of those are used for -E...
as well.  Note, BTW, that for userland code this bug mostly isn't - libc will
convert that value to int before returning to caller, so sign expansion or
not, we won't notice.  The things like strace will, though...