sendfile64() on x86_64 breaks at 2gb (MAX_NON_LFS limit)

sendfile64() on x86_64 breaks at 2gb (MAX_NON_LFS limit)

[Mr. Berkley Shands]

for the opteron, the value of MAX_NON_LFS (include/linux/fs.h) is fixed 
at (1UL<<31 -1).
Since ALL 64 bit boxes force O_LARGEFILE on, shouldn't this value be 
(1UL<<63 -1) so that
sendfile64() will proceed beyond the 2gb limit?
under 2.6.6, sendfile64() has no __NR_sendfile64 entry in asm*/unistd.h 
(same for 2.6.9-rc2)
so the syscall sendfile64() maps to sendfile(), which has MAX_NON_LFS 
hard coded in fs/read_write.c
as its limit, rather than 0ULL as in sendfile64().

So the fix is to make the correct entry for sendfile64 in unistd.h (note 
that this hoses /usr/include/.../syscalls.h :-)
and update fs.h as folows:


--- fs.h.old    2004-09-23 11:44:33.469481466 -0500
+++ fs.h        2004-09-23 11:29:56.823712018 -0500
@@ -589,7 +589,11 @@
 /* Release a private file and free its security structure. */
 extern void close_private_file(struct file *file);
 
+#if BITS_PER_LONG==32
 #define        MAX_NON_LFS     ((1UL<<31) - 1)
+#else
+#define        MAX_NON_LFS     ((1UL<<63) - 1)
+#endif
 
 /* Page cache limit. The filesystems should put that into their s_maxbytes
    limits, otherwise bad things can happen in VM. */


Please consider making these updates in the next rev of the kernel sources.

Berkley Shands

Re: sendfile64() on x86_64 breaks at 2gb (MAX_NON_LFS limit)

[Jakub Jelinek]

On Thu, Sep 23, 2004 at 12:09:29PM -0500, Mr. Berkley Shands wrote:
> for the opteron, the value of MAX_NON_LFS (include/linux/fs.h) is fixed 
> at (1UL<<31 -1).
> Since ALL 64 bit boxes force O_LARGEFILE on, shouldn't this value be 
> (1UL<<63 -1) so that
> sendfile64() will proceed beyond the 2gb limit?
> under 2.6.6, sendfile64() has no __NR_sendfile64 entry in asm*/unistd.h 
> (same for 2.6.9-rc2)
> so the syscall sendfile64() maps to sendfile(), which has MAX_NON_LFS 
> hard coded in fs/read_write.c
> as its limit, rather than 0ULL as in sendfile64().
> 
> So the fix is to make the correct entry for sendfile64 in unistd.h (note 
> that this hoses /usr/include/.../syscalls.h :-)
> and update fs.h as folows:

No.  You want the MAX_NON_LFS limit for 32-bit programs that call sendfile()
(as opposed to sendfile64()) on 64-bit arches.

The correct fix which is in the kernel for a few months is:
http://linux.bkbits.net:8080/linux-2.5/diffs/include/asm-x86_64/unistd.h@1.25

	Jakub