2.4.18 kernel lseek() bug

2.4.18 kernel lseek() bug

[Tomaz Susnik]

[1] 	Problem description 
----------------------------------

 	a call to lseek() fails with EINVAL under the following conditions:
		- it is called on a disk device file
		- required offset is larger than the target disk device size


[2] 	Kernel version
---------------------------
	This problem seems to be limited to version 2.4.18
	I tested also on kernel 2.4.2 but everythng works fine there.

[3] 	Reproducing the problem
-----------------------------------------

	Compile the following program (lseektest.c):

	#define _LARGEFILE64_SOURCE 
	#define _FILE_OFFSET_BITS 64  
	#include <unistd.h>
	#include <sys/types.h>
	#include <sys/stat.h>
	#include <fcntl.h>
	#include <errno.h>

	main(int argc, char *argv[])
	{
	    int fd;
	    int i;
	    off_t ret;

	    printf("Attempting to seek through file %s \n", argv[1]);
	    fd = open(argv[1], O_RDONLY | O_NONBLOCK, 0);
	    printf("Executed file open. errno = %d\n", errno);
    
	    for (i=1; i<10; i++ )
	    {
	        errno = 0;
	
	        ret = lseek (fd, (off_t)1024 *(off_t)1024 *(off_t)1024
*(off_t)i, SEEK_SET);
	        printf("lseek(%i Gb     ): errno = %i ret = %lli\n", i,
errno, ret );

	    };
	    close (fd);
	}
	

	Run it with 1 parameter being an existing disk device file. For
example:

	./lseektest /dev/hda3


	Sample output on a machine with kernel 2.4.2, displaying the CORRECT
behaviour
	(/dev/hda3 is a device file repersenting disk image of size 7517151
kb):

	Attempting to seek through file /dev/hda3 

	Executed file open. errno = 0
	lseek(1 Gb     ): errno = 0 ret = 1073741824
	lseek(2 Gb     ): errno = 0 ret = 2147483648
	lseek(3 Gb     ): errno = 0 ret = 3221225472
	lseek(4 Gb     ): errno = 0 ret = 4294967296
	lseek(5 Gb     ): errno = 0 ret = 5368709120
	lseek(6 Gb     ): errno = 0 ret = 6442450944
	lseek(7 Gb     ): errno = 0 ret = 7516192768
	lseek(8 Gb     ): errno = 0 ret = 8589934592
	lseek(9 Gb     ): errno = 0 ret = 9663676416

	
	Sample output on the same machine, but booted with kernel 2.4.18:

	Attempting to seek through file /dev/hda3 

	Executed file open. errno = 0
	lseek(1 Gb     ): errno = 0 ret = 1073741824
	lseek(2 Gb     ): errno = 0 ret = 2147483648
	lseek(3 Gb     ): errno = 0 ret = 3221225472
	lseek(4 Gb     ): errno = 0 ret = 4294967296
	lseek(5 Gb     ): errno = 0 ret = 5368709120
	lseek(6 Gb     ): errno = 0 ret = 6442450944
	lseek(7 Gb     ): errno = 0 ret = 7516192768
	lseek(8 Gb     ): errno = 22 ret = -1
	lseek(9 Gb     ): errno = 22 ret = -1



[5] 	Special NOTE
---------------------------
	Improper behaviour was noticed on device files only. When "normal"
files 
	are in question, everything seems to be fine.


[6] 	Reason for reporting this problem
---------------------------------------------------
	Our multi-platform backup product relies on proper behaviour of the
lseek() 
	command to calculate a rawdisk size.

	
[7]	Submitter
----------------------
	Tomaz Susnik, Hermes SoftLab, Slovenia
	tomaz.susnik@hermes.si
	

Re: 2.4.18 kernel lseek() bug

[Andreas Dilger]

On Jun 14, 2002  15:07 +0200, Tomaz Susnik wrote:
> [1] 	Problem description 
> ----------------------------------
> 
>  	a call to lseek() fails with EINVAL under the following conditions:
> 		- it is called on a disk device file
> 		- required offset is larger than the target disk device size

Is this behaviour mandated in a standard, or is it just different from
previous behaviour?  I'm not saying it _isn't_ a bug, but I don't see
how seeking past the end of a block device is very useful.

> 	Attempting to seek through file /dev/hda3 
> 
> 	lseek(6 Gb     ): errno = 0 ret = 6442450944
> 	lseek(7 Gb     ): errno = 0 ret = 7516192768
> 	lseek(8 Gb     ): errno = 0 ret = 8589934592
> 	lseek(9 Gb     ): errno = 0 ret = 9663676416
> 
> 	
> 	Sample output on the same machine, but booted with kernel 2.4.18:
> 
> 	Attempting to seek through file /dev/hda3 
> 
> 	lseek(6 Gb     ): errno = 0 ret = 6442450944
> 	lseek(7 Gb     ): errno = 0 ret = 7516192768
> 	lseek(8 Gb     ): errno = 22 ret = -1
> 	lseek(9 Gb     ): errno = 22 ret = -1
> 
> [6] 	Reason for reporting this problem
> ---------------------------------------------------
> 	Our multi-platform backup product relies on proper behaviour of the
> lseek() command to calculate a rawdisk size.

Well, e2fsprogs has a similar test that it uses if the BLKGETSZ ioctl
fails, but I don't see how this new behaviour is a real problem.  All you
have to do is check if _either_ lseek(offset) fails or read() from that
offset fails to know you are past the end of the block device.  It hardly
changes the algorithm at all.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: 2.4.18 kernel lseek() bug

[Andries.Brouwer]

    >      a call to lseek() fails with EINVAL under the following conditions:
    >         - it is called on a disk device file
    >         - required offset is larger than the target disk device size

    Is this behaviour mandated in a standard, or is it just different from
    previous behaviour?  I'm not saying it _isn't_ a bug, but I don't see
    how seeking past the end of a block device is very useful.

I know many programs that use this. They seek and do not expect
an error, because the standard says no error is to be expected,
and then try to read or write.

Let me quote POSIX 1003.1-2001.

...
The lseek() function shall allow the file offset to be set beyond the
end of the existing data in the file.
...

[EINVAL] 
          The whence argument is not a proper value, or the resulting
          file offset would be negative for a regular file, block special
          file, or directory. 


Andries

RE: 2.4.18 kernel lseek() bug

[Tomaz Susnik]

Thank you for your suggestion. I agree that the fix to our code should 
be quite simple. 
On the other hand, it's a multi-paltform piece of code that seems to work 
fine on many UNIX platforms. This problem came up at a rather bad time for 
us, because our product is closing up to a release deadline. At this stage 
I'd have hard time explaining to the Project Manager why I suddenly want 
to change code that worked just fine for years...

If the new lseek() behaviour really is to become accepted (that is if
no patch is to be released for kernel 2.4.18) I will of course do my best 
to force the necessary changes into our code.

BR, Tomaz 



-----Original Message-----
From: Andreas Dilger [mailto:adilger@clusterfs.com]
Sent: Saturday, June 15, 2002 5:37 AM
To: Tomaz Susnik
Cc: linux-kernel@vger.kernel.org
Subject: Re: 2.4.18 kernel lseek() bug


On Jun 14, 2002  15:07 +0200, Tomaz Susnik wrote:
> [1] 	Problem description 
> ----------------------------------
> 
>  	a call to lseek() fails with EINVAL under the following conditions:
> 		- it is called on a disk device file
> 		- required offset is larger than the target disk device size

Is this behaviour mandated in a standard, or is it just different from
previous behaviour?  I'm not saying it _isn't_ a bug, but I don't see
how seeking past the end of a block device is very useful.

> 	Attempting to seek through file /dev/hda3 
> 
> 	lseek(6 Gb     ): errno = 0 ret = 6442450944
> 	lseek(7 Gb     ): errno = 0 ret = 7516192768
> 	lseek(8 Gb     ): errno = 0 ret = 8589934592
> 	lseek(9 Gb     ): errno = 0 ret = 9663676416
> 
> 	
> 	Sample output on the same machine, but booted with kernel 2.4.18:
> 
> 	Attempting to seek through file /dev/hda3 
> 
> 	lseek(6 Gb     ): errno = 0 ret = 6442450944
> 	lseek(7 Gb     ): errno = 0 ret = 7516192768
> 	lseek(8 Gb     ): errno = 22 ret = -1
> 	lseek(9 Gb     ): errno = 22 ret = -1
> 
> [6] 	Reason for reporting this problem
> ---------------------------------------------------
> 	Our multi-platform backup product relies on proper behaviour of the
> lseek() command to calculate a rawdisk size.

Well, e2fsprogs has a similar test that it uses if the BLKGETSZ ioctl
fails, but I don't see how this new behaviour is a real problem.  All you
have to do is check if _either_ lseek(offset) fails or read() from that
offset fails to know you are past the end of the block device.  It hardly
changes the algorithm at all.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/