2.4: "access beyond end of device" after ext2 mount

2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

Hello,

mounting an ext2 (ext3 as well) filesystem seems to modify the
block device's EOF behaviour: before the mount the device returned
EOF, after the mount it doesn't anymore:

[on a fresh booted system]
root@darkside:~# uname -a
Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,157394 seconds (58699453 bytes/sec)
root@darkside:~# mke2fs /dev/hdg7
...
Block size=4096 (log=2)
Fragment size=4096 (log=2)
625248 inodes, 1249045 blocks
...
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,439332 seconds (58510184 bytes/sec)
root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
root@darkside:~# umount /dev/hdg7
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 87,145443 seconds (58707468 bytes/sec)
root@darkside:~# bc
1249045 * 4
4996180
1249045 * 4 * 2
9992360

Could somebody please explain this to me? Is this intentional?
I could partly imagine the ext2 mount shrinking the block device's
boundaries to the filesystem boundaries - for security reasons perhaps,
even if I personally think this isn't the best idea at all, since it
would violate layers encapsulation.
However, I have no idea why a) if so, this is not reverted at least
on umount and why b) the EOF behaviour of the block device changes -
before there were an EOF sent to the application (dd) while after there
isn't, but an I/O error instead.

Kernel is Debian's kernel-source-2.4.27 + kernel-patch-2.4-i2c +
kernel-patch-2.4-lm-sensors.

It seems there were already some mails regarding this issue suggesting
this could have shown up between 2.4.24 and 2.4.25, but it seems they
were misunderstood, for example:

From: "ProNIC Solutions" <mot@pronicsolutions.com>
Subject: 2.4.25: attempt to access beyond end of device
Date: Tue, 16 Mar 2004 09:06:31 -0500
Message-ID: <005201c40b5f$e21d34a0$0600a8c0@p17>

From: "Peter S. Mazinger" <ps.m@gmx.net>
Subject: BUG in 2.4.25-rc1: attempt to access beyond end of device
Date: Fri, 6 Feb 2004 13:53:40 +0100 (CET)
Message-ID: <Pine.LNX.4.44.0402061347160.27376-100000@lnx.bridge.intra>


regards
   Mario
-- 
Independence Day: Fortunately, the alien computer operating system works just
fine with the laptop. This proves an important point which Apple enthusiasts
have known for years. While the evil empire of Microsoft may dominate the
computers of Earth people, more advanced life forms clearly prefer Mac's.

Re: 2.4: "access beyond end of device" after ext2 mount

[Marcelo Tosatti]

On Sun, Jan 16, 2005 at 12:35:30AM +0100, Mario Holbe wrote:
> Hello,
> 
> mounting an ext2 (ext3 as well) filesystem seems to modify the
> block device's EOF behaviour: before the mount the device returned
> EOF, after the mount it doesn't anymore:
> 
> [on a fresh booted system]
> root@darkside:~# uname -a
> Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
> root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> 9992366+0 records in
> 9992366+0 records out
> 5116091392 bytes transferred in 87,157394 seconds (58699453 bytes/sec)
> root@darkside:~# mke2fs /dev/hdg7
> ...
> Block size=4096 (log=2)
> Fragment size=4096 (log=2)
> 625248 inodes, 1249045 blocks
> ...
> root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> 9992366+0 records in
> 9992366+0 records out
> 5116091392 bytes transferred in 87,439332 seconds (58510184 bytes/sec)
> root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
> root@darkside:~# umount /dev/hdg7
> root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> attempt to access beyond end of device
> 22:07: rw=0, want=4996184, limit=4996183
> dd: reading `/dev/hdg7': Input/output error
> 9992360+0 records in
> 9992360+0 records out
> 5116088320 bytes transferred in 87,145443 seconds (58707468 bytes/sec)
> root@darkside:~# bc
> 1249045 * 4
> 4996180
> 1249045 * 4 * 2
> 9992360
> 
> Could somebody please explain this to me? Is this intentional?

No

> I could partly imagine the ext2 mount shrinking the block device's
> boundaries to the filesystem boundaries - for security reasons perhaps,
> even if I personally think this isn't the best idea at all, since it
> would violate layers encapsulation.
> However, I have no idea why a) if so, this is not reverted at least
> on umount and why b) the EOF behaviour of the block device changes -
> before there were an EOF sent to the application (dd) while after there
> isn't, but an I/O error instead.

Its indeed strange.

> Kernel is Debian's kernel-source-2.4.27 + kernel-patch-2.4-i2c +
> kernel-patch-2.4-lm-sensors.
> 
> It seems there were already some mails regarding this issue suggesting
> this could have shown up between 2.4.24 and 2.4.25, but it seems they
> were misunderstood, for example:
> 
> From: "ProNIC Solutions" <mot@pronicsolutions.com>
> Subject: 2.4.25: attempt to access beyond end of device
> Date: Tue, 16 Mar 2004 09:06:31 -0500
> Message-ID: <005201c40b5f$e21d34a0$0600a8c0@p17>
> 
> From: "Peter S. Mazinger" <ps.m@gmx.net>
> Subject: BUG in 2.4.25-rc1: attempt to access beyond end of device
> Date: Fri, 6 Feb 2004 13:53:40 +0100 (CET)
> Message-ID: <Pine.LNX.4.44.0402061347160.27376-100000@lnx.bridge.intra>

Actually these were fixed during v2.4.26-pre.

Can you please turn readahead off (hdparm -a 0 /dev/hdg) and repeat the tests?

The kernel is trying to read one block after EOD.

> attempt to access beyond end of device
> 22:07: rw=0, want=4996184, limit=4996183

So maybe an off-by-one read-ahead?

Also I suggest a dump_stack() to see where the read is coming
from.

Any other ideas?

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Mon, Jan 17, 2005 at 05:46:35PM -0200, Marcelo Tosatti wrote:
> On Sun, Jan 16, 2005 at 12:35:30AM +0100, Mario Holbe wrote:
> > Could somebody please explain this to me? Is this intentional?
> No

Thanks :)

> Can you please turn readahead off (hdparm -a 0 /dev/hdg) and repeat the tests?
> The kernel is trying to read one block after EOD.

The same happens.

root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 117,573365 seconds (43514034 bytes/sec)
root@darkside:~# hdparm -a 0 /dev/hdg

/dev/hdg:
 setting fs readahead to 0
 readahead    =  0 (off)
root@darkside:~# mount -t ext3 -o ro /dev/hdg7 /mnt
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
root@darkside:~# umount /dev/hdg7
root@darkside:~# hdparm -a 0 /dev/hdg

/dev/hdg:
 setting fs readahead to 0
 readahead    =  0 (off)
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 100,455251 seconds (50929028 bytes/sec)
root@darkside:~#

> So maybe an off-by-one read-ahead?

Please also note dd's output: 9992366 vs. 9992360 records.

root@darkside:~# cat /proc/partitions
...
  34     7    4996183 hdg7 156620 6088613 19984760 368240 3 0 24 0 0 190720 368240
...
root@darkside:~# bc
4996183 * 2
9992366
root@darkside:~# tune2fs -l /dev/hdg7
...
Block count:              1249045
...
root@darkside:~# bc
4096 * 1249045 / 512
9992360

The block device (the partition) is 4996183 kB or 9992366 sectors,
while the ext3 fs is 9992360 sectors. Exactly these 9992360 sectors
could be read by dd in the second pass.

root@darkside:~# dd if=/dev/hdg7 of=/dev/null bs=512
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 92,603241 seconds (55247400 bytes/sec)
root@darkside:~#

Fixing dd's blocksize to 512 doesn't help either.

> Also I suggest a dump_stack() to see where the read is coming
> from.

I have no idea, how and where to do a dump_stack().


regards
   Mario
-- 
() Ascii Ribbon Campaign
/\ Support plain text e-mail

Re: 2.4: "access beyond end of device" after ext2 mount

[Andries Brouwer]

On Mon, Jan 17, 2005 at 05:46:35PM -0200, Marcelo Tosatti wrote:
> On Sun, Jan 16, 2005 at 12:35:30AM +0100, Mario Holbe wrote:

> > mounting an ext2 (ext3 as well) filesystem seems to modify the
> > block device's EOF behaviour: before the mount the device returned
> > EOF, after the mount it doesn't anymore:
> > 
> > [on a fresh booted system]
> > root@darkside:~# uname -a
> > Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
> > root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> > 9992366+0 records in
> > 9992366+0 records out
> > root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
> > root@darkside:~# umount /dev/hdg7
> > root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> > attempt to access beyond end of device
> > 22:07: rw=0, want=4996184, limit=4996183
> > dd: reading `/dev/hdg7': Input/output error
> > 9992360+0 records in
> > 9992360+0 records out
> > root@darkside:~# bc
> > 1249045 * 4
> > 4996180
> > 1249045 * 4 * 2
> > 9992360
> > 
> > Could somebody please explain this to me? Is this intentional?
> 
> No
> 
> Its indeed strange.

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading
the next 8 sectors and that fails because only 6 sectors are left.

Test that this is what happens using blockdev --getbsz.

If you want to restore the device to full size, use
blockdev --setbsz 512.

Andries

Re: 2.4: "access beyond end of device" after ext2 mount

[Marcelo Tosatti]

Hi Andries,

On Tue, Jan 18, 2005 at 11:55:47AM +0100, Andries Brouwer wrote:
> On Mon, Jan 17, 2005 at 05:46:35PM -0200, Marcelo Tosatti wrote:
> > On Sun, Jan 16, 2005 at 12:35:30AM +0100, Mario Holbe wrote:
> 
> > > mounting an ext2 (ext3 as well) filesystem seems to modify the
> > > block device's EOF behaviour: before the mount the device returned
> > > EOF, after the mount it doesn't anymore:
> > > 
> > > [on a fresh booted system]
> > > root@darkside:~# uname -a
> > > Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
> > > root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> > > 9992366+0 records in
> > > 9992366+0 records out
> > > root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
> > > root@darkside:~# umount /dev/hdg7
> > > root@darkside:~# dd if=/dev/hdg7 of=/dev/null
> > > attempt to access beyond end of device
> > > 22:07: rw=0, want=4996184, limit=4996183
> > > dd: reading `/dev/hdg7': Input/output error
> > > 9992360+0 records in
> > > 9992360+0 records out
> > > root@darkside:~# bc
> > > 1249045 * 4
> > > 4996180
> > > 1249045 * 4 * 2
> > > 9992360
> > > 
> > > Could somebody please explain this to me? Is this intentional?
> > 
> > No
> > 
> > Its indeed strange.
> 
> I suppose that what happens is the following:
> mounting sets the blocksize to 4096.
> After reading 9992360 sectors, reading the next block means reading
> the next 8 sectors and that fails because only 6 sectors are left.

So this is either not a Linux error and not a disk error, its just that the
"use with filesystem" then "direct access" is a unfortunate combination.

What would be the correct fix for this for this, if any? 

v2.6 should suffer from the same issues?

> Test that this is what happens using blockdev --getbsz.
> 
> If you want to restore the device to full size, use
> blockdev --setbsz 512.

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 06:45:26AM -0200, Marcelo Tosatti wrote:
> So this is either not a Linux error and not a disk error, its just that the
> "use with filesystem" then "direct access" is a unfortunate combination.
> What would be the correct fix for this for this, if any? 

Well, I personally think at least the EOF behaviour should be fixed
somehow.
I understand the point that the device's blocksize affects the device's
length... obviously a block device can only consist of full blocks,
not half a block or something like that.
However, if that's right, a block device's length should IMHO also be
affected by it's blocksize - thus, the block device should return an
EOF after the last block was read.

In my case this would mean, it should return an EOF if blocksize is
1024 and 4996183 blocks (9992366 sectors) are read or better, if the
4996184th block is attempted to read and it should return an EOF if
blocksize is 4096 and 1249045 blocks (9992360 sectors) are read or
better, if the 1249046th block is attempted to read.

Another approach could be to allow 'fractions of blocks' :)
Then, the device could have a blocksize of 4096 and consist of
1249045 full blocks and a 6/8 block. However, I'm not sure, if this
is possible at all.

> v2.6 should suffer from the same issues?

I don't know, but if it doesn't, one could just backport 2.6's
solution :)


Mario
-- 
Independence Day: Fortunately, the alien computer operating system works just
fine with the laptop. This proves an important point which Apple enthusiasts
have known for years. While the evil empire of Microsoft may dominate the
computers of Earth people, more advanced life forms clearly prefer Mac's.

Re: 2.4: "access beyond end of device" after ext2 mount

[Andries Brouwer]

On Tue, Jan 18, 2005 at 06:45:26AM -0200, Marcelo Tosatti wrote:

> > I suppose that what happens is the following:
> > mounting sets the blocksize to 4096.
> > After reading 9992360 sectors, reading the next block means reading
> > the next 8 sectors and that fails because only 6 sectors are left.
> 
> So this is either not a Linux error and not a disk error, its just that the
> "use with filesystem" then "direct access" is a unfortunate combination.

It is not a disk error, but I consider it a Linux error.

> What would be the correct fix for this for this, if any?

For 2.4 my reaction would be to say that it is a known property
of the system, possibly less fortunate, an unimportant flaw.
Of course a fix is possible if this is deemed important for some reason.

> v2.6 should suffer from the same issues?

I don't think so. But 2.6 details are rather different.

Andries

Re: 2.4: "access beyond end of device" after ext2 mount

[Marcelo Tosatti]

On Tue, Jan 18, 2005 at 01:37:08PM +0100, Andries Brouwer wrote:
> On Tue, Jan 18, 2005 at 06:45:26AM -0200, Marcelo Tosatti wrote:
> 
> > > I suppose that what happens is the following:
> > > mounting sets the blocksize to 4096.
> > > After reading 9992360 sectors, reading the next block means reading
> > > the next 8 sectors and that fails because only 6 sectors are left.
> > 
> > So this is either not a Linux error and not a disk error, its just that the
> > "use with filesystem" then "direct access" is a unfortunate combination.
> 
> It is not a disk error, but I consider it a Linux error.

OK.

> > What would be the correct fix for this for this, if any?
> 
> For 2.4 my reaction would be to say that it is a known property
> of the system, possibly less fortunate, an unimportant flaw.

This seems to be harmless, so, better do nothing about it.

> Of course a fix is possible if this is deemed important for some reason.
> 
> > v2.6 should suffer from the same issues?
> 
> I don't think so. But 2.6 details are rather different.

OK!

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 11:55:47AM +0100, Andries Brouwer wrote:
> I suppose that what happens is the following:
> mounting sets the blocksize to 4096.
> After reading 9992360 sectors, reading the next block means reading
> the next 8 sectors and that fails because only 6 sectors are left.
> Test that this is what happens using blockdev --getbsz.

Yes! This was the command I was looking for ;)

root@darkside:~# blockdev --getbsz /dev/hdg7
4096
root@darkside:~# blockdev --getbsz /dev/hdg6
1024
root@darkside:~# mount -t ext3 -o ro /dev/hdg6 /mnt
root@darkside:~# umount /dev/hdg6 
root@darkside:~# blockdev --getbsz /dev/hdg6
4096

> If you want to restore the device to full size, use
> blockdev --setbsz 512.

Does that in any way hurt, if a filesystem is just mounted?
I mean, what would happen, if I mount /dev/hdg7 and then
set the block size back to 1024? Would that perhaps corrupt
my filesystem or something like that?


Mario
-- 
<jv> Oh well, config
<jv> one actually wonders what force in the universe is holding it
<jv> and makes it working
<Beeth> chances and accidents :)

Re: 2.4: "access beyond end of device" after ext2 mount

[Andries Brouwer]

On Tue, Jan 18, 2005 at 12:47:01PM +0100, Mario Holbe wrote:

> > If you want to restore the device to full size, use
> > blockdev --setbsz 512.
> 
> Does that in any way hurt, if a filesystem is just mounted?

It is a bad idea to change the blocksize of a mounted filesystem.

RE: 2.4: "access beyond end of device" after ext2 mount

[Piszcz, Justin Michael]

Normally, this problem associated with drives over 32GB or 127GB on a
controller that cannot support it.  It was not discussed here, I was
wondering if that is the problem, if it is not, what type of Hard Drive
is giving you these problems?

Thanks.


-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Marcelo Tosatti
Sent: Tuesday, January 18, 2005 5:20 AM
To: Andries Brouwer
Cc: Mario Holbe; linux-kernel@vger.kernel.org
Subject: Re: 2.4: "access beyond end of device" after ext2 mount

On Tue, Jan 18, 2005 at 01:37:08PM +0100, Andries Brouwer wrote:
> On Tue, Jan 18, 2005 at 06:45:26AM -0200, Marcelo Tosatti wrote:
> 
> > > I suppose that what happens is the following:
> > > mounting sets the blocksize to 4096.
> > > After reading 9992360 sectors, reading the next block means
reading
> > > the next 8 sectors and that fails because only 6 sectors are left.
> > 
> > So this is either not a Linux error and not a disk error, its just
that the
> > "use with filesystem" then "direct access" is a unfortunate
combination.
> 
> It is not a disk error, but I consider it a Linux error.

OK.

> > What would be the correct fix for this for this, if any?
> 
> For 2.4 my reaction would be to say that it is a known property
> of the system, possibly less fortunate, an unimportant flaw.

This seems to be harmless, so, better do nothing about it.

> Of course a fix is possible if this is deemed important for some
reason.
> 
> > v2.6 should suffer from the same issues?
> 
> I don't think so. But 2.6 details are rather different.

OK!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 08:42:08AM -0500, Piszcz, Justin Michael wrote:
> Normally, this problem associated with drives over 32GB or 127GB on a
> controller that cannot support it.  It was not discussed here, I was
> wondering if that is the problem, if it is not, what type of Hard Drive
> is giving you these problems?

This does not depend on the type of the hard disk for sure.

root@darkside:/dev/shm# dd if=/dev/zero of=foo bs=1024 count=10001
10001+0 records in
10001+0 records out
10241024 bytes transferred in 0,062195 seconds (164659895 bytes/sec)
root@darkside:/dev/shm# losetup /dev/loop0 foo
loop: loaded (max 8 devices)
root@darkside:/dev/shm# mke2fs -b 4096 /dev/loop0
...
root@darkside:/dev/shm# blockdev --getbsz /dev/loop0
1024
root@darkside:/dev/shm# dd if=/dev/loop0 of=/dev/null
20002+0 records in
20002+0 records out
10241024 bytes transferred in 0,248255 seconds (41252031 bytes/sec)
root@darkside:/dev/shm# mount -o ro /dev/loop0 /mnt
root@darkside:/dev/shm# umount /dev/loop0
root@darkside:/dev/shm# blockdev --getbsz /dev/loop0
4096
root@darkside:/dev/shm# dd if=/dev/loop0 of=/dev/null
attempt to access beyond end of device
07:00: rw=0, want=10004, limit=10001
dd: reading `/dev/loop0': Input/output error
20000+0 records in
20000+0 records out
10240000 bytes transferred in 0,185949 seconds (55068833 bytes/sec)

Of course you could reproduce it much more simple without all
the ext2 stuff using blockdev --setbsz :)


Mario
-- 
() Ascii Ribbon Campaign
/\ Support plain text e-mail

Re: 2.4: "access beyond end of device" after ext2 mount

[Sytse Wielinga]

Why not just use dd if=/dev/xxx `blockdev --getbsz /dev/xxx` ...?

	Sytse

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 03:17:07PM +0100, Sytse Wielinga wrote:
> Why not just use dd if=/dev/xxx `blockdev --getbsz /dev/xxx` ...?

because it doesn't work, as I've demonstrated in
Message-ID: <20050118082022.GA2839@darkside.22.kls.lan>

> root@darkside:~# dd if=/dev/hdg7 of=/dev/null bs=512
> attempt to access beyond end of device
> 22:07: rw=0, want=4996184, limit=4996183
> dd: reading `/dev/hdg7': Input/output error
> 9992360+0 records in
> 9992360+0 records out
> 5116088320 bytes transferred in 92,603241 seconds (55247400 bytes/sec)
> root@darkside:~#
> 
> Fixing dd's blocksize to 512 doesn't help either.


-- 
*axiom* welcher sensorische input bewirkte die output-aktion,
        den irc-chatter mit dem nick "dus" des irc-servers
        mittels eines kills zu verweisen?

Re: 2.4: "access beyond end of device" after ext2 mount

[Sytse Wielinga]

On Tue, Jan 18, 2005 at 04:20:06PM +0100, Mario Holbe wrote:
> On Tue, Jan 18, 2005 at 03:17:07PM +0100, Sytse Wielinga wrote:
> > Why not just use dd if=/dev/xxx `blockdev --getbsz /dev/xxx` ...?
> 
> because it doesn't work, as I've demonstrated in
> Message-ID: <20050118082022.GA2839@darkside.22.kls.lan>
> 
> > root@darkside:~# dd if=/dev/hdg7 of=/dev/null bs=512
> > attempt to access beyond end of device
> > 22:07: rw=0, want=4996184, limit=4996183
> > dd: reading `/dev/hdg7': Input/output error
> > 9992360+0 records in
> > 9992360+0 records out
> > 5116088320 bytes transferred in 92,603241 seconds (55247400 bytes/sec)
> > root@darkside:~#
> > 
> > Fixing dd's blocksize to 512 doesn't help either.

That's not what I said; the block size of /dev/hdg7 there is 4096, not 512.
Besides, the blocksize of dd is 512 by default and independent of the blocksize
of the device.

Anyhow, the block size set for dd doesn't matter as even if the block size is
huge dd copies over the last partial block. The problem was that with the 2.4
kernel only whole blocks are usable, but it rounds off the device size to
multiples of the sector size (i.e., not; the device size is given in multiples
of the sector size, which, by default, is 512) instead of rounding off to
multiples of the block size. The 2.6 kernel does not have this problem; it
appears to accept partial blocks, and doesn't even appear to calculate the
device size (blockdev --getsz and --getsize return 0 on my machine.)

I think the fix could be as simple as rounding off the device size in one
location, but, as I haven't had a look at the source, I'm not sure, maybe every
driver needs a fix.

	Sytse

Re: 2.4: "access beyond end of device" after ext2 mount

[Sytse Wielinga]

On Tue, Jan 18, 2005 at 04:55:34PM +0100, I wrote:
> multiples of the block size. The 2.6 kernel does not have this problem; it
> appears to accept partial blocks, and doesn't even appear to calculate the
> device size (blockdev --getsz and --getsize return 0 on my machine.)
                                               ^^^^^^^^^^^
Please disregard the second half of that sentence; when I ran blockdev --getsz,
I had deleted my loop device already. The device size still reads 20002 sectors
in 2.6. This doesn't change anything though, since the last two sectors of the
device, or the first two sectors of the last block, are still usable in 2.6 and
not in 2.4.

	Sytse

RE: 2.4: "access beyond end of device" after ext2 mount

[Piszcz, Justin Michael]

Okay but what hard drive model and IDE Chipset/Controller are you using?

Thanks!

-----Original Message-----
From: Mario Holbe [mailto:Mario.Holbe@TU-Ilmenau.DE] 
Sent: Tuesday, January 18, 2005 9:02 AM
To: Piszcz, Justin Michael
Cc: Marcelo Tosatti; Andries Brouwer; linux-kernel@vger.kernel.org
Subject: Re: 2.4: "access beyond end of device" after ext2 mount

On Tue, Jan 18, 2005 at 08:42:08AM -0500, Piszcz, Justin Michael wrote:
> Normally, this problem associated with drives over 32GB or 127GB on a
> controller that cannot support it.  It was not discussed here, I was
> wondering if that is the problem, if it is not, what type of Hard
Drive
> is giving you these problems?

This does not depend on the type of the hard disk for sure.

root@darkside:/dev/shm# dd if=/dev/zero of=foo bs=1024 count=10001
10001+0 records in
10001+0 records out
10241024 bytes transferred in 0,062195 seconds (164659895 bytes/sec)
root@darkside:/dev/shm# losetup /dev/loop0 foo
loop: loaded (max 8 devices)
root@darkside:/dev/shm# mke2fs -b 4096 /dev/loop0
...
root@darkside:/dev/shm# blockdev --getbsz /dev/loop0
1024
root@darkside:/dev/shm# dd if=/dev/loop0 of=/dev/null
20002+0 records in
20002+0 records out
10241024 bytes transferred in 0,248255 seconds (41252031 bytes/sec)
root@darkside:/dev/shm# mount -o ro /dev/loop0 /mnt
root@darkside:/dev/shm# umount /dev/loop0
root@darkside:/dev/shm# blockdev --getbsz /dev/loop0
4096
root@darkside:/dev/shm# dd if=/dev/loop0 of=/dev/null
attempt to access beyond end of device
07:00: rw=0, want=10004, limit=10001
dd: reading `/dev/loop0': Input/output error
20000+0 records in
20000+0 records out
10240000 bytes transferred in 0,185949 seconds (55068833 bytes/sec)

Of course you could reproduce it much more simple without all
the ext2 stuff using blockdev --setbsz :)


Mario
-- 
() Ascii Ribbon Campaign
/\ Support plain text e-mail

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 09:05:05AM -0500, Piszcz, Justin Michael wrote:
> Okay but what hard drive model and IDE Chipset/Controller are you using?

VIA vt82c686b onboard
PDC20269 (Promise U133TX2) on PCI

hda: WDC WD400EB-00CPF0, ATA DISK drive
hdc: IC35L080AVVA07-0, ATA DISK drive
hdd: HL-DT-ST DVDRAM GSA-4082B, ATAPI CD/DVD-ROM drive
hdg: SAMSUNG SP1614N, ATA DISK drive

hda: 78165360 sectors (40021 MB) w/2048KiB Cache, CHS=4865/255/63, UDMA(100)
hdc: 160836480 sectors (82348 MB) w/1863KiB Cache, CHS=159560/16/63, UDMA(100)
hdg: 312581808 sectors (160042 MB) w/8192KiB Cache, CHS=19457/255/63, UDMA(100)

However, it doesn't matter :)


Mario
-- 
<delta> talk softly and carry a keen sword

RE: 2.4: "access beyond end of device" after ext2 mount

[Piszcz, Justin Michael]

Is the problem with the drive on the promise board or the drive on the
VIA chipset?

(from google)
Soyo SY-7VBA133U VIA 694T SY-7VBA133U : ComputerHQ.com3
... Main Specifications. Product Description, SOYO Socket 370
SY-7VBA133U - mainboard -
ATX - Pro133T. ... Audio Output, Sound card - VIA VT82C686B - 16-bit -
stereo. ...

I have the -EXACT- same chipset on an older Soyo Motherboard and have
the same problem you are having, the motherboard did not support drives
over 32GB or it was because I had the 32GB clip (pins on the back of the
hard drive) shorted.  Did you check your HDD manual to see if you have
the 32GB clip enabled?  If so, you need to disable this.

Justin.


-----Original Message-----
From: Mario Holbe [mailto:Mario.Holbe@TU-Ilmenau.DE] 
Sent: Tuesday, January 18, 2005 9:15 AM
To: Piszcz, Justin Michael
Cc: linux-kernel@vger.kernel.org
Subject: Re: 2.4: "access beyond end of device" after ext2 mount

On Tue, Jan 18, 2005 at 09:05:05AM -0500, Piszcz, Justin Michael wrote:
> Okay but what hard drive model and IDE Chipset/Controller are you
using?

VIA vt82c686b onboard
PDC20269 (Promise U133TX2) on PCI

hda: WDC WD400EB-00CPF0, ATA DISK drive
hdc: IC35L080AVVA07-0, ATA DISK drive
hdd: HL-DT-ST DVDRAM GSA-4082B, ATAPI CD/DVD-ROM drive
hdg: SAMSUNG SP1614N, ATA DISK drive

hda: 78165360 sectors (40021 MB) w/2048KiB Cache, CHS=4865/255/63,
UDMA(100)
hdc: 160836480 sectors (82348 MB) w/1863KiB Cache, CHS=159560/16/63,
UDMA(100)
hdg: 312581808 sectors (160042 MB) w/8192KiB Cache, CHS=19457/255/63,
UDMA(100)

However, it doesn't matter :)


Mario
-- 
<delta> talk softly and carry a keen sword

Re: 2.4: "access beyond end of device" after ext2 mount

[Andries Brouwer]

On Tue, Jan 18, 2005 at 09:24:03AM -0500, Piszcz, Justin Michael wrote:

> Is the problem with the drive on the promise board or the drive on the
> VIA chipset?

Oh, please - no FUD. There is no problem, and we understand in detail
what happens and why it happens. There is no need for any speculation.
There is no relation to disk hardware or indeed any hardware.

> Did you check your HDD manual to see if you have
> the 32GB clip enabled?  If so, you need to disable this.

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 09:24:03AM -0500, Piszcz, Justin Michael wrote:
> Is the problem with the drive on the promise board or the drive on the
> VIA chipset?

The problem is with each drive on each controller. The problem is even
with no drive on no controller - as I've shown to you with my loop
example, because it does have exactly nothing to do with drives or
controllers at all but just with block devices and their block size.

> the same problem you are having, the motherboard did not support drives
> over 32GB or it was because I had the 32GB clip (pins on the back of the

I'm quite sure my board supports drives bigger than 32G and also drives
biggern than 128G.

> hard drive) shorted.  Did you check your HDD manual to see if you have
> the 32GB clip enabled?  If so, you need to disable this.

And I'm horribly sure, that I don't have 32GB clipping enabled on
my 40, 80 or 160G drives :)

However - it has nothing to do with drives at all. Just with block
devices and block sizes. It's no physical problem but a logical one
and you can reproduce it on any drive you like just by creating
partitions big enough to force mke2fs to allocate (2048|4096) blocks
(or by creating small ones and force mke2fs manually) and with an
absolute size being a multiple of 1024 but none of (2048|4096).


Mario
-- 
<jv> Oh well, config
<jv> one actually wonders what force in the universe is holding it
<jv> and makes it working
<Beeth> chances and accidents :)

RE: 2.4: "access beyond end of device" after ext2 mount

[Piszcz, Justin Michael]

Not trying to spread FUD, I am just explaining I had the same issue and
that was the resolution.

-----Original Message-----
From: Andries Brouwer [mailto:aebr@win.tue.nl] 
Sent: Tuesday, January 18, 2005 10:04 AM
To: Piszcz, Justin Michael
Cc: Mario Holbe; linux-kernel@vger.kernel.org
Subject: Re: 2.4: "access beyond end of device" after ext2 mount

On Tue, Jan 18, 2005 at 09:24:03AM -0500, Piszcz, Justin Michael wrote:

> Is the problem with the drive on the promise board or the drive on the
> VIA chipset?

Oh, please - no FUD. There is no problem, and we understand in detail
what happens and why it happens. There is no need for any speculation.
There is no relation to disk hardware or indeed any hardware.

> Did you check your HDD manual to see if you have
> the 32GB clip enabled?  If so, you need to disable this.

Re: 2.4: "access beyond end of device" after ext2 mount

[Mario Holbe]

On Tue, Jan 18, 2005 at 10:07:27AM -0500, Piszcz, Justin Michael wrote:
> Not trying to spread FUD, I am just explaining I had the same issue and
> that was the resolution.

Well, *if* the reason of your issue was the same as the reason of
my issue (what could be, but must not be), you were in the lucky
position that after unclipping your drive your partition's size
was a multiple of 1024 *and* (2048|4096) while before it wasn't.
Chances of that are about 1:(1|3) - not that bad at all :)


Mario
-- 
It is practically impossible to teach good programming style to students
that have had prior exposure to BASIC: as potential programmers they are
mentally mutilated beyond hope of regeneration.  -- Dijkstra

Re: 2.4: "access beyond end of device" after ext2 mount

[Bodo Eggert]

Mario Holbe <Mario.Holbe@TU-Ilmenau.DE> wrote:

> I understand the point that the device's blocksize affects the device's
> length... obviously a block device can only consist of full blocks,
> not half a block or something like that.
> However, if that's right, a block device's length should IMHO also be
> affected by it's blocksize - thus, the block device should return an
> EOF after the last block was read.

This would shut up the error message while still hiding the possibly (0.001%)
valuable data in the last block. Even if it's the right thing for almost any
user, the error message should stay and report this error. Instead, the
cause of this message should be documented.