ramfs problem... (unlink of sparse file in "D" state)

ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

Hi !

I create a sparse file on ramfs (by writing 5 bytes at offset 3GB):

0.000269 open("./lfs.file", O_RDWR|O_CREAT|O_TRUNC|O_LARGEFILE, 0600) = 3
0.000214 _llseek(3, 18446744072635809792, [3221225472], SEEK_SET) = 0           
0.000159 write(3, "hallo", 5)      = 5                                          
0.000176 close(3)                  = 0                                          
0.000166 _exit(0)                  = ?                                          

Works fine.

Then I tried to unlink the file by running rm lfs.file log.

The rm process (and an ls process that I started after that)
are now in "D" state...

root      2934  0.0  0.2  1292  452 pts/5    D    05:38   0:00 ls /ramfs
root      2952  0.0  1.5  4028 2384 pts/3    S    05:40   0:00 vi sdlkhfd


(Note that the ls hangs because of "rm". before running "rm" ls worked
 fine. The same test (including rm) works fine on reiserfs 3.6.24 on
 the same 2.4.0 release kernel. But reiserfs needed 34 seconds on close
 and the system was not very responsible (including mouse updates within X)
 for that time....)


-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Sat, 6 Jan 2001, Stefan Traby wrote:

> Then I tried to unlink the file by running rm lfs.file log.
> 
> The rm process (and an ls process that I started after that)
> are now in "D" state...
> 
> root      2934  0.0  0.2  1292  452 pts/5    D    05:38   0:00 ls /ramfs
> root      2952  0.0  1.5  4028 2384 pts/3    S    05:40   0:00 vi sdlkhfd

Add UnlockPage(page) at the end of ramfs_writepage().

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Fri, Jan 05, 2001 at 11:52:31PM -0500, Alexander Viro wrote:
> On Sat, 6 Jan 2001, Stefan Traby wrote:
> 
> > Then I tried to unlink the file by running rm lfs.file log.
> > 
> > The rm process (and an ls process that I started after that)
> > are now in "D" state...
> > 
> > root      2934  0.0  0.2  1292  452 pts/5    D    05:38   0:00 ls /ramfs
> > root      2952  0.0  1.5  4028 2384 pts/3    S    05:40   0:00 vi sdlkhfd
> 
> Add UnlockPage(page) at the end of ramfs_writepage().

Shit. You are quite fast. Works.

It was the first D-state case here where sync(1) did not fall
into D-state, too. (ok, I know why :)

-- 

    Stefan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Sat, 6 Jan 2001, Stefan Traby wrote:

> On Fri, Jan 05, 2001 at 11:52:31PM -0500, Alexander Viro wrote:
> > On Sat, 6 Jan 2001, Stefan Traby wrote:
> > 
> > > Then I tried to unlink the file by running rm lfs.file log.
> > > 
> > > The rm process (and an ls process that I started after that)
> > > are now in "D" state...
> > > 
> > > root      2934  0.0  0.2  1292  452 pts/5    D    05:38   0:00 ls /ramfs
> > > root      2952  0.0  1.5  4028 2384 pts/3    S    05:40   0:00 vi sdlkhfd
> > 
> > Add UnlockPage(page) at the end of ramfs_writepage().
> 
> Shit. You are quite fast. Works.

	Sure, especially considering the fact that patch was sent to
Linus about a month ago (several times, actually)... ;-/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

modprobe ipv6 gives -1 usage count was [ramfs problem...]

[Stefan Traby]

On Sat, Jan 06, 2001 at 12:18:56AM -0500, Alexander Viro wrote:
> 
> 
> On Sat, 6 Jan 2001, Stefan Traby wrote:
> 
> > On Fri, Jan 05, 2001 at 11:52:31PM -0500, Alexander Viro wrote:
> > > On Sat, 6 Jan 2001, Stefan Traby wrote:
> > > 
> > > > Then I tried to unlink the file by running rm lfs.file log.
> > > > 
> > > > The rm process (and an ls process that I started after that)
> > > > are now in "D" state...
> > > > 
> > > > root      2934  0.0  0.2  1292  452 pts/5    D    05:38   0:00 ls /ramfs
> > > > root      2952  0.0  1.5  4028 2384 pts/3    S    05:40   0:00 vi sdlkhfd
> > > 
> > > Add UnlockPage(page) at the end of ramfs_writepage().
> > 
> > Shit. You are quite fast. Works.
> 
> 	Sure, especially considering the fact that patch was sent to
> Linus about a month ago (several times, actually)... ;-/

I bet that a fix for the following exists, too: :)

[0]--(06:19:49)-(root@stefan)-(~)-> lsmod |grep -i ipv6
[1]--(06:22:33)-(root@stefan)-(~)-> modprobe ipv6
[0]--(06:22:38)-(root@stefan)-(~)-> lsmod |grep -i ipv6
ipv6                  117424  -1 
[0]--(06:22:46)-(root@stefan)-(~)->

usage count: -1

-- 

    Stefan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: modprobe ipv6 gives -1 usage count was [ramfs problem...]

[Russell King]

Stefan Traby writes:
> I bet that a fix for the following exists, too: :)
> 
> [0]--(06:19:49)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> [1]--(06:22:33)-(root@stefan)-(~)-> modprobe ipv6
> [0]--(06:22:38)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> ipv6                  117424  -1 
> [0]--(06:22:46)-(root@stefan)-(~)->
> 
> usage count: -1

Why?  It is not a bug.  It means that the module itself determines when/if
it can be unloaded.  (an attempt to unload ipv6 calls ipv6_unload() which
determines if the module can be removed)
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |         Russell King        rmk@arm.linux.org.uk      --- ---
  | | | | http://www.arm.linux.org.uk/personal/aboutme.html   /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: modprobe ipv6 gives -1 usage count was [ramfs problem...]

[Matti Aarnio]

  Yes, it does same with 2.2.  Nothing new here.

  It just tells to the system that that module CAN NOT BE UNLOADED.

On Sat, Jan 06, 2001 at 06:24:48AM +0100, Stefan Traby wrote:
> > 	Sure, especially considering the fact that patch was sent to
> > Linus about a month ago (several times, actually)... ;-/
> 
> I bet that a fix for the following exists, too: :)
> 
> [0]--(06:19:49)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> [1]--(06:22:33)-(root@stefan)-(~)-> modprobe ipv6
> [0]--(06:22:38)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> ipv6                  117424  -1 
> [0]--(06:22:46)-(root@stefan)-(~)->
> 
> usage count: -1
> -- 
>     Stefan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> > > Add UnlockPage(page) at the end of ramfs_writepage().
> > Shit. You are quite fast. Works.
> 
> 	Sure, especially considering the fact that patch was sent to
> Linus about a month ago (several times, actually)... ;-/

Its in all the -ac trees 8)

BTW Al: We have another general vfs/fs problem to handle - which is exceeding
max file sizes on limited file systems. Pretty much nobody is getting it
right. Ext2 can be tricked to go past the limit, sys5 1k sits there emitting
printk messages etc.

Any objections to me putting max file size for an fs (in pages) into the
superblock ? An fs can still implement weird rules by putting large values
in that and doing its own checks.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: modprobe ipv6 gives -1 usage count was [ramfs problem...]

[Alan Cox]

> I bet that a fix for the following exists, too: :)
> 
> [0]--(06:19:49)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> [1]--(06:22:33)-(root@stefan)-(~)-> modprobe ipv6
> [0]--(06:22:38)-(root@stefan)-(~)-> lsmod |grep -i ipv6
> ipv6                  117424  -1 
> [0]--(06:22:46)-(root@stefan)-(~)->
> 
> usage count: -1

Not a bug.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Wedgwood]

On Sat, Jan 06, 2001 at 03:35:32PM +0000, Alan Cox wrote:

    BTW Al: We have another general vfs/fs problem to handle - which
    is exceeding max file sizes on limited file systems. Pretty much
    nobody is getting it right. Ext2 can be tricked to go past the
    limit, sys5 1k sits there emitting printk messages etc.

Which filesystems have limits other than 2^31 bytes?

I ask this because I was looking at LFS compliance and the way we
currently do things now isn't very smart. Only ext2 checks of
O_LARGEFILE at present (well, their is perhaps good reason for this
as it is one of the fre that supports multiGB files) whereas I think
this check should be done in the VFS.




  --cw

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> Which filesystems have limits other than 2^31 bytes?

Ext2 handles large files almost properly. (properly on 2.2 + patches)
NFSv3 handles large files but might be missing the O_LARGEFILE check.
I believe reiserfs went to at least 4Gig.

> O_LARGEFILE at present (well, their is perhaps good reason for this
> as it is one of the fre that supports multiGB files) whereas I think
> this check should be done in the VFS.

Possibly

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Wedgwood]

On Sat, Jan 06, 2001 at 03:58:20PM +0000, Alan Cox wrote:

    Ext2 handles large files almost properly. (properly on 2.2 +
    patches) NFSv3 handles large files but might be missing the
    O_LARGEFILE check.  I believe reiserfs went to at least 4Gig.

reiserfs 3.6.x under 2.4.x should go much higher unless i am reading
something wrong

<pause>

yup, it does.


as for NFS, I'm not sure how to pass O_LARGEFILE via the protocol and
since NFS isn't really POSIX like anyhow decided we might as well
just ingore it and have all sys_open calls for NFS look like
O_LARGEFILE was specified



move this check and also write/seek check into the VFS would make
sense; right now you can 


  f = open("blah",O_RDWR & ~O_LARGEFILE);
  lseek(f,1024*1024*1024,SEEK_CUR);
  lseek(f,1024*1024*1024,SEEK_CUR);
  lseek(f,1024*1024*1024,SEEK_CUR);
  write(f,"alan's beard",12);
  close(f);

and create a file you cannot open again with the same application




  --cw
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Eric W. Biederman]

Chris Wedgwood <cw@f00f.org> writes:

> On Sat, Jan 06, 2001 at 03:58:20PM +0000, Alan Cox wrote:
> 
>     Ext2 handles large files almost properly. (properly on 2.2 +
>     patches) NFSv3 handles large files but might be missing the
>     O_LARGEFILE check.  I believe reiserfs went to at least 4Gig.
> 
> reiserfs 3.6.x under 2.4.x should go much higher unless i am reading
> something wrong
> 
> <pause>
> 
> yup, it does.
> 
> 
> as for NFS, I'm not sure how to pass O_LARGEFILE via the protocol and
> since NFS isn't really POSIX like anyhow decided we might as well
> just ingore it and have all sys_open calls for NFS look like
> O_LARGEFILE was specified

Umm.  No.  The object of LFS stuff is so that programs that can't
handle large files don't shoot themselves in the foot.  You don't
need to pass O_LARGEFILE over the protocol and knfsd doesn't need
to handle it.  But with out specifying O_LARGEFILE you should
be limited to 2GB on 32bit systems.

Moving some of the LFS checks into the VFS does sound good.  

When I looked at one of the BSD's a while ago, they had
a max file size in (the superblock?) and the VFS did basic
max file size checking.  And I think it handled all of the LFS
API at the VFS layer as well.  Alan these are two seperate
but related issues.

Putting the LFS checks, & max filesize checks into the VFS sounds
right for 2.4.x because it fixes lots of filesystems, with just a
couple of lines of code. 

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Wedgwood]

On Sun, Jan 07, 2001 at 12:05:07AM -0700, Eric W. Biederman wrote:

    Umm.  No.  The object of LFS stuff is so that programs that can't
    handle large files don't shoot themselves in the foot.  You don't
    need to pass O_LARGEFILE over the protocol and knfsd doesn't need
    to handle it.  But with out specifying O_LARGEFILE you should be
    limited to 2GB on 32bit systems.

THis means we limit all NFS file sizes to 32-bits unless we have
NFSv3? (I assume v3 is where the 64-bit file sizes comes from? or
does it predate that?)


  --cw
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> Putting the LFS checks, & max filesize checks into the VFS sounds
> right for 2.4.x because it fixes lots of filesystems, with just a
> couple of lines of code. 

Rather more than that, and it only fixes those using generic_file_*

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> THis means we limit all NFS file sizes to 32-bits unless we have
> NFSv3? (I assume v3 is where the 64-bit file sizes comes from? or
> does it predate that?)

NFSv2 has a 32bit file offset. Thus the largest file on NFSv2 you want to 
touch is 2Gig.

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Wedgwood]

On Sun, Jan 07, 2001 at 01:57:17PM +0000, Alan Cox wrote:

    Rather more than that, and it only fixes those using
    generic_file_*

So we have common code that both generic_file_* calls and block_*
calls to check against -- how does that sound?

.. or we can check 'up one level' by adding another method to struct
file_operations perhaps (gross?).


  --cw

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> .. or we can check 'up one level' by adding another method to struct
> file_operations perhaps (gross?).

Not feasible. SOme work has to be done by the fs in certain cases. I can cover
the majority of them tho
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > Putting the LFS checks, & max filesize checks into the VFS sounds
> > right for 2.4.x because it fixes lots of filesystems, with just a
> > couple of lines of code. 
> 
> Rather more than that, and it only fixes those using generic_file_*

True.  But it is noticeable fewer lines of code than doing it all
once for each fs.

Eric


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Sat, 6 Jan 2001, Alan Cox wrote:

> > > > Add UnlockPage(page) at the end of ramfs_writepage().
> > > Shit. You are quite fast. Works.
> > 
> > 	Sure, especially considering the fact that patch was sent to
> > Linus about a month ago (several times, actually)... ;-/
> 
> Its in all the -ac trees 8)
> 
> BTW Al: We have another general vfs/fs problem to handle - which is exceeding
> max file sizes on limited file systems. Pretty much nobody is getting it
> right. Ext2 can be tricked to go past the limit, sys5 1k sits there emitting
> printk messages etc.
 
> Any objections to me putting max file size for an fs (in pages) into the
> superblock ? An fs can still implement weird rules by putting large values
> in that and doing its own checks.

Alan, it doesn't work that way. Maximal size depends on the type of object,
for one thing. Moreover, it's not always a multiple of page size, so you
still need foo_get_block() to be aware of the problem (it should return
-EFBIG). Besides, we need to take care of the situations when some of
get_block() calls fail in prepare_write() - that can happen due to other
problems. I've fixed all that stuff for ext2 (check the patches posted on
l-k after 12-pre6). We need to propagate it into other filesystems, but
I don't think that max size in pages is really worth the trouble.

I can pull these patches out of the mix and send them to you. ACK?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Sun, 7 Jan 2001, Chris Wedgwood wrote:

> On Sat, Jan 06, 2001 at 03:35:32PM +0000, Alan Cox wrote:
> 
>     BTW Al: We have another general vfs/fs problem to handle - which
>     is exceeding max file sizes on limited file systems. Pretty much
>     nobody is getting it right. Ext2 can be tricked to go past the
>     limit, sys5 1k sits there emitting printk messages etc.
> 
> Which filesystems have limits other than 2^31 bytes?

	Plenty. ext2, for one - e.g. with 4Kb blocks you have limit at
0x4010040c000 for files and 0x100000000 for directories. With 1Kb blocks
the limit for files is 0x404043000. Notice that the latter is not a
multiple of page size on Alpha.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Wedgwood]

On Mon, Jan 08, 2001 at 02:56:10AM -0500, Alexander Viro wrote:

    	Plenty. ext2, for one - e.g. with 4Kb blocks you have limit
    at 0x4010040c000 for files and 0x100000000 for directories. With
    1Kb blocks the limit for files is 0x404043000. Notice that the
    latter is not a multiple of page size on Alpha.

There is code to support this in 2.4.0-ac4 -- initially I didn't like
the way Alan had done things (I was think -EFBIG should be used only
for LFS violations) but after some thought has decided that what he
has makes a lot of sense.

The FS calculates the largest suitable byte-size for a file in put it
in the superblock; when the VFS checks in the generic_file_* paths.
The only time this won't work is if some complex criteria allows some
files to be larger than others, in which case -- we add a callback to
the fs.



  --cw

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

[PATCH(es)] Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Chris Wedgwood wrote:

> On Mon, Jan 08, 2001 at 02:56:10AM -0500, Alexander Viro wrote:
> 
>     	Plenty. ext2, for one - e.g. with 4Kb blocks you have limit
>     at 0x4010040c000 for files and 0x100000000 for directories. With
>     1Kb blocks the limit for files is 0x404043000. Notice that the
>     latter is not a multiple of page size on Alpha.
> 
> There is code to support this in 2.4.0-ac4 -- initially I didn't like
> the way Alan had done things (I was think -EFBIG should be used only
> for LFS violations) but after some thought has decided that what he
> has makes a lot of sense.
> 
> The FS calculates the largest suitable byte-size for a file in put it
> in the superblock; when the VFS checks in the generic_file_* paths.
> The only time this won't work is if some complex criteria allows some
> files to be larger than others, in which case -- we add a callback to
> the fs.

It's still not enough. get_block() can fail for a lot of reasons and
generic_file_write() should be prepared to deal with that. -EFBIG
is nothing special in that respect. We don't need new callbacks, get_block()
is quite enough.

IOW, the following is still needed:

diff -urN S0/fs/buffer.c S0-get_block_fail/fs/buffer.c
--- S0/fs/buffer.c	Wed Jan  3 23:45:26 2001
+++ S0-get_block_fail/fs/buffer.c	Mon Jan  8 03:12:07 2001
@@ -1494,6 +1494,7 @@
 	int err, i;
 	unsigned long block;
 	struct buffer_head *bh, *head;
+	int need_unlock = 1;
 
 	if (!PageLocked(page))
 		BUG();
@@ -1539,18 +1540,39 @@
 	} while (bh != head);
 
 	/* Stage 3: submit the IO */
+	SetPageUptodate(page);
 	do {
 		submit_bh(WRITE, bh);
 		bh = bh->b_this_page;		
 	} while (bh != head);
 
 	/* Done - end_buffer_io_async will unlock */
-	SetPageUptodate(page);
 	return 0;
 
 out:
 	ClearPageUptodate(page);
-	UnlockPage(page);
+	bh = head;
+	need_unlock = 1;
+	/* Recovery: lock and submit the mapped buffers */
+	do {
+		if (buffer_mapped(bh)) {
+			lock_buffer(bh);
+			need_unlock = 0;
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
+	do {
+		if (buffer_mapped(bh)) {
+			bh->b_end_io = end_buffer_io_async;
+			atomic_inc(&bh->b_count);
+			set_bit(BH_Uptodate, &bh->b_state);
+			set_bit(BH_Dirty, &bh->b_state);
+			submit_bh(WRITE, bh);
+		}
+		bh = bh->b_this_page;
+	}
+	if (need_unlock)
+		UnlockPage(page);
 	return err;
 }
 
@@ -1621,6 +1643,15 @@
 	}
 	return 0;
 out:
+	bh = head;
+	do {
+		if (buffer_new(bh) && !buffer_uptodate(bh)) {
+			memset(bh->b_data, 0, bh->b_size);
+			set_bit(BH_Uptodate, &bh->b_state);
+			mark_buffer_dirty(bh);
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
 	return err;
 }
 
diff -urN S0/mm/filemap.c S0-get_block_fail/mm/filemap.c
--- S0/mm/filemap.c	Tue Jan  2 21:59:45 2001
+++ S0-get_block_fail/mm/filemap.c	Mon Jan  8 03:13:37 2001
@@ -2449,6 +2449,7 @@
 	unsigned long	written;
 	long		status;
 	int		err;
+	unsigned	bytes;
 
 	cached_page = NULL;
 
@@ -2493,7 +2494,7 @@
 	}
 
 	while (count) {
-		unsigned long bytes, index, offset;
+		unsigned long index, offset;
 		char *kaddr;
 		int deactivate = 1;
 
@@ -2532,7 +2533,7 @@
 
 		status = mapping->a_ops->prepare_write(file, page, offset, offset+bytes);
 		if (status)
-			goto unlock;
+			goto sync_failure;
 		kaddr = page_address(page);
 		status = copy_from_user(kaddr+offset, buf, bytes);
 		flush_dcache_page(page);
@@ -2558,6 +2559,7 @@
 		if (status < 0)
 			break;
 	}
+done:
 	*ppos = pos;
 
 	if (cached_page)
@@ -2578,6 +2580,13 @@
 	ClearPageUptodate(page);
 	kunmap(page);
 	goto unlock;
+sync_failure:
+	UnlockPage(page);
+	deactivate_page(page);
+	page_cache_release(page);
+	if (pos + bytes > inode->i_size)
+		vmtruncate(inode, inode->i_size);
+	goto done;
 }
 
 void __init page_cache_init(unsigned long mempages)


and with that patch the rest of filesize limits goes into filesystems. For
ext2 it would be

diff -urN S0/fs/ext2/inode.c S0-ext2_get_block-EFBIG/fs/ext2/inode.c
--- S0/fs/ext2/inode.c	Fri Dec 29 17:36:44 2000
+++ S0-ext2_get_block-EFBIG/fs/ext2/inode.c	Mon Jan  8 03:22:58 2001
@@ -153,11 +153,13 @@
  *	This function translates the block number into path in that tree -
  *	return value is the path length and @offsets[n] is the offset of
  *	pointer to (n+1)th node in the nth one. If @block is out of range
- *	(negative or too large) warning is printed and zero returned.
+ *	(negative or too large) we return zero. Warning is printed if @block
+ *	is negative - that should never happen. Too large value is OK, it
+ *	just means that ext2_get_block() should return -%EFBIG.
  *
  *	Note: function doesn't find node addresses, so no IO is needed. All
  *	we need to know is the capacity of indirect blocks (taken from the
- *	inode->i_sb).
+ *	@inode->i_sb).
  */
 
 /*
@@ -196,7 +198,7 @@
 		offsets[n++] = (i_block >> ptrs_bits) & (ptrs - 1);
 		offsets[n++] = i_block & (ptrs - 1);
 	} else {
-		ext2_warning (inode->i_sb, "ext2_block_to_path", "block > big");
+		/* Too large, nothing to do here */
 	}
 	return n;
 }
@@ -505,7 +507,7 @@
 
 static int ext2_get_block(struct inode *inode, long iblock, struct buffer_head *bh_result, int create)
 {
-	int err = -EIO;
+	int err = -EFBIG;
 	int offsets[4];
 	Indirect chain[4];
 	Indirect *partial;

plus 

diff -urN S0/fs/ext2/inode.c S0-ext2-LFS/fs/ext2/inode.c
--- S0/fs/ext2/inode.c	Fri Dec 29 17:36:44 2000
+++ S0-ext2-LFS/fs/ext2/inode.c	Fri Jan  5 21:11:31 2001
@@ -1188,7 +1188,7 @@
 		raw_inode->i_dir_acl = cpu_to_le32(inode->u.ext2_i.i_dir_acl);
 	else {
 		raw_inode->i_size_high = cpu_to_le32(inode->i_size >> 32);
-		if (raw_inode->i_size_high) {
+		if (raw_inode->i_size_high || (inode->i_size & (1<<31))) {
 			struct super_block *sb = inode->i_sb;
 			if (!EXT2_HAS_RO_COMPAT_FEATURE(sb,
 					EXT2_FEATURE_RO_COMPAT_LARGE_FILE) ||

plus 

diff -urN S0/fs/ext2/file.c S0-ext2_max_size/fs/ext2/file.c
--- S0/fs/ext2/file.c	Wed Sep 27 16:41:33 2000
+++ S0-ext2_max_size/fs/ext2/file.c	Mon Jan  8 03:29:41 2001
@@ -25,17 +25,6 @@
 static loff_t ext2_file_lseek(struct file *, loff_t, int);
 static int ext2_open_file (struct inode *, struct file *);
 
-#define EXT2_MAX_SIZE(bits)							\
-	(((EXT2_NDIR_BLOCKS + (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) * (1LL << (bits - 2))) * 	\
-	  (1LL << bits)) - 1)
-
-static long long ext2_max_sizes[] = {
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
-EXT2_MAX_SIZE(10), EXT2_MAX_SIZE(11), EXT2_MAX_SIZE(12), EXT2_MAX_SIZE(13)
-};
-
 /*
  * Make sure the offset never goes beyond the 32-bit mark..
  */
@@ -56,7 +45,7 @@
 	if (offset<0)
 		return -EINVAL;
 	if (((unsigned long long) offset >> 32) != 0) {
-		if (offset > ext2_max_sizes[EXT2_BLOCK_SIZE_BITS(inode->i_sb)])
+		if (offset >= inode->i_sb->u.ext2_sb.s_max_size)
 			return -EINVAL;
 	} 
 	if (offset != file->f_pos) {
diff -urN S0/fs/ext2/super.c S0-ext2_max_size/fs/ext2/super.c
--- S0/fs/ext2/super.c	Fri Dec 29 17:36:44 2000
+++ S0-ext2_max_size/fs/ext2/super.c	Mon Jan  8 03:30:49 2001
@@ -380,6 +380,18 @@
 }
 
 #define log2(n) ffz(~(n))
+ 
+/*
+ * maximal file size.
+ */
+static loff_t ext2_max_size(int bits)
+{
+	loff_t res = EXT2_NDIR_BLOCKS;
+	res += 1LL << (bits-2);
+	res += 1LL << (2*(bits-2));
+	res += 1LL << (3*(bits-2));
+	return res << bits;
+}
 
 struct super_block * ext2_read_super (struct super_block * sb, void * data,
 				      int silent)
@@ -549,6 +561,7 @@
 		log2 (EXT2_ADDR_PER_BLOCK(sb));
 	sb->u.ext2_sb.s_desc_per_block_bits =
 		log2 (EXT2_DESC_PER_BLOCK(sb));
+	sb->u.ext2_sb.s_max_size = ext2_max_size(sb->s_blocksize_bits);
 	if (sb->s_magic != EXT2_SUPER_MAGIC) {
 		if (!silent)
 			printk ("VFS: Can't find an ext2 filesystem on dev "
diff -urN S0/include/linux/ext2_fs_sb.h S0-ext2_max_size/include/linux/ext2_fs_sb.h
--- S0/include/linux/ext2_fs_sb.h	Fri Dec 29 17:36:44 2000
+++ S0-ext2_max_size/include/linux/ext2_fs_sb.h	Mon Jan  8 03:31:19 2001
@@ -56,6 +56,7 @@
 	int s_desc_per_block_bits;
 	int s_inode_size;
 	int s_first_ino;
+	loff_t s_max_size;
 };
 
 #endif	/* _LINUX_EXT2_FS_SB */

plus


diff -urN S0-ext2_max_size/fs/ext2/file.c S0-ext2-LFS-full/fs/ext2/file.c
--- S0-ext2_max_size/fs/ext2/file.c	Mon Jan  8 03:29:41 2001
+++ S0-ext2-LFS-full/fs/ext2/file.c	Mon Jan  8 03:37:41 2001
@@ -99,4 +99,5 @@
 
 struct inode_operations ext2_file_inode_operations = {
 	truncate:	ext2_truncate,
+	setattr:	ext2_notify_change,
 };
diff -urN S0-ext2_max_size/fs/ext2/inode.c S0-ext2-LFS-full/fs/ext2/inode.c
--- S0-ext2_max_size/fs/ext2/inode.c	Fri Dec 29 17:36:44 2000
+++ S0-ext2-LFS-full/fs/ext2/inode.c	Mon Jan  8 03:37:41 2001
@@ -880,8 +880,6 @@
 	if (!(S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
 	    S_ISLNK(inode->i_mode)))
 		return;
-	if (IS_APPEND(inode) || IS_IMMUTABLE(inode))
-		return;
 
 	ext2_discard_prealloc(inode);
 
@@ -1239,11 +1237,17 @@
 	return ext2_update_inode (inode, 1);
 }
 
+static struct {unsigned attr, flag, ext2;} ext2_attr[] = {
+	{ATTR_FLAG_SYNCRONOUS,	S_SYNC,		EXT2_SYNC_FL},
+	{ATTR_FLAG_NOATIME,	S_NOATIME,	EXT2_NOATIME_FL},
+	{ATTR_FLAG_APPEND,	S_APPEND,	EXT2_APPEND_FL},
+	{ATTR_FLAG_IMMUTABLE,	S_IMMUTABLE,	EXT2_IMMUTABLE_FL}
+};
+	
 int ext2_notify_change(struct dentry *dentry, struct iattr *iattr)
 {
 	struct inode *inode = dentry->d_inode;
 	int		retval;
-	unsigned int	flags;
 	
 	retval = -EPERM;
 	if (iattr->ia_valid & ATTR_ATTR_FLAG &&
@@ -1260,36 +1264,27 @@
 	if (retval != 0)
 		goto out;
 
+	if (iattr->ia_valid & ATTR_SIZE) {
+		if (iattr->ia_size > inode->i_sb->u.ext2_sb.s_max_size) {
+			retval = -EFBIG;
+			goto out;
+		}
+	}
+
 	inode_setattr(inode, iattr);
 	
-	flags = iattr->ia_attr_flags;
-	if (flags & ATTR_FLAG_SYNCRONOUS) {
-		inode->i_flags |= S_SYNC;
-		inode->u.ext2_i.i_flags |= EXT2_SYNC_FL;
-	} else {
-		inode->i_flags &= ~S_SYNC;
-		inode->u.ext2_i.i_flags &= ~EXT2_SYNC_FL;
-	}
-	if (flags & ATTR_FLAG_NOATIME) {
-		inode->i_flags |= S_NOATIME;
-		inode->u.ext2_i.i_flags |= EXT2_NOATIME_FL;
-	} else {
-		inode->i_flags &= ~S_NOATIME;
-		inode->u.ext2_i.i_flags &= ~EXT2_NOATIME_FL;
-	}
-	if (flags & ATTR_FLAG_APPEND) {
-		inode->i_flags |= S_APPEND;
-		inode->u.ext2_i.i_flags |= EXT2_APPEND_FL;
-	} else {
-		inode->i_flags &= ~S_APPEND;
-		inode->u.ext2_i.i_flags &= ~EXT2_APPEND_FL;
-	}
-	if (flags & ATTR_FLAG_IMMUTABLE) {
-		inode->i_flags |= S_IMMUTABLE;
-		inode->u.ext2_i.i_flags |= EXT2_IMMUTABLE_FL;
-	} else {
-		inode->i_flags &= ~S_IMMUTABLE;
-		inode->u.ext2_i.i_flags &= ~EXT2_IMMUTABLE_FL;
+	if (iattr->ia_valid & ATTR_ATTR_FLAG) {
+		unsigned flags = iattr->ia_attr_flags;
+		int i;
+		for (i=0; i<sizeof(ext2_attr)/sizeof(ext2_attr[0]); i++) {
+			if (flags & ext2_attr[i].attr) {
+				inode->i_flags |= ext2_attr[i].flag;
+				inode->u.ext2_i.i_flags |= ext2_attr[i].ext2;
+			} else {
+				inode->i_flags &= ~ext2_attr[i].flag;
+				inode->u.ext2_i.i_flags &= ~ext2_attr[i].ext2;
+			}
+		}
 	}
 	mark_inode_dirty(inode);
 out:
diff -urN S0-ext2_max_size/include/linux/ext2_fs.h S0-ext2-LFS-full/include/linux/ext2_fs.h
--- S0-ext2_max_size/include/linux/ext2_fs.h	Thu Jan  4 17:50:47 2001
+++ S0-ext2-LFS-full/include/linux/ext2_fs.h	Mon Jan  8 03:37:41 2001
@@ -582,6 +582,8 @@
 extern int ext2_sync_inode (struct inode *);
 extern void ext2_discard_prealloc (struct inode *);
 
+extern int ext2_notify_change (struct dentry *, struct iattr *);
+
 /* ioctl.c */
 extern int ext2_ioctl (struct inode *, struct file *, unsigned int,
 		       unsigned long);

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> Alan, it doesn't work that way. Maximal size depends on the type of object,
> for one thing. Moreover, it's not always a multiple of page size, so you

Its a multiple of page size for all fs's we have but I did it in terms of
bytes anyway

> still need foo_get_block() to be aware of the problem (it should return
> -EFBIG). Besides, we need to take care of the situations when some of
> get_block() calls fail in prepare_write() - that can happen due to other
> problems. I've fixed all that stuff for ext2 (check the patches posted on
> l-k after 12-pre6). We need to propagate it into other filesystems, but

I put it into generic_file_write. That covers most fs's it seems. The jffs 
guys are going to switch to generic_file_write soon and the other fs's 
that dont are wacko ones I dont care about ;)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Alan Cox wrote:

> > Alan, it doesn't work that way. Maximal size depends on the type of object,
> > for one thing. Moreover, it's not always a multiple of page size, so you
> 
> Its a multiple of page size for all fs's we have but I did it in terms of
> bytes anyway

1Kb-block ext2 on Alpha. Count yourself and you'll see...

> > still need foo_get_block() to be aware of the problem (it should return
> > -EFBIG). Besides, we need to take care of the situations when some of
> > get_block() calls fail in prepare_write() - that can happen due to other
> > problems. I've fixed all that stuff for ext2 (check the patches posted on
> > l-k after 12-pre6). We need to propagate it into other filesystems, but
> 
> I put it into generic_file_write. That covers most fs's it seems. The jffs 
> guys are going to switch to generic_file_write soon and the other fs's 
> that dont are wacko ones I dont care about ;)

Alan, we have to deal with get_block() failures anyway. -ENOSPC, -EDQUOT,
not to mention plain and simple -EIO. -EFBIG handling is not different.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> There is code to support this in 2.4.0-ac4 -- initially I didn't like
> the way Alan had done things (I was think -EFBIG should be used only
> for LFS violations) but after some thought has decided that what he
> has makes a lot of sense.

Its based on the docs I have + a test program for the non LFS/limit cases that
sct provided. -ac4 isnt quite compliant (sends SIGXFZ on a short write case
which is wrong and also doesnt do a write partiall overlapping the disk limit)

-ac5 should fix those

> The only time this won't work is if some complex criteria allows some
> files to be larger than others, in which case -- we add a callback to
> the fs.

You don't need to add the callback. Set the limit infinite and handle it in your
truncate and writing paths.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> > I put it into generic_file_write. That covers most fs's it seems. The jffs 
> > guys are going to switch to generic_file_write soon and the other fs's 
> > that dont are wacko ones I dont care about ;)
> 
> Alan, we have to deal with get_block() failures anyway. -ENOSPC, -EDQUOT,
> not to mention plain and simple -EIO. -EFBIG handling is not different.

EFBIG is very different in several ways. To start with the get_block code
doesnt have enough information to correctly implement the SUS specification
rules.

The generic cases can be handled in generic code. They cannot cleanly be
handled by copying that code into every file system. Ditto with truncate.
Since the VFS can handle them nicely the VFS should handle them.

Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Alan Cox wrote:

> > > I put it into generic_file_write. That covers most fs's it seems. The jffs 
> > > guys are going to switch to generic_file_write soon and the other fs's 
> > > that dont are wacko ones I dont care about ;)
> > 
> > Alan, we have to deal with get_block() failures anyway. -ENOSPC, -EDQUOT,
> > not to mention plain and simple -EIO. -EFBIG handling is not different.
> 
> EFBIG is very different in several ways. To start with the get_block code
> doesnt have enough information to correctly implement the SUS specification
> rules.

Umm... Details, please? Are you talking about 2^32 or about fs layout limits?
The former may very well belong to VFS - no arguments here. The latter...
And yes, fs layout limits are visible - for ext2 they can be as low as 2^24
blocks.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> Umm... Details, please? Are you talking about 2^32 or about fs layout limits?
> The former may very well belong to VFS - no arguments here. The latter...
> And yes, fs layout limits are visible - for ext2 they can be as low as 2^24
> blocks.

The SuS rules require write checks

resource limit
	if that stops all of the write
			SIGXFZ
			EFBIG
	else short write

!O_LARGEFILE and > off_t limit
	if that stops all of the write
			SIGXFZ
			EFBIG
	else short write

over fs limit
	if that stops all the write
			SIGXFZ
			EFBIG
	else short write

truncate has to honour the resource and fs limit. ftruncate also checks
the O_LARGEFILE limit.

Finally some OS's actually let you F_SETFL O_LARGEFILE.

I can put all that in the VFS so I did (right now the ext2 size calculator is
wrong but thats proof of concept detail). Just need to shift if over from
ext2/file.c

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 12:26:17PM +0000, Alan Cox wrote:

> I can put all that in the VFS so I did (right now the ext2 size calculator is
> wrong but thats proof of concept detail). Just need to shift if over from
> ext2/file.c

Try 'getconf LINK_MAX /ramfs'.
While the result (127) is in some way SuS/POSIXLY_CORRECT,
it's not the truth.

Why not start to fix this problem outside the funny switch/case in glibc ?
The filesystem itself should able to handle this.

-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Stefan Traby wrote:

> On Mon, Jan 08, 2001 at 12:26:17PM +0000, Alan Cox wrote:
> 
> > I can put all that in the VFS so I did (right now the ext2 size calculator is
> > wrong but thats proof of concept detail). Just need to shift if over from
> > ext2/file.c
> 
> Try 'getconf LINK_MAX /ramfs'.
> While the result (127) is in some way SuS/POSIXLY_CORRECT,
> it's not the truth.
> 
> Why not start to fix this problem outside the funny switch/case in glibc ?
> The filesystem itself should able to handle this.

Sigh... And the API would be?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
>  I put it into generic_file_write. That covers most fs's it seems. The
> jffs  guys are going to switch to generic_file_write soon 

It's in CVS already. For 2.4, 'soon' == 'when Linus is ready to start taking
patches'

If you want it for 2.4-ac I can provide a patch which fixes both that and 
the oops on open() unlink() read().

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 08:35:10AM -0500, Alexander Viro wrote:
> On Mon, 8 Jan 2001, Stefan Traby wrote:
> > Try 'getconf LINK_MAX /ramfs'.
> > While the result (127) is in some way SuS/POSIXLY_CORRECT,
> > it's not the truth.
> > 
> > Why not start to fix this problem outside the funny switch/case in glibc ?
> > The filesystem itself should able to handle this.
> 
> Sigh... And the API would be?

Oh, IMHO that's not too important.
API should be accepted by you and Ulrich Drepper.
API should not be expected to return static values. (for example: reiserfs
may return different values dependend on filesystem-version)

I think the easiest way would be to define ioctls for this.
So VFS needs no changes; the only discussion would be about the
standard name for "request-names", and Ulrich could fall back to
his magic calculations if EINVAL is returned.

Because I have no knowledge on this I suggest that you and Ulrich fight
together on a more flexible solution than the current one. I guess
that Linus would accept this without thinking too much about it.

-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

Alan, consider applying the patch below.
Contents:
	* recovery from failing get_block() in __block_write_full_page()
and __block_prepare_write().
	* handling of partially mapped pages in generic_file_write().
	* use of ->s_maxbytes in default_llseek().
	* crapectomy in ext2/file.c
	* correct calculation of ->s_maxbytes for ext2.

diff -urN S0-AC4/fs/buffer.c S0-AC4-fixes/fs/buffer.c
--- S0-AC4/fs/buffer.c	Mon Jan  8 08:46:17 2001
+++ S0-AC4-fixes/fs/buffer.c	Mon Jan  8 08:28:55 2001
@@ -1493,6 +1493,7 @@
 	int err, i;
 	unsigned long block;
 	struct buffer_head *bh, *head;
+	int need_unlock = 1;
 
 	if (!PageLocked(page))
 		BUG();
@@ -1549,7 +1550,28 @@
 
 out:
 	ClearPageUptodate(page);
-	UnlockPage(page);
+	bh = head;
+	need_unlock = 1;
+	/* Recovery: lock and submit the mapped buffers */
+	do {
+		if (buffer_mapped(bh)) {
+			lock_buffer(bh);
+			need_unlock = 0;
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
+	do {
+		if (buffer_mapped(bh)) {
+			bh->b_end_io = end_buffer_io_async;
+			atomic_inc(&bh->b_count);
+			set_bit(BH_Uptodate, &bh->b_state);
+			set_bit(BH_Dirty, &bh->b_state);
+			submit_bh(WRITE, bh);
+		}
+		bh = bh->b_this_page;
+	}
+	if (need_unlock)
+		UnlockPage(page);
 	return err;
 }
 
@@ -1620,6 +1642,15 @@
 	}
 	return 0;
 out:
+	bh = head;
+	do {
+		if (buffer_new(bh) && !buffer_uptodate(bh)) {
+			memset(bh->b_data, 0, bh->b_size);
+			set_bit(BH_Uptodate, &bh->b_state);
+			mark_buffer_dirty(bh);
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
 	return err;
 }
 
diff -urN S0-AC4/fs/ext2/file.c S0-AC4-fixes/fs/ext2/file.c
--- S0-AC4/fs/ext2/file.c	Mon Jan  8 08:46:18 2001
+++ S0-AC4-fixes/fs/ext2/file.c	Mon Jan  8 08:34:04 2001
@@ -22,51 +22,6 @@
 #include <linux/ext2_fs.h>
 #include <linux/sched.h>
 
-static loff_t ext2_file_lseek(struct file *, loff_t, int);
-static int ext2_open_file (struct inode *, struct file *);
-
-#define EXT2_MAX_SIZE(bits)							\
-	(((EXT2_NDIR_BLOCKS + (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) * (1LL << (bits - 2))) * 	\
-	  (1LL << bits)) - 1)
-
-static long long ext2_max_sizes[] = {
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
-EXT2_MAX_SIZE(10), EXT2_MAX_SIZE(11), EXT2_MAX_SIZE(12), EXT2_MAX_SIZE(13)
-};
-
-/*
- * Make sure the offset never goes beyond the 32-bit mark..
- */
-static loff_t ext2_file_lseek(
-	struct file *file,
-	loff_t offset,
-	int origin)
-{
-	struct inode *inode = file->f_dentry->d_inode;
-
-	switch (origin) {
-		case 2:
-			offset += inode->i_size;
-			break;
-		case 1:
-			offset += file->f_pos;
-	}
-	if (offset<0)
-		return -EINVAL;
-	if (((unsigned long long) offset >> 32) != 0) {
-		if (offset > ext2_max_sizes[EXT2_BLOCK_SIZE_BITS(inode->i_sb)])
-			return -EINVAL;
-	} 
-	if (offset != file->f_pos) {
-		file->f_pos = offset;
-		file->f_reada = 0;
-		file->f_version = ++event;
-	}
-	return offset;
-}
-
 /*
  * Called when an inode is released. Note that this is different
  * from ext2_file_open: open gets called at every open, but release
@@ -84,7 +39,6 @@
  * the ext2 filesystem.
  */
 struct file_operations ext2_file_operations = {
-	llseek:		ext2_file_lseek,
 	read:		generic_file_read,
 	write:		generic_file_write,
 	ioctl:		ext2_ioctl,
diff -urN S0-AC4/fs/ext2/super.c S0-AC4-fixes/fs/ext2/super.c
--- S0-AC4/fs/ext2/super.c	Mon Jan  8 08:46:18 2001
+++ S0-AC4-fixes/fs/ext2/super.c	Mon Jan  8 08:35:16 2001
@@ -380,6 +380,20 @@
 }
 
 #define log2(n) ffz(~(n))
+ 
+/*
+ * maximal file size.
+ */
+static loff_t ext2_max_size(int bits)
+{
+	loff_t res = EXT2_NDIR_BLOCKS;
+	res += 1LL << (bits-2);
+	res += 1LL << (2*(bits-2));
+	res += 1LL << (3*(bits-2));
+	if (res > 1LL << 32)
+		res = 1LL << 32;
+	return res << bits;
+}
 
 struct super_block * ext2_read_super (struct super_block * sb, void * data,
 				      int silent)
@@ -476,8 +490,7 @@
 		le32_to_cpu(EXT2_SB(sb)->s_es->s_log_block_size) + 10;
 	sb->s_blocksize = 1 << sb->s_blocksize_bits;
 	
-	/* We allow 2^32 blocks - not pages */
-	sb->s_maxbytes = (1ULL<<(sb->s_blocksize_bits+32))-1;
+	sb->s_maxbytes = ext2_max_size(sb->s_blocksize_bits);
 	
 	if (sb->s_blocksize != BLOCK_SIZE &&
 	    (sb->s_blocksize == 1024 || sb->s_blocksize == 2048 ||
diff -urN S0-AC4/fs/read_write.c S0-AC4-fixes/fs/read_write.c
--- S0-AC4/fs/read_write.c	Fri Sep 22 17:21:19 2000
+++ S0-AC4-fixes/fs/read_write.c	Mon Jan  8 08:32:27 2001
@@ -36,7 +36,7 @@
 			offset += file->f_pos;
 	}
 	retval = -EINVAL;
-	if (offset >= 0) {
+	if (offset>=0 && offset<=file->f_dentry->d_inode->i_sb->s_maxbytes) {
 		if (offset != file->f_pos) {
 			file->f_pos = offset;
 			file->f_reada = 0;
diff -urN S0-AC4/mm/filemap.c S0-AC4-fixes/mm/filemap.c
--- S0-AC4/mm/filemap.c	Mon Jan  8 08:46:34 2001
+++ S0-AC4-fixes/mm/filemap.c	Mon Jan  8 08:26:21 2001
@@ -2452,6 +2452,7 @@
 	unsigned long	written;
 	long		status;
 	int		err;
+	unsigned	bytes;
 
 	cached_page = NULL;
 
@@ -2537,7 +2538,7 @@
 	mark_inode_dirty_sync(inode);
 
 	while (count) {
-		unsigned long bytes, index, offset;
+		unsigned long index, offset;
 		char *kaddr;
 		int deactivate = 1;
 
@@ -2577,7 +2578,7 @@
 
 		status = mapping->a_ops->prepare_write(file, page, offset, offset+bytes);
 		if (status)
-			goto unlock;
+			goto sync_failure;
 		kaddr = page_address(page);
 		status = copy_from_user(kaddr+offset, buf, bytes);
 		flush_dcache_page(page);
@@ -2603,6 +2604,7 @@
 		if (status < 0)
 			break;
 	}
+done:
 	*ppos = pos;
 
 	if (cached_page)
@@ -2627,6 +2629,13 @@
 	ClearPageUptodate(page);
 	kunmap(page);
 	goto unlock;
+sync_failure:
+	UnlockPage(page);
+	deactivate_page(page);
+	page_cache_release(page);
+	if (pos + bytes > inode->i_size)
+		vmtruncate(inode, inode->i_size);
+	goto done;
 }
 
 void __init page_cache_init(unsigned long mempages)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Stefan Traby wrote:

> Because I have no knowledge on this I suggest that you and Ulrich fight
> together on a more flexible solution than the current one. I guess
> that Linus would accept this without thinking too much about it.

Unfortunately, Ulrich's taste was incompatible with mine in almost all
cases I can recall. So "together" is very likely to be "hand-to-hand".
glibc is hopelessly ugly. If it can be made cleaner without making
the kernel API ugly - wonderful. If not - too bad. For glibc.

IOW, if you can propose clean API - do it. If Ulrich can do that - great,
I'm more than willing to listen. I'm not holding my breath on the last
one, though.

PS: The day when Linus will really switch to accepting API changes without
thinking about them will be the day when I stop work on his tree.
So far it had not happened and I like it that way, thank you very much.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> > Why not start to fix this problem outside the funny switch/case in glibc ?
> > The filesystem itself should able to handle this.
> 
> Sigh... And the API would be?

In SuS its pathconf()

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Alan Cox wrote:

> > > Why not start to fix this problem outside the funny switch/case in glibc ?
> > > The filesystem itself should able to handle this.
> > 
> > Sigh... And the API would be?
> 
> In SuS its pathconf()

Which happens to be remarkably ugly. And it will not get better tomoorow...

I _really_ don't think that this barfbag should make its way into the
kernel API. Some parts of information make sense. Some don't. API is...
well, "int name" with a bunch of constatnts is a dead giveaway in itself.

If anything, we might want a mount-ID created upon mount(2) and never
reused + pseudo-fs a-la /proc that would give access to per-mount data.
That might be more or less reasonable, but that would also require
returning such ID from stat()...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> Which happens to be remarkably ugly. And it will not get better tomoorow...

Its really only ugly in one way which is that you pass an int for the item
rather than having a struct of all the data

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Alan Cox wrote:

> > Which happens to be remarkably ugly. And it will not get better tomoorow...
> 
> Its really only ugly in one way which is that you pass an int for the item
> rather than having a struct of all the data

You know as well as I do that as soon as we add it glibc folks _will_
start whin^Wasking for "just one more field". So structure is out of
question for pretty obvious reasons... Wanna bet that they'll ask for
maximal size of symlink that could be created in directory? Or truncate
long names vs. reject long names policy. And everything from /proc/mounts,
since "binary data is easier to parse". And case sensitivity. And
NLS used. And timezone of that filesystem. And SGID policy (BSD vs. SysV).
And subset of mode bits available on that fs. And ability to create
device nodes. And...  There is a lot of crap that could be asked for.
General rule with GNU seems to be that _every_ piece of crap somebody
had thought about gets tossed into the mix.

Prediction:
	* Any attempt to decide on a fixed structure will generate a flamewar,
where everyone and his mom will advocate their pet features.
	* There will be regular requests to play syscall of the week game.
I.e. new version of pathconf(2) that will support more new features. Old
ones will not go away, indeed.
	* There will be regular requests to make syscall versioned (read:
magic number + pointer to structure with layout depending on that number).

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Mason]

On Monday, January 08, 2001 09:02:46 AM -0500 Alexander Viro
<viro@math.psu.edu> wrote:

> Alan, consider applying the patch below.
> Contents:
[snip]
> +	do {
> +		if (buffer_mapped(bh)) {
> +			bh->b_end_io = end_buffer_io_async;
> +			atomic_inc(&bh->b_count);
> +			set_bit(BH_Uptodate, &bh->b_state);
> +			set_bit(BH_Dirty, &bh->b_state);
> +			submit_bh(WRITE, bh);
> +		}
> +		bh = bh->b_this_page;
> +	}
          ^^^^^^^^^^^^^

This doesn't look right...

-chris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Chris Mason wrote:

> 
> 
> On Monday, January 08, 2001 09:02:46 AM -0500 Alexander Viro
> <viro@math.psu.edu> wrote:
> 
> > Alan, consider applying the patch below.
> > Contents:
> [snip]
> > +	do {
> > +		if (buffer_mapped(bh)) {
> > +			bh->b_end_io = end_buffer_io_async;
> > +			atomic_inc(&bh->b_count);
> > +			set_bit(BH_Uptodate, &bh->b_state);
> > +			set_bit(BH_Dirty, &bh->b_state);
> > +			submit_bh(WRITE, bh);
> > +		}
> > +		bh = bh->b_this_page;
> > +	}
>           ^^^^^^^^^^^^^

> This doesn't look right...

No, it doesn't. s/$/while(bh != head);/, indeed. Sorry about that -
cut-and-waste when I did rediff to 2.4.0. Corrected patch follows:

diff -urN S0-AC4/fs/buffer.c S0-AC4-fixes/fs/buffer.c
--- S0-AC4/fs/buffer.c	Mon Jan  8 08:46:17 2001
+++ S0-AC4-fixes/fs/buffer.c	Mon Jan  8 08:28:55 2001
@@ -1493,6 +1493,7 @@
 	int err, i;
 	unsigned long block;
 	struct buffer_head *bh, *head;
+	int need_unlock = 1;
 
 	if (!PageLocked(page))
 		BUG();
@@ -1549,7 +1550,28 @@
 
 out:
 	ClearPageUptodate(page);
-	UnlockPage(page);
+	bh = head;
+	need_unlock = 1;
+	/* Recovery: lock and submit the mapped buffers */
+	do {
+		if (buffer_mapped(bh)) {
+			lock_buffer(bh);
+			need_unlock = 0;
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
+	do {
+		if (buffer_mapped(bh)) {
+			bh->b_end_io = end_buffer_io_async;
+			atomic_inc(&bh->b_count);
+			set_bit(BH_Uptodate, &bh->b_state);
+			set_bit(BH_Dirty, &bh->b_state);
+			submit_bh(WRITE, bh);
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
+	if (need_unlock)
+		UnlockPage(page);
 	return err;
 }
 
@@ -1620,6 +1642,15 @@
 	}
 	return 0;
 out:
+	bh = head;
+	do {
+		if (buffer_new(bh) && !buffer_uptodate(bh)) {
+			memset(bh->b_data, 0, bh->b_size);
+			set_bit(BH_Uptodate, &bh->b_state);
+			mark_buffer_dirty(bh);
+		}
+		bh = bh->b_this_page;
+	} while (bh != head);
 	return err;
 }
 
diff -urN S0-AC4/fs/ext2/file.c S0-AC4-fixes/fs/ext2/file.c
--- S0-AC4/fs/ext2/file.c	Mon Jan  8 08:46:18 2001
+++ S0-AC4-fixes/fs/ext2/file.c	Mon Jan  8 08:34:04 2001
@@ -22,51 +22,6 @@
 #include <linux/ext2_fs.h>
 #include <linux/sched.h>
 
-static loff_t ext2_file_lseek(struct file *, loff_t, int);
-static int ext2_open_file (struct inode *, struct file *);
-
-#define EXT2_MAX_SIZE(bits)							\
-	(((EXT2_NDIR_BLOCKS + (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) + 				\
-	   (1LL << (bits - 2)) * (1LL << (bits - 2)) * (1LL << (bits - 2))) * 	\
-	  (1LL << bits)) - 1)
-
-static long long ext2_max_sizes[] = {
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
-EXT2_MAX_SIZE(10), EXT2_MAX_SIZE(11), EXT2_MAX_SIZE(12), EXT2_MAX_SIZE(13)
-};
-
-/*
- * Make sure the offset never goes beyond the 32-bit mark..
- */
-static loff_t ext2_file_lseek(
-	struct file *file,
-	loff_t offset,
-	int origin)
-{
-	struct inode *inode = file->f_dentry->d_inode;
-
-	switch (origin) {
-		case 2:
-			offset += inode->i_size;
-			break;
-		case 1:
-			offset += file->f_pos;
-	}
-	if (offset<0)
-		return -EINVAL;
-	if (((unsigned long long) offset >> 32) != 0) {
-		if (offset > ext2_max_sizes[EXT2_BLOCK_SIZE_BITS(inode->i_sb)])
-			return -EINVAL;
-	} 
-	if (offset != file->f_pos) {
-		file->f_pos = offset;
-		file->f_reada = 0;
-		file->f_version = ++event;
-	}
-	return offset;
-}
-
 /*
  * Called when an inode is released. Note that this is different
  * from ext2_file_open: open gets called at every open, but release
@@ -84,7 +39,6 @@
  * the ext2 filesystem.
  */
 struct file_operations ext2_file_operations = {
-	llseek:		ext2_file_lseek,
 	read:		generic_file_read,
 	write:		generic_file_write,
 	ioctl:		ext2_ioctl,
diff -urN S0-AC4/fs/ext2/super.c S0-AC4-fixes/fs/ext2/super.c
--- S0-AC4/fs/ext2/super.c	Mon Jan  8 08:46:18 2001
+++ S0-AC4-fixes/fs/ext2/super.c	Mon Jan  8 08:35:16 2001
@@ -380,6 +380,20 @@
 }
 
 #define log2(n) ffz(~(n))
+ 
+/*
+ * maximal file size.
+ */
+static loff_t ext2_max_size(int bits)
+{
+	loff_t res = EXT2_NDIR_BLOCKS;
+	res += 1LL << (bits-2);
+	res += 1LL << (2*(bits-2));
+	res += 1LL << (3*(bits-2));
+	if (res > 1LL << 32)
+		res = 1LL << 32;
+	return res << bits;
+}
 
 struct super_block * ext2_read_super (struct super_block * sb, void * data,
 				      int silent)
@@ -476,8 +490,7 @@
 		le32_to_cpu(EXT2_SB(sb)->s_es->s_log_block_size) + 10;
 	sb->s_blocksize = 1 << sb->s_blocksize_bits;
 	
-	/* We allow 2^32 blocks - not pages */
-	sb->s_maxbytes = (1ULL<<(sb->s_blocksize_bits+32))-1;
+	sb->s_maxbytes = ext2_max_size(sb->s_blocksize_bits) - 1;
 	
 	if (sb->s_blocksize != BLOCK_SIZE &&
 	    (sb->s_blocksize == 1024 || sb->s_blocksize == 2048 ||
diff -urN S0-AC4/fs/read_write.c S0-AC4-fixes/fs/read_write.c
--- S0-AC4/fs/read_write.c	Fri Sep 22 17:21:19 2000
+++ S0-AC4-fixes/fs/read_write.c	Mon Jan  8 08:32:27 2001
@@ -36,7 +36,7 @@
 			offset += file->f_pos;
 	}
 	retval = -EINVAL;
-	if (offset >= 0) {
+	if (offset>=0 && offset<=file->f_dentry->d_inode->i_sb->s_maxbytes) {
 		if (offset != file->f_pos) {
 			file->f_pos = offset;
 			file->f_reada = 0;
diff -urN S0-AC4/mm/filemap.c S0-AC4-fixes/mm/filemap.c
--- S0-AC4/mm/filemap.c	Mon Jan  8 08:46:34 2001
+++ S0-AC4-fixes/mm/filemap.c	Mon Jan  8 08:26:21 2001
@@ -2452,6 +2452,7 @@
 	unsigned long	written;
 	long		status;
 	int		err;
+	unsigned	bytes;
 
 	cached_page = NULL;
 
@@ -2537,7 +2538,7 @@
 	mark_inode_dirty_sync(inode);
 
 	while (count) {
-		unsigned long bytes, index, offset;
+		unsigned long index, offset;
 		char *kaddr;
 		int deactivate = 1;
 
@@ -2577,7 +2578,7 @@
 
 		status = mapping->a_ops->prepare_write(file, page, offset, offset+bytes);
 		if (status)
-			goto unlock;
+			goto sync_failure;
 		kaddr = page_address(page);
 		status = copy_from_user(kaddr+offset, buf, bytes);
 		flush_dcache_page(page);
@@ -2603,6 +2604,7 @@
 		if (status < 0)
 			break;
 	}
+done:
 	*ppos = pos;
 
 	if (cached_page)
@@ -2627,6 +2629,13 @@
 	ClearPageUptodate(page);
 	kunmap(page);
 	goto unlock;
+sync_failure:
+	UnlockPage(page);
+	deactivate_page(page);
+	page_cache_release(page);
+	if (pos + bytes > inode->i_size)
+		vmtruncate(inode, inode->i_size);
+	goto done;
 }
 
 void __init page_cache_init(unsigned long mempages)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 02:40:45PM +0000, Alan Cox wrote:
> > > Why not start to fix this problem outside the funny switch/case in glibc ?
> > > The filesystem itself should able to handle this.
> > 
> > Sigh... And the API would be?
> 
> In SuS its pathconf()

I prefer SuS fpathconf(), pathconf() is just a wrapper to fpathconf();

IMHO like sysconf - which is clearly a userland (libc) thing - fpathconf
should be also a userland thing.

long int fpathconf(int fildes, int name);
long int pathconf(const char *path, int name);

I still think that ioctl is the easiest and best way and to add
another syscall is a bad idea.
    
-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> I prefer SuS fpathconf(), pathconf() is just a wrapper to fpathconf();

You can't implement it that way in the corner cases.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Chris Mason]

On Monday, January 08, 2001 10:47:41 AM -0500 Alexander Viro
<viro@math.psu.edu> wrote:
> +	do {
> +		if (buffer_mapped(bh)) {
> +			bh->b_end_io = end_buffer_io_async;
> +			atomic_inc(&bh->b_count);
> +			set_bit(BH_Uptodate, &bh->b_state);
> +			set_bit(BH_Dirty, &bh->b_state);
                        ^^^^^^^

Sorry, missed this the first time I read it.  We need to clear the dirty bit
before sending to submit_bh, otherwise it stays dirty until
try_to_free_buffers writes it again.

> +			submit_bh(WRITE, bh);
> +		}
> +		bh = bh->b_this_page;
> +	} while (bh != head);

-chris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 04:01:10PM +0000, Alan Cox wrote:
> > I prefer SuS fpathconf(), pathconf() is just a wrapper to fpathconf();
> 
> You can't implement it that way in the corner cases.

I reread SuSv2 again and didn't found corner cases.
Do you mean FIFO/pipe stuff ? I can't see the problem in this area.

In which case is an emulation of pathconf by fpathconf impossible ?

-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

> I reread SuSv2 again and didn't found corner cases.
> Do you mean FIFO/pipe stuff ? I can't see the problem in this area.
> 
> In which case is an emulation of pathconf by fpathconf impossible ?

use all your file descriptors up. Now try

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Chris Mason wrote:

> 
> On Monday, January 08, 2001 10:47:41 AM -0500 Alexander Viro
> <viro@math.psu.edu> wrote:
> > +	do {
> > +		if (buffer_mapped(bh)) {
> > +			bh->b_end_io = end_buffer_io_async;
> > +			atomic_inc(&bh->b_count);
> > +			set_bit(BH_Uptodate, &bh->b_state);
> > +			set_bit(BH_Dirty, &bh->b_state);
>                         ^^^^^^^
> 
> Sorry, missed this the first time I read it.  We need to clear the dirty bit
> before sending to submit_bh, otherwise it stays dirty until
> try_to_free_buffers writes it again.

Umm... It does recovery, but we do extra write on blocks we've managed to
allocate. Thanks for spotting - looks like a bug that had been in the patch
for a month or so ;-/ clear_bit it is...

Looking one more time... generic_file_write() is very odd:
        if (pos > inode->i_sb->s_maxbytes)
        {
                send_sig(SIGXFSZ, current, 0);
                err = -EFBIG;
                goto out;
        }

        if (pos + count > inode->i_sb->s_maxbytes)
        {
                count = inode->i_sb->s_maxbytes - count;
                goto out;
        }
 
looks funny - goto out means that new (and rather meaningless) value of
count goes to hell. Shouldn't we remove that line and s/- count/- pos/
in the previous one? Besides, do we really want the limit on size ber
0x.......f for ext2? 0x7fffffff comes from s32 returned by lseek(), and
thus - for size. ext2 limit should be a multiple of block size...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alan Cox]

>         if (pos + count > inode->i_sb->s_maxbytes)
>         {
>                 count = inode->i_sb->s_maxbytes - count;
>                 goto out;
>         }
>  
> looks funny - goto out means that new (and rather meaningless) value of
> count goes to hell. Shouldn't we remove that line and s/- count/- pos/

I already fixed that in my tree. We also send signals in a couple of spots
we shouldnt tht I fixed


Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Stefan Traby wrote:

> On Mon, Jan 08, 2001 at 04:01:10PM +0000, Alan Cox wrote:
> > > I prefer SuS fpathconf(), pathconf() is just a wrapper to fpathconf();
> > 
> > You can't implement it that way in the corner cases.
> 
> I reread SuSv2 again and didn't found corner cases.
> Do you mean FIFO/pipe stuff ? I can't see the problem in this area.
> 
> In which case is an emulation of pathconf by fpathconf impossible ?

Damnit, symlinks for one thing.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 01:05:49PM -0500, Alexander Viro wrote:
> 
> 
> On Mon, 8 Jan 2001, Stefan Traby wrote:
> 
> > On Mon, Jan 08, 2001 at 04:01:10PM +0000, Alan Cox wrote:
> > > > I prefer SuS fpathconf(), pathconf() is just a wrapper to fpathconf();
> > > 
> > > You can't implement it that way in the corner cases.
> > 
> > I reread SuSv2 again and didn't found corner cases.
> > Do you mean FIFO/pipe stuff ? I can't see the problem in this area.
> > 
> > In which case is an emulation of pathconf by fpathconf impossible ?
> 
> Damnit, symlinks for one thing.

No this is no problem.

 For pathconf(), the path argument points to the pathname of a file
      or directory.

IMHO lstat can dedect this case.
Where is the problem ?
Calling pathconf with a symlink is not defined. I suggest
an implementation of "yankee doodle" for that case.
Anyway the broken SuS standard wants that pathconf follow symlinks.
Or how do you interpret this:

 [ELOOP]
           Too many symbolic links were encountered in resolving path.

But Alan's case "out of filedescriptor" fully counts.
Anyway, I personally would ignore it, but I agree, it's a completely
valid argument.

-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Stefan Traby wrote:

> Calling pathconf with a symlink is not defined. I suggest
> an implementation of "yankee doodle" for that case.
> Anyway the broken SuS standard wants that pathconf follow symlinks.
> Or how do you interpret this:
> 
>  [ELOOP]
>            Too many symbolic links were encountered in resolving path.


/a/b/c where b is a symlink to itself. Why?

> But Alan's case "out of filedescriptor" fully counts.
> Anyway, I personally would ignore it, but I agree, it's a completely
> valid argument.

Here's another one: suppose that /foo is a mountpoint and you have
no read permissions on it. Try to open the thing...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 01:22:49PM -0500, Alexander Viro wrote:

> Here's another one: suppose that /foo is a mountpoint and you have
> no read permissions on it. Try to open the thing...

I would return EACCESS.
[EACCES]
          Search permission is denied for a component of the path prefix.
             


-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Stefan Traby wrote:

> On Mon, Jan 08, 2001 at 01:22:49PM -0500, Alexander Viro wrote:
> 
> > Here's another one: suppose that /foo is a mountpoint and you have
> > no read permissions on it. Try to open the thing...
> 
> I would return EACCESS.
> [EACCES]
>           Search permission is denied for a component of the path prefix.

And prefix would be what? "/"? Besides, I said that you don't have
read permissions on /foo, not search ones.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Stefan Traby]

On Mon, Jan 08, 2001 at 01:33:50PM -0500, Alexander Viro wrote:
> 
> On Mon, 8 Jan 2001, Stefan Traby wrote:
> 
> > On Mon, Jan 08, 2001 at 01:22:49PM -0500, Alexander Viro wrote:
> > 
> > > Here's another one: suppose that /foo is a mountpoint and you have
> > > no read permissions on it. Try to open the thing...
> > 
> > I would return EACCESS.
> > [EACCES]
> >           Search permission is denied for a component of the path prefix.
> 
> And prefix would be what? "/"? Besides, I said that you don't have
> read permissions on /foo, not search ones.

And what ?

--------------------------------------------------
changing errno. If the implementation needs to use path to
determine the value of name and the implementation does not support
the association of name with the file specified by path, or if the
process did not have appropriate privileges to query the file
specified by path, or path does not exist, pathconf() returns -1
and errno is set to indicate the error.
--------------------------------------------------
                                
So this is case is covered.

-- 

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://st.traby@opengroup.org
Europe                                   mailto://stefan@hello-penguin.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Marc Lehmann]

On Mon, Jan 08, 2001 at 01:33:50PM -0500, Alexander Viro <viro@math.psu.edu> wrote:
> And prefix would be what? "/"? Besides, I said that you don't have
> read permissions on /foo, not search ones.

You do not need read permissions on /foo to make pathconf on it. This
makes sense: you are not reading the directory...

-- 
      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@opengroup.org |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Marc Lehmann wrote:

> On Mon, Jan 08, 2001 at 01:33:50PM -0500, Alexander Viro <viro@math.psu.edu> wrote:
> > And prefix would be what? "/"? Besides, I said that you don't have
> > read permissions on /foo, not search ones.
> 
> You do not need read permissions on /foo to make pathconf on it. This
> makes sense: you are not reading the directory...

Exactly. You do need them for open(). You need successful open() to
be able to use fpathconf(). Ergo, there are cases when pathconf()
can't be implemented via fpathconf(). QED.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Andreas Dilger]

Al Viro writes:
> No, it doesn't. s/$/while(bh != head);/, indeed. Sorry about that -
> cut-and-waste when I did rediff to 2.4.0. Corrected patch follows:
> 
> diff -urN S0-AC4/fs/ext2/super.c S0-AC4-fixes/fs/ext2/super.c
> --- S0-AC4/fs/ext2/super.c	Mon Jan  8 08:46:18 2001
> +++ S0-AC4-fixes/fs/ext2/super.c	Mon Jan  8 08:35:16 2001
> @@ -380,6 +380,20 @@
>  }
>  
>  #define log2(n) ffz(~(n))
> + 
> +/*
> + * maximal file size.
> + */
> +static loff_t ext2_max_size(int bits)
> +{
> +	loff_t res = EXT2_NDIR_BLOCKS;
> +	res += 1LL << (bits-2);
> +	res += 1LL << (2*(bits-2));
> +	res += 1LL << (3*(bits-2));
> +	if (res > 1LL << 32)
> +		res = 1LL << 32;
> +	return res << bits;
> +}

Actually, this is wrong.  The ext2 inode limit is 2^32 512-byte sectors,
not 2^32 blocksize blocks.  Yes this is a wart and Ted wants to fix it, as
soon as we have something else important enough to require an incompatible
change to ext2.  As it is, we are limited to 2TB files for now, which is no
loss because we only support 2TB devices in 2.4 anyways.  When we change to
2^32 filesystem blocks, we will again be limited by indirect blocks, except
for 8kB block filesystems on Alpha/ia64 at 32TB.

+/*
+ * Maximal file size.  There is a direct, and {,double-,triple-}indirect
+ * block limit, and also a limit of (2^32 - 1) 512-byte sectors in i_blocks.
+ * We need to be 1 filesystem block less than the 2^32 sector limit.
+ */
+static loff_t ext2_max_size(int bits)
+{
+	loff_t res = EXT2_NDIR_BLOCKS;
+	res += 1LL << (bits-2);
+	res += 1LL << (2*(bits-2));
+	res += 1LL << (3*(bits-2));
+	res << bits;
+	if (res > (512LL << 32) - (1 << bits))
+		res = (512LL << 32) - (1 << bits);
+	return res;
+}

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Alexander Viro]

On Mon, 8 Jan 2001, Andreas Dilger wrote:

> Actually, this is wrong.  The ext2 inode limit is 2^32 512-byte sectors,
> not 2^32 blocksize blocks.  Yes this is a wart and Ted wants to fix it, as

??? Where? Oh, wait... ->i_blocks? I'ld rather refuse to grow past 2^32 -
sparse files can legitimately have large offsets and very low ->i_blocks.
But yes, we need to check for overflows. BTW, 2^32-1 is not good enough -
indirect blocks also contribute, so limiting ->i_size by 2Tb is not
guaranteed to keep ->i_blocks low.

IMO it's bogus - so far we are protected by inability to allocate that
much and once we are past that limit we should rather refuse to do
allocation than refuse to get large sparse files.

Or was it about something entirely different? I don't see other limiting
factors that would give 2Tb boundary for ->i_size, but ICBW.

Al, going down...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Andreas Dilger]

Al Viro writes:
> Andreas Dilger wrote:
> > Actually, this is wrong.  The ext2 inode limit is 2^32 512-byte sectors,
> > not 2^32 blocksize blocks.  Yes this is a wart and Ted wants to fix it, as
> 
> ??? Where? Oh, wait... ->i_blocks? I'ld rather refuse to grow past 2^32 -
> sparse files can legitimately have large offsets and very low ->i_blocks.
> But yes, we need to check for overflows. BTW, 2^32-1 is not good enough -
> indirect blocks also contribute, so limiting ->i_size by 2Tb is not
> guaranteed to keep ->i_blocks low.

Yes, I was thinking i_blocks, but you are correct - I wasn't accounting for
the indirect blocks.  This limit is still {2,4,8,16} times smaller than the
limit you were calculating for i_size.  If we do the i_blocks limit checking
at block allocation time (for large sparse files) this is even better, but
so far it wasn't done...

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ramfs problem... (unlink of sparse file in "D" state)

[Andries.Brouwer]

> Calling pathconf with a symlink is not defined.

The Austin draft requires pathconf to follow symlinks.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/