JBD: ext2online wants too many credits (744 > 256)

JBD: ext2online wants too many credits (744 > 256)

[Frank van Maarseveen]

2.6.20.6, FC4:

I created a 91248k ext3 fs with 4k blocksize:

| mke2fs -j -b 4096 /dev/vol1/project 
| mke2fs 1.38 (30-Jun-2005)
| Filesystem label=
| OS type: Linux
| Block size=4096 (log=2)
| Fragment size=4096 (log=2)
| 23552 inodes, 23552 blocks
| 1177 blocks (5.00%) reserved for the super user
| First data block=0
| Maximum filesystem blocks=25165824
| 1 block group
| 32768 blocks per group, 32768 fragments per group
| 23552 inodes per group

Writing inode tables: done                            
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

Next, I tried to resize it to about 3G using ext2online while mounted:

| # ext2online /dev/vol1/project 
| ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
| ext2online: ext2_ioctl: No space left on device
|
| ext2online: unable to resize /dev/mapper/vol1-project

At that time the kernel said:

|JBD: ext2online wants too many credits (744 > 256)

What is the limitation I should be aware of? Has it something to do with
the journal log size?

The size actually did increase a bit, to 128112k.


Steps to reproduce:
Create a 3G partition, say /dev/vol1/project
mke2fs -j -b 4096 /dev/vol1/project 22812
mount it
ext2online /dev/vol1/project said:

| ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
| ext2online: ext2_ioctl: No space left on device
| 
| ext2online: unable to resize /dev/mapper/vol1-project

kernel said:

| JBD: ext2online wants too many credits (721 > 256)

-- 
Frank

Re: JBD: ext2online wants too many credits (744 > 256)

[Andrew Morton]

On Mon, 7 May 2007 00:26:26 +0200 Frank van Maarseveen <frankvm@frankvm.com> wrote:

> 2.6.20.6, FC4:
> 
> I created a 91248k ext3 fs with 4k blocksize:
> 
> | mke2fs -j -b 4096 /dev/vol1/project 
> | mke2fs 1.38 (30-Jun-2005)
> | Filesystem label=
> | OS type: Linux
> | Block size=4096 (log=2)
> | Fragment size=4096 (log=2)
> | 23552 inodes, 23552 blocks
> | 1177 blocks (5.00%) reserved for the super user
> | First data block=0
> | Maximum filesystem blocks=25165824
> | 1 block group
> | 32768 blocks per group, 32768 fragments per group
> | 23552 inodes per group
> 
> Writing inode tables: done                            
> Creating journal (1024 blocks): done
> Writing superblocks and filesystem accounting information: done
> 
> Next, I tried to resize it to about 3G using ext2online while mounted:
> 
> | # ext2online /dev/vol1/project 
> | ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
> | ext2online: ext2_ioctl: No space left on device
> |
> | ext2online: unable to resize /dev/mapper/vol1-project
> 
> At that time the kernel said:
> 
> |JBD: ext2online wants too many credits (744 > 256)
> 
> What is the limitation I should be aware of? Has it something to do with
> the journal log size?
> 
> The size actually did increase a bit, to 128112k.
> 
> 
> Steps to reproduce:
> Create a 3G partition, say /dev/vol1/project
> mke2fs -j -b 4096 /dev/vol1/project 22812
> mount it
> ext2online /dev/vol1/project said:
> 
> | ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
> | ext2online: ext2_ioctl: No space left on device
> | 
> | ext2online: unable to resize /dev/mapper/vol1-project
> 
> kernel said:
> 
> | JBD: ext2online wants too many credits (721 > 256)
> 

(added linux-ext4 to cc)

Re: JBD: ext2online wants too many credits (744 > 256)

[Frank van Maarseveen]

On Sun, May 06, 2007 at 09:40:14PM -0700, Andrew Morton wrote:
> On Mon, 7 May 2007 00:26:26 +0200 Frank van Maarseveen <frankvm@frankvm.com> wrote:
> 
> > Steps to reproduce:
> > Create a 3G partition, say /dev/vol1/project
> > mke2fs -j -b 4096 /dev/vol1/project 22812
> > mount it
> > ext2online /dev/vol1/project said:
> > 
> > | ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
> > | ext2online: ext2_ioctl: No space left on device
> > | 
> > | ext2online: unable to resize /dev/mapper/vol1-project
> > 
> > kernel said:
> > 
> > | JBD: ext2online wants too many credits (721 > 256)

There's a threshold for the problem depending on the initial
size. This one fails:

	mke2fs -j -b 4096 /dev/<3GB-blockdev> 32768
	(mount + ext2online or resize2fs)
	kernel: JBD: resize2fs wants too many credits (1034 > 1024)

Add one block to the initial mke2fs (32768+1 == 32769) and the
problem is gone.

Without the -b 4096 there's another resize problem

	mke2fs -j /dev/loop1 2048
	mount /dev/loop1 /1
	resize2fs /dev/loop1

says:
	resize2fs 1.40-WIP (14-Nov-2006)
	Filesystem at /dev/loop1 is mounted on /1; on-line resizing required
	old desc_blocks = 1, new_desc_blocks = 12
	Performing an on-line resize of /dev/loop1 to 3072000 (1k) blocks.
	resize2fs: Invalid argument While trying to add group #256

and the kernel says:

May  7 15:36:08 lokka EXT3-fs warning (device loop1): verify_reserved_gdb: 
May  7 15:36:08 lokka reserved GDT 10 missing grp 1 (8202)
May  7 15:36:08 lokka  

After that, the filesystem has been resized to 2GB. I recall a 2G
(?) limit for ext3 resizing with 1k blocksize but trying the above with
4096 1k blocks seems to work. fsck says it's ok all the time.

-- 
Frank

Re: JBD: ext2online wants too many credits (744 > 256)

[Andreas Dilger]

On May 06, 2007  21:40 -0700, Andrew Morton wrote:
> On Mon, 7 May 2007 00:26:26 +0200 Frank van Maarseveen <frankvm@frankvm.com> wrote:
> 
> > 2.6.20.6, FC4:
> > 
> > I created a 91248k ext3 fs with 4k blocksize:
> > 
> > Next, I tried to resize it to about 3G using ext2online while mounted:
> > 
> > At that time the kernel said:
> > 
> > |JBD: ext2online wants too many credits (744 > 256)
> > 
> > What is the limitation I should be aware of? Has it something to do with
> > the journal log size?

Yes, for very small filesystems the default journal size is only 4MB, and
the maximum transaction size is 1MB (256 blocks).

If the filesystem was 128MB or larger in the initial mke2fs then the
journal would be 16MB and the resize would get up to 1024 blocks for
the transaction.

I'm not sure what the right solution is for this.  If you know you will
be resizing the fs you could increase the initial journal size at mke2fs
time (-J size=16).  Alternately, it is possible resize to an intermediate
size and then delete and recreate the journal via tune2fs (which would
be the larger size by default) but that can only be done offline.  

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: JBD: ext2online wants too many credits (744 > 256)

[Theodore Tso]

On Mon, May 07, 2007 at 12:26:26AM +0200, Frank van Maarseveen wrote:
> 2.6.20.6, FC4:
> 
> |JBD: ext2online wants too many credits (744 > 256)

You would be better off using resize2fs from e2fsprogs 1.39, BTW....

About the only reason to keep using the ext2resize packages is
ext2prepare, which is the off-line tool to add a resizing inode after
the fact, and the main reason that functionality hasn't been absorbed
into e2fsprogs is that it needs to be completely rewritten before I'm
willing to trust and maintain it...

> What is the limitation I should be aware of? Has it something to do with
> the journal log size?

Yes, the problem is the journal log size.  Since the original
filesystem was so small, the journal was kept small so as not to use a
huge percentage of the filesystem size.  Of course, with a larger
filesystem you probably want a larger journal anyway to get better
performance.  So that brings up a related problem, which is we don't
have a way of dynamically increasing the size of the journal, which we
probably would want to do as part of resizing the filesystem size.

So the short-term workaround for now, if you know that you're going to
take a filesystem which is 22M and increase it to 3G, is to create it
with largish journal in the first place:

	mke2fs -j -b 4096 -J size=16M /dev/vol1/project 22812

Note that if you were creating a 3G filesystem from scratch, by
default it would be using a 128M journal --- but it's a little hard to
fit a 128M journal in a 22M filesystem.  :-/

We're going to have to rethink the defaults of the journal size as it
relates to on-line resizing, but there might not be a lot of great
answers here.  We'll also have to look at the on-line resizing code
and see if there's a way to break up the resize operation into smaller
transactions as a way of avoiding this problem --- but that would
still leave the user stuck with a pathetically small 4M journal on a
3G filesystem.

						- Ted

Re: JBD: ext2online wants too many credits (744 > 256)

[david]

On Mon, 7 May 2007, Theodore Tso wrote:

> We're going to have to rethink the defaults of the journal size as it
> relates to on-line resizing, but there might not be a lot of great
> answers here.  We'll also have to look at the on-line resizing code
> and see if there's a way to break up the resize operation into smaller
> transactions as a way of avoiding this problem --- but that would
> still leave the user stuck with a pathetically small 4M journal on a
> 3G filesystem.

why not just allocate a new journal (something along the lines of convert 
the ext3 to ext2 then back to ext3 would be the ugly, scripted way of 
doing it)

or is the problem that you are trying to resize things without remounting 
them (and therefor without flushing the journal)

David Lang

Re: JBD: ext2online wants too many credits (744 > 256)

[Theodore Tso]

On Mon, May 07, 2007 at 07:46:38AM -0700, david@lang.hm wrote:
> On Mon, 7 May 2007, Theodore Tso wrote:
> 
> >We're going to have to rethink the defaults of the journal size as it
> >relates to on-line resizing, but there might not be a lot of great
> >answers here.  We'll also have to look at the on-line resizing code
> >and see if there's a way to break up the resize operation into smaller
> >transactions as a way of avoiding this problem --- but that would
> >still leave the user stuck with a pathetically small 4M journal on a
> >3G filesystem.
> 
> why not just allocate a new journal (something along the lines of convert 
> the ext3 to ext2 then back to ext3 would be the ugly, scripted way of 
> doing it)
> 
> or is the problem that you are trying to resize things without remounting 
> them (and therefor without flushing the journal)

Yes, I'm referring to the difficulties of getting this right when
doing online (mounted) resizing, as that was the context of the
original bug report.

Clearly the right thing for resize2fs to do when it's doing an
off-line resize is to just adjust the journal inode and make it
bigger.  I'll add that to my todo list, but of course, patches to do
that would be gratefully accepted...

						- Ted

Re: JBD: ext2online wants too many credits (744 > 256)

[Andreas Dilger]

On May 07, 2007  11:50 -0400, Theodore Tso wrote:
> On Mon, May 07, 2007 at 07:46:38AM -0700, david@lang.hm wrote:
> > On Mon, 7 May 2007, Theodore Tso wrote:
> > >We're going to have to rethink the defaults of the journal size as it
> > >relates to on-line resizing, but there might not be a lot of great
> > >answers here.  We'll also have to look at the on-line resizing code
> > >and see if there's a way to break up the resize operation into smaller
> > >transactions as a way of avoiding this problem --- but that would
> > >still leave the user stuck with a pathetically small 4M journal on a
> > >3G filesystem.
> 
> Clearly the right thing for resize2fs to do when it's doing an
> off-line resize is to just adjust the journal inode and make it
> bigger.  I'll add that to my todo list, but of course, patches to do
> that would be gratefully accepted...

For that matter, there was a paper from U. Wisconsin showing that
having the journal in the middle of the filesystem gave noticably
better performance due to lower average seek times.  If anyone is
looking at messing with the journal that is probably also a good
and easy place to start (e.g. may be as easy as just giving a goal
block of s_blocks_count / 2 to the journal create code).

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.