Re: XFS internal error xfs_trans_cancel at line 1150 of file fs/xfs/xfs_trans.c

[David Chinner <dgc@sgi.com>]

On Fri, Mar 07, 2008 at 12:19:28PM +0100, Christian Røsnes wrote:
> >  Actually, a single mkdir command is enough to trigger the filesystem
> >  shutdown when its 99% full (according to df -k):
> >
> >  /data# mkdir test
> >  mkdir: cannot create directory `test': No space left on device

Ok, that's helpful ;)

So, can you dump the directory inode with xfs_db? i.e.

# ls -ia /data

The directory inode is the inode at ".", and if this is the root of
the filesystem it will probably be 128.

Then run:

# xfs_db -r -c 'inode 128' -c p /dev/sdb1

> >  --------
> >  meta-data=/dev/sdb1              isize=512    agcount=16, agsize=4476752 blks
> >          =                       sectsz=512   attr=0
> >  data     =                       bsize=4096   blocks=71627792, imaxpct=25
> >          =                       sunit=16     swidth=32 blks, unwritten=1
> >  naming   =version 2              bsize=4096
> >  log      =internal               bsize=4096   blocks=32768, version=2
> >          =                       sectsz=512   sunit=16 blks, lazy-count=0
> >  realtime =none                   extsz=65536  blocks=0, rtextents=0
> >
> >  xfs_db -r -c 'sb 0' -c p /dev/sdb1
> >  ----------------------------------
.....
> >  fdblocks = 847484

Apparently there are still lots of free blocks. I wonder if you are out of
space in the metadata AGs.

Can you do this for me:

-------
#!/bin/bash

for i in `seq 0 1 15`; do
	echo freespace histogram for AG $i
	xfs_db -r -c "freesp -bs -a $i" /dev/sdb1
done
------

> Instrumenting the code, I found that this occurs on my system when I
> do a 'mkdir /data/test' on the partition in question:
> 
> in xfs_mkdir  (xfs_vnodeops.c):
> 
>   error = xfs_dir_ialloc(&tp, dp, mode, 2,
>                         0, credp, prid, resblks > 0,
>                 &cdp, NULL);
> 
>         if (error) {
>                 if (error == ENOSPC)
>                         goto error_return;   <=== this is hit and then
> execution jumps to error_return
>                 goto abort_return;
>         }

Ah - you can ignore my last email, then. You're already one step ahead
of me ;)

This does not appear to be the case I was expecting, though I can
see how we can get an ENOSPC here with plenty of blocks free - none
are large enough to allocate an inode chunk. What would be worth
knowing is the value of resblks when this error is reported.

This tends to imply we are returning an ENOSPC with a dirty
transaction. Right now I can't see how that would occur, though
the fragmented free space is something I can try to reproduce with.

> Is this the correct behavior for this type of situation: mkdir command
> fails due to no available space on filesystem,
> and xfs_mkdir goes to label error_return  ? (And after this the
> filesystem is shutdown)

The filesystem should not be shutdown here. We need to trace through
further to the source of the error....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group