libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Louis-David Mitterrand]

Package: xfsprogs
Version: 2.8.11-1
Followup-For: Bug #382935

Hello,

Using xfs_repair on a 2TB partition I get:

	Phase 1 - find and verify superblock...
	Phase 2 - using internal log
			- zero log...
			- scan filesystem freespace and inode maps...
			- found root inode chunk
	Phase 3 - for each AG...
			- scan and clear agi unlinked lists...
			- process known inodes and perform inode discovery...
			- agno = 0
			- agno = 1
			- agno = 2
			- agno = 3
			- agno = 4
			- agno = 5
			- agno = 6
			- agno = 7
			- agno = 8
			- agno = 9
			- agno = 10
	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory



-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (499, 'experimental')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17-6-debian-styx
Locale: LANG=en_CA, LC_CTYPE=fr_FR@euro (charmap=ISO-8859-15)

Versions of packages xfsprogs depends on:
ii  libc6                        2.3.6.ds1-4 GNU C Library: Shared libraries
ii  libreadline5                 5.1-7       GNU readline and history libraries
ii  libuuid1                     1.39-1      universally unique id library

xfsprogs recommends no packages.

-- no debconf information

libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Louis-David Mitterrand]

Package: xfsprogs
Version: 2.8.11-1
Followup-For: Bug #382935

Hello,

Using xfs_repair on a 2TB partition I get:

	Phase 1 - find and verify superblock...
	Phase 2 - using internal log
			- zero log...
			- scan filesystem freespace and inode maps...
			- found root inode chunk
	Phase 3 - for each AG...
			- scan and clear agi unlinked lists...
			- process known inodes and perform inode discovery...
			- agno = 0
			- agno = 1
			- agno = 2
			- agno = 3
			- agno = 4
			- agno = 5
			- agno = 6
			- agno = 7
			- agno = 8
			- agno = 9
			- agno = 10
	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory



-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (499, 'experimental')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17-6-debian-styx
Locale: LANG=en_CA, LC_CTYPE=fr_FR@euro (charmap=ISO-8859-15)

Versions of packages xfsprogs depends on:
ii  libc6                        2.3.6.ds1-4 GNU C Library: Shared libraries
ii  libreadline5                 5.1-7       GNU readline and history libraries
ii  libuuid1                     1.39-1      universally unique id library

xfsprogs recommends no packages.

-- no debconf information

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Peter Grandi]

>>> On Sun, 27 Aug 2006 09:28:10 +0200, Louis-David Mitterrand
>>> <vindex+lists-xfs@apartia.org> said:

[ ... ]

> xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

Most probably this is not a bug. The system probably does not
have enough memory. "Cannot allocate memory" is probably what
is happening. This has been discussed many times previously...

   http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html

[ ... ]

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Chris Wedgwood]

On Sun, Aug 27, 2006 at 09:28:10AM +0200, Louis-David Mitterrand wrote:

> 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

Not enough memory, try adding swap space if that's possible?  (BTW,
how much memory do you have in this case?)

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Louis-David Mitterrand]

On Sun, Aug 27, 2006 at 01:42:22PM -0700, Chris Wedgwood wrote:
> On Sun, Aug 27, 2006 at 09:28:10AM +0200, Louis-David Mitterrand wrote:
> 
> > 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory
> 
> Not enough memory, try adding swap space if that's possible?  (BTW,
> how much memory do you have in this case?)

I had 2GB RAM and a 300GB swap partition and was monitoring memory 
consumption with "top": it never went over 3BG before failing.

With the 2.7.x version of xfs_repair found on a live CD distro, memory 
consumption never exceeds 1GB on the same filesystem. Why has the 2.8.11 
version become so greedy?

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[David Chinner]

On Sun, Aug 27, 2006 at 09:28:10AM +0200, Louis-David Mitterrand wrote:
> Package: xfsprogs
> Version: 2.8.11-1
> Followup-For: Bug #382935
> 
> Hello,
> 
> Using xfs_repair on a 2TB partition I get:
.....
> 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory
> 
> 
> 
> -- System Information:
> Debian Release: testing/unstable
>   APT prefers unstable
>   APT policy: (500, 'unstable'), (500, 'testing'), (499, 'experimental')
> Architecture: i386 (i686)

ia32, multi-terabyte filesystem, so as Peter pointed out:

On Sun, Aug 27, 2006 at 12:36:05PM +0100, Peter Grandi wrote:
> Most probably this is not a bug. The system probably does not
> have enough memory. "Cannot allocate memory" is probably what
> is happening. This has been discussed many times previously...
> 
>    http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html

This will be the problem.

On Sun, Aug 27, 2006 at 10:51:19PM +0200, Louis-David Mitterrand wrote:
> On Sun, Aug 27, 2006 at 01:42:22PM -0700, Chris Wedgwood wrote:
> > On Sun, Aug 27, 2006 at 09:28:10AM +0200, Louis-David Mitterrand wrote:
> > 
> > > 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory
> > 
> > Not enough memory, try adding swap space if that's possible?  (BTW,
> > how much memory do you have in this case?)
> 
> I had 2GB RAM and a 300GB swap partition and was monitoring memory 
> consumption with "top": it never went over 3BG before failing.

It' won't matter how much swap you add - ia32 has a per-process memory
limit of 1-4GB RAM depending on kernel build options.

> With the 2.7.x version of xfs_repair found on a live CD distro, memory 
> consumption never exceeds 1GB on the same filesystem. Why has the 2.8.11 
> version become so greedy?

xfs_repair is undergoing a lot of change at the moment and so memory usage
may have blown out under some circumstances. Do you have figures on how much
more memory xfs_repair is using?

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Peter Grandi]

>>> On Sun, 27 Aug 2006 22:51:19 +0200, Louis-David Mitterrand
>>> <vindex+lists-xfs@apartia.org> said:

>> [ ... ] xfs_repair: libxfs_initbuf can't memalign 4096 bytes:
>> Cannot allocate memory [ ... ]

> I had 2GB RAM and a 300GB swap partition and was monitoring
> memory consumption with "top": it never went over 3BG before
> failing.

The default 3GiB address space limit per process on a 32 bit
system is rather well documented, for a summary:

  http://WWW.sabi.co.UK/Notes/#060821c

Let's mention this classic entry again:

  http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html

   «> > Your filesystem (8TiB) may simply bee too large for your
    > > system to be able to repair. Try mounting it on a 64bit
    > > system with more RAM in it and repairing it from there.

    Now that linux supports larger than 2TiB filesystems on 32
    bit systems, this is true for Linux as well.»

A previous entry in the same thread might help too:

  http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00037.html

   «> I try xfs_check and xfs_ncheck (and more progs) with
    > +200GB swap, but no different! less than 1 second and get
    > out of memory.

    Swap won't help if you're running an ia32 (32bit) kernel -
    you have a per-process memory limit of 1-4GiB (depending on
    kernel and config). The amount of physical memory and swap
    does not change this limitation.»

Trying hard to avoid looking at what «has been discussed many
times previously» is a great time saving strategy! :-)

RE: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Barry Naujok]

 

> -----Original Message-----
> From: xfs-bounce@oss.sgi.com [mailto:xfs-bounce@oss.sgi.com] 
> On Behalf Of Louis-David Mitterrand
> Sent: Sunday, 27 August 2006 5:21 PM
> To: xfs@oss.sgi.com
> Subject: libxfs_initbuf can't memalign 4096 bytes: Cannot 
> allocate memory
> 
> Package: xfsprogs
> Version: 2.8.11-1
> Followup-For: Bug #382935
> 
> Hello,
> 
> Using xfs_repair on a 2TB partition I get:
[snip] 
> 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: 
> Cannot allocate memory
 
Can you tell me how many inodes you have on your filesystem?

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Louis-David Mitterrand]

On Mon, Aug 28, 2006 at 03:57:02PM +1000, Barry Naujok wrote:
> > 
> > Hello,
> > 
> > Using xfs_repair on a 2TB partition I get:
> [snip] 
> > 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: 
> > Cannot allocate memory
>  
> Can you tell me how many inodes you have on your filesystem?
> 

	sylla:~# xfs_info /dev/md1
	meta-data=/dev/root              isize=256    agcount=32, agsize=15252656 blks
			 =                       sectsz=4096  attr=0
	data     =                       bsize=4096   blocks=488084704, imaxpct=25
			 =                       sunit=16     swidth=80 blks, unwritten=1
	naming   =version 2              bsize=4096  
	log      =internal               bsize=4096   blocks=32768, version=2
			 =                       sectsz=4096  sunit=1 blks
	realtime =none                   extsz=327680 blocks=0, rtextents=0

Hmmm, no inode info here... How do I find it?

Re: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory

[Louis-David Mitterrand]

On Mon, Aug 28, 2006 at 04:57:51PM +0200, Louis-David Mitterrand wrote:
> On Mon, Aug 28, 2006 at 03:57:02PM +1000, Barry Naujok wrote:
> > > 
> > > Hello,
> > > 
> > > Using xfs_repair on a 2TB partition I get:
> > [snip] 
> > > 	xfs_repair: libxfs_initbuf can't memalign 4096 bytes: 
> > > Cannot allocate memory
> >  
> > Can you tell me how many inodes you have on your filesystem?
> > 
> 	sylla:~# xfs_info /dev/md1
> 	meta-data=/dev/root              isize=256    agcount=32, agsize=15252656 blks
> 			 =                       sectsz=4096  attr=0
> 	data     =                       bsize=4096   blocks=488084704, imaxpct=25
> 			 =                       sunit=16     swidth=80 blks, unwritten=1
> 	naming   =version 2              bsize=4096  
> 	log      =internal               bsize=4096   blocks=32768, version=2
> 			 =                       sectsz=4096  sunit=1 blks
> 	realtime =none                   extsz=327680 blocks=0, rtextents=0
> 
> Hmmm, no inode info here... How do I find it?

Ok, got it:

sylla:~# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/md1             895571792 7921236 887650556    1% /