* Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? @ 2011-12-31 21:50 Michael Lueck 2012-01-02 21:37 ` Dave Chinner 0 siblings, 1 reply; 8+ messages in thread From: Michael Lueck @ 2011-12-31 21:50 UTC (permalink / raw) To: linux-xfs Greetings, Happy old year! ;-) Running XFS on Ununtu 10.04 LTS, I have noticed more warnings with XFS dump the longer the system has been up. A reboot cuts down on the number of warnings xfsdump encounters. An example run of xfsdump is as follows: /usr/sbin/xfsdump: using file dump (drive_simple) strategy /usr/sbin/xfsdump: version 3.0.4 (dump format 3.0) - Running single-threaded /usr/sbin/xfsdump: level 0 dump of ldslnx01:/srv /usr/sbin/xfsdump: dump date: Sat Dec 31 15:25:42 2011 /usr/sbin/xfsdump: session id: b56b3cd7-30d6-44b4-a8a4-e131ee70b60d /usr/sbin/xfsdump: session label: "data" /usr/sbin/xfsdump: ino map phase 1: constructing initial dump list /usr/sbin/xfsdump: ino map phase 2: skipping (no pruning necessary) /usr/sbin/xfsdump: ino map phase 3: skipping (only one dump stream) /usr/sbin/xfsdump: ino map construction complete /usr/sbin/xfsdump: estimated dump size: 115786343168 bytes /usr/sbin/xfsdump: WARNING: no media label specified /usr/sbin/xfsdump: creating dump session media file 0 (media 0, file 0) /usr/sbin/xfsdump: dumping ino map /usr/sbin/xfsdump: dumping directories /usr/sbin/xfsdump: dumping non-directory files /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 234778: Cannot allocate memory (12) /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 234813: Cannot allocate memory (12) /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 1828443: Cannot allocate memory (12) /usr/sbin/xfsdump: WARNING: could not get list of root attributes for nondir ino 1828443: Cannot allocate memory (12) /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 2234019: Cannot allocate memory (12) /usr/sbin/xfsdump: ending media file /usr/sbin/xfsdump: media file size 115563111376 bytes /usr/sbin/xfsdump: dump size (non-dir files) : 115472209824 bytes /usr/sbin/xfsdump: dump complete: 4099 seconds elapsed /usr/sbin/xfsdump: Dump Status: SUCCESS Could someone please explain what is going on, and why a freshly rebooted system would result in less / no errors? Sincerely, -- Michael Lueck Lueck Data Systems http://www.lueckdatasystems.com/ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2011-12-31 21:50 Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? Michael Lueck @ 2012-01-02 21:37 ` Dave Chinner 2012-01-03 3:24 ` Michael Lueck 2012-01-11 9:55 ` Christoph Hellwig 0 siblings, 2 replies; 8+ messages in thread From: Dave Chinner @ 2012-01-02 21:37 UTC (permalink / raw) To: Michael Lueck; +Cc: linux-xfs On Sat, Dec 31, 2011 at 04:50:27PM -0500, Michael Lueck wrote: > Greetings, > > Happy old year! ;-) > > Running XFS on Ununtu 10.04 LTS, I have noticed more warnings with > XFS dump the longer the system has been up. A reboot cuts down on > the number of warnings xfsdump encounters. An example run of xfsdump > is as follows: > > /usr/sbin/xfsdump: using file dump (drive_simple) strategy > /usr/sbin/xfsdump: version 3.0.4 (dump format 3.0) - Running single-threaded > /usr/sbin/xfsdump: level 0 dump of ldslnx01:/srv > /usr/sbin/xfsdump: dump date: Sat Dec 31 15:25:42 2011 > /usr/sbin/xfsdump: session id: b56b3cd7-30d6-44b4-a8a4-e131ee70b60d > /usr/sbin/xfsdump: session label: "data" > /usr/sbin/xfsdump: ino map phase 1: constructing initial dump list > /usr/sbin/xfsdump: ino map phase 2: skipping (no pruning necessary) > /usr/sbin/xfsdump: ino map phase 3: skipping (only one dump stream) > /usr/sbin/xfsdump: ino map construction complete > /usr/sbin/xfsdump: estimated dump size: 115786343168 bytes > /usr/sbin/xfsdump: WARNING: no media label specified > /usr/sbin/xfsdump: creating dump session media file 0 (media 0, file 0) > /usr/sbin/xfsdump: dumping ino map > /usr/sbin/xfsdump: dumping directories > /usr/sbin/xfsdump: dumping non-directory files > /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 234778: Cannot allocate memory (12) > /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 234813: Cannot allocate memory (12) > /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 1828443: Cannot allocate memory (12) > /usr/sbin/xfsdump: WARNING: could not get list of root attributes for nondir ino 1828443: Cannot allocate memory (12) > /usr/sbin/xfsdump: WARNING: could not get list of non-root attributes for nondir ino 2234019: Cannot allocate memory (12) It's getting ENOMEM from the kernel. xfsdump is asking for the list of attributes from the file, and passing in a buffer of XATTR_LIST_MAX bytes, which the xfs_attrmulti_attr_get() function immeidately kmalloc()s for a working buffer. That kmalloc is likely to be failing randomly as it is asking for a contiguous 64k allocation, and that may not be available given memory gets fragmented over time. > /usr/sbin/xfsdump: ending media file > /usr/sbin/xfsdump: media file size 115563111376 bytes > /usr/sbin/xfsdump: dump size (non-dir files) : 115472209824 bytes > /usr/sbin/xfsdump: dump complete: 4099 seconds elapsed > /usr/sbin/xfsdump: Dump Status: SUCCESS > > Could someone please explain what is going on, and why a freshly rebooted system would result in less / no errors? Memory is less fragmented, hence more likely to have a order-5 allocation succeed. Try the patch below, and see if that helps. Cheers, Dave. -- Dave Chinner david@fromorbit.com xfs: handle kmalloc failure when reading attrs From: Dave Chinner <dchinner@redhat.com> xfsdump uses for a large buffer for extended attributes, which has a kmalloc'd shadow buffer in the kernel. This can fail after the system has been running for some time as it is a high order allocation. Convert this to a vmalloc so that it doesn't require contiguous memory and so won't randomly fail while xfsdump is running. Signed-off-by: Dave Chinner <dchinner@redhat.com> --- fs/xfs/xfs_ioctl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c index 2f3f56a..0f3fab1 100644 --- a/fs/xfs/xfs_ioctl.c +++ b/fs/xfs/xfs_ioctl.c @@ -448,7 +448,7 @@ xfs_attrmulti_attr_get( if (*len > XATTR_SIZE_MAX) return EINVAL; - kbuf = kmalloc(*len, GFP_KERNEL); + kbuf = vmalloc(*len, GFP_KERNEL); if (!kbuf) return ENOMEM; @@ -460,7 +460,7 @@ xfs_attrmulti_attr_get( error = EFAULT; out_kfree: - kfree(kbuf); + vfree(kbuf); return error; } _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-02 21:37 ` Dave Chinner @ 2012-01-03 3:24 ` Michael Lueck 2012-01-11 9:55 ` Christoph Hellwig 1 sibling, 0 replies; 8+ messages in thread From: Michael Lueck @ 2012-01-03 3:24 UTC (permalink / raw) To: linux-xfs Dave Chinner wrote: > Memory is less fragmented, hence more likely to have a order-5 > allocation succeed. Try the patch below, and see if that helps. More I was feeling a bit guilty that I was bouncing the machines to have the backup work without error... as "Linux Administrators NEVER reboot their boxes!" I have a few too many things in the test tubes at the moment to take on custom patching a kernel build. So, do these mean that there is anything wrong with the backup being made? Is the final SUCCESS return code in error? Thank you for providing an explanation and patch. Sincerely, -- Michael Lueck Lueck Data Systems http://www.lueckdatasystems.com/ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-02 21:37 ` Dave Chinner 2012-01-03 3:24 ` Michael Lueck @ 2012-01-11 9:55 ` Christoph Hellwig 2012-01-11 10:03 ` Dave Chinner 1 sibling, 1 reply; 8+ messages in thread From: Christoph Hellwig @ 2012-01-11 9:55 UTC (permalink / raw) To: Dave Chinner; +Cc: linux-xfs, Michael Lueck On Tue, Jan 03, 2012 at 08:37:06AM +1100, Dave Chinner wrote: > xfs: handle kmalloc failure when reading attrs > > From: Dave Chinner <dchinner@redhat.com> > > xfsdump uses for a large buffer for extended attributes, which has a > kmalloc'd shadow buffer in the kernel. This can fail after the > system has been running for some time as it is a high order > allocation. Convert this to a vmalloc so that it doesn't require > contiguous memory and so won't randomly fail while xfsdump is > running. I think this should use kmalloc for allocation smaller than a page to avoid the vmalloc overhead for the usual case of small attributes. Also the fs/xattr.c code for the normal xattr system calls needs the same treatment. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-11 9:55 ` Christoph Hellwig @ 2012-01-11 10:03 ` Dave Chinner 2012-01-11 10:04 ` Christoph Hellwig 0 siblings, 1 reply; 8+ messages in thread From: Dave Chinner @ 2012-01-11 10:03 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs, Michael Lueck On Wed, Jan 11, 2012 at 04:55:14AM -0500, Christoph Hellwig wrote: > On Tue, Jan 03, 2012 at 08:37:06AM +1100, Dave Chinner wrote: > > xfs: handle kmalloc failure when reading attrs > > > > From: Dave Chinner <dchinner@redhat.com> > > > > xfsdump uses for a large buffer for extended attributes, which has a > > kmalloc'd shadow buffer in the kernel. This can fail after the > > system has been running for some time as it is a high order > > allocation. Convert this to a vmalloc so that it doesn't require > > contiguous memory and so won't randomly fail while xfsdump is > > running. > > I think this should use kmalloc for allocation smaller than a page to > avoid the vmalloc overhead for the usual case of small attributes. > > Also the fs/xattr.c code for the normal xattr system calls needs the > same treatment. Sounds like a generic kvmalloc()/kvfree() call pair is the answer then. i.e. if kmalloc fails, then use vmalloc()... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-11 10:03 ` Dave Chinner @ 2012-01-11 10:04 ` Christoph Hellwig 2012-01-11 10:14 ` Dave Chinner 0 siblings, 1 reply; 8+ messages in thread From: Christoph Hellwig @ 2012-01-11 10:04 UTC (permalink / raw) To: Dave Chinner; +Cc: Christoph Hellwig, linux-xfs, Michael Lueck On Wed, Jan 11, 2012 at 09:03:16PM +1100, Dave Chinner wrote: > Sounds like a generic kvmalloc()/kvfree() call pair is the answer > then. i.e. if kmalloc fails, then use vmalloc()... If my memory serves me right that's been discussed and rejected a few times before. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-11 10:04 ` Christoph Hellwig @ 2012-01-11 10:14 ` Dave Chinner 2012-01-11 10:29 ` Christoph Hellwig 0 siblings, 1 reply; 8+ messages in thread From: Dave Chinner @ 2012-01-11 10:14 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs, Michael Lueck On Wed, Jan 11, 2012 at 05:04:12AM -0500, Christoph Hellwig wrote: > On Wed, Jan 11, 2012 at 09:03:16PM +1100, Dave Chinner wrote: > > Sounds like a generic kvmalloc()/kvfree() call pair is the answer > > then. i.e. if kmalloc fails, then use vmalloc()... > > If my memory serves me right that's been discussed and rejected a few > times before. Which has lead to several implementations of duplicated functionality like: - ext4_kvmalloc/ext4_kvfree - drm_malloc_ab/drm_calloc_large/drm_free_large - ceph_buffer_new/ceph_buffer_release - gfs2_alloc_sort_buffer/gfs2_free_sort_buffer - __ntfs_malloc/ntfs_free I don't think adding another two sets of identical wrappers (or open coding it) is the right thing given that it is a repeating pattern. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? 2012-01-11 10:14 ` Dave Chinner @ 2012-01-11 10:29 ` Christoph Hellwig 0 siblings, 0 replies; 8+ messages in thread From: Christoph Hellwig @ 2012-01-11 10:29 UTC (permalink / raw) To: Dave Chinner; +Cc: Christoph Hellwig, linux-xfs, Michael Lueck On Wed, Jan 11, 2012 at 09:14:27PM +1100, Dave Chinner wrote: > Which has lead to several implementations of duplicated > functionality like: > > - ext4_kvmalloc/ext4_kvfree > - drm_malloc_ab/drm_calloc_large/drm_free_large > - ceph_buffer_new/ceph_buffer_release > - gfs2_alloc_sort_buffer/gfs2_free_sort_buffer > - __ntfs_malloc/ntfs_free > > I don't think adding another two sets of identical wrappers (or open > coding it) is the right thing given that it is a repeating pattern. Give it a try, I'm not against it myself. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-01-11 10:29 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-12-31 21:50 Why does xfsdump encounter "WARNING: could not get list of non-root attributes for nondir ino nnnn: Cannot allocate memory (12)"? Michael Lueck 2012-01-02 21:37 ` Dave Chinner 2012-01-03 3:24 ` Michael Lueck 2012-01-11 9:55 ` Christoph Hellwig 2012-01-11 10:03 ` Dave Chinner 2012-01-11 10:04 ` Christoph Hellwig 2012-01-11 10:14 ` Dave Chinner 2012-01-11 10:29 ` Christoph Hellwig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox